Re: [DISCUSS] KIP-516: Topic Identifiers

[Justine Olshan <jo...@confluent.io>]

Hello all, it's been almost a year! I've made some changes to this KIP and hope to continue the discussion. 

One of the main changes I've added is now the metadata response will include the topic ID (as Colin suggested). Clients can obtain the topicID of a given topic through a TopicDescription. The topicId will also be included with the UpdateMetadata request. 

Let me know what you all think.
Thank you,
Justine

On 2019/09/13 16:38:26, "Colin McCabe" <cm...@apache.org> wrote: 
> Hi Lucas,
> 
> Thanks for tackling this.  Topic IDs are a great idea, and this is a really good writeup.
> 
> For /brokers/topics/[topic], the schema version should be bumped to version 3, rather than 2.  KIP-455 bumped the version of this znode to 2 already :)
> 
> Given that we're going to be seeing these things as strings as lot (in logs, in ZooKeeper, on the command-line, etc.), does it make sense to use base64 when converting them to strings?
> 
> Here is an example of the hex representation:
> 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> 
> And here is an example in base64.
> b8tRS7h4TJ2Vt43Dp85v2A
> 
> The base64 version saves 15 letters (to be fair, 4 of those were dashes that we could have elided in the hex representation.)
> 
> Another thing to consider is that we should specify that the all-zeroes UUID is not a valid topic UUID.   We can't use null for this because we can't pass a null UUID over the RPC protocol (there is no special pattern for null, nor do we want to waste space reserving such a pattern.)
> 
> Maybe I missed it, but did you describe "migration of... existing topic[s] without topic IDs" in detail in any section?  It seems like when the new controller becomes active, it should just generate random UUIDs for these, and write the random UUIDs back to ZooKeeper.  It would be good to spell that out.  We should make it clear that this happens regardless of the inter-broker protocol version (it's a compatible change).
> 
> "LeaderAndIsrRequests including an is_every_partition flag" seems a bit wordy.  Can we just call these "full LeaderAndIsrRequests"?  Then the RPC field could be named "full".  Also, it would probably be better for the RPC field to be an enum of { UNSPECIFIED, INCREMENTAL, FULL }, so that we can cleanly handle old versions (by treating them as UNSPECIFIED)
> 
> In the LeaderAndIsrRequest section, you write "A final deletion event will be secheduled for X ms after the LeaderAndIsrRequest was first received..."  I guess the X was a placeholder that you intended to replace before posting? :)  In any case, this seems like the kind of thing we'd want a configuration for.  Let's describe that configuration key somewhere in this KIP, including what its default value is.
> 
> We should probably also log a bunch of messages at WARN level when something is scheduled for deletion, as well.  (Maybe this was assumed, but it would be good to mention it).
> 
> I feel like there are a few sections that should be moved to "rejected alternatives."  For example, in the DeleteTopics section, since we're not going to do option 1 or 2, these should be moved into "rejected alternatives,"  rather than appearing inline.  Another case is the "Should we remove topic name from the protocol where possible" section.  This is clearly discussing a design alternative that we're not proposing to implement: removing the topic name from those protocols.
> 
> Is it really necessary to have a new /admin/delete_topics_by_id path in ZooKeeper?  It seems like we don't really need this.  Whenever there is a new controller, we'll send out full LeaderAndIsrRequests which will trigger the stale topics to be cleaned up.   The active controller will also send the full LeaderAndIsrRequest to brokers that are just starting up.    So we don't really need this kind of two-phase commit (send out StopReplicasRequest, get ACKs from all nodes, commit by removing /admin/delete_topics node) any more.
> 
> You mention that FetchRequest will now include UUID to avoid issues where requests are made to stale partitions.  However, adding a UUID to MetadataRequest is listed as future work, out of scope for this KIP.  How will the client learn what the topic UUID is, if the metadata response doesn't include that information?  It seems like adding the UUID to MetadataResponse would be an improvement here that might not be too hard to make.
> 
> best,
> Colin
> 
> 
> On Mon, Sep 9, 2019, at 17:48, Ryanne Dolan wrote:
> > Lucas, this would be great. I've run into issues with topics being
> > resurrected accidentally, since a client cannot easily distinguish between
> > a deleted topic and a new topic with the same name. I'd need the ID
> > accessible from the client to solve that issue, but this is a good first
> > step.
> > 
> > Ryanne
> > 
> > On Wed, Sep 4, 2019 at 1:41 PM Lucas Bradstreet <lu...@confluent.io> wrote:
> > 
> > > Hi all,
> > >
> > > I would like to kick off discussion of KIP-516, an implementation of topic
> > > IDs for Kafka. Topic IDs aim to solve topic uniqueness problems in Kafka,
> > > where referring to a topic by name alone is insufficient. Such cases
> > > include when a topic has been deleted and recreated with the same name.
> > >
> > > Unique identifiers will help simplify and improve Kafka's topic deletion
> > > process, as well as prevent cases where brokers may incorrectly interact
> > > with stale versions of topics.
> > >
> > >
> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > >
> > > Looking forward to your thoughts.
> > >
> > > Lucas
> > >
> >
> 

Re: [DISCUSS] KIP-516: Topic Identifiers

[Justine Olshan <jo...@confluent.io>]

Hi all,

Thanks for the discussion. I'm glad we are able to get our best ideas out
there.

David Jacot
1. I apologize for the incorrect information. I have fixed the KIP.
2. Yes. The difference between full and incremental is that on full we
check for the two types of stale request—topics on the broker that are not
contained in the request, and topics in the request that do not match the
id stored on the broker. In incremental, we can only delete in the second
scenario since not all topics are in the request.
3. Yes we should use delete.stale.topic.delay.ms in both FULL and
INCREMENTAL requests.

I’ve updated the KIP to make some of these things more clear.


Ismael
Removing the topic ID has been requested by a few in this discussion so
I’ve decided to do this. I’ve updated the KIP to reflect this new protocol
and explained the reasoning.

As for partition metadata file. On LeaderAndIsr requests, we will check
this file for all the topic partitions included the request. If the topic
ID in the file does not match the topic ID in the request, it implies that
the local topic partition is stale, as the topic must have been deleted to
create a new topic with a different topic ID. We will mark this topic for
deletion. I’ve updated the KIP to make this more clear in the partition
metadata file section.

Jun
I’ve updated the KIP to fix some of the points you brought up in 30, 31,
and 32. 33 will require a bit more thought, so I will get back to you on
that.

Thanks,
Justine

On Thu, Sep 24, 2020 at 11:10 AM Jun Rao <ju...@confluent.io> wrote:

> Hi, Justine,
>
> Thanks for the updated KIP. A few more comments below.
>
> 30.  {"name": "id", "type": "string", "doc": "version id"}}: The doc should
> say UUID.
>
> 31. LeaderAndIsrResponse v5 and StopReplicaResponse v4 : It seems there is
> no need to add topic_id at partitions level.
>
> 32. Regarding partition metadata file. Perhaps the key can be a single
> word, sth like the following.
> version: 0
> topic_id: 46bdb63f-9e8d-4a38-bf7b-ee4eb2a794e4
>
> 33. Another tricky thing that I realized is how to support the metadata
> topic introduced in KIP-595. It's a bit tricky to assign a UUID to the
> metadata topic since we have a chicken-and-egg problem. The controller
> needs to persist the UUID in the metadata topic in order to assign one
> successfully, but the metadata topic is needed to elect a controller
> (KIP-631). So, this probably needs a bit more thought.
>
> Jun
>
> On Thu, Sep 24, 2020 at 3:04 AM Ismael Juma <is...@juma.me.uk> wrote:
>
> > Also, can we provide more details on how the Partition Metadata file will
> > be used?
> >
> > Ismael
> >
> > On Thu, Sep 24, 2020 at 3:01 AM Ismael Juma <is...@juma.me.uk> wrote:
> >
> > > Hi Justine,
> > >
> > > I think we need to update the "Rejected Alternatives" section to take
> > into
> > > account that the proposal now removes the topic name from the fetch
> > > request. Also, if we are removing it from the Fetch request, does it
> make
> > > sense not to remove it from similar requests like ListOffsetRequest?
> > >
> > > Ismael
> > >
> > > On Thu, Sep 24, 2020 at 2:46 AM David Jacot <dj...@confluent.io>
> wrote:
> > >
> > >> Hi Justine,
> > >>
> > >> Thanks for the KIP. I finally had time to read it :). It is a great
> > >> improvement.
> > >>
> > >> I have a few comments/questions:
> > >>
> > >> 1. It seems that the schema of the StopReplicaRequest is slightly
> > >> outdated.
> > >> We
> > >> did some changes as part of KIP-570. V3 is already organized by
> topics.
> > >>
> > >> 2. I just want to make sure that I understand the reconciliation
> > >> logic correctly. When
> > >> an "INCREMENTAL" LeaderAndIsr Request is received, the broker will
> also
> > >> reconcile
> > >> when the local uuid does not match the uuid in the request, right? In
> > this
> > >> case, the
> > >> local log is staged for deletion.
> > >>
> > >> 3. In the documentation of the `delete.stale.topic.delay.ms` config,
> it
> > >> says "When a
> > >> FULL LeaderAndIsrRequest is received..." but I suppose it applies to
> > both
> > >> types.
> > >>
> > >> Best,
> > >> David
> > >>
> > >> On Thu, Sep 24, 2020 at 1:40 AM Justine Olshan <jo...@confluent.io>
> > >> wrote:
> > >>
> > >> > Hi Jun,
> > >> >
> > >> > Thanks for the comments. I apologize for some of the typos and
> > >> confusion.
> > >> > I’ve updated the KIP to fix some of the issues you mentioned.
> > >> >
> > >> > 20.2 I’ve changed the type to String.
> > >> > 20.1/3 I’ve updated the TopicZNode to fix formatting and reflect the
> > >> latest
> > >> > version before this change.
> > >> >
> > >> > 21. You are correct and I’ve removed this line. I’ve also added a
> line
> > >> > mentioning an IBP bump is necessary for migration
> > >> >
> > >> > 22. I think the wording was unclear but your summary is what was
> > >> intended
> > >> > by this line. I’ve updated to make this point more clear. “Remove
> > >> deleted
> > >> > topics from replicas by sending StopReplicaRequest V3 before the IBP
> > >> bump
> > >> > using the old logic, and using V4 and the new logic with topic IDs
> > after
> > >> > the IBP bump.”
> > >> >
> > >> > 23. I’ve removed the unspecified type from the KIP and mention that
> > IBP
> > >> > will be used to handle this request. “IBP will be used to determine
> > >> whether
> > >> > this new form of the request will be used. For older requests, we
> will
> > >> > ignore this field and default to previous behavior.”
> > >> >
> > >> > 24. I’ve fixed this typo.
> > >> >
> > >> > 25. I've added a topics at a higher level for LeaderAndIsrResponse
> v5,
> > >> > StopReplicaResponse v4. I've also changed StopReplicaRequest v4 to
> > have
> > >> > topics at a higher level.
> > >> >
> > >> > 26. I’ve updated forgotten_topics_data--added the topic ID and
> removed
> > >> the
> > >> > topic name
> > >> >
> > >> > 27. I’ve decided on plain text, and I’ve added an example.
> > >> >
> > >> > 28. I’ve added this idea to future work.
> > >> >
> > >> > Thanks again for taking a look,
> > >> >
> > >> > Justine
> > >> >
> > >> > On Wed, Sep 23, 2020 at 10:28 AM Jun Rao <ju...@confluent.io> wrote:
> > >> >
> > >> > > Hi, Justine,
> > >> > >
> > >> > > Thanks for the response. Made another pass. A few more comments
> > below.
> > >> > >
> > >> > > 20. znode schema:
> > >> > > 20.1 It seems that {"name": "version", "type": "int", "id":
> "UUID",
> > >> > "doc":
> > >> > > "version id"} should be {"name": "version", "type": "int"},
> {"name":
> > >> > "id",
> > >> > > "type": "UUID", "doc": "version id"}.
> > >> > > 20.2 The znode format is JSON which doesn't have UUID type. So the
> > >> type
> > >> > > probably should be string?
> > >> > > 20.3 Also, the existing format used seems outdated. It should have
> > the
> > >> > > following format.
> > >> > >     Json.encodeAsBytes(Map(
> > >> > >       "version" -> 2,
> > >> > >       "partitions" -> replicaAssignmentJson.asJava,
> > >> > >       "adding_replicas" -> addingReplicasAssignmentJson.asJava,
> > >> > >       "removing_replicas" -> removingReplicasAssignmentJson.asJava
> > >> > >     ).asJava)
> > >> > >   }
> > >> > >
> > >> > > 21. Migration: The KIP says "The migration process can take place
> > >> without
> > >> > > an inter-broker protocol bump, as the format stored in
> > >> > > /brokers/topics/[topic] will be compatible with older broker
> > >> versions."
> > >> > > However, since we are bumping up the version of inter-broker
> > >> requests, it
> > >> > > seems that we need to use IBP for migration.
> > >> > >
> > >> > > 22. The KIP says "Remove deleted topics from replicas by sending
> > >> > > StopReplicaRequest V3 for any topics which do not contain a topic
> > ID,
> > >> and
> > >> > > V4 for any topics which do contain a topic ID.". However, if we
> use
> > >> IBP,
> > >> > it
> > >> > > seems that the controller will either send StopReplicaRequest V3
> > >> > > or StopReplicaRequest V4, but never mixed V3 and V4 for different
> > >> topics.
> > >> > > Basically, before the IBP bump, V3 will be used. After the IBP
> bump,
> > >> > > topicId will be created and V4 will be used.
> > >> > >
> > >> > > 23. Given that we depend on IBP, do we still need "0 UNSPECIFIED"
> > >> > > in LeaderAndIsr?
> > >> > >
> > >> > > 24. LeaderAndIsrResponse v5 : It still has the topic field.
> > >> > >
> > >> > > 25. LeaderAndIsrResponse v5, StopReplicaResponse v4: Could we use
> > this
> > >> > > opportunity to organize the response in 2 levels, first by topic,
> > >> then by
> > >> > > partition, as most other requests/responses?
> > >> > >
> > >> > > 26. FetchRequest v13 : Should forgotten_topics_data use topicId
> too?
> > >> > >
> > >> > > 27. "This file can either be plain text (key/value pairs) or
> JSON."
> > >> Have
> > >> > we
> > >> > > decided which one to use? Also, it would be helpful to provide an
> > >> > example.
> > >> > >
> > >> > > 28. Future improvement: Another future improvement opportunity is
> to
> > >> use
> > >> > > topicId in GroupMetadataManager.offsetCommitKey in the
> offset_commit
> > >> > topic.
> > >> > > This may save some space.
> > >> > >
> > >> > > Thanks,
> > >> > >
> > >> > > Jun
> > >> > >
> > >> > > On Wed, Sep 23, 2020 at 8:50 AM Justine Olshan <
> > jolshan@confluent.io>
> > >> > > wrote:
> > >> > >
> > >> > > > Hi Tom,
> > >> > > >
> > >> > > > Thanks for the comment. I think this is a really good idea and
> it
> > >> has
> > >> > > been
> > >> > > > added to the KIP under the newly added tooling section.
> > >> > > >
> > >> > > > Thanks again,
> > >> > > > Justine
> > >> > > >
> > >> > > > On Wed, Sep 23, 2020 at 3:17 AM Tom Bentley <
> tbentley@redhat.com>
> > >> > wrote:
> > >> > > >
> > >> > > > > Hi Justine,
> > >> > > > >
> > >> > > > > I know you started the vote thread, but on re-reading the KIP
> I
> > >> > noticed
> > >> > > > > that although the topic id is included in the MetadataResponse
> > >> it's
> > >> > not
> > >> > > > > surfaced in the output from `kafka-topics.sh --describe`.
> Maybe
> > >> that
> > >> > > was
> > >> > > > > intentional because ids are intentionally not really something
> > the
> > >> > user
> > >> > > > > should care deeply about, but it would also make life harder
> for
> > >> > anyone
> > >> > > > > debugging Kafka and this would likely get worse the more topic
> > ids
> > >> > got
> > >> > > > > rolled out across the protocols, clients etc. It seems likely
> > that
> > >> > > > > `kafka-topics.sh` will eventually need the ability to show the
> > id
> > >> of
> > >> > a
> > >> > > > > topic and perhaps find a topic name given an id. Is there any
> > >> reason
> > >> > > not
> > >> > > > to
> > >> > > > > implement that in this KIP?
> > >> > > > >
> > >> > > > > Many thanks,
> > >> > > > >
> > >> > > > > Tom
> > >> > > > >
> > >> > > > > On Mon, Sep 21, 2020 at 9:54 PM Justine Olshan <
> > >> jolshan@confluent.io
> > >> > >
> > >> > > > > wrote:
> > >> > > > >
> > >> > > > > > Hi all,
> > >> > > > > >
> > >> > > > > > After thinking about it, I've decided to remove the topic
> name
> > >> from
> > >> > > the
> > >> > > > > > Fetch Request and Response after all. Since there are so
> many
> > of
> > >> > > these
> > >> > > > > > requests per second, it is worth removing the extra
> > information.
> > >> > I've
> > >> > > > > > updated the KIP to reflect this change.
> > >> > > > > >
> > >> > > > > > Please let me know if there is anything else we should
> discuss
> > >> > before
> > >> > > > > > voting.
> > >> > > > > >
> > >> > > > > > Thank you,
> > >> > > > > > Justine
> > >> > > > > >
> > >> > > > > > On Fri, Sep 18, 2020 at 9:46 AM Justine Olshan <
> > >> > jolshan@confluent.io
> > >> > > >
> > >> > > > > > wrote:
> > >> > > > > >
> > >> > > > > > > Hi Jun,
> > >> > > > > > >
> > >> > > > > > > I see what you are saying. For now we can remove the extra
> > >> > > > information.
> > >> > > > > > > I'll leave the option to add more fields to the file in
> the
> > >> > future.
> > >> > > > The
> > >> > > > > > KIP
> > >> > > > > > > has been updated to reflect this change.
> > >> > > > > > >
> > >> > > > > > > Thanks,
> > >> > > > > > > Justine
> > >> > > > > > >
> > >> > > > > > > On Fri, Sep 18, 2020 at 8:46 AM Jun Rao <jun@confluent.io
> >
> > >> > wrote:
> > >> > > > > > >
> > >> > > > > > >> Hi, Justine,
> > >> > > > > > >>
> > >> > > > > > >> Thanks for the reply.
> > >> > > > > > >>
> > >> > > > > > >> 13. If the log directory is the source of truth, it means
> > >> that
> > >> > the
> > >> > > > > > >> redundant info in the metadata file will be ignored. Then
> > the
> > >> > > > question
> > >> > > > > > is
> > >> > > > > > >> why do we need to put the redundant info in the metadata
> > file
> > >> > now?
> > >> > > > > > >>
> > >> > > > > > >> Thanks,
> > >> > > > > > >>
> > >> > > > > > >> Jun
> > >> > > > > > >>
> > >> > > > > > >> On Thu, Sep 17, 2020 at 5:07 PM Justine Olshan <
> > >> > > > jolshan@confluent.io>
> > >> > > > > > >> wrote:
> > >> > > > > > >>
> > >> > > > > > >> > Hi Jun,
> > >> > > > > > >> > Thanks for the quick response!
> > >> > > > > > >> >
> > >> > > > > > >> > 12. I've decided to bump up the versions on the
> requests
> > >> and
> > >> > > > updated
> > >> > > > > > the
> > >> > > > > > >> > KIP. I think it's good we thoroughly discussed the
> > options
> > >> > here,
> > >> > > > so
> > >> > > > > we
> > >> > > > > > >> know
> > >> > > > > > >> > we made a good choice. :)
> > >> > > > > > >> >
> > >> > > > > > >> > 13. This is an interesting situation. I think if this
> > does
> > >> > occur
> > >> > > > we
> > >> > > > > > >> should
> > >> > > > > > >> > give a warning. I agree that it's hard to know the
> source
> > >> of
> > >> > > truth
> > >> > > > > for
> > >> > > > > > >> sure
> > >> > > > > > >> > since the directory or the file could be manually
> > >> modified. I
> > >> > > > guess
> > >> > > > > > the
> > >> > > > > > >> > directory could be used as the source of truth. To be
> > >> honest,
> > >> > > I'm
> > >> > > > > not
> > >> > > > > > >> > really sure what happens in kafka when the log
> directory
> > is
> > >> > > > renamed
> > >> > > > > > >> > manually in such a way. I'm also wondering if the
> > >> situation is
> > >> > > > > > >> recoverable
> > >> > > > > > >> > in this scenario.
> > >> > > > > > >> >
> > >> > > > > > >> > Thanks,
> > >> > > > > > >> > Justine
> > >> > > > > > >> >
> > >> > > > > > >> > On Thu, Sep 17, 2020 at 4:28 PM Jun Rao <
> > jun@confluent.io>
> > >> > > wrote:
> > >> > > > > > >> >
> > >> > > > > > >> > > Hi, Justine,
> > >> > > > > > >> > >
> > >> > > > > > >> > > Thanks for the reply.
> > >> > > > > > >> > >
> > >> > > > > > >> > > 12. I don't have a strong preference either. However,
> > if
> > >> we
> > >> > > need
> > >> > > > > IBP
> > >> > > > > > >> > > anyway, maybe it's easier to just bump up the version
> > for
> > >> > all
> > >> > > > > inter
> > >> > > > > > >> > broker
> > >> > > > > > >> > > requests and add the topic id field as a regular
> > field. A
> > >> > > > regular
> > >> > > > > > >> field
> > >> > > > > > >> > is
> > >> > > > > > >> > > a bit more concise in wire transfer than a flexible
> > >> field.
> > >> > > > > > >> > >
> > >> > > > > > >> > > 13. The confusion that I was referring to is between
> > the
> > >> > topic
> > >> > > > > name
> > >> > > > > > >> and
> > >> > > > > > >> > > partition number between the log dir and the metadata
> > >> file.
> > >> > > For
> > >> > > > > > >> example,
> > >> > > > > > >> > if
> > >> > > > > > >> > > the log dir is topicA-1 and the metadata file in it
> has
> > >> > topicB
> > >> > > > and
> > >> > > > > > >> > > partition 0 (say due to a bug or manual
> modification),
> > >> which
> > >> > > one
> > >> > > > > do
> > >> > > > > > we
> > >> > > > > > >> > use
> > >> > > > > > >> > > as the source of truth?
> > >> > > > > > >> > >
> > >> > > > > > >> > > Jun
> > >> > > > > > >> > >
> > >> > > > > > >> > > On Thu, Sep 17, 2020 at 3:43 PM Justine Olshan <
> > >> > > > > > jolshan@confluent.io>
> > >> > > > > > >> > > wrote:
> > >> > > > > > >> > >
> > >> > > > > > >> > > > Hi Jun,
> > >> > > > > > >> > > > Thanks for the comments.
> > >> > > > > > >> > > >
> > >> > > > > > >> > > > 12. I bumped the LeaderAndIsrRequest because I
> > removed
> > >> the
> > >> > > > topic
> > >> > > > > > >> name
> > >> > > > > > >> > > field
> > >> > > > > > >> > > > in the response. It may be possible to avoid
> bumping
> > >> the
> > >> > > > version
> > >> > > > > > >> > without
> > >> > > > > > >> > > > that change, but I may be missing something.
> > >> > > > > > >> > > > I believe StopReplica is actually on version 3 now,
> > but
> > >> > > > because
> > >> > > > > > >> > version 2
> > >> > > > > > >> > > > is flexible, I kept that listed as version 2 on the
> > KIP
> > >> > > page.
> > >> > > > > > >> However,
> > >> > > > > > >> > > you
> > >> > > > > > >> > > > may be right in that we may need to bump the
> version
> > on
> > >> > > > > > StopReplica
> > >> > > > > > >> to
> > >> > > > > > >> > > deal
> > >> > > > > > >> > > > with deletion differently as mentioned above. I
> don't
> > >> know
> > >> > > if
> > >> > > > I
> > >> > > > > > >> have a
> > >> > > > > > >> > > big
> > >> > > > > > >> > > > preference over used tagged fields or not.
> > >> > > > > > >> > > >
> > >> > > > > > >> > > > 13. I was thinking that in the case where the file
> > and
> > >> the
> > >> > > > > request
> > >> > > > > > >> > topic
> > >> > > > > > >> > > > ids don't match, it means that the broker's
> topic/the
> > >> one
> > >> > in
> > >> > > > the
> > >> > > > > > >> file
> > >> > > > > > >> > has
> > >> > > > > > >> > > > been deleted. In that case, we would need to delete
> > the
> > >> > old
> > >> > > > > topic
> > >> > > > > > >> and
> > >> > > > > > >> > > start
> > >> > > > > > >> > > > receiving the new version. If the topic name were
> to
> > >> > change,
> > >> > > > but
> > >> > > > > > the
> > >> > > > > > >> > ids
> > >> > > > > > >> > > > still matched, the file would also need to update.
> > Am I
> > >> > > > missing
> > >> > > > > a
> > >> > > > > > >> case
> > >> > > > > > >> > > > where the file would be correct and not the
> request?
> > >> > > > > > >> > > >
> > >> > > > > > >> > > > Thanks,
> > >> > > > > > >> > > > Justine
> > >> > > > > > >> > > >
> > >> > > > > > >> > > > On Thu, Sep 17, 2020 at 3:18 PM Jun Rao <
> > >> jun@confluent.io
> > >> > >
> > >> > > > > wrote:
> > >> > > > > > >> > > >
> > >> > > > > > >> > > > > Hi, Justine,
> > >> > > > > > >> > > > >
> > >> > > > > > >> > > > > Thanks for the reply. A couple of more comments
> > >> below.
> > >> > > > > > >> > > > >
> > >> > > > > > >> > > > > 12. ListOffset and OffsetForLeader currently
> don't
> > >> > support
> > >> > > > > > >> flexible
> > >> > > > > > >> > > > fields.
> > >> > > > > > >> > > > > So, we have to bump up the version number and use
> > >> IBP at
> > >> > > > least
> > >> > > > > > for
> > >> > > > > > >> > > these
> > >> > > > > > >> > > > > two requests. Note that it seems 2.7.0 will
> require
> > >> IBP
> > >> > > > anyway
> > >> > > > > > >> > because
> > >> > > > > > >> > > of
> > >> > > > > > >> > > > > changes in KAFKA-10435. Also, it seems that the
> > >> version
> > >> > > for
> > >> > > > > > >> > > > > LeaderAndIsrRequest and StopReplica are bumped
> even
> > >> > though
> > >> > > > we
> > >> > > > > > only
> > >> > > > > > >> > > added
> > >> > > > > > >> > > > a
> > >> > > > > > >> > > > > tagged field. But since IBP is needed anyway, we
> > may
> > >> > want
> > >> > > to
> > >> > > > > > >> revisit
> > >> > > > > > >> > > the
> > >> > > > > > >> > > > > overall tagged field choice.
> > >> > > > > > >> > > > >
> > >> > > > > > >> > > > > 13. The only downside is the potential confusion
> on
> > >> > which
> > >> > > > one
> > >> > > > > is
> > >> > > > > > >> the
> > >> > > > > > >> > > > source
> > >> > > > > > >> > > > > of truth if they don't match. Another option is
> to
> > >> > include
> > >> > > > > those
> > >> > > > > > >> > fields
> > >> > > > > > >> > > > in
> > >> > > > > > >> > > > > the metadata file when we actually change the
> > >> directory
> > >> > > > > > structure.
> > >> > > > > > >> > > > >
> > >> > > > > > >> > > > > Thanks,
> > >> > > > > > >> > > > >
> > >> > > > > > >> > > > > Jun
> > >> > > > > > >> > > > >
> > >> > > > > > >> > > > > On Thu, Sep 17, 2020 at 2:01 PM Justine Olshan <
> > >> > > > > > >> jolshan@confluent.io
> > >> > > > > > >> > >
> > >> > > > > > >> > > > > wrote:
> > >> > > > > > >> > > > >
> > >> > > > > > >> > > > > > Hello all,
> > >> > > > > > >> > > > > >
> > >> > > > > > >> > > > > > I've thought some more about removing the topic
> > >> name
> > >> > > field
> > >> > > > > > from
> > >> > > > > > >> > some
> > >> > > > > > >> > > of
> > >> > > > > > >> > > > > the
> > >> > > > > > >> > > > > > requests. On closer inspection of the
> > >> > > requests/responses,
> > >> > > > it
> > >> > > > > > >> seems
> > >> > > > > > >> > > that
> > >> > > > > > >> > > > > the
> > >> > > > > > >> > > > > > internal changes would be much larger than I
> > >> expected.
> > >> > > > Some
> > >> > > > > > >> > protocols
> > >> > > > > > >> > > > > > involve clients, so they would require changes
> > too.
> > >> > I'm
> > >> > > > > > thinking
> > >> > > > > > >> > that
> > >> > > > > > >> > > > for
> > >> > > > > > >> > > > > > now, removing the topic name from these
> requests
> > >> and
> > >> > > > > responses
> > >> > > > > > >> are
> > >> > > > > > >> > > out
> > >> > > > > > >> > > > of
> > >> > > > > > >> > > > > > scope.
> > >> > > > > > >> > > > > >
> > >> > > > > > >> > > > > > I have decided to just keep the change
> > >> > > > LeaderAndIsrResponse
> > >> > > > > to
> > >> > > > > > >> > remove
> > >> > > > > > >> > > > the
> > >> > > > > > >> > > > > > topic name, and have updated the KIP to reflect
> > >> this
> > >> > > > > change. I
> > >> > > > > > >> have
> > >> > > > > > >> > > > also
> > >> > > > > > >> > > > > > mentioned the other requests and responses in
> > >> future
> > >> > > work.
> > >> > > > > > >> > > > > >
> > >> > > > > > >> > > > > > I'm hoping to start the voting process soon, so
> > >> let me
> > >> > > > know
> > >> > > > > if
> > >> > > > > > >> > there
> > >> > > > > > >> > > is
> > >> > > > > > >> > > > > > anything else we should discuss.
> > >> > > > > > >> > > > > >
> > >> > > > > > >> > > > > > Thank you,
> > >> > > > > > >> > > > > > Justine
> > >> > > > > > >> > > > > >
> > >> > > > > > >> > > > > > On Tue, Sep 15, 2020 at 3:57 PM Justine Olshan
> <
> > >> > > > > > >> > jolshan@confluent.io
> > >> > > > > > >> > > >
> > >> > > > > > >> > > > > > wrote:
> > >> > > > > > >> > > > > >
> > >> > > > > > >> > > > > > > Hello again,
> > >> > > > > > >> > > > > > > To follow up on some of the other comments:
> > >> > > > > > >> > > > > > >
> > >> > > > > > >> > > > > > > 10/11) We can remove the topic name from
> these
> > >> > > > > > >> > requests/responses,
> > >> > > > > > >> > > > and
> > >> > > > > > >> > > > > > > that means we will just have to make a few
> > >> internal
> > >> > > > > changes
> > >> > > > > > to
> > >> > > > > > >> > make
> > >> > > > > > >> > > > > > > partitions accessible by topic id and
> > partition.
> > >> I
> > >> > can
> > >> > > > > > update
> > >> > > > > > >> the
> > >> > > > > > >> > > KIP
> > >> > > > > > >> > > > > to
> > >> > > > > > >> > > > > > > remove them unless anyone thinks they should
> > >> stay.
> > >> > > > > > >> > > > > > >
> > >> > > > > > >> > > > > > > 12) Addressed in the previous email. I've
> > updated
> > >> > the
> > >> > > > KIP
> > >> > > > > to
> > >> > > > > > >> > > include
> > >> > > > > > >> > > > > > > tagged fields for the requests and responses.
> > >> (More
> > >> > on
> > >> > > > > that
> > >> > > > > > >> > below)
> > >> > > > > > >> > > > > > >
> > >> > > > > > >> > > > > > > 13) I think part of the idea for including
> this
> > >> > > > > information
> > >> > > > > > >> is to
> > >> > > > > > >> > > > > prepare
> > >> > > > > > >> > > > > > > for future changes. Perhaps the directory
> > >> structure
> > >> > > > might
> > >> > > > > > >> change
> > >> > > > > > >> > > from
> > >> > > > > > >> > > > > > > topicName_partitionNumber to something like
> > >> > > > > > >> > > topicID_partitionNumber.
> > >> > > > > > >> > > > > Then
> > >> > > > > > >> > > > > > > it would be useful to have the topic name in
> > the
> > >> > file
> > >> > > > > since
> > >> > > > > > it
> > >> > > > > > >> > > would
> > >> > > > > > >> > > > > not
> > >> > > > > > >> > > > > > be
> > >> > > > > > >> > > > > > > in the directory structure. Supporting topic
> > >> renames
> > >> > > > might
> > >> > > > > > be
> > >> > > > > > >> > > easier
> > >> > > > > > >> > > > if
> > >> > > > > > >> > > > > > the
> > >> > > > > > >> > > > > > > other fields are included. Would there be any
> > >> > > downsides
> > >> > > > to
> > >> > > > > > >> > > including
> > >> > > > > > >> > > > > this
> > >> > > > > > >> > > > > > > information?
> > >> > > > > > >> > > > > > >
> > >> > > > > > >> > > > > > > 14)  Yes, we would need to copy the partition
> > >> > metadata
> > >> > > > > file
> > >> > > > > > in
> > >> > > > > > >> > this
> > >> > > > > > >> > > > > > > process. I've updated the KIP to include
> this.
> > >> > > > > > >> > > > > > >
> > >> > > > > > >> > > > > > > 15) I believe Lucas meant v1 and v2 here. He
> > was
> > >> > > > referring
> > >> > > > > > to
> > >> > > > > > >> how
> > >> > > > > > >> > > the
> > >> > > > > > >> > > > > > > requests would fall under different IBP and
> > meant
> > >> > that
> > >> > > > > older
> > >> > > > > > >> > > brokers
> > >> > > > > > >> > > > > > would
> > >> > > > > > >> > > > > > > have to use the older version of the request
> > and
> > >> the
> > >> > > > > > existing
> > >> > > > > > >> > topic
> > >> > > > > > >> > > > > > > deletion process. At first, it seemed like
> > tagged
> > >> > > fields
> > >> > > > > > would
> > >> > > > > > >> > > > resolve
> > >> > > > > > >> > > > > > > the IBP issue. However, we may need IBP for
> > this
> > >> > > request
> > >> > > > > > after
> > >> > > > > > >> > all
> > >> > > > > > >> > > > > since
> > >> > > > > > >> > > > > > > the controller handles the topic deletion
> > >> > differently
> > >> > > > > > >> depending
> > >> > > > > > >> > on
> > >> > > > > > >> > > > the
> > >> > > > > > >> > > > > > IBP
> > >> > > > > > >> > > > > > > version. In an older version, we can't just
> > send
> > >> a
> > >> > > > > > StopReplica
> > >> > > > > > >> > > delete
> > >> > > > > > >> > > > > the
> > >> > > > > > >> > > > > > > topic immediately like we'd want to for this
> > KIP.
> > >> > > > > > >> > > > > > >
> > >> > > > > > >> > > > > > > This makes me wonder if we want tagged fields
> > on
> > >> all
> > >> > > the
> > >> > > > > > >> requests
> > >> > > > > > >> > > > after
> > >> > > > > > >> > > > > > > all. Let me know your thoughts!
> > >> > > > > > >> > > > > > >
> > >> > > > > > >> > > > > > > Justine
> > >> > > > > > >> > > > > > >
> > >> > > > > > >> > > > > > > On Tue, Sep 15, 2020 at 1:03 PM Justine
> Olshan
> > <
> > >> > > > > > >> > > jolshan@confluent.io
> > >> > > > > > >> > > > >
> > >> > > > > > >> > > > > > > wrote:
> > >> > > > > > >> > > > > > >
> > >> > > > > > >> > > > > > >> Hi all,
> > >> > > > > > >> > > > > > >> Jun brought up a good point in his last
> email
> > >> about
> > >> > > > > tagged
> > >> > > > > > >> > fields,
> > >> > > > > > >> > > > and
> > >> > > > > > >> > > > > > >> I've updated the KIP to reflect that the
> > >> changes to
> > >> > > > > > requests
> > >> > > > > > >> and
> > >> > > > > > >> > > > > > responses
> > >> > > > > > >> > > > > > >> will be in the form of tagged fields to
> avoid
> > >> > > changing
> > >> > > > > IBP.
> > >> > > > > > >> > > > > > >>
> > >> > > > > > >> > > > > > >> Jun: I plan on sending a followup email to
> > >> address
> > >> > > some
> > >> > > > > of
> > >> > > > > > >> the
> > >> > > > > > >> > > other
> > >> > > > > > >> > > > > > >> points.
> > >> > > > > > >> > > > > > >>
> > >> > > > > > >> > > > > > >> Thanks,
> > >> > > > > > >> > > > > > >> Justine
> > >> > > > > > >> > > > > > >>
> > >> > > > > > >> > > > > > >> On Mon, Sep 14, 2020 at 4:25 PM Jun Rao <
> > >> > > > > jun@confluent.io>
> > >> > > > > > >> > wrote:
> > >> > > > > > >> > > > > > >>
> > >> > > > > > >> > > > > > >>> Hi, Justine,
> > >> > > > > > >> > > > > > >>>
> > >> > > > > > >> > > > > > >>> Thanks for the updated KIP. A few comments
> > >> below.
> > >> > > > > > >> > > > > > >>>
> > >> > > > > > >> > > > > > >>> 10. LeaderAndIsr Response: Do we need the
> > topic
> > >> > > name?
> > >> > > > > > >> > > > > > >>>
> > >> > > > > > >> > > > > > >>> 11. For the changed request/response, other
> > >> than
> > >> > > > > > >> LeaderAndIsr,
> > >> > > > > > >> > > > > > >>> UpdateMetadata, Metadata, do we need to
> > include
> > >> > the
> > >> > > > > topic
> > >> > > > > > >> name?
> > >> > > > > > >> > > > > > >>>
> > >> > > > > > >> > > > > > >>> 12. It seems that upgrades don't require
> IBP.
> > >> Does
> > >> > > > that
> > >> > > > > > mean
> > >> > > > > > >> > the
> > >> > > > > > >> > > > new
> > >> > > > > > >> > > > > > >>> fields
> > >> > > > > > >> > > > > > >>> in all the request/response are added as
> > tagged
> > >> > > fields
> > >> > > > > > >> without
> > >> > > > > > >> > > > > bumping
> > >> > > > > > >> > > > > > up
> > >> > > > > > >> > > > > > >>> the request version? It would be useful to
> > make
> > >> > that
> > >> > > > > > clear.
> > >> > > > > > >> > > > > > >>>
> > >> > > > > > >> > > > > > >>> 13. Partition Metadata file: Do we need to
> > >> include
> > >> > > the
> > >> > > > > > topic
> > >> > > > > > >> > name
> > >> > > > > > >> > > > and
> > >> > > > > > >> > > > > > the
> > >> > > > > > >> > > > > > >>> partition id since they are implied in the
> > >> > directory
> > >> > > > > name?
> > >> > > > > > >> > > > > > >>>
> > >> > > > > > >> > > > > > >>> 14. In the JBOD mode, we support moving a
> > >> > > partition's
> > >> > > > > data
> > >> > > > > > >> from
> > >> > > > > > >> > > one
> > >> > > > > > >> > > > > > disk
> > >> > > > > > >> > > > > > >>> to
> > >> > > > > > >> > > > > > >>> another. Will the new partition metadata
> file
> > >> be
> > >> > > > copied
> > >> > > > > > >> during
> > >> > > > > > >> > > that
> > >> > > > > > >> > > > > > >>> process?
> > >> > > > > > >> > > > > > >>>
> > >> > > > > > >> > > > > > >>> 15. The KIP says "Remove deleted topics
> from
> > >> > > replicas
> > >> > > > by
> > >> > > > > > >> > sending
> > >> > > > > > >> > > > > > >>> StopReplicaRequest V2 for any topics which
> do
> > >> not
> > >> > > > > contain
> > >> > > > > > a
> > >> > > > > > >> > topic
> > >> > > > > > >> > > > ID,
> > >> > > > > > >> > > > > > and
> > >> > > > > > >> > > > > > >>> V3 for any topics which do contain a topic
> > >> ID.".
> > >> > > > > However,
> > >> > > > > > it
> > >> > > > > > >> > > seems
> > >> > > > > > >> > > > > the
> > >> > > > > > >> > > > > > >>> updated controller will create all missing
> > >> topic
> > >> > IDs
> > >> > > > > first
> > >> > > > > > >> > before
> > >> > > > > > >> > > > > doing
> > >> > > > > > >> > > > > > >>> other actions. So, is StopReplicaRequest V2
> > >> > needed?
> > >> > > > > > >> > > > > > >>>
> > >> > > > > > >> > > > > > >>> Jun
> > >> > > > > > >> > > > > > >>>
> > >> > > > > > >> > > > > > >>> On Fri, Sep 11, 2020 at 10:31 AM John
> > Roesler <
> > >> > > > > > >> > > vvcephei@apache.org
> > >> > > > > > >> > > > >
> > >> > > > > > >> > > > > > >>> wrote:
> > >> > > > > > >> > > > > > >>>
> > >> > > > > > >> > > > > > >>> > Thanks, Justine!
> > >> > > > > > >> > > > > > >>> >
> > >> > > > > > >> > > > > > >>> > Your response seems compelling to me.
> > >> > > > > > >> > > > > > >>> >
> > >> > > > > > >> > > > > > >>> > -John
> > >> > > > > > >> > > > > > >>> >
> > >> > > > > > >> > > > > > >>> > On Fri, 2020-09-11 at 10:17 -0700,
> Justine
> > >> > Olshan
> > >> > > > > wrote:
> > >> > > > > > >> > > > > > >>> > > Hello all,
> > >> > > > > > >> > > > > > >>> > > Thanks for continuing the discussion! I
> > >> have a
> > >> > > few
> > >> > > > > > >> > responses
> > >> > > > > > >> > > to
> > >> > > > > > >> > > > > > your
> > >> > > > > > >> > > > > > >>> > points.
> > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > >> > > > > > >>> > > Tom: You are correct in that this KIP
> has
> > >> not
> > >> > > > > > mentioned
> > >> > > > > > >> the
> > >> > > > > > >> > > > > > >>> > > DeleteTopicsRequest. I think that this
> > >> would
> > >> > be
> > >> > > > out
> > >> > > > > of
> > >> > > > > > >> > scope
> > >> > > > > > >> > > > for
> > >> > > > > > >> > > > > > >>> now, but
> > >> > > > > > >> > > > > > >>> > > may be something worth adding in the
> > >> future.
> > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > >> > > > > > >>> > > John: We did consider sequence ids, but
> > >> there
> > >> > > are
> > >> > > > a
> > >> > > > > > few
> > >> > > > > > >> > > reasons
> > >> > > > > > >> > > > > to
> > >> > > > > > >> > > > > > >>> favor
> > >> > > > > > >> > > > > > >>> > > UUIDs. There are several cases where
> > topics
> > >> > from
> > >> > > > > > >> different
> > >> > > > > > >> > > > > clusters
> > >> > > > > > >> > > > > > >>> may
> > >> > > > > > >> > > > > > >>> > > interact now and in the future. For
> > >> example,
> > >> > > > Mirror
> > >> > > > > > >> Maker 2
> > >> > > > > > >> > > may
> > >> > > > > > >> > > > > > >>> benefit
> > >> > > > > > >> > > > > > >>> > > from being able to detect when a
> cluster
> > >> being
> > >> > > > > > mirrored
> > >> > > > > > >> is
> > >> > > > > > >> > > > > deleted
> > >> > > > > > >> > > > > > >>> and
> > >> > > > > > >> > > > > > >>> > > recreated and globally unique
> identifiers
> > >> > would
> > >> > > > make
> > >> > > > > > >> > > resolving
> > >> > > > > > >> > > > > > issues
> > >> > > > > > >> > > > > > >>> > > easier than sequence IDs which may
> > collide
> > >> > > between
> > >> > > > > > >> > clusters.
> > >> > > > > > >> > > > > > KIP-405
> > >> > > > > > >> > > > > > >>> > > (tiered storage) will also benefit from
> > >> > globally
> > >> > > > > > unique
> > >> > > > > > >> IDs
> > >> > > > > > >> > > as
> > >> > > > > > >> > > > > > shared
> > >> > > > > > >> > > > > > >>> > > buckets may be used between clusters.
> > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > >> > > > > > >>> > > Globally unique IDs would also make
> > >> > > functionality
> > >> > > > > like
> > >> > > > > > >> > moving
> > >> > > > > > >> > > > > > topics
> > >> > > > > > >> > > > > > >>> > > between disparate clusters easier in
> the
> > >> > future,
> > >> > > > > > >> simplify
> > >> > > > > > >> > any
> > >> > > > > > >> > > > > > future
> > >> > > > > > >> > > > > > >>> > > implementations of backups and
> restores,
> > >> and
> > >> > > more.
> > >> > > > > In
> > >> > > > > > >> > > general,
> > >> > > > > > >> > > > > > >>> unique IDs
> > >> > > > > > >> > > > > > >>> > > would ensure that the source cluster
> > >> topics do
> > >> > > not
> > >> > > > > > >> conflict
> > >> > > > > > >> > > > with
> > >> > > > > > >> > > > > > the
> > >> > > > > > >> > > > > > >>> > > destination cluster topics.
> > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > >> > > > > > >>> > > If we were to use sequence ids, we
> would
> > >> need
> > >> > > > > > >> sufficiently
> > >> > > > > > >> > > > large
> > >> > > > > > >> > > > > > >>> cluster
> > >> > > > > > >> > > > > > >>> > > ids to be stored with the topic
> > >> identifiers or
> > >> > > we
> > >> > > > > run
> > >> > > > > > >> the
> > >> > > > > > >> > > risk
> > >> > > > > > >> > > > of
> > >> > > > > > >> > > > > > >>> > > collisions. This will give up any
> > >> advantage in
> > >> > > > > > >> compactness
> > >> > > > > > >> > > that
> > >> > > > > > >> > > > > > >>> sequence
> > >> > > > > > >> > > > > > >>> > > numbers may bring. Given these
> > advantages I
> > >> > > think
> > >> > > > it
> > >> > > > > > >> makes
> > >> > > > > > >> > > > sense
> > >> > > > > > >> > > > > to
> > >> > > > > > >> > > > > > >>> use
> > >> > > > > > >> > > > > > >>> > > UUIDs.
> > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > >> > > > > > >>> > > Gokul: This is an interesting idea, but
> > >> this
> > >> > is
> > >> > > a
> > >> > > > > > >> breaking
> > >> > > > > > >> > > > > change.
> > >> > > > > > >> > > > > > >>> Out of
> > >> > > > > > >> > > > > > >>> > > scope for now, but maybe worth
> discussing
> > >> in
> > >> > the
> > >> > > > > > future.
> > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > >> > > > > > >>> > > Hope this explains some of the
> decisions,
> > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > >> > > > > > >>> > > Justine
> > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > >> > > > > > >>> > > On Fri, Sep 11, 2020 at 8:27 AM Gokul
> > >> Ramanan
> > >> > > > > > >> Subramanian <
> > >> > > > > > >> > > > > > >>> > > gokul2411s@gmail.com> wrote:
> > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > >> > > > > > >>> > > > Hi.
> > >> > > > > > >> > > > > > >>> > > >
> > >> > > > > > >> > > > > > >>> > > > Thanks for the KIP.
> > >> > > > > > >> > > > > > >>> > > >
> > >> > > > > > >> > > > > > >>> > > > Have you thought about whether it
> makes
> > >> > sense
> > >> > > to
> > >> > > > > > >> support
> > >> > > > > > >> > > > > > >>> authorizing a
> > >> > > > > > >> > > > > > >>> > > > principal for a topic ID rather than
> a
> > >> topic
> > >> > > > name
> > >> > > > > to
> > >> > > > > > >> > > achieve
> > >> > > > > > >> > > > > > >>> tighter
> > >> > > > > > >> > > > > > >>> > > > security?
> > >> > > > > > >> > > > > > >>> > > >
> > >> > > > > > >> > > > > > >>> > > > Or is the topic ID fundamentally an
> > >> internal
> > >> > > > > detail
> > >> > > > > > >> > similar
> > >> > > > > > >> > > > to
> > >> > > > > > >> > > > > > >>> epochs
> > >> > > > > > >> > > > > > >>> > used
> > >> > > > > > >> > > > > > >>> > > > in a bunch of other places in Kafka?
> > >> > > > > > >> > > > > > >>> > > >
> > >> > > > > > >> > > > > > >>> > > > Thanks.
> > >> > > > > > >> > > > > > >>> > > >
> > >> > > > > > >> > > > > > >>> > > > On Fri, Sep 11, 2020 at 4:06 PM John
> > >> > Roesler <
> > >> > > > > > >> > > > > > vvcephei@apache.org>
> > >> > > > > > >> > > > > > >>> > wrote:
> > >> > > > > > >> > > > > > >>> > > >
> > >> > > > > > >> > > > > > >>> > > > > Hello Justine,
> > >> > > > > > >> > > > > > >>> > > > >
> > >> > > > > > >> > > > > > >>> > > > > Thanks for the KIP!
> > >> > > > > > >> > > > > > >>> > > > >
> > >> > > > > > >> > > > > > >>> > > > > I happen to have been confronted
> > >> recently
> > >> > > with
> > >> > > > > the
> > >> > > > > > >> need
> > >> > > > > > >> > > to
> > >> > > > > > >> > > > > keep
> > >> > > > > > >> > > > > > >>> > track of
> > >> > > > > > >> > > > > > >>> > > > a
> > >> > > > > > >> > > > > > >>> > > > > large number of topics as compactly
> > as
> > >> > > > > possible. I
> > >> > > > > > >> was
> > >> > > > > > >> > > > going
> > >> > > > > > >> > > > > to
> > >> > > > > > >> > > > > > >>> come
> > >> > > > > > >> > > > > > >>> > up
> > >> > > > > > >> > > > > > >>> > > > > with some way to dictionary encode
> > the
> > >> > topic
> > >> > > > > names
> > >> > > > > > >> as
> > >> > > > > > >> > > > > integers,
> > >> > > > > > >> > > > > > >>> but
> > >> > > > > > >> > > > > > >>> > this
> > >> > > > > > >> > > > > > >>> > > > > seems much better!
> > >> > > > > > >> > > > > > >>> > > > >
> > >> > > > > > >> > > > > > >>> > > > > Apologies if this has been raised
> > >> before,
> > >> > > but
> > >> > > > > I’m
> > >> > > > > > >> > > wondering
> > >> > > > > > >> > > > > > >>> about the
> > >> > > > > > >> > > > > > >>> > > > > choice of UUID vs sequence number
> for
> > >> the
> > >> > > ids.
> > >> > > > > > >> > Typically,
> > >> > > > > > >> > > > > I’ve
> > >> > > > > > >> > > > > > >>> seen
> > >> > > > > > >> > > > > > >>> > UUIDs
> > >> > > > > > >> > > > > > >>> > > > > in two situations:
> > >> > > > > > >> > > > > > >>> > > > > 1. When processes need to generate
> > >> > > > non-colliding
> > >> > > > > > >> > > > identifiers
> > >> > > > > > >> > > > > > >>> without
> > >> > > > > > >> > > > > > >>> > > > > coordination.
> > >> > > > > > >> > > > > > >>> > > > > 2. When the identifier needs to be
> > >> > > > “universally
> > >> > > > > > >> > unique”;
> > >> > > > > > >> > > > > I.e.,
> > >> > > > > > >> > > > > > >>> the
> > >> > > > > > >> > > > > > >>> > > > > identifier needs to distinguish the
> > >> entity
> > >> > > > from
> > >> > > > > > all
> > >> > > > > > >> > other
> > >> > > > > > >> > > > > > >>> entities
> > >> > > > > > >> > > > > > >>> > that
> > >> > > > > > >> > > > > > >>> > > > > could ever exist. This is useful in
> > >> cases
> > >> > > > where
> > >> > > > > > >> > entities
> > >> > > > > > >> > > > from
> > >> > > > > > >> > > > > > all
> > >> > > > > > >> > > > > > >>> > kinds
> > >> > > > > > >> > > > > > >>> > > > of
> > >> > > > > > >> > > > > > >>> > > > > systems get mixed together, such as
> > >> when
> > >> > > > dumping
> > >> > > > > > >> logs
> > >> > > > > > >> > > from
> > >> > > > > > >> > > > > all
> > >> > > > > > >> > > > > > >>> > processes
> > >> > > > > > >> > > > > > >>> > > > in
> > >> > > > > > >> > > > > > >>> > > > > a company into a common system.
> > >> > > > > > >> > > > > > >>> > > > >
> > >> > > > > > >> > > > > > >>> > > > > Maybe I’m being short-sighted, but
> it
> > >> > > doesn’t
> > >> > > > > seem
> > >> > > > > > >> like
> > >> > > > > > >> > > > > either
> > >> > > > > > >> > > > > > >>> really
> > >> > > > > > >> > > > > > >>> > > > > applies here. It seems like the
> > brokers
> > >> > > could
> > >> > > > > and
> > >> > > > > > >> would
> > >> > > > > > >> > > > > achieve
> > >> > > > > > >> > > > > > >>> > consensus
> > >> > > > > > >> > > > > > >>> > > > > when creating a topic anyway, which
> > is
> > >> all
> > >> > > > > that’s
> > >> > > > > > >> > > required
> > >> > > > > > >> > > > to
> > >> > > > > > >> > > > > > >>> > generate
> > >> > > > > > >> > > > > > >>> > > > > non-colliding sequence ids. For the
> > >> > second,
> > >> > > as
> > >> > > > > you
> > >> > > > > > >> > > mention,
> > >> > > > > > >> > > > > we
> > >> > > > > > >> > > > > > >>> could
> > >> > > > > > >> > > > > > >>> > > > assign
> > >> > > > > > >> > > > > > >>> > > > > a UUID to the cluster as a whole,
> > which
> > >> > > would
> > >> > > > > > render
> > >> > > > > > >> > any
> > >> > > > > > >> > > > > > resource
> > >> > > > > > >> > > > > > >>> > scoped
> > >> > > > > > >> > > > > > >>> > > > to
> > >> > > > > > >> > > > > > >>> > > > > the broker universally unique as
> > well.
> > >> > > > > > >> > > > > > >>> > > > >
> > >> > > > > > >> > > > > > >>> > > > > The reason I mention this is that,
> > >> > although
> > >> > > a
> > >> > > > > UUID
> > >> > > > > > >> is
> > >> > > > > > >> > way
> > >> > > > > > >> > > > > more
> > >> > > > > > >> > > > > > >>> > compact
> > >> > > > > > >> > > > > > >>> > > > > than topic names, it’s still 16
> > bytes.
> > >> In
> > >> > > > > > contrast,
> > >> > > > > > >> a
> > >> > > > > > >> > > > 4-byte
> > >> > > > > > >> > > > > > >>> integer
> > >> > > > > > >> > > > > > >>> > > > > sequence id would give us 4 billion
> > >> unique
> > >> > > > > topics
> > >> > > > > > >> per
> > >> > > > > > >> > > > > cluster,
> > >> > > > > > >> > > > > > >>> which
> > >> > > > > > >> > > > > > >>> > > > seems
> > >> > > > > > >> > > > > > >>> > > > > like enough ;)
> > >> > > > > > >> > > > > > >>> > > > >
> > >> > > > > > >> > > > > > >>> > > > > Considering the number of different
> > >> times
> > >> > > > these
> > >> > > > > > >> topic
> > >> > > > > > >> > > > > > >>> identifiers are
> > >> > > > > > >> > > > > > >>> > > > sent
> > >> > > > > > >> > > > > > >>> > > > > over the wire or stored in memory,
> it
> > >> > seems
> > >> > > > like
> > >> > > > > > it
> > >> > > > > > >> > might
> > >> > > > > > >> > > > be
> > >> > > > > > >> > > > > > >>> worth
> > >> > > > > > >> > > > > > >>> > the
> > >> > > > > > >> > > > > > >>> > > > > additional 4x space savings.
> > >> > > > > > >> > > > > > >>> > > > >
> > >> > > > > > >> > > > > > >>> > > > > What do you think about this?
> > >> > > > > > >> > > > > > >>> > > > >
> > >> > > > > > >> > > > > > >>> > > > > Thanks,
> > >> > > > > > >> > > > > > >>> > > > > John
> > >> > > > > > >> > > > > > >>> > > > >
> > >> > > > > > >> > > > > > >>> > > > > On Fri, Sep 11, 2020, at 03:20, Tom
> > >> > Bentley
> > >> > > > > wrote:
> > >> > > > > > >> > > > > > >>> > > > > > Hi Justine,
> > >> > > > > > >> > > > > > >>> > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > This looks like a very welcome
> > >> > > improvement.
> > >> > > > > > >> Thanks!
> > >> > > > > > >> > > > > > >>> > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > Maybe I missed it, but the KIP
> > >> doesn't
> > >> > > seem
> > >> > > > to
> > >> > > > > > >> > mention
> > >> > > > > > >> > > > > > changing
> > >> > > > > > >> > > > > > >>> > > > > > DeleteTopicsRequest to identify
> the
> > >> > topic
> > >> > > > > using
> > >> > > > > > an
> > >> > > > > > >> > id.
> > >> > > > > > >> > > > > Maybe
> > >> > > > > > >> > > > > > >>> > that's out
> > >> > > > > > >> > > > > > >>> > > > > of
> > >> > > > > > >> > > > > > >>> > > > > > scope, but DeleteTopicsRequest is
> > not
> > >> > > listed
> > >> > > > > > among
> > >> > > > > > >> > the
> > >> > > > > > >> > > > > Future
> > >> > > > > > >> > > > > > >>> Work
> > >> > > > > > >> > > > > > >>> > APIs
> > >> > > > > > >> > > > > > >>> > > > > > either.
> > >> > > > > > >> > > > > > >>> > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > Kind regards,
> > >> > > > > > >> > > > > > >>> > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > Tom
> > >> > > > > > >> > > > > > >>> > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > On Thu, Sep 10, 2020 at 3:59 PM
> > >> Satish
> > >> > > > > Duggana <
> > >> > > > > > >> > > > > > >>> > > > satish.duggana@gmail.com
> > >> > > > > > >> > > > > > >>> > > > > > wrote:
> > >> > > > > > >> > > > > > >>> > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > > Thanks Lucas/Justine for the
> nice
> > >> KIP.
> > >> > > > > > >> > > > > > >>> > > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > > It has several benefits which
> > also
> > >> > > include
> > >> > > > > > >> > > simplifying
> > >> > > > > > >> > > > > the
> > >> > > > > > >> > > > > > >>> topic
> > >> > > > > > >> > > > > > >>> > > > > > > deletion process by controller
> > and
> > >> > logs
> > >> > > > > > cleanup
> > >> > > > > > >> by
> > >> > > > > > >> > > > > brokers
> > >> > > > > > >> > > > > > in
> > >> > > > > > >> > > > > > >>> > corner
> > >> > > > > > >> > > > > > >>> > > > > > > cases.
> > >> > > > > > >> > > > > > >>> > > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > > Best,
> > >> > > > > > >> > > > > > >>> > > > > > > Satish.
> > >> > > > > > >> > > > > > >>> > > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > > On Wed, Sep 9, 2020 at 10:07 PM
> > >> > Justine
> > >> > > > > > Olshan <
> > >> > > > > > >> > > > > > >>> > jolshan@confluent.io
> > >> > > > > > >> > > > > > >>> > > > > > > wrote:
> > >> > > > > > >> > > > > > >>> > > > > > > > Hello all, it's been almost a
> > >> year!
> > >> > > I've
> > >> > > > > > made
> > >> > > > > > >> > some
> > >> > > > > > >> > > > > > changes
> > >> > > > > > >> > > > > > >>> to
> > >> > > > > > >> > > > > > >>> > this
> > >> > > > > > >> > > > > > >>> > > > > KIP
> > >> > > > > > >> > > > > > >>> > > > > > > and hope to continue the
> > >> discussion.
> > >> > > > > > >> > > > > > >>> > > > > > > > One of the main changes I've
> > >> added
> > >> > is
> > >> > > > now
> > >> > > > > > the
> > >> > > > > > >> > > > metadata
> > >> > > > > > >> > > > > > >>> response
> > >> > > > > > >> > > > > > >>> > > > will
> > >> > > > > > >> > > > > > >>> > > > > > > include the topic ID (as Colin
> > >> > > suggested).
> > >> > > > > > >> Clients
> > >> > > > > > >> > > can
> > >> > > > > > >> > > > > > >>> obtain the
> > >> > > > > > >> > > > > > >>> > > > > topicID
> > >> > > > > > >> > > > > > >>> > > > > > > of a given topic through a
> > >> > > > TopicDescription.
> > >> > > > > > The
> > >> > > > > > >> > > > topicId
> > >> > > > > > >> > > > > > will
> > >> > > > > > >> > > > > > >>> > also be
> > >> > > > > > >> > > > > > >>> > > > > > > included with the
> UpdateMetadata
> > >> > > request.
> > >> > > > > > >> > > > > > >>> > > > > > > > Let me know what you all
> think.
> > >> > > > > > >> > > > > > >>> > > > > > > > Thank you,
> > >> > > > > > >> > > > > > >>> > > > > > > > Justine
> > >> > > > > > >> > > > > > >>> > > > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > > > On 2019/09/13 16:38:26,
> "Colin
> > >> > > McCabe" <
> > >> > > > > > >> > > > > > cmccabe@apache.org
> > >> > > > > > >> > > > > > >>> >
> > >> > > > > > >> > > > > > >>> > wrote:
> > >> > > > > > >> > > > > > >>> > > > > > > > > Hi Lucas,
> > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > > > > Thanks for tackling this.
> > >> Topic
> > >> > IDs
> > >> > > > > are a
> > >> > > > > > >> > great
> > >> > > > > > >> > > > > idea,
> > >> > > > > > >> > > > > > >>> and
> > >> > > > > > >> > > > > > >>> > this
> > >> > > > > > >> > > > > > >>> > > > is
> > >> > > > > > >> > > > > > >>> > > > > a
> > >> > > > > > >> > > > > > >>> > > > > > > really good writeup.
> > >> > > > > > >> > > > > > >>> > > > > > > > > For
> /brokers/topics/[topic],
> > >> the
> > >> > > > schema
> > >> > > > > > >> version
> > >> > > > > > >> > > > > should
> > >> > > > > > >> > > > > > be
> > >> > > > > > >> > > > > > >>> > bumped
> > >> > > > > > >> > > > > > >>> > > > to
> > >> > > > > > >> > > > > > >>> > > > > > > version 3, rather than 2.
> > KIP-455
> > >> > > bumped
> > >> > > > > the
> > >> > > > > > >> > version
> > >> > > > > > >> > > > of
> > >> > > > > > >> > > > > > this
> > >> > > > > > >> > > > > > >>> > znode
> > >> > > > > > >> > > > > > >>> > > > to
> > >> > > > > > >> > > > > > >>> > > > > 2
> > >> > > > > > >> > > > > > >>> > > > > > > already :)
> > >> > > > > > >> > > > > > >>> > > > > > > > > Given that we're going to
> be
> > >> > seeing
> > >> > > > > these
> > >> > > > > > >> > things
> > >> > > > > > >> > > as
> > >> > > > > > >> > > > > > >>> strings
> > >> > > > > > >> > > > > > >>> > as
> > >> > > > > > >> > > > > > >>> > > > lot
> > >> > > > > > >> > > > > > >>> > > > > (in
> > >> > > > > > >> > > > > > >>> > > > > > > logs, in ZooKeeper, on the
> > >> > command-line,
> > >> > > > > > etc.),
> > >> > > > > > >> > does
> > >> > > > > > >> > > it
> > >> > > > > > >> > > > > > make
> > >> > > > > > >> > > > > > >>> > sense to
> > >> > > > > > >> > > > > > >>> > > > > use
> > >> > > > > > >> > > > > > >>> > > > > > > base64 when converting them to
> > >> > strings?
> > >> > > > > > >> > > > > > >>> > > > > > > > > Here is an example of the
> hex
> > >> > > > > > >> representation:
> > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >> > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > > > > And here is an example in
> > >> base64.
> > >> > > > > > >> > > > > > >>> > > > > > > > > b8tRS7h4TJ2Vt43Dp85v2A
> > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > > > > The base64 version saves 15
> > >> > letters
> > >> > > > (to
> > >> > > > > be
> > >> > > > > > >> > fair,
> > >> > > > > > >> > > 4
> > >> > > > > > >> > > > of
> > >> > > > > > >> > > > > > >>> those
> > >> > > > > > >> > > > > > >>> > were
> > >> > > > > > >> > > > > > >>> > > > > > > dashes that we could have
> elided
> > in
> > >> > the
> > >> > > > hex
> > >> > > > > > >> > > > > > representation.)
> > >> > > > > > >> > > > > > >>> > > > > > > > > Another thing to consider
> is
> > >> that
> > >> > we
> > >> > > > > > should
> > >> > > > > > >> > > specify
> > >> > > > > > >> > > > > > that
> > >> > > > > > >> > > > > > >>> the
> > >> > > > > > >> > > > > > >>> > > > > > > all-zeroes UUID is not a valid
> > >> topic
> > >> > > UUID.
> > >> > > > > >  We
> > >> > > > > > >> > can't
> > >> > > > > > >> > > > use
> > >> > > > > > >> > > > > > >>> null
> > >> > > > > > >> > > > > > >>> > for
> > >> > > > > > >> > > > > > >>> > > > this
> > >> > > > > > >> > > > > > >>> > > > > > > because we can't pass a null
> UUID
> > >> over
> > >> > > the
> > >> > > > > RPC
> > >> > > > > > >> > > protocol
> > >> > > > > > >> > > > > > >>> (there
> > >> > > > > > >> > > > > > >>> > is no
> > >> > > > > > >> > > > > > >>> > > > > > > special pattern for null, nor
> do
> > we
> > >> > want
> > >> > > > to
> > >> > > > > > >> waste
> > >> > > > > > >> > > space
> > >> > > > > > >> > > > > > >>> reserving
> > >> > > > > > >> > > > > > >>> > > > such
> > >> > > > > > >> > > > > > >>> > > > > a
> > >> > > > > > >> > > > > > >>> > > > > > > pattern.)
> > >> > > > > > >> > > > > > >>> > > > > > > > > Maybe I missed it, but did
> > you
> > >> > > > describe
> > >> > > > > > >> > > "migration
> > >> > > > > > >> > > > > > of...
> > >> > > > > > >> > > > > > >>> > existing
> > >> > > > > > >> > > > > > >>> > > > > > > topic[s] without topic IDs" in
> > >> detail
> > >> > in
> > >> > > > any
> > >> > > > > > >> > section?
> > >> > > > > > >> > > > It
> > >> > > > > > >> > > > > > >>> seems
> > >> > > > > > >> > > > > > >>> > like
> > >> > > > > > >> > > > > > >>> > > > > when
> > >> > > > > > >> > > > > > >>> > > > > > > the new controller becomes
> > active,
> > >> it
> > >> > > > should
> > >> > > > > > >> just
> > >> > > > > > >> > > > > generate
> > >> > > > > > >> > > > > > >>> random
> > >> > > > > > >> > > > > > >>> > > > > UUIDs for
> > >> > > > > > >> > > > > > >>> > > > > > > these, and write the random
> UUIDs
> > >> back
> > >> > > to
> > >> > > > > > >> > ZooKeeper.
> > >> > > > > > >> > > > It
> > >> > > > > > >> > > > > > >>> would be
> > >> > > > > > >> > > > > > >>> > > > good
> > >> > > > > > >> > > > > > >>> > > > > to
> > >> > > > > > >> > > > > > >>> > > > > > > spell that out.  We should make
> > it
> > >> > clear
> > >> > > > > that
> > >> > > > > > >> this
> > >> > > > > > >> > > > > happens
> > >> > > > > > >> > > > > > >>> > regardless
> > >> > > > > > >> > > > > > >>> > > > > of
> > >> > > > > > >> > > > > > >>> > > > > > > the inter-broker protocol
> version
> > >> > (it's
> > >> > > a
> > >> > > > > > >> > compatible
> > >> > > > > > >> > > > > > change).
> > >> > > > > > >> > > > > > >>> > > > > > > > > "LeaderAndIsrRequests
> > >> including an
> > >> > > > > > >> > > > is_every_partition
> > >> > > > > > >> > > > > > >>> flag"
> > >> > > > > > >> > > > > > >>> > > > seems a
> > >> > > > > > >> > > > > > >>> > > > > > > bit wordy.  Can we just call
> > these
> > >> > "full
> > >> > > > > > >> > > > > > >>> LeaderAndIsrRequests"?
> > >> > > > > > >> > > > > > >>> > Then
> > >> > > > > > >> > > > > > >>> > > > > the
> > >> > > > > > >> > > > > > >>> > > > > > > RPC field could be named
> "full".
> > >> > Also,
> > >> > > it
> > >> > > > > > would
> > >> > > > > > >> > > > probably
> > >> > > > > > >> > > > > > be
> > >> > > > > > >> > > > > > >>> > better
> > >> > > > > > >> > > > > > >>> > > > > for the
> > >> > > > > > >> > > > > > >>> > > > > > > RPC field to be an enum of {
> > >> > > UNSPECIFIED,
> > >> > > > > > >> > > INCREMENTAL,
> > >> > > > > > >> > > > > FULL
> > >> > > > > > >> > > > > > >>> }, so
> > >> > > > > > >> > > > > > >>> > > > that
> > >> > > > > > >> > > > > > >>> > > > > we
> > >> > > > > > >> > > > > > >>> > > > > > > can cleanly handle old versions
> > (by
> > >> > > > treating
> > >> > > > > > >> them
> > >> > > > > > >> > as
> > >> > > > > > >> > > > > > >>> UNSPECIFIED)
> > >> > > > > > >> > > > > > >>> > > > > > > > > In the LeaderAndIsrRequest
> > >> > section,
> > >> > > > you
> > >> > > > > > >> write
> > >> > > > > > >> > "A
> > >> > > > > > >> > > > > final
> > >> > > > > > >> > > > > > >>> > deletion
> > >> > > > > > >> > > > > > >>> > > > > event
> > >> > > > > > >> > > > > > >>> > > > > > > will be secheduled for X ms
> after
> > >> the
> > >> > > > > > >> > > > LeaderAndIsrRequest
> > >> > > > > > >> > > > > > was
> > >> > > > > > >> > > > > > >>> > first
> > >> > > > > > >> > > > > > >>> > > > > > > received..."  I guess the X
> was a
> > >> > > > > placeholder
> > >> > > > > > >> that
> > >> > > > > > >> > > you
> > >> > > > > > >> > > > > > >>> intended
> > >> > > > > > >> > > > > > >>> > to
> > >> > > > > > >> > > > > > >>> > > > > replace
> > >> > > > > > >> > > > > > >>> > > > > > > before posting? :)  In any
> case,
> > >> this
> > >> > > > seems
> > >> > > > > > like
> > >> > > > > > >> > the
> > >> > > > > > >> > > > kind
> > >> > > > > > >> > > > > > of
> > >> > > > > > >> > > > > > >>> > thing
> > >> > > > > > >> > > > > > >>> > > > we'd
> > >> > > > > > >> > > > > > >>> > > > > > > want a configuration for.
> Let's
> > >> > > describe
> > >> > > > > that
> > >> > > > > > >> > > > > > configuration
> > >> > > > > > >> > > > > > >>> key
> > >> > > > > > >> > > > > > >>> > > > > somewhere
> > >> > > > > > >> > > > > > >>> > > > > > > in this KIP, including what its
> > >> > default
> > >> > > > > value
> > >> > > > > > >> is.
> > >> > > > > > >> > > > > > >>> > > > > > > > > We should probably also
> log a
> > >> > bunch
> > >> > > of
> > >> > > > > > >> messages
> > >> > > > > > >> > > at
> > >> > > > > > >> > > > > WARN
> > >> > > > > > >> > > > > > >>> level
> > >> > > > > > >> > > > > > >>> > > > when
> > >> > > > > > >> > > > > > >>> > > > > > > something is scheduled for
> > >> deletion,
> > >> > as
> > >> > > > > well.
> > >> > > > > > >> > (Maybe
> > >> > > > > > >> > > > > this
> > >> > > > > > >> > > > > > >>> was
> > >> > > > > > >> > > > > > >>> > > > > assumed, but
> > >> > > > > > >> > > > > > >>> > > > > > > it would be good to mention
> it).
> > >> > > > > > >> > > > > > >>> > > > > > > > > I feel like there are a few
> > >> > sections
> > >> > > > > that
> > >> > > > > > >> > should
> > >> > > > > > >> > > be
> > >> > > > > > >> > > > > > >>> moved to
> > >> > > > > > >> > > > > > >>> > > > > "rejected
> > >> > > > > > >> > > > > > >>> > > > > > > alternatives."  For example, in
> > the
> > >> > > > > > DeleteTopics
> > >> > > > > > >> > > > section,
> > >> > > > > > >> > > > > > >>> since
> > >> > > > > > >> > > > > > >>> > we're
> > >> > > > > > >> > > > > > >>> > > > > not
> > >> > > > > > >> > > > > > >>> > > > > > > going to do option 1 or 2,
> these
> > >> > should
> > >> > > be
> > >> > > > > > moved
> > >> > > > > > >> > into
> > >> > > > > > >> > > > > > >>> "rejected
> > >> > > > > > >> > > > > > >>> > > > > > > alternatives,"  rather than
> > >> appearing
> > >> > > > > inline.
> > >> > > > > > >> > > Another
> > >> > > > > > >> > > > > case
> > >> > > > > > >> > > > > > >>> is
> > >> > > > > > >> > > > > > >>> > the
> > >> > > > > > >> > > > > > >>> > > > > "Should
> > >> > > > > > >> > > > > > >>> > > > > > > we remove topic name from the
> > >> protocol
> > >> > > > where
> > >> > > > > > >> > > possible"
> > >> > > > > > >> > > > > > >>> section.
> > >> > > > > > >> > > > > > >>> > This
> > >> > > > > > >> > > > > > >>> > > > > is
> > >> > > > > > >> > > > > > >>> > > > > > > clearly discussing a design
> > >> > alternative
> > >> > > > that
> > >> > > > > > >> we're
> > >> > > > > > >> > > not
> > >> > > > > > >> > > > > > >>> proposing
> > >> > > > > > >> > > > > > >>> > to
> > >> > > > > > >> > > > > > >>> > > > > > > implement: removing the topic
> > name
> > >> > from
> > >> > > > > those
> > >> > > > > > >> > > > protocols.
> > >> > > > > > >> > > > > > >>> > > > > > > > > Is it really necessary to
> > have
> > >> a
> > >> > new
> > >> > > > > > >> > > > > > >>> > /admin/delete_topics_by_id
> > >> > > > > > >> > > > > > >>> > > > > path
> > >> > > > > > >> > > > > > >>> > > > > > > in ZooKeeper?  It seems like we
> > >> don't
> > >> > > > really
> > >> > > > > > >> need
> > >> > > > > > >> > > this.
> > >> > > > > > >> > > > > > >>> Whenever
> > >> > > > > > >> > > > > > >>> > > > > there is
> > >> > > > > > >> > > > > > >>> > > > > > > a new controller, we'll send
> out
> > >> full
> > >> > > > > > >> > > > > LeaderAndIsrRequests
> > >> > > > > > >> > > > > > >>> which
> > >> > > > > > >> > > > > > >>> > will
> > >> > > > > > >> > > > > > >>> > > > > > > trigger the stale topics to be
> > >> cleaned
> > >> > > up.
> > >> > > > > >  The
> > >> > > > > > >> > > active
> > >> > > > > > >> > > > > > >>> > controller
> > >> > > > > > >> > > > > > >>> > > > will
> > >> > > > > > >> > > > > > >>> > > > > > > also send the full
> > >> LeaderAndIsrRequest
> > >> > > to
> > >> > > > > > >> brokers
> > >> > > > > > >> > > that
> > >> > > > > > >> > > > > are
> > >> > > > > > >> > > > > > >>> just
> > >> > > > > > >> > > > > > >>> > > > > starting
> > >> > > > > > >> > > > > > >>> > > > > > > up.    So we don't really need
> > this
> > >> > kind
> > >> > > > of
> > >> > > > > > >> > two-phase
> > >> > > > > > >> > > > > > commit
> > >> > > > > > >> > > > > > >>> > (send
> > >> > > > > > >> > > > > > >>> > > > out
> > >> > > > > > >> > > > > > >>> > > > > > > StopReplicasRequest, get ACKs
> > from
> > >> all
> > >> > > > > nodes,
> > >> > > > > > >> > commit
> > >> > > > > > >> > > by
> > >> > > > > > >> > > > > > >>> removing
> > >> > > > > > >> > > > > > >>> > > > > > > /admin/delete_topics node) any
> > >> more.
> > >> > > > > > >> > > > > > >>> > > > > > > > > You mention that
> FetchRequest
> > >> will
> > >> > > now
> > >> > > > > > >> include
> > >> > > > > > >> > > UUID
> > >> > > > > > >> > > > > to
> > >> > > > > > >> > > > > > >>> avoid
> > >> > > > > > >> > > > > > >>> > > > issues
> > >> > > > > > >> > > > > > >>> > > > > > > where requests are made to
> stale
> > >> > > > partitions.
> > >> > > > > > >> > > However,
> > >> > > > > > >> > > > > > >>> adding a
> > >> > > > > > >> > > > > > >>> > UUID
> > >> > > > > > >> > > > > > >>> > > > to
> > >> > > > > > >> > > > > > >>> > > > > > > MetadataRequest is listed as
> > future
> > >> > > work,
> > >> > > > > out
> > >> > > > > > of
> > >> > > > > > >> > > scope
> > >> > > > > > >> > > > > for
> > >> > > > > > >> > > > > > >>> this
> > >> > > > > > >> > > > > > >>> > KIP.
> > >> > > > > > >> > > > > > >>> > > > > How
> > >> > > > > > >> > > > > > >>> > > > > > > will the client learn what the
> > >> topic
> > >> > > UUID
> > >> > > > > is,
> > >> > > > > > if
> > >> > > > > > >> > the
> > >> > > > > > >> > > > > > metadata
> > >> > > > > > >> > > > > > >>> > > > response
> > >> > > > > > >> > > > > > >>> > > > > > > doesn't include that
> information?
> > >> It
> > >> > > > seems
> > >> > > > > > like
> > >> > > > > > >> > > adding
> > >> > > > > > >> > > > > the
> > >> > > > > > >> > > > > > >>> UUID
> > >> > > > > > >> > > > > > >>> > to
> > >> > > > > > >> > > > > > >>> > > > > > > MetadataResponse would be an
> > >> > improvement
> > >> > > > > here
> > >> > > > > > >> that
> > >> > > > > > >> > > > might
> > >> > > > > > >> > > > > > not
> > >> > > > > > >> > > > > > >>> be
> > >> > > > > > >> > > > > > >>> > too
> > >> > > > > > >> > > > > > >>> > > > > hard to
> > >> > > > > > >> > > > > > >>> > > > > > > make.
> > >> > > > > > >> > > > > > >>> > > > > > > > > best,
> > >> > > > > > >> > > > > > >>> > > > > > > > > Colin
> > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > > > > On Mon, Sep 9, 2019, at
> > 17:48,
> > >> > > Ryanne
> > >> > > > > > Dolan
> > >> > > > > > >> > > wrote:
> > >> > > > > > >> > > > > > >>> > > > > > > > > > Lucas, this would be
> great.
> > >> I've
> > >> > > run
> > >> > > > > > into
> > >> > > > > > >> > > issues
> > >> > > > > > >> > > > > with
> > >> > > > > > >> > > > > > >>> > topics
> > >> > > > > > >> > > > > > >>> > > > > being
> > >> > > > > > >> > > > > > >>> > > > > > > > > > resurrected accidentally,
> > >> since
> > >> > a
> > >> > > > > client
> > >> > > > > > >> > cannot
> > >> > > > > > >> > > > > > easily
> > >> > > > > > >> > > > > > >>> > > > > distinguish
> > >> > > > > > >> > > > > > >>> > > > > > > between
> > >> > > > > > >> > > > > > >>> > > > > > > > > > a deleted topic and a new
> > >> topic
> > >> > > with
> > >> > > > > the
> > >> > > > > > >> same
> > >> > > > > > >> > > > name.
> > >> > > > > > >> > > > > > I'd
> > >> > > > > > >> > > > > > >>> > need
> > >> > > > > > >> > > > > > >>> > > > the
> > >> > > > > > >> > > > > > >>> > > > > ID
> > >> > > > > > >> > > > > > >>> > > > > > > > > > accessible from the
> client
> > to
> > >> > > solve
> > >> > > > > that
> > >> > > > > > >> > issue,
> > >> > > > > > >> > > > but
> > >> > > > > > >> > > > > > >>> this
> > >> > > > > > >> > > > > > >>> > is a
> > >> > > > > > >> > > > > > >>> > > > > good
> > >> > > > > > >> > > > > > >>> > > > > > > first
> > >> > > > > > >> > > > > > >>> > > > > > > > > > step.
> > >> > > > > > >> > > > > > >>> > > > > > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > > > > > Ryanne
> > >> > > > > > >> > > > > > >>> > > > > > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > > > > > On Wed, Sep 4, 2019 at
> 1:41
> > >> PM
> > >> > > Lucas
> > >> > > > > > >> > > Bradstreet <
> > >> > > > > > >> > > > > > >>> > > > > lucas@confluent.io>
> > >> > > > > > >> > > > > > >>> > > > > > > wrote:
> > >> > > > > > >> > > > > > >>> > > > > > > > > > > Hi all,
> > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > > > > > > I would like to kick
> off
> > >> > > > discussion
> > >> > > > > of
> > >> > > > > > >> > > KIP-516,
> > >> > > > > > >> > > > > an
> > >> > > > > > >> > > > > > >>> > > > > implementation
> > >> > > > > > >> > > > > > >>> > > > > > > of topic
> > >> > > > > > >> > > > > > >>> > > > > > > > > > > IDs for Kafka. Topic
> IDs
> > >> aim
> > >> > to
> > >> > > > > solve
> > >> > > > > > >> topic
> > >> > > > > > >> > > > > > >>> uniqueness
> > >> > > > > > >> > > > > > >>> > > > > problems in
> > >> > > > > > >> > > > > > >>> > > > > > > Kafka,
> > >> > > > > > >> > > > > > >>> > > > > > > > > > > where referring to a
> > topic
> > >> by
> > >> > > name
> > >> > > > > > >> alone is
> > >> > > > > > >> > > > > > >>> insufficient.
> > >> > > > > > >> > > > > > >>> > > > Such
> > >> > > > > > >> > > > > > >>> > > > > > > cases
> > >> > > > > > >> > > > > > >>> > > > > > > > > > > include when a topic
> has
> > >> been
> > >> > > > > deleted
> > >> > > > > > >> and
> > >> > > > > > >> > > > > recreated
> > >> > > > > > >> > > > > > >>> with
> > >> > > > > > >> > > > > > >>> > the
> > >> > > > > > >> > > > > > >>> > > > > same
> > >> > > > > > >> > > > > > >>> > > > > > > name.
> > >> > > > > > >> > > > > > >>> > > > > > > > > > > Unique identifiers will
> > >> help
> > >> > > > > simplify
> > >> > > > > > >> and
> > >> > > > > > >> > > > improve
> > >> > > > > > >> > > > > > >>> Kafka's
> > >> > > > > > >> > > > > > >>> > > > topic
> > >> > > > > > >> > > > > > >>> > > > > > > deletion
> > >> > > > > > >> > > > > > >>> > > > > > > > > > > process, as well as
> > prevent
> > >> > > cases
> > >> > > > > > where
> > >> > > > > > >> > > brokers
> > >> > > > > > >> > > > > may
> > >> > > > > > >> > > > > > >>> > > > incorrectly
> > >> > > > > > >> > > > > > >>> > > > > > > interact
> > >> > > > > > >> > > > > > >>> > > > > > > > > > > with stale versions of
> > >> topics.
> > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > >> > > > > > >> > > > > > >>> > > >
> > >> > > > > > >> > > > > > >>> >
> > >> > > > > > >> > > > > > >>>
> > >> > > > > > >> > > > > >
> > >> > > > > > >> > > > >
> > >> > > > > > >> > > >
> > >> > > > > > >> > >
> > >> > > > > > >> >
> > >> > > > > > >>
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > >> > > > > > >> > > > > > >>> > > > > > > > > > > Looking forward to your
> > >> > > thoughts.
> > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > > > > > > Lucas
> > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > >> > > > > > >> > > > > > >>> >
> > >> > > > > > >> > > > > > >>> >
> > >> > > > > > >> > > > > > >>>
> > >> > > > > > >> > > > > > >>
> > >> > > > > > >> > > > > >
> > >> > > > > > >> > > > >
> > >> > > > > > >> > > >
> > >> > > > > > >> > >
> > >> > > > > > >> >
> > >> > > > > > >>
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> >
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Justine Olshan <jo...@confluent.io>]

Hi all,

This KIP has been accepted, but I just wanted to share some information
about Topic IDs that I have just added to the KIP. Part of the KIP includes
support for supplying a topic ID with no name when asking to describe
topics. I'd like to draw attention to the way we will handle topic IDs when
they are not found on the broker.

*MetadataRequest must also be modified. Topic name will be left in to allow
requests to be made either by topic name or topic ID. Requests should only
use one or the other.*


*ID will be checked first, but if the value is the default zero UUID, topic
name will be used instead.  If an ID is specified and the ID does not
exist, the request will fail regardless of allow_auto_topic_creation. If
the topic ID is not found, the request will return
an UNKNOWN_TOPIC_ID error for the topic indicating the topic ID did not
exist. The check for the topic ID will occur before checking authorization
on the topic. Thus, topic IDs are not considered sensitive information.*


A delete topics request will in the same way return an UNKNOWN_TOPIC_ID error
if the topic ID is not found.

Since UNKNOWN_TOPIC_ID is a public facing API, I thought it was important
to update everyone with this change.

Thanks,

Justine



On Tue, Oct 13, 2020 at 3:18 PM Justine Olshan <jo...@confluent.io> wrote:

> Hi all,
>
> I'm moving my response to the VOTE thread here as to not start filling the
> vote thread with more discussion.
>
> Here's what was said:
>
>
>
>
> *1. It seems that we should remove the topic field from
> theStopReplicaResponse below?StopReplica Response (Version: 4) =>
> error_code [topics]  error_code => INT16    topics => topic topic_id*
> [partitions]*
>
> *Fixed*
>
> *2. "*After controller* election, upon receiving the result, assign the*
> *metadata topic its unique topic ID". Will the UUID for the metadata topic*
> *be written to the metadata topic itself?*
>
> * I believe so. The topic ID would be written to ZooKeeper, so when
> ZooKeeper is replaced, it makes sense to write the topic ID for the
> metadata topic in the metadata topic.*
>
> *3. The vote request is designed to support multiple topics, each of them*
> *may require a different sentinel ID. Should we reserve more than one*
> *sentinel ID for future usage?*
>
>
> *The plan was that only the metadata topic will use the sentinel ID before
> the controller is elected. Once the controller is elected, the metadata
> topic will simply use a unique id, so I believe these other topics can just
> use a unique ID given by the controller.*
> *4. UUID.randomUUID(): Could we clarify whether this method returns any*
> *sentinel ID? Also, how do we expect the user to use it?*
>
>
> *The method does not return the sentinel ID. I wasn't sure if this was
> something worth mentioning since the sentinel ID is only used internally,
> and it is only one out of many values that could be obtained. (In normal
> use, the user would not notice this value is missing.) I thought this
> method would be useful for testing, since a user might not want to
> specify their own UUID values.*
>
> Thanks,
> Justine
>
>
> On Fri, Oct 9, 2020 at 2:00 PM Justine Olshan <jo...@confluent.io>
> wrote:
>
>> Hi all,
>>
>> I think it is almost time to vote on this KIP. Please let me know if
>> there are any outstanding issues.
>>
>> Thank you,
>> Justine
>>
>> On Fri, Oct 9, 2020 at 7:56 AM Justine Olshan <jo...@confluent.io>
>> wrote:
>>
>>> Hi Jun,
>>>
>>> 50. Right. That's what I was thinking but wanted to make sure. Works for
>>> me!
>>>
>>> Thanks again,
>>> Justine
>>>
>>> On Thu, Oct 8, 2020 at 3:50 PM Jun Rao <ju...@confluent.io> wrote:
>>>
>>>> Hi, Justine,
>>>>
>>>> 50. We can't change the format of existing versions since it will break
>>>> compatibility. Since we are bumping the version for MetadataRequest
>>>> and DeleteTopicsRequest, we can make the topic name field nullable from
>>>> the
>>>> new version.
>>>>
>>>> Thanks,
>>>>
>>>> Jun
>>>>
>>>> On Thu, Oct 8, 2020 at 2:19 PM Justine Olshan <jo...@confluent.io>
>>>> wrote:
>>>>
>>>> > Hi Jun,
>>>> >
>>>> > 50. Works for me. Is the idea that it will be nullable from this
>>>> version
>>>> > onwards, but not past versions?
>>>> >
>>>> > 51. Added some information and example methods for AdminClient future
>>>> work
>>>> >
>>>> > Thanks,
>>>> > Justine
>>>> >
>>>> >
>>>> > On Thu, Oct 8, 2020 at 10:59 AM Jun Rao <ju...@confluent.io> wrote:
>>>> >
>>>> > > Hi, Justine,
>>>> > >
>>>> > > The changes sound reasonable to me.
>>>> > >
>>>> > > 50. Should we make the topic name in the new MetadataRequest
>>>> > > and DeleteTopicsRequest nullable?
>>>> > >
>>>> > > 51. In the future work section, perhaps we could mention
>>>> opportunities
>>>> > for
>>>> > > using topicId in other methods in AdminClient where topic name is
>>>> used.
>>>> > >
>>>> > > Thanks,
>>>> > >
>>>> > > Jun
>>>> > >
>>>> > > On Wed, Oct 7, 2020 at 2:44 PM Justine Olshan <jolshan@confluent.io
>>>> >
>>>> > > wrote:
>>>> > >
>>>> > > > Hi all,
>>>> > > >
>>>> > > > Another update on the KIP. I've added some information about how
>>>> topic
>>>> > > IDs
>>>> > > > will be supported in AdminClient
>>>> > > >
>>>> > > >
>>>> > > >
>>>> > >
>>>> >
>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-AdminClientSupport
>>>> > > >
>>>> > > >
>>>> > > > One specific decision I wanted to focus on is the inclusion of
>>>> both
>>>> > topic
>>>> > > > names and topic Ids in metadata and deleteTopics requests.
>>>> > > >
>>>> > > > The idea is that when the request is received the topic id field
>>>> is
>>>> > > checked
>>>> > > > first, but if it contains the default (zero) uuid, then the topic
>>>> name
>>>> > > > field will be used.
>>>> > > >
>>>> > > > In the case where ids are used, if the ids do not have a
>>>> corresponding
>>>> > > > topic, the request will fail. In the case where both ids and
>>>> names are
>>>> > > > included, the request will fail if the name does not match the id.
>>>> > > >
>>>> > > >
>>>> > > > Initially the id field will be used to support the admin client
>>>> use
>>>> > cases
>>>> > > > as described in the KIP, but over time, we can transfer to using
>>>> topic
>>>> > > ids
>>>> > > > in all uses of the request.
>>>> > > >
>>>> > > >
>>>> > > > There will also be some support for returning the topic Id on the
>>>> > > > createtopics, deletetopics and metadata responses. I think it
>>>> makes
>>>> > sense
>>>> > > > to include both the name and the id in the successful response for
>>>> > these
>>>> > > > cases.
>>>> > > >
>>>> > > >
>>>> > > > Let me know if you have any comments or concerns regarding the
>>>> Admin
>>>> > > Client
>>>> > > > changes.
>>>> > > >
>>>> > > > On Wed, Oct 7, 2020 at 11:05 AM Jun Rao <ju...@confluent.io> wrote:
>>>> > > >
>>>> > > > > Hi, Justine,
>>>> > > > >
>>>> > > > > I was just wondering if we expose topicId publicly, how would
>>>> people
>>>> > > use
>>>> > > > > it. One possibility is to use topicId in all those public
>>>> interfaces
>>>> > > > where
>>>> > > > > we use topic name. I am not sure if the need is very strong
>>>> though.
>>>> > So,
>>>> > > > > maybe we can list this as future considerations.
>>>> > > > >
>>>> > > > > Thanks,
>>>> > > > >
>>>> > > > > Jun
>>>> > > > >
>>>> > > > > On Wed, Oct 7, 2020 at 8:57 AM Justine Olshan <
>>>> jolshan@confluent.io>
>>>> > > > > wrote:
>>>> > > > >
>>>> > > > > > Hi all,
>>>> > > > > >
>>>> > > > > > After some careful consideration, I've decided that the
>>>> > > > TopicIdPartition
>>>> > > > > > will be out of scope for this KIP. After investigating what
>>>> it will
>>>> > > > look
>>>> > > > > > like using it in only the Fetch Request, I found that I would
>>>> need
>>>> > to
>>>> > > > > > modify several public APIs in the consumer and I would need
>>>> to take
>>>> > > > > special
>>>> > > > > > care in many situations to handle the request correctly based
>>>> on
>>>> > what
>>>> > > > > > version of the request was sent. For these reasons, I've added
>>>> > > > > > TopicIdPartition to potential future work along with an
>>>> alternative
>>>> > > to
>>>> > > > > > modify TopicPartition to include a topic ID field.
>>>> > > > > >
>>>> > > > > > Jun- I was also wondering about some of the details for
>>>> supporting
>>>> > > > topic
>>>> > > > > > IDs for delete and describe topics in the admin client. Is
>>>> the idea
>>>> > > > that
>>>> > > > > > instead of supplying topic names to describe or delete, we
>>>> would
>>>> > > > supply a
>>>> > > > > > collection of ids? If that is the case, I'm wondering if it
>>>> makes
>>>> > > sense
>>>> > > > > to
>>>> > > > > > add topic Ids to the metadata and delete topics requests.
>>>> > > > > >
>>>> > > > > > Thanks,
>>>> > > > > > Justine
>>>> > > > > >
>>>> > > > > > On Mon, Oct 5, 2020 at 9:25 PM Satish Duggana <
>>>> > > > satish.duggana@gmail.com>
>>>> > > > > > wrote:
>>>> > > > > >
>>>> > > > > > > Hi Jun, Justine,
>>>> > > > > > > +1 to have TopicIdPartition in public packages. This is
>>>> useful
>>>> > for
>>>> > > > > > > KIP-405 tiered storage too. We currently use TopicPartition
>>>> for
>>>> > > > > > > pluggable log storage and metadata storage apis. We will be
>>>> using
>>>> > > > > > > TopicIdPartition once the proposed changes in this KIP are
>>>> > merged.
>>>> > > > > > >
>>>> > > > > > > Thanks,
>>>> > > > > > > Satish.
>>>> > > > > > >
>>>> > > > > > > On Tue, Oct 6, 2020 at 5:21 AM Jun Rao <ju...@confluent.io>
>>>> wrote:
>>>> > > > > > > >
>>>> > > > > > > > Hi, Justine,
>>>> > > > > > > >
>>>> > > > > > > > I guess we have to maintain the mapping between topic
>>>> name and
>>>> > > > > topicId
>>>> > > > > > in
>>>> > > > > > > > both the client and the server. If TopicIdPartition is
>>>> meant
>>>> > for
>>>> > > > > > internal
>>>> > > > > > > > usage only, we can create it under a package
>>>> > > > > > > > like org/apache/kafka/common/internal and we don't need to
>>>> > > document
>>>> > > > > it
>>>> > > > > > in
>>>> > > > > > > > the KIP. If it's meant for external public usage (and
>>>> > potentially
>>>> > > > > > > internal
>>>> > > > > > > > usage too), we can create it in org/apache/kafka/common
>>>> and
>>>> > need
>>>> > > to
>>>> > > > > > > > document it in the KIP.
>>>> > > > > > > >
>>>> > > > > > > > Since topicId is now part of the topic metadata, perhaps
>>>> it's
>>>> > > > useful
>>>> > > > > to
>>>> > > > > > > > expose it in TopicMetadataAndConfig too.
>>>> > > > > > > >
>>>> > > > > > > > Thanks,
>>>> > > > > > > >
>>>> > > > > > > > Jun
>>>> > > > > > > >
>>>> > > > > > > > On Mon, Oct 5, 2020 at 3:35 PM Justine Olshan <
>>>> > > > jolshan@confluent.io>
>>>> > > > > > > wrote:
>>>> > > > > > > >
>>>> > > > > > > > > Hi Jun,
>>>> > > > > > > > >
>>>> > > > > > > > > This KIP removes the topic name from many requests. A
>>>> lot of
>>>> > > > > requests
>>>> > > > > > > keep
>>>> > > > > > > > > track of the partitions internally with a
>>>> TopicPartition. So
>>>> > > when
>>>> > > > > we
>>>> > > > > > > remove
>>>> > > > > > > > > the topic name, we can not differentiate unless we can
>>>> also
>>>> > use
>>>> > > > > topic
>>>> > > > > > > ID.
>>>> > > > > > > > > The Fetch request relies heavily on this, but
>>>> ListOffset,
>>>> > > > > > > > > OffsetsForLeaderEpoch, etc all use TopicPartition as a
>>>> key
>>>> > > value
>>>> > > > > for
>>>> > > > > > > > > internal maps. Digging deeper into Fetch, I've noticed
>>>> that
>>>> > the
>>>> > > > > code
>>>> > > > > > > path
>>>> > > > > > > > > runs into some public APIs that use the TopicPartition.
>>>> Some
>>>> > > > > examples
>>>> > > > > > > are
>>>> > > > > > > > > Cluster.java or RecordTooLargeException.
>>>> > > > > > > > >
>>>> > > > > > > > > A related issue is that in some places like
>>>> > > > > > > TopicAuthorizationException,
>>>> > > > > > > > > the topic name is required and without the name in the
>>>> > request,
>>>> > > > we
>>>> > > > > > > won't
>>>> > > > > > > > > know it.
>>>> > > > > > > > >
>>>> > > > > > > > > I guess an alternative would be to have some mapping
>>>> from
>>>> > topic
>>>> > > > > name
>>>> > > > > > to
>>>> > > > > > > > > topic ID, but I would have to think about that more.
>>>> > > > > > > > >
>>>> > > > > > > > > The KIP mentions exposing TopicId in TopicDescription.
>>>> (The
>>>> > > font
>>>> > > > > was
>>>> > > > > > > > > weirdly colored, so I've made it easier to read.)
>>>> > > > > > > > >
>>>> > > > > > > > > Would the idea for the TopicMetadataAndConfig be that
>>>> the
>>>> > topic
>>>> > > > ID
>>>> > > > > > > would be
>>>> > > > > > > > > returned as a result?
>>>> > > > > > > > > As for the other methods describe/deletetopics, it makes
>>>> > sense
>>>> > > to
>>>> > > > > > > include
>>>> > > > > > > > > them at some point, but I'm not sure how far we should
>>>> go in
>>>> > > > terms
>>>> > > > > of
>>>> > > > > > > > > supporting topic IDs everywhere.
>>>> > > > > > > > >
>>>> > > > > > > > > Thank you for continuing to look at the KIP!
>>>> > > > > > > > > Justine
>>>> > > > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > On Mon, Oct 5, 2020 at 2:53 PM Jun Rao <
>>>> jun@confluent.io>
>>>> > > wrote:
>>>> > > > > > > > >
>>>> > > > > > > > > > Hi, Justine,
>>>> > > > > > > > > >
>>>> > > > > > > > > > Adding a TopicIdPartition in the public interface may
>>>> be
>>>> > > > useful.
>>>> > > > > > > Where
>>>> > > > > > > > > > would you expect it to be used?
>>>> > > > > > > > > >
>>>> > > > > > > > > > Related to this, since we are exposing topicId in the
>>>> tool,
>>>> > > it
>>>> > > > > > seems
>>>> > > > > > > that
>>>> > > > > > > > > > we may want to expose topicId in at least
>>>> TopicDescription
>>>> > > used
>>>> > > > > by
>>>> > > > > > > > > > AdminClient.describeTopics() and
>>>> TopicMetadataAndConfig
>>>> > used
>>>> > > in
>>>> > > > > > > > > > AdminClient.createTopics(Collection<NewTopic>
>>>> newTopics). I
>>>> > > am
>>>> > > > > also
>>>> > > > > > > not
>>>> > > > > > > > > > sure if we plan to expose topicId in other AdminClient
>>>> > > methods
>>>> > > > in
>>>> > > > > > > this
>>>> > > > > > > > > KIP
>>>> > > > > > > > > > or in some future KIP. For example, it might be
>>>> useful to
>>>> > > > support
>>>> > > > > > > topicId
>>>> > > > > > > > > > in the following methods.
>>>> > > > > > > > > >
>>>> > > > > > > > > >     DescribeTopicsResult
>>>> describeTopics(Collection<String>
>>>> > > > > > > topicNames,
>>>> > > > > > > > > > DescribeTopicsOptions options);
>>>> > > > > > > > > >     DeleteTopicsResult deleteTopics(Collection<String>
>>>> > > topics,
>>>> > > > > > > > > > DeleteTopicsOptions options);
>>>> > > > > > > > > >
>>>> > > > > > > > > > Thanks,
>>>> > > > > > > > > >
>>>> > > > > > > > > > Jun
>>>> > > > > > > > > >
>>>> > > > > > > > > > On Mon, Oct 5, 2020 at 11:03 AM Justine Olshan <
>>>> > > > > > jolshan@confluent.io
>>>> > > > > > > >
>>>> > > > > > > > > > wrote:
>>>> > > > > > > > > >
>>>> > > > > > > > > > > Hi all,
>>>> > > > > > > > > > >
>>>> > > > > > > > > > > I was thinking about this KIP some more. We are
>>>> removing
>>>> > > the
>>>> > > > > > topic
>>>> > > > > > > name
>>>> > > > > > > > > > > from many of these request types, and some of them
>>>> (eg.
>>>> > > > Fetch)
>>>> > > > > > use
>>>> > > > > > > > > > > TopicPartition as keys in a map when handling the
>>>> > request.
>>>> > > I
>>>> > > > > was
>>>> > > > > > > > > thinking
>>>> > > > > > > > > > > it would make sense to make some sort of
>>>> TopicIdPartition
>>>> > > > with
>>>> > > > > > > topic
>>>> > > > > > > > > > name,
>>>> > > > > > > > > > > topic id, and partition fields (name and id would
>>>> > > potentially
>>>> > > > > be
>>>> > > > > > > > > optional
>>>> > > > > > > > > > > types since different request versions will contain
>>>> only
>>>> > > one
>>>> > > > of
>>>> > > > > > the
>>>> > > > > > > > > two)
>>>> > > > > > > > > > I
>>>> > > > > > > > > > > was wondering if it made sense to include this as a
>>>> > public
>>>> > > > > facing
>>>> > > > > > > API
>>>> > > > > > > > > > > similar to TopicPartition. If so, I will add this
>>>> class
>>>> > to
>>>> > > > the
>>>> > > > > > KIP
>>>> > > > > > > as
>>>> > > > > > > > > > well.
>>>> > > > > > > > > > >
>>>> > > > > > > > > > > Thanks,
>>>> > > > > > > > > > > Justine
>>>> > > > > > > > > > >
>>>> > > > > > > > > > > On Mon, Oct 5, 2020 at 8:52 AM Justine Olshan <
>>>> > > > > > > jolshan@confluent.io>
>>>> > > > > > > > > > > wrote:
>>>> > > > > > > > > > >
>>>> > > > > > > > > > > > Hi all,
>>>> > > > > > > > > > > > I've made a few minor changes to the UUID class.
>>>> I've
>>>> > > > added a
>>>> > > > > > few
>>>> > > > > > > > > more
>>>> > > > > > > > > > > > methods to support UUIDs more generally, by
>>>> including a
>>>> > > > > > > constructor
>>>> > > > > > > > > > and a
>>>> > > > > > > > > > > > method to generate random UUIDs. Both of these
>>>> will be
>>>> > > > useful
>>>> > > > > > for
>>>> > > > > > > > > > > testing.
>>>> > > > > > > > > > > >
>>>> > > > > > > > > > > > Let me know if there are any other areas for
>>>> > discussion.
>>>> > > > > > > > > > > >
>>>> > > > > > > > > > > > Thank you,
>>>> > > > > > > > > > > > Justine
>>>> > > > > > > > > > > >
>>>> > > > > > > > > > > > On Thu, Oct 1, 2020 at 5:29 PM Justine Olshan <
>>>> > > > > > > jolshan@confluent.io>
>>>> > > > > > > > > > > > wrote:
>>>> > > > > > > > > > > >
>>>> > > > > > > > > > > >> Hi Jun,
>>>> > > > > > > > > > > >> Thanks for looking at it again. Here's the new
>>>> spec.
>>>> > (I
>>>> > > > > fixed
>>>> > > > > > > the
>>>> > > > > > > > > typo
>>>> > > > > > > > > > > in
>>>> > > > > > > > > > > >> it too.)
>>>> > > > > > > > > > > >> {"name": "id", "type": "string", "doc":
>>>> option[UUID]}
>>>> > > > > > > > > > > >>
>>>> > > > > > > > > > > >> Justine
>>>> > > > > > > > > > > >>
>>>> > > > > > > > > > > >>
>>>> > > > > > > > > > > >> On Thu, Oct 1, 2020 at 5:03 PM Jun Rao <
>>>> > > jun@confluent.io>
>>>> > > > > > > wrote:
>>>> > > > > > > > > > > >>
>>>> > > > > > > > > > > >>> Hi, Justine,
>>>> > > > > > > > > > > >>>
>>>> > > > > > > > > > > >>> Thanks for the update. The KIP looks good to me
>>>> now.
>>>> > > > Just a
>>>> > > > > > > minor
>>>> > > > > > > > > > > comment
>>>> > > > > > > > > > > >>> below.
>>>> > > > > > > > > > > >>>
>>>> > > > > > > > > > > >>> 30. Perhaps "option[UUID]" can be put in the
>>>> doc.
>>>> > > > > > > > > > > >>>
>>>> > > > > > > > > > > >>> Jun
>>>> > > > > > > > > > > >>>
>>>> > > > > > > > > > > >>> On Thu, Oct 1, 2020 at 3:28 PM Justine Olshan <
>>>> > > > > > > > > jolshan@confluent.io>
>>>> > > > > > > > > > > >>> wrote:
>>>> > > > > > > > > > > >>>
>>>> > > > > > > > > > > >>> > Hi Jun,
>>>> > > > > > > > > > > >>> > Thanks for the response!
>>>> > > > > > > > > > > >>> >
>>>> > > > > > > > > > > >>> > 30. I think I might have changed this in
>>>> between.
>>>> > The
>>>> > > > > > current
>>>> > > > > > > > > > version
>>>> > > > > > > > > > > >>> > says:  {"name":
>>>> > > > > > > > > > > >>> > "id", "type": "option[UUID]"}, "doc": topic
>>>> id}
>>>> > > > > > > > > > > >>> > I have switched to the option type to cover
>>>> the
>>>> > > > migration
>>>> > > > > > > case
>>>> > > > > > > > > > where
>>>> > > > > > > > > > > a
>>>> > > > > > > > > > > >>> > TopicZNode does not yet have a topic ID.
>>>> > > > > > > > > > > >>> > I understand that due to json, this field is
>>>> > written
>>>> > > > as a
>>>> > > > > > > string,
>>>> > > > > > > > > > so
>>>> > > > > > > > > > > >>> if I
>>>> > > > > > > > > > > >>> > should move the "option[uuid]" to the doc
>>>> field and
>>>> > > the
>>>> > > > > > type
>>>> > > > > > > > > should
>>>> > > > > > > > > > > be
>>>> > > > > > > > > > > >>> > "string" please let me know.
>>>> > > > > > > > > > > >>> >
>>>> > > > > > > > > > > >>> > 40. I've added a definition for UUID.
>>>> > > > > > > > > > > >>> > 41,42. Fixed
>>>> > > > > > > > > > > >>> >
>>>> > > > > > > > > > > >>> > Thank you,
>>>> > > > > > > > > > > >>> > Justine
>>>> > > > > > > > > > > >>> >
>>>> > > > > > > > > > > >>> > On Wed, Sep 30, 2020 at 1:15 PM Jun Rao <
>>>> > > > > jun@confluent.io>
>>>> > > > > > > > > wrote:
>>>> > > > > > > > > > > >>> >
>>>> > > > > > > > > > > >>> > > Hi, Justine,
>>>> > > > > > > > > > > >>> > >
>>>> > > > > > > > > > > >>> > > Thanks for the summary. Just a few minor
>>>> comments
>>>> > > > blow.
>>>> > > > > > > > > > > >>> > >
>>>> > > > > > > > > > > >>> > > 30.  {"name": "id", "type": "string", "doc":
>>>> > > "version
>>>> > > > > > > id"}}:
>>>> > > > > > > > > The
>>>> > > > > > > > > > > doc
>>>> > > > > > > > > > > >>> > should
>>>> > > > > > > > > > > >>> > > say UUID. The issue seems unfixed.
>>>> > > > > > > > > > > >>> > >
>>>> > > > > > > > > > > >>> > > 40. Since UUID is public facing, could you
>>>> > include
>>>> > > > its
>>>> > > > > > > > > > definition?
>>>> > > > > > > > > > > >>> > >
>>>> > > > > > > > > > > >>> > > 41. StopReplicaResponse still includes the
>>>> topic
>>>> > > > field.
>>>> > > > > > > > > > > >>> > >
>>>> > > > > > > > > > > >>> > > 42. "It is unnecessary to include the name
>>>> of the
>>>> > > > topic
>>>> > > > > > in
>>>> > > > > > > the
>>>> > > > > > > > > > > >>> following
>>>> > > > > > > > > > > >>> > > Request/Response calls" It would be useful
>>>> to
>>>> > > include
>>>> > > > > all
>>>> > > > > > > > > > modified
>>>> > > > > > > > > > > >>> > requests
>>>> > > > > > > > > > > >>> > > (e.g. produce) in the list below.
>>>> > > > > > > > > > > >>> > >
>>>> > > > > > > > > > > >>> > > Thanks,
>>>> > > > > > > > > > > >>> > >
>>>> > > > > > > > > > > >>> > > Jun
>>>> > > > > > > > > > > >>> > >
>>>> > > > > > > > > > > >>> > >
>>>> > > > > > > > > > > >>> > > On Wed, Sep 30, 2020 at 10:17 AM Justine
>>>> Olshan <
>>>> > > > > > > > > > > >>> jolshan@confluent.io>
>>>> > > > > > > > > > > >>> > > wrote:
>>>> > > > > > > > > > > >>> > >
>>>> > > > > > > > > > > >>> > > > Hello again,
>>>> > > > > > > > > > > >>> > > >
>>>> > > > > > > > > > > >>> > > > I've taken some time to discuss some of
>>>> the
>>>> > > > remaining
>>>> > > > > > > points
>>>> > > > > > > > > > > >>> brought up
>>>> > > > > > > > > > > >>> > > by
>>>> > > > > > > > > > > >>> > > > the previous emails offline. Here are
>>>> some of
>>>> > the
>>>> > > > > > > > > conclusions.
>>>> > > > > > > > > > > >>> > > >
>>>> > > > > > > > > > > >>> > > > 1. Directory Structure:
>>>> > > > > > > > > > > >>> > > > There was some talk about whether the
>>>> directory
>>>> > > > > > structure
>>>> > > > > > > > > > should
>>>> > > > > > > > > > > be
>>>> > > > > > > > > > > >>> > > changed
>>>> > > > > > > > > > > >>> > > > to replace all topic names with topic IDs.
>>>> > > > > > > > > > > >>> > > > This will be a useful change, but will
>>>> prevent
>>>> > > > > > > downgrades. It
>>>> > > > > > > > > > > will
>>>> > > > > > > > > > > >>> be
>>>> > > > > > > > > > > >>> > > best
>>>> > > > > > > > > > > >>> > > > to wait until a major release, likely
>>>> alongside
>>>> > > > > KIP-500
>>>> > > > > > > > > changes
>>>> > > > > > > > > > > >>> that
>>>> > > > > > > > > > > >>> > will
>>>> > > > > > > > > > > >>> > > > prevent downgrades. I've updated the KIP
>>>> to
>>>> > > include
>>>> > > > > > this
>>>> > > > > > > > > change
>>>> > > > > > > > > > > >>> with
>>>> > > > > > > > > > > >>> > the
>>>> > > > > > > > > > > >>> > > > note about migration and deprecation.
>>>> > > > > > > > > > > >>> > > >
>>>> > > > > > > > > > > >>> > > >
>>>> > > > > > > > > > > >>> > >
>>>> > > > > > > > > > > >>> >
>>>> > > > > > > > > > > >>>
>>>> > > > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > >
>>>> > > > > > >
>>>> > > > > >
>>>> > > > >
>>>> > > >
>>>> > >
>>>> >
>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-log.dirlayout
>>>> > > > > > > > > > > >>> > > >
>>>> > > > > > > > > > > >>> > > > 2. Partition Metadata file
>>>> > > > > > > > > > > >>> > > > There was some disagreement about the
>>>> inclusion
>>>> > > of
>>>> > > > > the
>>>> > > > > > > > > > partition
>>>> > > > > > > > > > > >>> > metadata
>>>> > > > > > > > > > > >>> > > > file.
>>>> > > > > > > > > > > >>> > > > This file will be needed to persist the
>>>> topic
>>>> > ID,
>>>> > > > > > > especially
>>>> > > > > > > > > > > while
>>>> > > > > > > > > > > >>> we
>>>> > > > > > > > > > > >>> > > still
>>>> > > > > > > > > > > >>> > > > have the old directory structure. Even
>>>> after
>>>> > the
>>>> > > > > > changes,
>>>> > > > > > > > > this
>>>> > > > > > > > > > > >>> file can
>>>> > > > > > > > > > > >>> > > be
>>>> > > > > > > > > > > >>> > > > useful for debugging and recovery.
>>>> > > > > > > > > > > >>> > > > Creating a single mapping file from topic
>>>> names
>>>> > > to
>>>> > > > > > topic
>>>> > > > > > > IDs
>>>> > > > > > > > > > was
>>>> > > > > > > > > > > >>> > > > considered, but ultimately scrapped as it
>>>> would
>>>> > > not
>>>> > > > > be
>>>> > > > > > as
>>>> > > > > > > > > easy
>>>> > > > > > > > > > to
>>>> > > > > > > > > > > >>> > > maintain.
>>>> > > > > > > > > > > >>> > > >
>>>> > > > > > > > > > > >>> > > >
>>>> > > > > > > > > > > >>> > >
>>>> > > > > > > > > > > >>> >
>>>> > > > > > > > > > > >>>
>>>> > > > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > >
>>>> > > > > > >
>>>> > > > > >
>>>> > > > >
>>>> > > >
>>>> > >
>>>> >
>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PartitionMetadatafile
>>>> > > > > > > > > > > >>> > > >
>>>> > > > > > > > > > > >>> > > >
>>>> > > > > > > > > > > >>> > > >
>>>> > > > > > > > > > > >>> > >
>>>> > > > > > > > > > > >>> >
>>>> > > > > > > > > > > >>>
>>>> > > > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > >
>>>> > > > > > >
>>>> > > > > >
>>>> > > > >
>>>> > > >
>>>> > >
>>>> >
>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PersistingTopicIDs
>>>> > > > > > > > > > > >>> > > >
>>>> > > > > > > > > > > >>> > > > 3. Produce Protocols
>>>> > > > > > > > > > > >>> > > > After some further discussion, this
>>>> replacing
>>>> > the
>>>> > > > > topic
>>>> > > > > > > name
>>>> > > > > > > > > > with
>>>> > > > > > > > > > > >>> topic
>>>> > > > > > > > > > > >>> > > ID
>>>> > > > > > > > > > > >>> > > > in these protocols has been added to the
>>>> KIP.
>>>> > > > > > > > > > > >>> > > > This will cut down on the size of the
>>>> protocol.
>>>> > > > Since
>>>> > > > > > > changes
>>>> > > > > > > > > > to
>>>> > > > > > > > > > > >>> fetch
>>>> > > > > > > > > > > >>> > > are
>>>> > > > > > > > > > > >>> > > > included, it does make sense to update
>>>> these
>>>> > > > > protocols.
>>>> > > > > > > > > > > >>> > > >
>>>> > > > > > > > > > > >>> > > >
>>>> > > > > > > > > > > >>> > >
>>>> > > > > > > > > > > >>> >
>>>> > > > > > > > > > > >>>
>>>> > > > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > >
>>>> > > > > > >
>>>> > > > > >
>>>> > > > >
>>>> > > >
>>>> > >
>>>> >
>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-Produce
>>>> > > > > > > > > > > >>> > > >
>>>> > > > > > > > > > > >>> > > > 4. KIP-500 Compatibility
>>>> > > > > > > > > > > >>> > > > After some discussion with those involved
>>>> with
>>>> > > > > KIP-500,
>>>> > > > > > > it
>>>> > > > > > > > > > seems
>>>> > > > > > > > > > > >>> best
>>>> > > > > > > > > > > >>> > to
>>>> > > > > > > > > > > >>> > > > use a sentinel topic ID for the metadata
>>>> topic
>>>> > > that
>>>> > > > > is
>>>> > > > > > > used
>>>> > > > > > > > > > > before
>>>> > > > > > > > > > > >>> the
>>>> > > > > > > > > > > >>> > > > first controller election.
>>>> > > > > > > > > > > >>> > > > We had an issue where this metadata topic
>>>> may
>>>> > not
>>>> > > > be
>>>> > > > > > > assigned
>>>> > > > > > > > > > an
>>>> > > > > > > > > > > ID
>>>> > > > > > > > > > > >>> > > before
>>>> > > > > > > > > > > >>> > > > utilizing the Vote and Fetch protocols.
>>>> It was
>>>> > > > > decided
>>>> > > > > > to
>>>> > > > > > > > > > > reserve a
>>>> > > > > > > > > > > >>> > > unique
>>>> > > > > > > > > > > >>> > > > ID for this topic to be used until the
>>>> > controller
>>>> > > > > could
>>>> > > > > > > give
>>>> > > > > > > > > > the
>>>> > > > > > > > > > > >>> topic
>>>> > > > > > > > > > > >>> > a
>>>> > > > > > > > > > > >>> > > > unique ID.
>>>> > > > > > > > > > > >>> > > > Having the topic keep the sentinel ID
>>>> (not be
>>>> > > > > > reassigned
>>>> > > > > > > to a
>>>> > > > > > > > > > > >>> unique
>>>> > > > > > > > > > > >>> > ID)
>>>> > > > > > > > > > > >>> > > > was considered, but it seemed like a good
>>>> idea
>>>> > to
>>>> > > > > leave
>>>> > > > > > > the
>>>> > > > > > > > > > > option
>>>> > > > > > > > > > > >>> open
>>>> > > > > > > > > > > >>> > > for
>>>> > > > > > > > > > > >>> > > > the metadata topic to have a unique ID in
>>>> cases
>>>> > > > where
>>>> > > > > > it
>>>> > > > > > > > > would
>>>> > > > > > > > > > > >>> need to
>>>> > > > > > > > > > > >>> > be
>>>> > > > > > > > > > > >>> > > > differentiated from other clusters'
>>>> metadata
>>>> > > > topics.
>>>> > > > > > (ex.
>>>> > > > > > > > > > tiered
>>>> > > > > > > > > > > >>> > storage)
>>>> > > > > > > > > > > >>> > > >
>>>> > > > > > > > > > > >>> > > >
>>>> > > > > > > > > > > >>> > >
>>>> > > > > > > > > > > >>> >
>>>> > > > > > > > > > > >>>
>>>> > > > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > >
>>>> > > > > > >
>>>> > > > > >
>>>> > > > >
>>>> > > >
>>>> > >
>>>> >
>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-CompatibilitywithKIP-500
>>>> > > > > > > > > > > >>> > > >
>>>> > > > > > > > > > > >>> > > > I've also split up the KIP into sub-tasks
>>>> on
>>>> > the
>>>> > > > > JIRA.
>>>> > > > > > > > > > Hopefully
>>>> > > > > > > > > > > >>> this
>>>> > > > > > > > > > > >>> > > will
>>>> > > > > > > > > > > >>> > > > provide a better idea about what tasks we
>>>> have,
>>>> > > and
>>>> > > > > > > > > eventually
>>>> > > > > > > > > > > >>> provide
>>>> > > > > > > > > > > >>> > a
>>>> > > > > > > > > > > >>> > > > place to see what's done and what is left.
>>>> > > > > > > > > > > >>> > > > If there is a task I am missing, please
>>>> let me
>>>> > > > know!
>>>> > > > > > > > > > > >>> > > >
>>>> > https://issues.apache.org/jira/browse/KAFKA-8872
>>>> > > > > > > > > > > >>> > > >
>>>> > > > > > > > > > > >>> > > > Of course, these decisions are not set in
>>>> > stone,
>>>> > > > and
>>>> > > > > I
>>>> > > > > > > would
>>>> > > > > > > > > > love
>>>> > > > > > > > > > > >>> to
>>>> > > > > > > > > > > >>> > hear
>>>> > > > > > > > > > > >>> > > > any feedback.
>>>> > > > > > > > > > > >>> > > >
>>>> > > > > > > > > > > >>> > > > Thanks,
>>>> > > > > > > > > > > >>> > > > Justine
>>>> > > > > > > > > > > >>> > > >
>>>> > > > > > > > > > > >>> > > > On Mon, Sep 28, 2020 at 11:38 AM Justine
>>>> > Olshan <
>>>> > > > > > > > > > > >>> jolshan@confluent.io>
>>>> > > > > > > > > > > >>> > > > wrote:
>>>> > > > > > > > > > > >>> > > >
>>>> > > > > > > > > > > >>> > > > > Hello all,
>>>> > > > > > > > > > > >>> > > > >
>>>> > > > > > > > > > > >>> > > > > I just wanted to follow up on this
>>>> > discussion.
>>>> > > > Did
>>>> > > > > we
>>>> > > > > > > come
>>>> > > > > > > > > to
>>>> > > > > > > > > > > an
>>>> > > > > > > > > > > >>> > > > > understanding about the directory
>>>> structure?
>>>> > > > > > > > > > > >>> > > > >
>>>> > > > > > > > > > > >>> > > > > I think the biggest question here is
>>>> what is
>>>> > > > > > > acceptable to
>>>> > > > > > > > > > > leave
>>>> > > > > > > > > > > >>> out
>>>> > > > > > > > > > > >>> > > due
>>>> > > > > > > > > > > >>> > > > > to scope vs. what is considered to be
>>>> too
>>>> > much
>>>> > > > tech
>>>> > > > > > > debt.
>>>> > > > > > > > > > > >>> > > > > This KIP is already pretty large with
>>>> the
>>>> > > number
>>>> > > > of
>>>> > > > > > > > > changes,
>>>> > > > > > > > > > > but
>>>> > > > > > > > > > > >>> it
>>>> > > > > > > > > > > >>> > > also
>>>> > > > > > > > > > > >>> > > > > makes sense to do things correctly, so
>>>> I'd
>>>> > love
>>>> > > > to
>>>> > > > > > hear
>>>> > > > > > > > > > > >>> everyone's
>>>> > > > > > > > > > > >>> > > > thoughts.
>>>> > > > > > > > > > > >>> > > > >
>>>> > > > > > > > > > > >>> > > > > Thanks,
>>>> > > > > > > > > > > >>> > > > > Justine
>>>> > > > > > > > > > > >>> > > > >
>>>> > > > > > > > > > > >>> > > > > On Fri, Sep 25, 2020 at 8:19 AM Lucas
>>>> > > Bradstreet
>>>> > > > <
>>>> > > > > > > > > > > >>> lucas@confluent.io
>>>> > > > > > > > > > > >>> > >
>>>> > > > > > > > > > > >>> > > > > wrote:
>>>> > > > > > > > > > > >>> > > > >
>>>> > > > > > > > > > > >>> > > > >> Hi Ismael,
>>>> > > > > > > > > > > >>> > > > >>
>>>> > > > > > > > > > > >>> > > > >> If you do not store it in a metadata
>>>> file or
>>>> > > in
>>>> > > > > the
>>>> > > > > > > > > > directory
>>>> > > > > > > > > > > >>> > > structure
>>>> > > > > > > > > > > >>> > > > >> would you then
>>>> > > > > > > > > > > >>> > > > >> require the LeaderAndIsrRequest
>>>> following
>>>> > > > startup
>>>> > > > > to
>>>> > > > > > > give
>>>> > > > > > > > > > you
>>>> > > > > > > > > > > >>> some
>>>> > > > > > > > > > > >>> > > > notion
>>>> > > > > > > > > > > >>> > > > >> of
>>>> > > > > > > > > > > >>> > > > >> topic name in memory? We would still
>>>> need
>>>> > this
>>>> > > > > > > information
>>>> > > > > > > > > > for
>>>> > > > > > > > > > > >>> the
>>>> > > > > > > > > > > >>> > > older
>>>> > > > > > > > > > > >>> > > > >> protocols, but
>>>> > > > > > > > > > > >>> > > > >> perhaps this is what's meant by tech
>>>> debt.
>>>> > > > > > > > > > > >>> > > > >>
>>>> > > > > > > > > > > >>> > > > >> Once we're free of the old non-topicID
>>>> > > requests
>>>> > > > > > then I
>>>> > > > > > > > > think
>>>> > > > > > > > > > > you
>>>> > > > > > > > > > > >>> > > > wouldn't
>>>> > > > > > > > > > > >>> > > > >> need to retain the topic name.
>>>> > > > > > > > > > > >>> > > > >> I think the ability to easily look up
>>>> topic
>>>> > > > names
>>>> > > > > > > > > associated
>>>> > > > > > > > > > > >>> with
>>>> > > > > > > > > > > >>> > > > >> partition
>>>> > > > > > > > > > > >>> > > > >> directories would still be missed
>>>> > > > > > > > > > > >>> > > > >> when diagnosing issues, though maybe it
>>>> > > wouldn't
>>>> > > > > be
>>>> > > > > > a
>>>> > > > > > > deal
>>>> > > > > > > > > > > >>> breaker
>>>> > > > > > > > > > > >>> > > with
>>>> > > > > > > > > > > >>> > > > >> the
>>>> > > > > > > > > > > >>> > > > >> right tooling.
>>>> > > > > > > > > > > >>> > > > >>
>>>> > > > > > > > > > > >>> > > > >> Thanks,
>>>> > > > > > > > > > > >>> > > > >>
>>>> > > > > > > > > > > >>> > > > >> Lucas
>>>> > > > > > > > > > > >>> > > > >>
>>>> > > > > > > > > > > >>> > > > >> On Fri, Sep 25, 2020 at 7:55 AM Ismael
>>>> Juma
>>>> > <
>>>> > > > > > > > > > > ismael@juma.me.uk>
>>>> > > > > > > > > > > >>> > > wrote:
>>>> > > > > > > > > > > >>> > > > >>
>>>> > > > > > > > > > > >>> > > > >> > Hi Lucas,
>>>> > > > > > > > > > > >>> > > > >> >
>>>> > > > > > > > > > > >>> > > > >> > Why would you include the name and
>>>> id? I
>>>> > > think
>>>> > > > > > you'd
>>>> > > > > > > > > want
>>>> > > > > > > > > > to
>>>> > > > > > > > > > > >>> > remove
>>>> > > > > > > > > > > >>> > > > the
>>>> > > > > > > > > > > >>> > > > >> > name from the directory name right?
>>>> > Jason's
>>>> > > > > > > suggestion
>>>> > > > > > > > > was
>>>> > > > > > > > > > > >>> that if
>>>> > > > > > > > > > > >>> > > you
>>>> > > > > > > > > > > >>> > > > >> > remove the name from the directory,
>>>> then
>>>> > why
>>>> > > > > would
>>>> > > > > > > you
>>>> > > > > > > > > > need
>>>> > > > > > > > > > > >>> the id
>>>> > > > > > > > > > > >>> > > > name
>>>> > > > > > > > > > > >>> > > > >> > mapping file?
>>>> > > > > > > > > > > >>> > > > >> >
>>>> > > > > > > > > > > >>> > > > >> > Ismael
>>>> > > > > > > > > > > >>> > > > >> >
>>>> > > > > > > > > > > >>> > > > >> > On Thu, Sep 24, 2020 at 4:24 PM Lucas
>>>> > > > > Bradstreet <
>>>> > > > > > > > > > > >>> > > lucas@confluent.io>
>>>> > > > > > > > > > > >>> > > > >> > wrote:
>>>> > > > > > > > > > > >>> > > > >> >
>>>> > > > > > > > > > > >>> > > > >> > > > 2. Part of the usage of the file
>>>> is to
>>>> > > > have
>>>> > > > > > > > > persistent
>>>> > > > > > > > > > > >>> storage
>>>> > > > > > > > > > > >>> > > of
>>>> > > > > > > > > > > >>> > > > >> the
>>>> > > > > > > > > > > >>> > > > >> > > topic
>>>> > > > > > > > > > > >>> > > > >> > > ID and use it to compare with the
>>>> ID
>>>> > > > supplied
>>>> > > > > in
>>>> > > > > > > the
>>>> > > > > > > > > > > >>> > LeaderAndIsr
>>>> > > > > > > > > > > >>> > > > >> > Request.
>>>> > > > > > > > > > > >>> > > > >> > > There is some discussion in the KIP
>>>> > about
>>>> > > > > > changes
>>>> > > > > > > to
>>>> > > > > > > > > the
>>>> > > > > > > > > > > >>> > directory
>>>> > > > > > > > > > > >>> > > > >> > > structure, but I believe directory
>>>> > changes
>>>> > > > > were
>>>> > > > > > > > > > considered
>>>> > > > > > > > > > > >>> to be
>>>> > > > > > > > > > > >>> > > out
>>>> > > > > > > > > > > >>> > > > >> of
>>>> > > > > > > > > > > >>> > > > >> > > scope when the KIP was written.
>>>> > > > > > > > > > > >>> > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> > > Yeah, I was hoping to get a better
>>>> > > > > understanding
>>>> > > > > > > of
>>>> > > > > > > > > why
>>>> > > > > > > > > > it
>>>> > > > > > > > > > > >>> was
>>>> > > > > > > > > > > >>> > > taken
>>>> > > > > > > > > > > >>> > > > >> out
>>>> > > > > > > > > > > >>> > > > >> > of
>>>> > > > > > > > > > > >>> > > > >> > > scope. Perhaps Lucas Bradstreet
>>>> might
>>>> > have
>>>> > > > > more
>>>> > > > > > > > > insight
>>>> > > > > > > > > > > >>> about
>>>> > > > > > > > > > > >>> > the
>>>> > > > > > > > > > > >>> > > > >> > decision.
>>>> > > > > > > > > > > >>> > > > >> > > Basically my point is that we have
>>>> to
>>>> > > create
>>>> > > > > > > > > additional
>>>> > > > > > > > > > > >>> > > > infrastructure
>>>> > > > > > > > > > > >>> > > > >> > here
>>>> > > > > > > > > > > >>> > > > >> > > to support the name/id mapping, so
>>>> I
>>>> > > wanted
>>>> > > > to
>>>> > > > > > > > > > understand
>>>> > > > > > > > > > > >>> if we
>>>> > > > > > > > > > > >>> > > just
>>>> > > > > > > > > > > >>> > > > >> > > consider this a sort of tech debt.
>>>> It
>>>> > > would
>>>> > > > be
>>>> > > > > > > useful
>>>> > > > > > > > > to
>>>> > > > > > > > > > > >>> cover
>>>> > > > > > > > > > > >>> > how
>>>> > > > > > > > > > > >>> > > > we
>>>> > > > > > > > > > > >>> > > > >> > > handle the case when this file gets
>>>> > > > corrupted.
>>>> > > > > > > Seems
>>>> > > > > > > > > > like
>>>> > > > > > > > > > > we
>>>> > > > > > > > > > > >>> > just
>>>> > > > > > > > > > > >>> > > > >> have to
>>>> > > > > > > > > > > >>> > > > >> > > trust that it matches whatever the
>>>> > > > controller
>>>> > > > > > > tells us
>>>> > > > > > > > > > and
>>>> > > > > > > > > > > >>> > rewrite
>>>> > > > > > > > > > > >>> > > > it?
>>>> > > > > > > > > > > >>> > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> > > Hi Jason, Justine,
>>>> > > > > > > > > > > >>> > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> > > My thought process is that we will
>>>> > likely
>>>> > > > need
>>>> > > > > > the
>>>> > > > > > > > > > > metadata
>>>> > > > > > > > > > > >>> file
>>>> > > > > > > > > > > >>> > > > >> whether
>>>> > > > > > > > > > > >>> > > > >> > we
>>>> > > > > > > > > > > >>> > > > >> > > rename the directories or not.
>>>> > > > > > > > > > > >>> > > > >> > > Linux supports filenames of up to
>>>> 255
>>>> > > bytes
>>>> > > > > and
>>>> > > > > > > that
>>>> > > > > > > > > > would
>>>> > > > > > > > > > > >>> not
>>>> > > > > > > > > > > >>> > be
>>>> > > > > > > > > > > >>> > > > >> enough
>>>> > > > > > > > > > > >>> > > > >> > to
>>>> > > > > > > > > > > >>> > > > >> > > support a directory name
>>>> > > > > > > > > > > >>> > > > >> > >  including both the name and topic
>>>> ID.
>>>> > > Given
>>>> > > > > > that
>>>> > > > > > > > > fact,
>>>> > > > > > > > > > it
>>>> > > > > > > > > > > >>> > seemed
>>>> > > > > > > > > > > >>> > > > >> > > reasonable to add the metadata file
>>>> > > > > > > > > > > >>> > > > >> > > and leave the directory rename
>>>> until
>>>> > some
>>>> > > > time
>>>> > > > > > in
>>>> > > > > > > the
>>>> > > > > > > > > > > future
>>>> > > > > > > > > > > >>> > > > (possibly
>>>> > > > > > > > > > > >>> > > > >> > > never).
>>>> > > > > > > > > > > >>> > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> > > If the file does get corrupted I
>>>> think
>>>> > > > you're
>>>> > > > > > > right.
>>>> > > > > > > > > We
>>>> > > > > > > > > > > >>> would
>>>> > > > > > > > > > > >>> > > either
>>>> > > > > > > > > > > >>> > > > >> have
>>>> > > > > > > > > > > >>> > > > >> > > to trust it matches what the
>>>> controller
>>>> > > > tells
>>>> > > > > us
>>>> > > > > > > > > > > >>> > > > >> > >  or error out and let an
>>>> administrator
>>>> > > > resolve
>>>> > > > > > it
>>>> > > > > > > by
>>>> > > > > > > > > > > >>> checking
>>>> > > > > > > > > > > >>> > > across
>>>> > > > > > > > > > > >>> > > > >> > > replicas for consistency.
>>>> > > > > > > > > > > >>> > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> > > Lucas
>>>> > > > > > > > > > > >>> > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> > > On Thu, Sep 24, 2020 at 3:41 PM
>>>> Jason
>>>> > > > > Gustafson
>>>> > > > > > <
>>>> > > > > > > > > > > >>> > > jason@confluent.io
>>>> > > > > > > > > > > >>> > > > >
>>>> > > > > > > > > > > >>> > > > >> > > wrote:
>>>> > > > > > > > > > > >>> > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> > > > Thanks Justine. Responses below:
>>>> > > > > > > > > > > >>> > > > >> > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > 1. Yes, the directory will
>>>> still be
>>>> > > > based
>>>> > > > > on
>>>> > > > > > > the
>>>> > > > > > > > > > topic
>>>> > > > > > > > > > > >>> > names.
>>>> > > > > > > > > > > >>> > > > >> > > > LeaderAndIsrRequest is one of
>>>> the few
>>>> > > > > requests
>>>> > > > > > > that
>>>> > > > > > > > > > will
>>>> > > > > > > > > > > >>> still
>>>> > > > > > > > > > > >>> > > > >> contain
>>>> > > > > > > > > > > >>> > > > >> > > the
>>>> > > > > > > > > > > >>> > > > >> > > > topic name. So I think we have
>>>> this
>>>> > > > covered.
>>>> > > > > > > Sorry
>>>> > > > > > > > > for
>>>> > > > > > > > > > > >>> > > confusion.
>>>> > > > > > > > > > > >>> > > > >> > > >
>>>> > > > > > > > > > > >>> > > > >> > > > Ah, you're right. My eyes passed
>>>> right
>>>> > > > over
>>>> > > > > > the
>>>> > > > > > > > > field.
>>>> > > > > > > > > > > >>> > > > >> > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > 2. Part of the usage of the
>>>> file is
>>>> > to
>>>> > > > > have
>>>> > > > > > > > > > persistent
>>>> > > > > > > > > > > >>> > storage
>>>> > > > > > > > > > > >>> > > > of
>>>> > > > > > > > > > > >>> > > > >> the
>>>> > > > > > > > > > > >>> > > > >> > > > topic
>>>> > > > > > > > > > > >>> > > > >> > > > ID and use it to compare with
>>>> the ID
>>>> > > > > supplied
>>>> > > > > > > in the
>>>> > > > > > > > > > > >>> > > LeaderAndIsr
>>>> > > > > > > > > > > >>> > > > >> > > Request.
>>>> > > > > > > > > > > >>> > > > >> > > > There is some discussion in the
>>>> KIP
>>>> > > about
>>>> > > > > > > changes to
>>>> > > > > > > > > > the
>>>> > > > > > > > > > > >>> > > directory
>>>> > > > > > > > > > > >>> > > > >> > > > structure, but I believe
>>>> directory
>>>> > > changes
>>>> > > > > > were
>>>> > > > > > > > > > > >>> considered to
>>>> > > > > > > > > > > >>> > be
>>>> > > > > > > > > > > >>> > > > >> out of
>>>> > > > > > > > > > > >>> > > > >> > > > scope when the KIP was written.
>>>> > > > > > > > > > > >>> > > > >> > > >
>>>> > > > > > > > > > > >>> > > > >> > > > Yeah, I was hoping to get a
>>>> better
>>>> > > > > > > understanding of
>>>> > > > > > > > > > why
>>>> > > > > > > > > > > >>> it was
>>>> > > > > > > > > > > >>> > > > taken
>>>> > > > > > > > > > > >>> > > > >> > out
>>>> > > > > > > > > > > >>> > > > >> > > of
>>>> > > > > > > > > > > >>> > > > >> > > > scope. Perhaps Lucas Bradstreet
>>>> might
>>>> > > have
>>>> > > > > > more
>>>> > > > > > > > > > insight
>>>> > > > > > > > > > > >>> about
>>>> > > > > > > > > > > >>> > > the
>>>> > > > > > > > > > > >>> > > > >> > > decision.
>>>> > > > > > > > > > > >>> > > > >> > > > Basically my point is that we
>>>> have to
>>>> > > > create
>>>> > > > > > > > > > additional
>>>> > > > > > > > > > > >>> > > > >> infrastructure
>>>> > > > > > > > > > > >>> > > > >> > > here
>>>> > > > > > > > > > > >>> > > > >> > > > to support the name/id mapping,
>>>> so I
>>>> > > > wanted
>>>> > > > > to
>>>> > > > > > > > > > > understand
>>>> > > > > > > > > > > >>> if
>>>> > > > > > > > > > > >>> > we
>>>> > > > > > > > > > > >>> > > > just
>>>> > > > > > > > > > > >>> > > > >> > > > consider this a sort of tech
>>>> debt. It
>>>> > > > would
>>>> > > > > be
>>>> > > > > > > > > useful
>>>> > > > > > > > > > to
>>>> > > > > > > > > > > >>> cover
>>>> > > > > > > > > > > >>> > > how
>>>> > > > > > > > > > > >>> > > > >> we
>>>> > > > > > > > > > > >>> > > > >> > > > handle the case when this file
>>>> gets
>>>> > > > > corrupted.
>>>> > > > > > > Seems
>>>> > > > > > > > > > > like
>>>> > > > > > > > > > > >>> we
>>>> > > > > > > > > > > >>> > > just
>>>> > > > > > > > > > > >>> > > > >> have
>>>> > > > > > > > > > > >>> > > > >> > to
>>>> > > > > > > > > > > >>> > > > >> > > > trust that it matches whatever
>>>> the
>>>> > > > > controller
>>>> > > > > > > tells
>>>> > > > > > > > > us
>>>> > > > > > > > > > > and
>>>> > > > > > > > > > > >>> > > rewrite
>>>> > > > > > > > > > > >>> > > > >> it?
>>>> > > > > > > > > > > >>> > > > >> > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > 3. I think this is a good
>>>> point,
>>>> > but I
>>>> > > > > > again I
>>>> > > > > > > > > > wonder
>>>> > > > > > > > > > > >>> about
>>>> > > > > > > > > > > >>> > > the
>>>> > > > > > > > > > > >>> > > > >> scope
>>>> > > > > > > > > > > >>> > > > >> > > of
>>>> > > > > > > > > > > >>> > > > >> > > > the KIP. Most of the changes
>>>> mentioned
>>>> > > in
>>>> > > > > the
>>>> > > > > > > KIP
>>>> > > > > > > > > are
>>>> > > > > > > > > > > for
>>>> > > > > > > > > > > >>> > > > supporting
>>>> > > > > > > > > > > >>> > > > >> > > topic
>>>> > > > > > > > > > > >>> > > > >> > > > deletion and I believe that is
>>>> why the
>>>> > > > > produce
>>>> > > > > > > > > request
>>>> > > > > > > > > > > was
>>>> > > > > > > > > > > >>> > > listed
>>>> > > > > > > > > > > >>> > > > >> under
>>>> > > > > > > > > > > >>> > > > >> > > > future work.
>>>> > > > > > > > > > > >>> > > > >> > > >
>>>> > > > > > > > > > > >>> > > > >> > > > That's fair. I brought it up
>>>> since
>>>> > > `Fetch`
>>>> > > > > is
>>>> > > > > > > > > already
>>>> > > > > > > > > > > >>> > included.
>>>> > > > > > > > > > > >>> > > If
>>>> > > > > > > > > > > >>> > > > >> > we've
>>>> > > > > > > > > > > >>> > > > >> > > > got `Metadata` and `Fetch`,
>>>> seems we
>>>> > may
>>>> > > > as
>>>> > > > > > > well do
>>>> > > > > > > > > > > >>> `Produce`
>>>> > > > > > > > > > > >>> > > and
>>>> > > > > > > > > > > >>> > > > >> save
>>>> > > > > > > > > > > >>> > > > >> > an
>>>> > > > > > > > > > > >>> > > > >> > > > extra kip. No strong objection
>>>> though
>>>> > if
>>>> > > > you
>>>> > > > > > > want to
>>>> > > > > > > > > > > >>> leave it
>>>> > > > > > > > > > > >>> > > out.
>>>> > > > > > > > > > > >>> > > > >> > > >
>>>> > > > > > > > > > > >>> > > > >> > > >
>>>> > > > > > > > > > > >>> > > > >> > > > -Jason
>>>> > > > > > > > > > > >>> > > > >> > > >
>>>> > > > > > > > > > > >>> > > > >> > > >
>>>> > > > > > > > > > > >>> > > > >> > > > On Thu, Sep 24, 2020 at 3:26 PM
>>>> > Justine
>>>> > > > > > Olshan <
>>>> > > > > > > > > > > >>> > > > >> jolshan@confluent.io>
>>>> > > > > > > > > > > >>> > > > >> > > > wrote:
>>>> > > > > > > > > > > >>> > > > >> > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > Hi Jason,
>>>> > > > > > > > > > > >>> > > > >> > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > Thanks for your comments.
>>>> > > > > > > > > > > >>> > > > >> > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > 1. Yes, the directory will
>>>> still be
>>>> > > > based
>>>> > > > > on
>>>> > > > > > > the
>>>> > > > > > > > > > topic
>>>> > > > > > > > > > > >>> > names.
>>>> > > > > > > > > > > >>> > > > >> > > > > LeaderAndIsrRequest is one of
>>>> the
>>>> > few
>>>> > > > > > requests
>>>> > > > > > > > > that
>>>> > > > > > > > > > > will
>>>> > > > > > > > > > > >>> > still
>>>> > > > > > > > > > > >>> > > > >> > contain
>>>> > > > > > > > > > > >>> > > > >> > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > topic name. So I think we have
>>>> this
>>>> > > > > covered.
>>>> > > > > > > Sorry
>>>> > > > > > > > > > for
>>>> > > > > > > > > > > >>> > > > confusion.
>>>> > > > > > > > > > > >>> > > > >> > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > 2. Part of the usage of the
>>>> file is
>>>> > to
>>>> > > > > have
>>>> > > > > > > > > > persistent
>>>> > > > > > > > > > > >>> > storage
>>>> > > > > > > > > > > >>> > > > of
>>>> > > > > > > > > > > >>> > > > >> the
>>>> > > > > > > > > > > >>> > > > >> > > > topic
>>>> > > > > > > > > > > >>> > > > >> > > > > ID and use it to compare with
>>>> the ID
>>>> > > > > > supplied
>>>> > > > > > > in
>>>> > > > > > > > > the
>>>> > > > > > > > > > > >>> > > > LeaderAndIsr
>>>> > > > > > > > > > > >>> > > > >> > > > Request.
>>>> > > > > > > > > > > >>> > > > >> > > > > There is some discussion in
>>>> the KIP
>>>> > > > about
>>>> > > > > > > changes
>>>> > > > > > > > > to
>>>> > > > > > > > > > > the
>>>> > > > > > > > > > > >>> > > > directory
>>>> > > > > > > > > > > >>> > > > >> > > > > structure, but I believe
>>>> directory
>>>> > > > changes
>>>> > > > > > > were
>>>> > > > > > > > > > > >>> considered
>>>> > > > > > > > > > > >>> > to
>>>> > > > > > > > > > > >>> > > be
>>>> > > > > > > > > > > >>> > > > >> out
>>>> > > > > > > > > > > >>> > > > >> > of
>>>> > > > > > > > > > > >>> > > > >> > > > > scope when the KIP was written.
>>>> > > > > > > > > > > >>> > > > >> > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > 3. I think this is a good
>>>> point,
>>>> > but I
>>>> > > > > > again I
>>>> > > > > > > > > > wonder
>>>> > > > > > > > > > > >>> about
>>>> > > > > > > > > > > >>> > > the
>>>> > > > > > > > > > > >>> > > > >> scope
>>>> > > > > > > > > > > >>> > > > >> > > of
>>>> > > > > > > > > > > >>> > > > >> > > > > the KIP. Most of the changes
>>>> > mentioned
>>>> > > > in
>>>> > > > > > the
>>>> > > > > > > KIP
>>>> > > > > > > > > > are
>>>> > > > > > > > > > > >>> for
>>>> > > > > > > > > > > >>> > > > >> supporting
>>>> > > > > > > > > > > >>> > > > >> > > > topic
>>>> > > > > > > > > > > >>> > > > >> > > > > deletion and I believe that is
>>>> why
>>>> > the
>>>> > > > > > produce
>>>> > > > > > > > > > request
>>>> > > > > > > > > > > >>> was
>>>> > > > > > > > > > > >>> > > > listed
>>>> > > > > > > > > > > >>> > > > >> > under
>>>> > > > > > > > > > > >>> > > > >> > > > > future work.
>>>> > > > > > > > > > > >>> > > > >> > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > 4. This sounds like it might
>>>> be a
>>>> > good
>>>> > > > > > > solution,
>>>> > > > > > > > > > but I
>>>> > > > > > > > > > > >>> will
>>>> > > > > > > > > > > >>> > > need
>>>> > > > > > > > > > > >>> > > > >> to
>>>> > > > > > > > > > > >>> > > > >> > > > discuss
>>>> > > > > > > > > > > >>> > > > >> > > > > more with KIP-500 folks to get
>>>> the
>>>> > > > details
>>>> > > > > > > right.
>>>> > > > > > > > > > > >>> > > > >> > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > Thanks,
>>>> > > > > > > > > > > >>> > > > >> > > > > Justine
>>>> > > > > > > > > > > >>> > > > >> > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > On Thu, Sep 24, 2020 at 12:30
>>>> PM
>>>> > Jason
>>>> > > > > > > Gustafson <
>>>> > > > > > > > > > > >>> > > > >> jason@confluent.io
>>>> > > > > > > > > > > >>> > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > wrote:
>>>> > > > > > > > > > > >>> > > > >> > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > Hi Justine,
>>>> > > > > > > > > > > >>> > > > >> > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > Thanks for picking up this
>>>> work. I
>>>> > > > have
>>>> > > > > a
>>>> > > > > > > few
>>>> > > > > > > > > > > >>> > > > >> questions/comments:
>>>> > > > > > > > > > > >>> > > > >> > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > 1. It sounds like the
>>>> directory
>>>> > > > > structure
>>>> > > > > > is
>>>> > > > > > > > > still
>>>> > > > > > > > > > > >>> going
>>>> > > > > > > > > > > >>> > to
>>>> > > > > > > > > > > >>> > > be
>>>> > > > > > > > > > > >>> > > > >> > based
>>>> > > > > > > > > > > >>> > > > >> > > on
>>>> > > > > > > > > > > >>> > > > >> > > > > > topic names. Do I have that
>>>> right?
>>>> > > One
>>>> > > > > > > > > > complication
>>>> > > > > > > > > > > is
>>>> > > > > > > > > > > >>> > that
>>>> > > > > > > > > > > >>> > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > LeaderAndIsr request does not
>>>> > > include
>>>> > > > > the
>>>> > > > > > > topic
>>>> > > > > > > > > > name
>>>> > > > > > > > > > > >>> any
>>>> > > > > > > > > > > >>> > > > longer.
>>>> > > > > > > > > > > >>> > > > >> > This
>>>> > > > > > > > > > > >>> > > > >> > > > > means
>>>> > > > > > > > > > > >>> > > > >> > > > > > that a replica must wait for
>>>> the
>>>> > > > > > > UpdateMetadata
>>>> > > > > > > > > > > >>> request to
>>>> > > > > > > > > > > >>> > > > >> arrive
>>>> > > > > > > > > > > >>> > > > >> > > with
>>>> > > > > > > > > > > >>> > > > >> > > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > topic name mapping before it
>>>> can
>>>> > > > create
>>>> > > > > > the
>>>> > > > > > > > > > > >>> directory. I
>>>> > > > > > > > > > > >>> > > > wonder
>>>> > > > > > > > > > > >>> > > > >> if
>>>> > > > > > > > > > > >>> > > > >> > it
>>>> > > > > > > > > > > >>> > > > >> > > > > would
>>>> > > > > > > > > > > >>> > > > >> > > > > > be simpler to avoid
>>>> assumptions on
>>>> > > the
>>>> > > > > > > ordering
>>>> > > > > > > > > of
>>>> > > > > > > > > > > >>> > > > >> UpdateMetadata
>>>> > > > > > > > > > > >>> > > > >> > and
>>>> > > > > > > > > > > >>> > > > >> > > > let
>>>> > > > > > > > > > > >>> > > > >> > > > > > LeaderAndIsr include the
>>>> topic
>>>> > name
>>>> > > as
>>>> > > > > > well.
>>>> > > > > > > > > Feels
>>>> > > > > > > > > > > >>> like we
>>>> > > > > > > > > > > >>> > > are
>>>> > > > > > > > > > > >>> > > > >> not
>>>> > > > > > > > > > > >>> > > > >> > > > saving
>>>> > > > > > > > > > > >>> > > > >> > > > > > that much by excluding it.
>>>> > > > > > > > > > > >>> > > > >> > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > 2. On a related note, it
>>>> seems
>>>> > that
>>>> > > > the
>>>> > > > > > > reason
>>>> > > > > > > > > we
>>>> > > > > > > > > > > >>> have the
>>>> > > > > > > > > > > >>> > > > >> > partition
>>>> > > > > > > > > > > >>> > > > >> > > > > > metadata file is because we
>>>> are
>>>> > not
>>>> > > > > > > changing the
>>>> > > > > > > > > > > >>> directory
>>>> > > > > > > > > > > >>> > > > >> > structure.
>>>> > > > > > > > > > > >>> > > > >> > > > We
>>>> > > > > > > > > > > >>> > > > >> > > > > > need it so that we remember
>>>> which
>>>> > > > > > > directories
>>>> > > > > > > > > map
>>>> > > > > > > > > > to
>>>> > > > > > > > > > > >>> which
>>>> > > > > > > > > > > >>> > > > topic
>>>> > > > > > > > > > > >>> > > > >> > id.
>>>> > > > > > > > > > > >>> > > > >> > > I
>>>> > > > > > > > > > > >>> > > > >> > > > > > think it would be useful to
>>>> add
>>>> > some
>>>> > > > > > detail
>>>> > > > > > > > > about
>>>> > > > > > > > > > > why
>>>> > > > > > > > > > > >>> > > changing
>>>> > > > > > > > > > > >>> > > > >> the
>>>> > > > > > > > > > > >>> > > > >> > > > > > directory layout was
>>>> rejected.
>>>> > > > Basically
>>>> > > > > > > trying
>>>> > > > > > > > > to
>>>> > > > > > > > > > > >>> > > understand
>>>> > > > > > > > > > > >>> > > > >> how
>>>> > > > > > > > > > > >>> > > > >> > > much
>>>> > > > > > > > > > > >>> > > > >> > > > > > effort we are saving by
>>>> skipping
>>>> > > this
>>>> > > > > step
>>>> > > > > > > if we
>>>> > > > > > > > > > > have
>>>> > > > > > > > > > > >>> to
>>>> > > > > > > > > > > >>> > > > >> implement
>>>> > > > > > > > > > > >>> > > > >> > > this
>>>> > > > > > > > > > > >>> > > > >> > > > > > additional bookkeeping. One
>>>> > concern
>>>> > > I
>>>> > > > > have
>>>> > > > > > > for
>>>> > > > > > > > > > > >>> example is
>>>> > > > > > > > > > > >>> > > that
>>>> > > > > > > > > > > >>> > > > >> the
>>>> > > > > > > > > > > >>> > > > >> > > > > > partition metadata file gets
>>>> > corrupt
>>>> > > > and
>>>> > > > > > > then
>>>> > > > > > > > > all
>>>> > > > > > > > > > > >>> bets are
>>>> > > > > > > > > > > >>> > > off
>>>> > > > > > > > > > > >>> > > > >> as
>>>> > > > > > > > > > > >>> > > > >> > far
>>>> > > > > > > > > > > >>> > > > >> > > > as
>>>> > > > > > > > > > > >>> > > > >> > > > > > topic consistency.
>>>> > > > > > > > > > > >>> > > > >> > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > 3. Is it worth adding
>>>> support now
>>>> > > for
>>>> > > > > > topic
>>>> > > > > > > ids
>>>> > > > > > > > > in
>>>> > > > > > > > > > > the
>>>> > > > > > > > > > > >>> > > > `Produce`
>>>> > > > > > > > > > > >>> > > > >> > > > request
>>>> > > > > > > > > > > >>> > > > >> > > > > > now? Overhead is mentioned
>>>> as one
>>>> > of
>>>> > > > the
>>>> > > > > > > > > > motivations
>>>> > > > > > > > > > > >>> and
>>>> > > > > > > > > > > >>> > the
>>>> > > > > > > > > > > >>> > > > >> best
>>>> > > > > > > > > > > >>> > > > >> > > APIs
>>>> > > > > > > > > > > >>> > > > >> > > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > get savings from are
>>>> `Produce` and
>>>> > > > > > `Fetch`.
>>>> > > > > > > > > > > >>> > > > >> > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > 4. When it comes to
>>>> bootstrapping
>>>> > > the
>>>> > > > > > > metadata
>>>> > > > > > > > > > topic
>>>> > > > > > > > > > > >>> with
>>>> > > > > > > > > > > >>> > > > >> respect
>>>> > > > > > > > > > > >>> > > > >> > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > KIP-500, one suggestion
>>>> would be
>>>> > to
>>>> > > > > > reserve
>>>> > > > > > > a
>>>> > > > > > > > > > > sentinel
>>>> > > > > > > > > > > >>> > topic
>>>> > > > > > > > > > > >>> > > > ID
>>>> > > > > > > > > > > >>> > > > >> > which
>>>> > > > > > > > > > > >>> > > > >> > > > can
>>>> > > > > > > > > > > >>> > > > >> > > > > > be used initially by new
>>>> brokers.
>>>> > > When
>>>> > > > > the
>>>> > > > > > > first
>>>> > > > > > > > > > > >>> > controller
>>>> > > > > > > > > > > >>> > > is
>>>> > > > > > > > > > > >>> > > > >> > > elected,
>>>> > > > > > > > > > > >>> > > > >> > > > > it
>>>> > > > > > > > > > > >>> > > > >> > > > > > can assign a topicId to the
>>>> > metadata
>>>> > > > > > topic.
>>>> > > > > > > This
>>>> > > > > > > > > > is
>>>> > > > > > > > > > > a
>>>> > > > > > > > > > > >>> bit
>>>> > > > > > > > > > > >>> > > > >> similar
>>>> > > > > > > > > > > >>> > > > >> > to
>>>> > > > > > > > > > > >>> > > > >> > > > how
>>>> > > > > > > > > > > >>> > > > >> > > > > we
>>>> > > > > > > > > > > >>> > > > >> > > > > > were planning to generate the
>>>> > > > clusterId.
>>>> > > > > > > > > > > >>> > > > >> > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > Thanks,
>>>> > > > > > > > > > > >>> > > > >> > > > > > Jason
>>>> > > > > > > > > > > >>> > > > >> > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > On Thu, Sep 24, 2020 at
>>>> 11:10 AM
>>>> > Jun
>>>> > > > > Rao <
>>>> > > > > > > > > > > >>> > jun@confluent.io>
>>>> > > > > > > > > > > >>> > > > >> wrote:
>>>> > > > > > > > > > > >>> > > > >> > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > Hi, Justine,
>>>> > > > > > > > > > > >>> > > > >> > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > Thanks for the updated
>>>> KIP. A
>>>> > few
>>>> > > > more
>>>> > > > > > > > > comments
>>>> > > > > > > > > > > >>> below.
>>>> > > > > > > > > > > >>> > > > >> > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > 30.  {"name": "id", "type":
>>>> > > > "string",
>>>> > > > > > > "doc":
>>>> > > > > > > > > > > >>> "version
>>>> > > > > > > > > > > >>> > > id"}}:
>>>> > > > > > > > > > > >>> > > > >> The
>>>> > > > > > > > > > > >>> > > > >> > > doc
>>>> > > > > > > > > > > >>> > > > >> > > > > > should
>>>> > > > > > > > > > > >>> > > > >> > > > > > > say UUID.
>>>> > > > > > > > > > > >>> > > > >> > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > 31. LeaderAndIsrResponse
>>>> v5 and
>>>> > > > > > > > > > > StopReplicaResponse
>>>> > > > > > > > > > > >>> v4 :
>>>> > > > > > > > > > > >>> > > It
>>>> > > > > > > > > > > >>> > > > >> seems
>>>> > > > > > > > > > > >>> > > > >> > > > there
>>>> > > > > > > > > > > >>> > > > >> > > > > > is
>>>> > > > > > > > > > > >>> > > > >> > > > > > > no need to add topic_id at
>>>> > > > partitions
>>>> > > > > > > level.
>>>> > > > > > > > > > > >>> > > > >> > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > 32. Regarding partition
>>>> metadata
>>>> > > > file.
>>>> > > > > > > Perhaps
>>>> > > > > > > > > > the
>>>> > > > > > > > > > > >>> key
>>>> > > > > > > > > > > >>> > can
>>>> > > > > > > > > > > >>> > > > be
>>>> > > > > > > > > > > >>> > > > >> a
>>>> > > > > > > > > > > >>> > > > >> > > > single
>>>> > > > > > > > > > > >>> > > > >> > > > > > > word, sth like the
>>>> following.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > version: 0
>>>> > > > > > > > > > > >>> > > > >> > > > > > > topic_id:
>>>> > > > > > > 46bdb63f-9e8d-4a38-bf7b-ee4eb2a794e4
>>>> > > > > > > > > > > >>> > > > >> > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > 33. Another tricky thing
>>>> that I
>>>> > > > > realized
>>>> > > > > > > is
>>>> > > > > > > > > how
>>>> > > > > > > > > > to
>>>> > > > > > > > > > > >>> > support
>>>> > > > > > > > > > > >>> > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > metadata
>>>> > > > > > > > > > > >>> > > > >> > > > > > > topic introduced in
>>>> KIP-595.
>>>> > It's
>>>> > > a
>>>> > > > > bit
>>>> > > > > > > tricky
>>>> > > > > > > > > > to
>>>> > > > > > > > > > > >>> > assign a
>>>> > > > > > > > > > > >>> > > > >> UUID
>>>> > > > > > > > > > > >>> > > > >> > to
>>>> > > > > > > > > > > >>> > > > >> > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > metadata topic since we
>>>> have a
>>>> > > > > > > chicken-and-egg
>>>> > > > > > > > > > > >>> problem.
>>>> > > > > > > > > > > >>> > > The
>>>> > > > > > > > > > > >>> > > > >> > > > controller
>>>> > > > > > > > > > > >>> > > > >> > > > > > > needs to persist the UUID
>>>> in the
>>>> > > > > > metadata
>>>> > > > > > > > > topic
>>>> > > > > > > > > > in
>>>> > > > > > > > > > > >>> order
>>>> > > > > > > > > > > >>> > > to
>>>> > > > > > > > > > > >>> > > > >> > assign
>>>> > > > > > > > > > > >>> > > > >> > > > one
>>>> > > > > > > > > > > >>> > > > >> > > > > > > successfully, but the
>>>> metadata
>>>> > > topic
>>>> > > > > is
>>>> > > > > > > needed
>>>> > > > > > > > > > to
>>>> > > > > > > > > > > >>> elect
>>>> > > > > > > > > > > >>> > a
>>>> > > > > > > > > > > >>> > > > >> > > controller
>>>> > > > > > > > > > > >>> > > > >> > > > > > > (KIP-631). So, this
>>>> probably
>>>> > > needs a
>>>> > > > > bit
>>>> > > > > > > more
>>>> > > > > > > > > > > >>> thought.
>>>> > > > > > > > > > > >>> > > > >> > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > Jun
>>>> > > > > > > > > > > >>> > > > >> > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > On Thu, Sep 24, 2020 at
>>>> 3:04 AM
>>>> > > > Ismael
>>>> > > > > > > Juma <
>>>> > > > > > > > > > > >>> > > > >> ismael@juma.me.uk>
>>>> > > > > > > > > > > >>> > > > >> > > > wrote:
>>>> > > > > > > > > > > >>> > > > >> > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > Also, can we provide more
>>>> > > details
>>>> > > > on
>>>> > > > > > > how the
>>>> > > > > > > > > > > >>> Partition
>>>> > > > > > > > > > > >>> > > > >> Metadata
>>>> > > > > > > > > > > >>> > > > >> > > > file
>>>> > > > > > > > > > > >>> > > > >> > > > > > will
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > be used?
>>>> > > > > > > > > > > >>> > > > >> > > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > Ismael
>>>> > > > > > > > > > > >>> > > > >> > > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > On Thu, Sep 24, 2020 at
>>>> 3:01
>>>> > AM
>>>> > > > > Ismael
>>>> > > > > > > Juma
>>>> > > > > > > > > <
>>>> > > > > > > > > > > >>> > > > >> ismael@juma.me.uk
>>>> > > > > > > > > > > >>> > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > wrote:
>>>> > > > > > > > > > > >>> > > > >> > > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > > Hi Justine,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > > I think we need to
>>>> update
>>>> > the
>>>> > > > > > > "Rejected
>>>> > > > > > > > > > > >>> > Alternatives"
>>>> > > > > > > > > > > >>> > > > >> section
>>>> > > > > > > > > > > >>> > > > >> > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > take
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > into
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > > account that the
>>>> proposal
>>>> > now
>>>> > > > > > removes
>>>> > > > > > > the
>>>> > > > > > > > > > > topic
>>>> > > > > > > > > > > >>> name
>>>> > > > > > > > > > > >>> > > > from
>>>> > > > > > > > > > > >>> > > > >> the
>>>> > > > > > > > > > > >>> > > > >> > > > fetch
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > > request. Also, if we
>>>> are
>>>> > > > removing
>>>> > > > > it
>>>> > > > > > > from
>>>> > > > > > > > > > the
>>>> > > > > > > > > > > >>> Fetch
>>>> > > > > > > > > > > >>> > > > >> request,
>>>> > > > > > > > > > > >>> > > > >> > > does
>>>> > > > > > > > > > > >>> > > > >> > > > > it
>>>> > > > > > > > > > > >>> > > > >> > > > > > > make
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > > sense not to remove it
>>>> from
>>>> > > > > similar
>>>> > > > > > > > > requests
>>>> > > > > > > > > > > >>> like
>>>> > > > > > > > > > > >>> > > > >> > > > > ListOffsetRequest?
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > > Ismael
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > > On Thu, Sep 24, 2020
>>>> at 2:46
>>>> > > AM
>>>> > > > > > David
>>>> > > > > > > > > Jacot
>>>> > > > > > > > > > <
>>>> > > > > > > > > > > >>> > > > >> > > djacot@confluent.io
>>>> > > > > > > > > > > >>> > > > >> > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > wrote:
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> Hi Justine,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> Thanks for the KIP. I
>>>> > finally
>>>> > > > had
>>>> > > > > > > time to
>>>> > > > > > > > > > > read
>>>> > > > > > > > > > > >>> it
>>>> > > > > > > > > > > >>> > :).
>>>> > > > > > > > > > > >>> > > > It
>>>> > > > > > > > > > > >>> > > > >> is
>>>> > > > > > > > > > > >>> > > > >> > a
>>>> > > > > > > > > > > >>> > > > >> > > > > great
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> improvement.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> I have a few
>>>> > > > comments/questions:
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> 1. It seems that the
>>>> schema
>>>> > > of
>>>> > > > > the
>>>> > > > > > > > > > > >>> > StopReplicaRequest
>>>> > > > > > > > > > > >>> > > > is
>>>> > > > > > > > > > > >>> > > > >> > > > slightly
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> outdated.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> We
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> did some changes as
>>>> part of
>>>> > > > > > KIP-570.
>>>> > > > > > > V3
>>>> > > > > > > > > is
>>>> > > > > > > > > > > >>> already
>>>> > > > > > > > > > > >>> > > > >> organized
>>>> > > > > > > > > > > >>> > > > >> > > by
>>>> > > > > > > > > > > >>> > > > >> > > > > > > topics.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> 2. I just want to
>>>> make sure
>>>> > > > that
>>>> > > > > I
>>>> > > > > > > > > > understand
>>>> > > > > > > > > > > >>> the
>>>> > > > > > > > > > > >>> > > > >> > > reconciliation
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> logic correctly. When
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> an "INCREMENTAL"
>>>> > LeaderAndIsr
>>>> > > > > > > Request is
>>>> > > > > > > > > > > >>> received,
>>>> > > > > > > > > > > >>> > > the
>>>> > > > > > > > > > > >>> > > > >> > broker
>>>> > > > > > > > > > > >>> > > > >> > > > will
>>>> > > > > > > > > > > >>> > > > >> > > > > > > also
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> reconcile
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> when the local uuid
>>>> does
>>>> > not
>>>> > > > > match
>>>> > > > > > > the
>>>> > > > > > > > > uuid
>>>> > > > > > > > > > > in
>>>> > > > > > > > > > > >>> the
>>>> > > > > > > > > > > >>> > > > >> request,
>>>> > > > > > > > > > > >>> > > > >> > > > right?
>>>> > > > > > > > > > > >>> > > > >> > > > > > In
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > this
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> case, the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> local log is staged
>>>> for
>>>> > > > deletion.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> 3. In the
>>>> documentation of
>>>> > > the
>>>> > > > `
>>>> > > > > > > > > > > >>> > > > >> delete.stale.topic.delay.ms
>>>> > > > > > > > > > > >>> > > > >> > `
>>>> > > > > > > > > > > >>> > > > >> > > > > > config,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > it
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> says "When a
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> FULL
>>>> LeaderAndIsrRequest is
>>>> > > > > > > received..."
>>>> > > > > > > > > > but
>>>> > > > > > > > > > > I
>>>> > > > > > > > > > > >>> > > suppose
>>>> > > > > > > > > > > >>> > > > it
>>>> > > > > > > > > > > >>> > > > >> > > > applies
>>>> > > > > > > > > > > >>> > > > >> > > > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > both
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> types.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> Best,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> David
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> On Thu, Sep 24, 2020
>>>> at
>>>> > 1:40
>>>> > > AM
>>>> > > > > > > Justine
>>>> > > > > > > > > > > Olshan
>>>> > > > > > > > > > > >>> <
>>>> > > > > > > > > > > >>> > > > >> > > > > > jolshan@confluent.io>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> wrote:
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > Hi Jun,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > Thanks for the
>>>> comments.
>>>> > I
>>>> > > > > > > apologize
>>>> > > > > > > > > for
>>>> > > > > > > > > > > >>> some of
>>>> > > > > > > > > > > >>> > > the
>>>> > > > > > > > > > > >>> > > > >> typos
>>>> > > > > > > > > > > >>> > > > >> > > and
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> confusion.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > I’ve updated the
>>>> KIP to
>>>> > fix
>>>> > > > > some
>>>> > > > > > > of the
>>>> > > > > > > > > > > >>> issues
>>>> > > > > > > > > > > >>> > you
>>>> > > > > > > > > > > >>> > > > >> > > mentioned.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > 20.2 I’ve changed
>>>> the
>>>> > type
>>>> > > to
>>>> > > > > > > String.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > 20.1/3 I’ve updated
>>>> the
>>>> > > > > > TopicZNode
>>>> > > > > > > to
>>>> > > > > > > > > fix
>>>> > > > > > > > > > > >>> > > formatting
>>>> > > > > > > > > > > >>> > > > >> and
>>>> > > > > > > > > > > >>> > > > >> > > > reflect
>>>> > > > > > > > > > > >>> > > > >> > > > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> latest
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > version before this
>>>> > change.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > 21. You are correct
>>>> and
>>>> > > I’ve
>>>> > > > > > > removed
>>>> > > > > > > > > this
>>>> > > > > > > > > > > >>> line.
>>>> > > > > > > > > > > >>> > > I’ve
>>>> > > > > > > > > > > >>> > > > >> also
>>>> > > > > > > > > > > >>> > > > >> > > > added
>>>> > > > > > > > > > > >>> > > > >> > > > > a
>>>> > > > > > > > > > > >>> > > > >> > > > > > > line
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > mentioning an IBP
>>>> bump is
>>>> > > > > > > necessary for
>>>> > > > > > > > > > > >>> migration
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > 22. I think the
>>>> wording
>>>> > was
>>>> > > > > > > unclear but
>>>> > > > > > > > > > > your
>>>> > > > > > > > > > > >>> > > summary
>>>> > > > > > > > > > > >>> > > > is
>>>> > > > > > > > > > > >>> > > > >> > what
>>>> > > > > > > > > > > >>> > > > >> > > > was
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> intended
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > by this line. I’ve
>>>> > updated
>>>> > > to
>>>> > > > > > make
>>>> > > > > > > this
>>>> > > > > > > > > > > point
>>>> > > > > > > > > > > >>> > more
>>>> > > > > > > > > > > >>> > > > >> clear.
>>>> > > > > > > > > > > >>> > > > >> > > > > “Remove
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> deleted
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > topics from
>>>> replicas by
>>>> > > > sending
>>>> > > > > > > > > > > >>> > StopReplicaRequest
>>>> > > > > > > > > > > >>> > > V3
>>>> > > > > > > > > > > >>> > > > >> > before
>>>> > > > > > > > > > > >>> > > > >> > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > IBP
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> bump
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > using the old
>>>> logic, and
>>>> > > > using
>>>> > > > > V4
>>>> > > > > > > and
>>>> > > > > > > > > the
>>>> > > > > > > > > > > new
>>>> > > > > > > > > > > >>> > logic
>>>> > > > > > > > > > > >>> > > > >> with
>>>> > > > > > > > > > > >>> > > > >> > > topic
>>>> > > > > > > > > > > >>> > > > >> > > > > IDs
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > after
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > the IBP bump.”
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > 23. I’ve removed the
>>>> > > > > unspecified
>>>> > > > > > > type
>>>> > > > > > > > > > from
>>>> > > > > > > > > > > >>> the
>>>> > > > > > > > > > > >>> > KIP
>>>> > > > > > > > > > > >>> > > > and
>>>> > > > > > > > > > > >>> > > > >> > > mention
>>>> > > > > > > > > > > >>> > > > >> > > > > > that
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > IBP
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > will be used to
>>>> handle
>>>> > this
>>>> > > > > > > request.
>>>> > > > > > > > > “IBP
>>>> > > > > > > > > > > >>> will be
>>>> > > > > > > > > > > >>> > > > used
>>>> > > > > > > > > > > >>> > > > >> to
>>>> > > > > > > > > > > >>> > > > >> > > > > > determine
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> whether
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > this new form of the
>>>> > > request
>>>> > > > > will
>>>> > > > > > > be
>>>> > > > > > > > > > used.
>>>> > > > > > > > > > > >>> For
>>>> > > > > > > > > > > >>> > > older
>>>> > > > > > > > > > > >>> > > > >> > > requests,
>>>> > > > > > > > > > > >>> > > > >> > > > > we
>>>> > > > > > > > > > > >>> > > > >> > > > > > > will
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > ignore this field
>>>> and
>>>> > > default
>>>> > > > > to
>>>> > > > > > > > > previous
>>>> > > > > > > > > > > >>> > > behavior.”
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > 24. I’ve fixed this
>>>> typo.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > 25. I've added a
>>>> topics
>>>> > at
>>>> > > a
>>>> > > > > > higher
>>>> > > > > > > > > level
>>>> > > > > > > > > > > for
>>>> > > > > > > > > > > >>> > > > >> > > > > LeaderAndIsrResponse
>>>> > > > > > > > > > > >>> > > > >> > > > > > > v5,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > StopReplicaResponse
>>>> v4.
>>>> > > I've
>>>> > > > > also
>>>> > > > > > > > > changed
>>>> > > > > > > > > > > >>> > > > >> > StopReplicaRequest
>>>> > > > > > > > > > > >>> > > > >> > > > v4
>>>> > > > > > > > > > > >>> > > > >> > > > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > have
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > topics at a higher
>>>> level.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > 26. I’ve updated
>>>> > > > > > > > > > > >>> forgotten_topics_data--added the
>>>> > > > > > > > > > > >>> > > > >> topic ID
>>>> > > > > > > > > > > >>> > > > >> > > and
>>>> > > > > > > > > > > >>> > > > >> > > > > > > removed
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > topic name
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > 27. I’ve decided on
>>>> plain
>>>> > > > text,
>>>> > > > > > and
>>>> > > > > > > > > I’ve
>>>> > > > > > > > > > > >>> added an
>>>> > > > > > > > > > > >>> > > > >> example.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > 28. I’ve added this
>>>> idea
>>>> > to
>>>> > > > > > future
>>>> > > > > > > > > work.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > Thanks again for
>>>> taking a
>>>> > > > look,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > Justine
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > On Wed, Sep 23,
>>>> 2020 at
>>>> > > 10:28
>>>> > > > > AM
>>>> > > > > > > Jun
>>>> > > > > > > > > Rao
>>>> > > > > > > > > > <
>>>> > > > > > > > > > > >>> > > > >> > jun@confluent.io>
>>>> > > > > > > > > > > >>> > > > >> > > > > > wrote:
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > Hi, Justine,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > Thanks for the
>>>> > response.
>>>> > > > Made
>>>> > > > > > > another
>>>> > > > > > > > > > > >>> pass. A
>>>> > > > > > > > > > > >>> > few
>>>> > > > > > > > > > > >>> > > > >> more
>>>> > > > > > > > > > > >>> > > > >> > > > > comments
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > below.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 20. znode schema:
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 20.1 It seems that
>>>> > > {"name":
>>>> > > > > > > > > "version",
>>>> > > > > > > > > > > >>> "type":
>>>> > > > > > > > > > > >>> > > > "int",
>>>> > > > > > > > > > > >>> > > > >> > > "id":
>>>> > > > > > > > > > > >>> > > > >> > > > > > > "UUID",
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > "doc":
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > "version id"}
>>>> should be
>>>> > > > > > {"name":
>>>> > > > > > > > > > > "version",
>>>> > > > > > > > > > > >>> > > "type":
>>>> > > > > > > > > > > >>> > > > >> > > "int"},
>>>> > > > > > > > > > > >>> > > > >> > > > > > > {"name":
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > "id",
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > "type": "UUID",
>>>> "doc":
>>>> > > > > "version
>>>> > > > > > > id"}.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 20.2 The znode
>>>> format
>>>> > is
>>>> > > > JSON
>>>> > > > > > > which
>>>> > > > > > > > > > > doesn't
>>>> > > > > > > > > > > >>> > have
>>>> > > > > > > > > > > >>> > > > UUID
>>>> > > > > > > > > > > >>> > > > >> > > type.
>>>> > > > > > > > > > > >>> > > > >> > > > So
>>>> > > > > > > > > > > >>> > > > >> > > > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> type
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > probably should be
>>>> > > string?
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 20.3 Also, the
>>>> existing
>>>> > > > > format
>>>> > > > > > > used
>>>> > > > > > > > > > seems
>>>> > > > > > > > > > > >>> > > outdated.
>>>> > > > > > > > > > > >>> > > > >> It
>>>> > > > > > > > > > > >>> > > > >> > > > should
>>>> > > > > > > > > > > >>> > > > >> > > > > > have
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > following format.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>>> > >  Json.encodeAsBytes(Map(
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >       "version"
>>>> -> 2,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>>>  "partitions" ->
>>>> > > > > > > > > > > >>> > > replicaAssignmentJson.asJava,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>>>  "adding_replicas"
>>>> > > ->
>>>> > > > > > > > > > > >>> > > > >> > > > >
>>>> addingReplicasAssignmentJson.asJava,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>>> >  "removing_replicas"
>>>> > > > ->
>>>> > > > > > > > > > > >>> > > > >> > > > > >
>>>> > > removingReplicasAssignmentJson.asJava
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >     ).asJava)
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >   }
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 21. Migration:
>>>> The KIP
>>>> > > says
>>>> > > > > > "The
>>>> > > > > > > > > > > migration
>>>> > > > > > > > > > > >>> > > process
>>>> > > > > > > > > > > >>> > > > >> can
>>>> > > > > > > > > > > >>> > > > >> > > take
>>>> > > > > > > > > > > >>> > > > >> > > > > > place
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> without
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > an inter-broker
>>>> > protocol
>>>> > > > > bump,
>>>> > > > > > > as the
>>>> > > > > > > > > > > >>> format
>>>> > > > > > > > > > > >>> > > stored
>>>> > > > > > > > > > > >>> > > > >> in
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>>> /brokers/topics/[topic]
>>>> > > > will
>>>> > > > > be
>>>> > > > > > > > > > > compatible
>>>> > > > > > > > > > > >>> with
>>>> > > > > > > > > > > >>> > > > older
>>>> > > > > > > > > > > >>> > > > >> > > broker
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> versions."
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > However, since we
>>>> are
>>>> > > > bumping
>>>> > > > > > up
>>>> > > > > > > the
>>>> > > > > > > > > > > >>> version of
>>>> > > > > > > > > > > >>> > > > >> > > inter-broker
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> requests, it
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > seems that we
>>>> need to
>>>> > use
>>>> > > > IBP
>>>> > > > > > for
>>>> > > > > > > > > > > >>> migration.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 22. The KIP says
>>>> > "Remove
>>>> > > > > > deleted
>>>> > > > > > > > > topics
>>>> > > > > > > > > > > >>> from
>>>> > > > > > > > > > > >>> > > > >> replicas by
>>>> > > > > > > > > > > >>> > > > >> > > > > sending
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>>> StopReplicaRequest V3
>>>> > for
>>>> > > > any
>>>> > > > > > > topics
>>>> > > > > > > > > > > which
>>>> > > > > > > > > > > >>> do
>>>> > > > > > > > > > > >>> > not
>>>> > > > > > > > > > > >>> > > > >> > contain
>>>> > > > > > > > > > > >>> > > > >> > > a
>>>> > > > > > > > > > > >>> > > > >> > > > > > topic
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > ID,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> and
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > V4 for any topics
>>>> which
>>>> > > do
>>>> > > > > > > contain a
>>>> > > > > > > > > > > topic
>>>> > > > > > > > > > > >>> > ID.".
>>>> > > > > > > > > > > >>> > > > >> > However,
>>>> > > > > > > > > > > >>> > > > >> > > if
>>>> > > > > > > > > > > >>> > > > >> > > > > we
>>>> > > > > > > > > > > >>> > > > >> > > > > > > use
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> IBP,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > it
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > seems that the
>>>> > controller
>>>> > > > > will
>>>> > > > > > > either
>>>> > > > > > > > > > > send
>>>> > > > > > > > > > > >>> > > > >> > > > StopReplicaRequest
>>>> > > > > > > > > > > >>> > > > >> > > > > V3
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > or
>>>> StopReplicaRequest
>>>> > V4,
>>>> > > > but
>>>> > > > > > > never
>>>> > > > > > > > > > mixed
>>>> > > > > > > > > > > >>> V3
>>>> > > > > > > > > > > >>> > and
>>>> > > > > > > > > > > >>> > > V4
>>>> > > > > > > > > > > >>> > > > >> for
>>>> > > > > > > > > > > >>> > > > >> > > > > > different
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> topics.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > Basically, before
>>>> the
>>>> > IBP
>>>> > > > > bump,
>>>> > > > > > > V3
>>>> > > > > > > > > will
>>>> > > > > > > > > > > be
>>>> > > > > > > > > > > >>> > used.
>>>> > > > > > > > > > > >>> > > > >> After
>>>> > > > > > > > > > > >>> > > > >> > the
>>>> > > > > > > > > > > >>> > > > >> > > > IBP
>>>> > > > > > > > > > > >>> > > > >> > > > > > > bump,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > topicId will be
>>>> created
>>>> > > and
>>>> > > > > V4
>>>> > > > > > > will
>>>> > > > > > > > > be
>>>> > > > > > > > > > > >>> used.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 23. Given that we
>>>> > depend
>>>> > > on
>>>> > > > > > IBP,
>>>> > > > > > > do
>>>> > > > > > > > > we
>>>> > > > > > > > > > > >>> still
>>>> > > > > > > > > > > >>> > need
>>>> > > > > > > > > > > >>> > > > "0
>>>> > > > > > > > > > > >>> > > > >> > > > > > UNSPECIFIED"
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > in LeaderAndIsr?
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 24.
>>>> > LeaderAndIsrResponse
>>>> > > > v5 :
>>>> > > > > > It
>>>> > > > > > > > > still
>>>> > > > > > > > > > > has
>>>> > > > > > > > > > > >>> the
>>>> > > > > > > > > > > >>> > > > topic
>>>> > > > > > > > > > > >>> > > > >> > > field.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 25.
>>>> > LeaderAndIsrResponse
>>>> > > > v5,
>>>> > > > > > > > > > > >>> > StopReplicaResponse
>>>> > > > > > > > > > > >>> > > > v4:
>>>> > > > > > > > > > > >>> > > > >> > Could
>>>> > > > > > > > > > > >>> > > > >> > > > we
>>>> > > > > > > > > > > >>> > > > >> > > > > > use
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > this
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > opportunity to
>>>> organize
>>>> > > the
>>>> > > > > > > response
>>>> > > > > > > > > > in 2
>>>> > > > > > > > > > > >>> > levels,
>>>> > > > > > > > > > > >>> > > > >> first
>>>> > > > > > > > > > > >>> > > > >> > by
>>>> > > > > > > > > > > >>> > > > >> > > > > > topic,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> then by
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > partition, as most
>>>> > other
>>>> > > > > > > > > > > >>> requests/responses?
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 26. FetchRequest
>>>> v13 :
>>>> > > > Should
>>>> > > > > > > > > > > >>> > > forgotten_topics_data
>>>> > > > > > > > > > > >>> > > > >> use
>>>> > > > > > > > > > > >>> > > > >> > > > > topicId
>>>> > > > > > > > > > > >>> > > > >> > > > > > > too?
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 27. "This file can
>>>> > either
>>>> > > > be
>>>> > > > > > > plain
>>>> > > > > > > > > text
>>>> > > > > > > > > > > >>> > > (key/value
>>>> > > > > > > > > > > >>> > > > >> > pairs)
>>>> > > > > > > > > > > >>> > > > >> > > or
>>>> > > > > > > > > > > >>> > > > >> > > > > > > JSON."
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> Have
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > we
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > decided which one
>>>> to
>>>> > use?
>>>> > > > > Also,
>>>> > > > > > > it
>>>> > > > > > > > > > would
>>>> > > > > > > > > > > be
>>>> > > > > > > > > > > >>> > > helpful
>>>> > > > > > > > > > > >>> > > > >> to
>>>> > > > > > > > > > > >>> > > > >> > > > provide
>>>> > > > > > > > > > > >>> > > > >> > > > > > an
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > example.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 28. Future
>>>> improvement:
>>>> > > > > Another
>>>> > > > > > > > > future
>>>> > > > > > > > > > > >>> > > improvement
>>>> > > > > > > > > > > >>> > > > >> > > > opportunity
>>>> > > > > > > > > > > >>> > > > >> > > > > > is
>>>> > > > > > > > > > > >>> > > > >> > > > > > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> use
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > topicId in
>>>> > > > > > > > > > > >>> GroupMetadataManager.offsetCommitKey
>>>> > > > > > > > > > > >>> > > in
>>>> > > > > > > > > > > >>> > > > >> the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > offset_commit
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > topic.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > This may save some
>>>> > space.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > Thanks,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > Jun
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > On Wed, Sep 23,
>>>> 2020 at
>>>> > > > 8:50
>>>> > > > > AM
>>>> > > > > > > > > Justine
>>>> > > > > > > > > > > >>> Olshan
>>>> > > > > > > > > > > >>> > <
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > jolshan@confluent.io>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > wrote:
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > Hi Tom,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > Thanks for the
>>>> > > comment. I
>>>> > > > > > think
>>>> > > > > > > > > this
>>>> > > > > > > > > > > is a
>>>> > > > > > > > > > > >>> > > really
>>>> > > > > > > > > > > >>> > > > >> good
>>>> > > > > > > > > > > >>> > > > >> > > idea
>>>> > > > > > > > > > > >>> > > > >> > > > > and
>>>> > > > > > > > > > > >>> > > > >> > > > > > > it
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> has
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > been
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > added to the KIP
>>>> > under
>>>> > > > the
>>>> > > > > > > newly
>>>> > > > > > > > > > added
>>>> > > > > > > > > > > >>> > tooling
>>>> > > > > > > > > > > >>> > > > >> > section.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > Thanks again,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > Justine
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > On Wed, Sep 23,
>>>> 2020
>>>> > at
>>>> > > > > 3:17
>>>> > > > > > > AM Tom
>>>> > > > > > > > > > > >>> Bentley <
>>>> > > > > > > > > > > >>> > > > >> > > > > > > tbentley@redhat.com>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > wrote:
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > Hi Justine,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > I know you
>>>> started
>>>> > > the
>>>> > > > > vote
>>>> > > > > > > > > thread,
>>>> > > > > > > > > > > >>> but on
>>>> > > > > > > > > > > >>> > > > >> > re-reading
>>>> > > > > > > > > > > >>> > > > >> > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > KIP
>>>> > > > > > > > > > > >>> > > > >> > > > > > > I
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > noticed
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that although
>>>> the
>>>> > > topic
>>>> > > > > id
>>>> > > > > > is
>>>> > > > > > > > > > > included
>>>> > > > > > > > > > > >>> in
>>>> > > > > > > > > > > >>> > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > MetadataResponse
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> it's
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > not
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > surfaced in
>>>> the
>>>> > > output
>>>> > > > > from
>>>> > > > > > > > > > > >>> > `kafka-topics.sh
>>>> > > > > > > > > > > >>> > > > >> > > > --describe`.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > Maybe
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> that
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > was
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > intentional
>>>> because
>>>> > > ids
>>>> > > > > are
>>>> > > > > > > > > > > >>> intentionally
>>>> > > > > > > > > > > >>> > not
>>>> > > > > > > > > > > >>> > > > >> really
>>>> > > > > > > > > > > >>> > > > >> > > > > > something
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > user
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > should care
>>>> deeply
>>>> > > > about,
>>>> > > > > > > but it
>>>> > > > > > > > > > > would
>>>> > > > > > > > > > > >>> also
>>>> > > > > > > > > > > >>> > > > make
>>>> > > > > > > > > > > >>> > > > >> > life
>>>> > > > > > > > > > > >>> > > > >> > > > > harder
>>>> > > > > > > > > > > >>> > > > >> > > > > > > for
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > anyone
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > debugging
>>>> Kafka and
>>>> > > > this
>>>> > > > > > > would
>>>> > > > > > > > > > likely
>>>> > > > > > > > > > > >>> get
>>>> > > > > > > > > > > >>> > > worse
>>>> > > > > > > > > > > >>> > > > >> the
>>>> > > > > > > > > > > >>> > > > >> > > more
>>>> > > > > > > > > > > >>> > > > >> > > > > > topic
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > ids
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > got
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > rolled out
>>>> across
>>>> > the
>>>> > > > > > > protocols,
>>>> > > > > > > > > > > >>> clients
>>>> > > > > > > > > > > >>> > etc.
>>>> > > > > > > > > > > >>> > > > It
>>>> > > > > > > > > > > >>> > > > >> > seems
>>>> > > > > > > > > > > >>> > > > >> > > > > > likely
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > that
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
>>>> `kafka-topics.sh`
>>>> > > will
>>>> > > > > > > eventually
>>>> > > > > > > > > > > need
>>>> > > > > > > > > > > >>> the
>>>> > > > > > > > > > > >>> > > > >> ability
>>>> > > > > > > > > > > >>> > > > >> > to
>>>> > > > > > > > > > > >>> > > > >> > > > show
>>>> > > > > > > > > > > >>> > > > >> > > > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > id
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> of
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > a
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > topic and
>>>> perhaps
>>>> > > find
>>>> > > > a
>>>> > > > > > > topic
>>>> > > > > > > > > name
>>>> > > > > > > > > > > >>> given
>>>> > > > > > > > > > > >>> > an
>>>> > > > > > > > > > > >>> > > > id.
>>>> > > > > > > > > > > >>> > > > >> Is
>>>> > > > > > > > > > > >>> > > > >> > > > there
>>>> > > > > > > > > > > >>> > > > >> > > > > > any
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> reason
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > not
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > implement
>>>> that in
>>>> > > this
>>>> > > > > KIP?
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > Many thanks,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > Tom
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > On Mon, Sep
>>>> 21,
>>>> > 2020
>>>> > > at
>>>> > > > > > 9:54
>>>> > > > > > > PM
>>>> > > > > > > > > > > Justine
>>>> > > > > > > > > > > >>> > > Olshan
>>>> > > > > > > > > > > >>> > > > <
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> jolshan@confluent.io
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > wrote:
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Hi all,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > After
>>>> thinking
>>>> > > about
>>>> > > > > it,
>>>> > > > > > > I've
>>>> > > > > > > > > > > >>> decided to
>>>> > > > > > > > > > > >>> > > > remove
>>>> > > > > > > > > > > >>> > > > >> > the
>>>> > > > > > > > > > > >>> > > > >> > > > > topic
>>>> > > > > > > > > > > >>> > > > >> > > > > > > name
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> from
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Fetch
>>>> Request and
>>>> > > > > > Response
>>>> > > > > > > > > after
>>>> > > > > > > > > > > all.
>>>> > > > > > > > > > > >>> > Since
>>>> > > > > > > > > > > >>> > > > >> there
>>>> > > > > > > > > > > >>> > > > >> > > are
>>>> > > > > > > > > > > >>> > > > >> > > > so
>>>> > > > > > > > > > > >>> > > > >> > > > > > > many
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > of
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > these
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > requests per
>>>> > > second,
>>>> > > > it
>>>> > > > > > is
>>>> > > > > > > > > worth
>>>> > > > > > > > > > > >>> removing
>>>> > > > > > > > > > > >>> > > the
>>>> > > > > > > > > > > >>> > > > >> > extra
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > information.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > I've
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > updated the
>>>> KIP
>>>> > to
>>>> > > > > > reflect
>>>> > > > > > > this
>>>> > > > > > > > > > > >>> change.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Please let
>>>> me
>>>> > know
>>>> > > if
>>>> > > > > > > there is
>>>> > > > > > > > > > > >>> anything
>>>> > > > > > > > > > > >>> > > else
>>>> > > > > > > > > > > >>> > > > we
>>>> > > > > > > > > > > >>> > > > >> > > should
>>>> > > > > > > > > > > >>> > > > >> > > > > > > discuss
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > before
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > voting.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Thank you,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Justine
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > On Fri, Sep
>>>> 18,
>>>> > > 2020
>>>> > > > at
>>>> > > > > > > 9:46 AM
>>>> > > > > > > > > > > >>> Justine
>>>> > > > > > > > > > > >>> > > > Olshan
>>>> > > > > > > > > > > >>> > > > >> <
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>>>> jolshan@confluent.io
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > wrote:
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > Hi Jun,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > I see
>>>> what you
>>>> > > are
>>>> > > > > > > saying.
>>>> > > > > > > > > For
>>>> > > > > > > > > > > now
>>>> > > > > > > > > > > >>> we
>>>> > > > > > > > > > > >>> > can
>>>> > > > > > > > > > > >>> > > > >> remove
>>>> > > > > > > > > > > >>> > > > >> > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > extra
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > information.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > I'll
>>>> leave the
>>>> > > > option
>>>> > > > > > to
>>>> > > > > > > add
>>>> > > > > > > > > > more
>>>> > > > > > > > > > > >>> > fields
>>>> > > > > > > > > > > >>> > > to
>>>> > > > > > > > > > > >>> > > > >> the
>>>> > > > > > > > > > > >>> > > > >> > > file
>>>> > > > > > > > > > > >>> > > > >> > > > > in
>>>> > > > > > > > > > > >>> > > > >> > > > > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > future.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > The
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > KIP
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > has been
>>>> > updated
>>>> > > to
>>>> > > > > > > reflect
>>>> > > > > > > > > > this
>>>> > > > > > > > > > > >>> > change.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > Thanks,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > Justine
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > On Fri,
>>>> Sep 18,
>>>> > > > 2020
>>>> > > > > at
>>>> > > > > > > 8:46
>>>> > > > > > > > > AM
>>>> > > > > > > > > > > Jun
>>>> > > > > > > > > > > >>> > Rao <
>>>> > > > > > > > > > > >>> > > > >> > > > > > jun@confluent.io
>>>> > > > > > > > > > > >>> > > > >> > > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > wrote:
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Hi,
>>>> Justine,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Thanks
>>>> for the
>>>> > > > > reply.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> 13. If
>>>> the log
>>>> > > > > > > directory is
>>>> > > > > > > > > > the
>>>> > > > > > > > > > > >>> source
>>>> > > > > > > > > > > >>> > > of
>>>> > > > > > > > > > > >>> > > > >> > truth,
>>>> > > > > > > > > > > >>> > > > >> > > it
>>>> > > > > > > > > > > >>> > > > >> > > > > > means
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> that
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>>>> redundant info
>>>> > > in
>>>> > > > > the
>>>> > > > > > > > > metadata
>>>> > > > > > > > > > > >>> file
>>>> > > > > > > > > > > >>> > will
>>>> > > > > > > > > > > >>> > > > be
>>>> > > > > > > > > > > >>> > > > >> > > > ignored.
>>>> > > > > > > > > > > >>> > > > >> > > > > > Then
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > question
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > is
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> why do
>>>> we need
>>>> > > to
>>>> > > > > put
>>>> > > > > > > the
>>>> > > > > > > > > > > >>> redundant
>>>> > > > > > > > > > > >>> > info
>>>> > > > > > > > > > > >>> > > > in
>>>> > > > > > > > > > > >>> > > > >> the
>>>> > > > > > > > > > > >>> > > > >> > > > > > metadata
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > file
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > now?
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Thanks,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Jun
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> On Thu,
>>>> Sep
>>>> > 17,
>>>> > > > 2020
>>>> > > > > > at
>>>> > > > > > > 5:07
>>>> > > > > > > > > > PM
>>>> > > > > > > > > > > >>> > Justine
>>>> > > > > > > > > > > >>> > > > >> Olshan
>>>> > > > > > > > > > > >>> > > > >> > <
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
>>>> jolshan@confluent.io
>>>> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> wrote:
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Hi Jun,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Thanks
>>>> for
>>>> > the
>>>> > > > > quick
>>>> > > > > > > > > > response!
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > 12.
>>>> I've
>>>> > > decided
>>>> > > > > to
>>>> > > > > > > bump
>>>> > > > > > > > > up
>>>> > > > > > > > > > > the
>>>> > > > > > > > > > > >>> > > versions
>>>> > > > > > > > > > > >>> > > > >> on
>>>> > > > > > > > > > > >>> > > > >> > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > requests
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> and
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > updated
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > KIP. I
>>>> think
>>>> > > > it's
>>>> > > > > > > good we
>>>> > > > > > > > > > > >>> thoroughly
>>>> > > > > > > > > > > >>> > > > >> > discussed
>>>> > > > > > > > > > > >>> > > > >> > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > options
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > here,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > so
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > we
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> know
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > we
>>>> made a
>>>> > good
>>>> > > > > > > choice. :)
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > 13.
>>>> This is
>>>> > an
>>>> > > > > > > interesting
>>>> > > > > > > > > > > >>> > situation.
>>>> > > > > > > > > > > >>> > > I
>>>> > > > > > > > > > > >>> > > > >> think
>>>> > > > > > > > > > > >>> > > > >> > > if
>>>> > > > > > > > > > > >>> > > > >> > > > > this
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > does
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > occur
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > we
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> should
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > give a
>>>> > > warning.
>>>> > > > I
>>>> > > > > > > agree
>>>> > > > > > > > > that
>>>> > > > > > > > > > > >>> it's
>>>> > > > > > > > > > > >>> > hard
>>>> > > > > > > > > > > >>> > > > to
>>>> > > > > > > > > > > >>> > > > >> > know
>>>> > > > > > > > > > > >>> > > > >> > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > source
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> of
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > truth
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > for
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> sure
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > since
>>>> the
>>>> > > > > directory
>>>> > > > > > > or the
>>>> > > > > > > > > > > file
>>>> > > > > > > > > > > >>> > could
>>>> > > > > > > > > > > >>> > > be
>>>> > > > > > > > > > > >>> > > > >> > > manually
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> modified. I
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > guess
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>>>> directory
>>>> > > could
>>>> > > > be
>>>> > > > > > > used as
>>>> > > > > > > > > > the
>>>> > > > > > > > > > > >>> > source
>>>> > > > > > > > > > > >>> > > of
>>>> > > > > > > > > > > >>> > > > >> > truth.
>>>> > > > > > > > > > > >>> > > > >> > > > To
>>>> > > > > > > > > > > >>> > > > >> > > > > be
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> honest,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > I'm
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > not
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > really
>>>> sure
>>>> > > what
>>>> > > > > > > happens
>>>> > > > > > > > > in
>>>> > > > > > > > > > > >>> kafka
>>>> > > > > > > > > > > >>> > when
>>>> > > > > > > > > > > >>> > > > the
>>>> > > > > > > > > > > >>> > > > >> > log
>>>> > > > > > > > > > > >>> > > > >> > > > > > > directory
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > is
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > renamed
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>>>> manually in
>>>> > > > such a
>>>> > > > > > > way.
>>>> > > > > > > > > I'm
>>>> > > > > > > > > > > also
>>>> > > > > > > > > > > >>> > > > >> wondering if
>>>> > > > > > > > > > > >>> > > > >> > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> situation is
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>>>> recoverable
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > in this
>>>> > > > scenario.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Thanks,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Justine
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > On
>>>> Thu, Sep
>>>> > > 17,
>>>> > > > > 2020
>>>> > > > > > > at
>>>> > > > > > > > > 4:28
>>>> > > > > > > > > > > PM
>>>> > > > > > > > > > > >>> Jun
>>>> > > > > > > > > > > >>> > > Rao
>>>> > > > > > > > > > > >>> > > > <
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > jun@confluent.io>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > wrote:
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Hi,
>>>> > Justine,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> Thanks for
>>>> > > the
>>>> > > > > > > reply.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > 12. I
>>>> > don't
>>>> > > > > have a
>>>> > > > > > > > > strong
>>>> > > > > > > > > > > >>> > preference
>>>> > > > > > > > > > > >>> > > > >> > either.
>>>> > > > > > > > > > > >>> > > > >> > > > > > However,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > if
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> we
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > need
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > IBP
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> anyway,
>>>> > > maybe
>>>> > > > > it's
>>>> > > > > > > > > easier
>>>> > > > > > > > > > to
>>>> > > > > > > > > > > >>> just
>>>> > > > > > > > > > > >>> > > bump
>>>> > > > > > > > > > > >>> > > > >> up
>>>> > > > > > > > > > > >>> > > > >> > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > version
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > for
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > all
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > inter
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > broker
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> requests
>>>> > and
>>>> > > > add
>>>> > > > > > the
>>>> > > > > > > > > topic
>>>> > > > > > > > > > > id
>>>> > > > > > > > > > > >>> > field
>>>> > > > > > > > > > > >>> > > > as a
>>>> > > > > > > > > > > >>> > > > >> > > > regular
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > field. A
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > regular
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> field
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > is
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > a
>>>> bit more
>>>> > > > > concise
>>>> > > > > > > in
>>>> > > > > > > > > wire
>>>> > > > > > > > > > > >>> > transfer
>>>> > > > > > > > > > > >>> > > > >> than a
>>>> > > > > > > > > > > >>> > > > >> > > > > flexible
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> field.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > 13.
>>>> The
>>>> > > > > confusion
>>>> > > > > > > that I
>>>> > > > > > > > > > was
>>>> > > > > > > > > > > >>> > > referring
>>>> > > > > > > > > > > >>> > > > >> to
>>>> > > > > > > > > > > >>> > > > >> > is
>>>> > > > > > > > > > > >>> > > > >> > > > > > between
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > topic
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > name
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> partition
>>>> > > > number
>>>> > > > > > > between
>>>> > > > > > > > > > the
>>>> > > > > > > > > > > >>> log
>>>> > > > > > > > > > > >>> > dir
>>>> > > > > > > > > > > >>> > > > and
>>>> > > > > > > > > > > >>> > > > >> > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > metadata
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> file.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > For
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> example,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > if
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > the
>>>> log
>>>> > dir
>>>> > > is
>>>> > > > > > > topicA-1
>>>> > > > > > > > > > and
>>>> > > > > > > > > > > >>> the
>>>> > > > > > > > > > > >>> > > > metadata
>>>> > > > > > > > > > > >>> > > > >> > file
>>>> > > > > > > > > > > >>> > > > >> > > > in
>>>> > > > > > > > > > > >>> > > > >> > > > > it
>>>> > > > > > > > > > > >>> > > > >> > > > > > > has
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > topicB
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > and
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> partition
>>>> > 0
>>>> > > > (say
>>>> > > > > > > due to
>>>> > > > > > > > > a
>>>> > > > > > > > > > > bug
>>>> > > > > > > > > > > >>> or
>>>> > > > > > > > > > > >>> > > > manual
>>>> > > > > > > > > > > >>> > > > >> > > > > > > modification),
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> which
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > one
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > do
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > we
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > use
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > as
>>>> the
>>>> > > source
>>>> > > > of
>>>> > > > > > > truth?
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Jun
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > On
>>>> Thu,
>>>> > Sep
>>>> > > > 17,
>>>> > > > > > > 2020 at
>>>> > > > > > > > > > 3:43
>>>> > > > > > > > > > > >>> PM
>>>> > > > > > > > > > > >>> > > > Justine
>>>> > > > > > > > > > > >>> > > > >> > > Olshan
>>>> > > > > > > > > > > >>> > > > >> > > > <
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
>>>> > > jolshan@confluent.io
>>>> > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> wrote:
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > Hi
>>>> Jun,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> Thanks
>>>> > for
>>>> > > > the
>>>> > > > > > > > > comments.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> 12. I
>>>> > > bumped
>>>> > > > > the
>>>> > > > > > > > > > > >>> > > LeaderAndIsrRequest
>>>> > > > > > > > > > > >>> > > > >> > > because
>>>> > > > > > > > > > > >>> > > > >> > > > I
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > removed
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > topic
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> name
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > field
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > in
>>>> the
>>>> > > > > response.
>>>> > > > > > > It
>>>> > > > > > > > > may
>>>> > > > > > > > > > be
>>>> > > > > > > > > > > >>> > > possible
>>>> > > > > > > > > > > >>> > > > to
>>>> > > > > > > > > > > >>> > > > >> > > avoid
>>>> > > > > > > > > > > >>> > > > >> > > > > > > bumping
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > version
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > without
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> that
>>>> > > change,
>>>> > > > > but
>>>> > > > > > > I may
>>>> > > > > > > > > > be
>>>> > > > > > > > > > > >>> > missing
>>>> > > > > > > > > > > >>> > > > >> > > something.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > I
>>>> > believe
>>>> > > > > > > StopReplica
>>>> > > > > > > > > is
>>>> > > > > > > > > > > >>> > actually
>>>> > > > > > > > > > > >>> > > on
>>>> > > > > > > > > > > >>> > > > >> > > version
>>>> > > > > > > > > > > >>> > > > >> > > > 3
>>>> > > > > > > > > > > >>> > > > >> > > > > > now,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > but
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > because
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>>>> version 2
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > is
>>>> > > > flexible, I
>>>> > > > > > > kept
>>>> > > > > > > > > that
>>>> > > > > > > > > > > >>> listed
>>>> > > > > > > > > > > >>> > as
>>>> > > > > > > > > > > >>> > > > >> > version
>>>> > > > > > > > > > > >>> > > > >> > > 2
>>>> > > > > > > > > > > >>> > > > >> > > > on
>>>> > > > > > > > > > > >>> > > > >> > > > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > KIP
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > page.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> However,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > you
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> may be
>>>> > > right
>>>> > > > > in
>>>> > > > > > > that
>>>> > > > > > > > > we
>>>> > > > > > > > > > > may
>>>> > > > > > > > > > > >>> need
>>>> > > > > > > > > > > >>> > > to
>>>> > > > > > > > > > > >>> > > > >> bump
>>>> > > > > > > > > > > >>> > > > >> > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > version
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > on
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > StopReplica
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > deal
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> with
>>>> > > > deletion
>>>> > > > > > > > > > differently
>>>> > > > > > > > > > > as
>>>> > > > > > > > > > > >>> > > > mentioned
>>>> > > > > > > > > > > >>> > > > >> > > > above. I
>>>> > > > > > > > > > > >>> > > > >> > > > > > > don't
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> know
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > if
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > I
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> have a
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > big
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> > preference
>>>> > > > > over
>>>> > > > > > > used
>>>> > > > > > > > > > > tagged
>>>> > > > > > > > > > > >>> > fields
>>>> > > > > > > > > > > >>> > > > or
>>>> > > > > > > > > > > >>> > > > >> > not.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> 13. I
>>>> > was
>>>> > > > > > thinking
>>>> > > > > > > > > that
>>>> > > > > > > > > > in
>>>> > > > > > > > > > > >>> the
>>>> > > > > > > > > > > >>> > > case
>>>> > > > > > > > > > > >>> > > > >> where
>>>> > > > > > > > > > > >>> > > > >> > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > file
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > and
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > request
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > topic
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > ids
>>>> > don't
>>>> > > > > match,
>>>> > > > > > > it
>>>> > > > > > > > > > means
>>>> > > > > > > > > > > >>> that
>>>> > > > > > > > > > > >>> > the
>>>> > > > > > > > > > > >>> > > > >> > broker's
>>>> > > > > > > > > > > >>> > > > >> > > > > > > topic/the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> one
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > in
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> file
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > has
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> been
>>>> > > > deleted.
>>>> > > > > In
>>>> > > > > > > that
>>>> > > > > > > > > > > case,
>>>> > > > > > > > > > > >>> we
>>>> > > > > > > > > > > >>> > > would
>>>> > > > > > > > > > > >>> > > > >> need
>>>> > > > > > > > > > > >>> > > > >> > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > delete
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > old
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > topic
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > start
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> > receiving
>>>> > > > the
>>>> > > > > > new
>>>> > > > > > > > > > version.
>>>> > > > > > > > > > > >>> If
>>>> > > > > > > > > > > >>> > the
>>>> > > > > > > > > > > >>> > > > >> topic
>>>> > > > > > > > > > > >>> > > > >> > > name
>>>> > > > > > > > > > > >>> > > > >> > > > > were
>>>> > > > > > > > > > > >>> > > > >> > > > > > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > change,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > but
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > ids
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> still
>>>> > > > matched,
>>>> > > > > > the
>>>> > > > > > > > > file
>>>> > > > > > > > > > > >>> would
>>>> > > > > > > > > > > >>> > also
>>>> > > > > > > > > > > >>> > > > >> need
>>>> > > > > > > > > > > >>> > > > >> > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > update.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > Am I
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > missing
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > a
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> case
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> where
>>>> > the
>>>> > > > file
>>>> > > > > > > would
>>>> > > > > > > > > be
>>>> > > > > > > > > > > >>> correct
>>>> > > > > > > > > > > >>> > > and
>>>> > > > > > > > > > > >>> > > > >> not
>>>> > > > > > > > > > > >>> > > > >> > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > request?
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> Thanks,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> Justine
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > On
>>>> Thu,
>>>> > > Sep
>>>> > > > > 17,
>>>> > > > > > > 2020
>>>> > > > > > > > > at
>>>> > > > > > > > > > > >>> 3:18 PM
>>>> > > > > > > > > > > >>> > > Jun
>>>> > > > > > > > > > > >>> > > > >> Rao <
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> jun@confluent.io
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > wrote:
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> Hi,
>>>> > > > Justine,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> Thanks
>>>> > > for
>>>> > > > > the
>>>> > > > > > > > > reply.
>>>> > > > > > > > > > A
>>>> > > > > > > > > > > >>> couple
>>>> > > > > > > > > > > >>> > > of
>>>> > > > > > > > > > > >>> > > > >> more
>>>> > > > > > > > > > > >>> > > > >> > > > > comments
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> below.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> 12.
>>>> > > > > ListOffset
>>>> > > > > > > and
>>>> > > > > > > > > > > >>> > > OffsetForLeader
>>>> > > > > > > > > > > >>> > > > >> > > > currently
>>>> > > > > > > > > > > >>> > > > >> > > > > > > don't
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > support
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> flexible
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> fields.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> So, we
>>>> > > > have
>>>> > > > > to
>>>> > > > > > > bump
>>>> > > > > > > > > up
>>>> > > > > > > > > > > the
>>>> > > > > > > > > > > >>> > > version
>>>> > > > > > > > > > > >>> > > > >> > number
>>>> > > > > > > > > > > >>> > > > >> > > > and
>>>> > > > > > > > > > > >>> > > > >> > > > > > use
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> IBP at
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > least
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > for
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > these
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> two
>>>> > > > > requests.
>>>> > > > > > > Note
>>>> > > > > > > > > > that
>>>> > > > > > > > > > > it
>>>> > > > > > > > > > > >>> > seems
>>>> > > > > > > > > > > >>> > > > >> 2.7.0
>>>> > > > > > > > > > > >>> > > > >> > > will
>>>> > > > > > > > > > > >>> > > > >> > > > > > > require
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> IBP
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > anyway
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > because
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > of
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > changes
>>>> > > in
>>>> > > > > > > > > > KAFKA-10435.
>>>> > > > > > > > > > > >>> Also,
>>>> > > > > > > > > > > >>> > it
>>>> > > > > > > > > > > >>> > > > >> seems
>>>> > > > > > > > > > > >>> > > > >> > > that
>>>> > > > > > > > > > > >>> > > > >> > > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> version
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > for
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > > > > > > LeaderAndIsrRequest
>>>> > > > > > > > > > and
>>>> > > > > > > > > > > >>> > > > StopReplica
>>>> > > > > > > > > > > >>> > > > >> are
>>>> > > > > > > > > > > >>> > > > >> > > > > bumped
>>>> > > > > > > > > > > >>> > > > >> > > > > > > even
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > though
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > we
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > only
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > added
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > a
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> tagged
>>>> > > > > field.
>>>> > > > > > > But
>>>> > > > > > > > > > since
>>>> > > > > > > > > > > >>> IBP is
>>>> > > > > > > > > > > >>> > > > >> needed
>>>> > > > > > > > > > > >>> > > > >> > > > anyway,
>>>> > > > > > > > > > > >>> > > > >> > > > > > we
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > may
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > want
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> revisit
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > overall
>>>> > > > > tagged
>>>> > > > > > > field
>>>> > > > > > > > > > > >>> choice.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> 13.
>>>> > The
>>>> > > > only
>>>> > > > > > > > > downside
>>>> > > > > > > > > > is
>>>> > > > > > > > > > > >>> the
>>>> > > > > > > > > > > >>> > > > >> potential
>>>> > > > > > > > > > > >>> > > > >> > > > > > confusion
>>>> > > > > > > > > > > >>> > > > >> > > > > > > on
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > which
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > one
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > is
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> source
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> of
>>>> > truth
>>>> > > > if
>>>> > > > > > they
>>>> > > > > > > > > don't
>>>> > > > > > > > > > > >>> match.
>>>> > > > > > > > > > > >>> > > > >> Another
>>>> > > > > > > > > > > >>> > > > >> > > > option
>>>> > > > > > > > > > > >>> > > > >> > > > > is
>>>> > > > > > > > > > > >>> > > > >> > > > > > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > include
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > those
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > fields
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > in
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> the
>>>> > > > metadata
>>>> > > > > > > file
>>>> > > > > > > > > when
>>>> > > > > > > > > > > we
>>>> > > > > > > > > > > >>> > > actually
>>>> > > > > > > > > > > >>> > > > >> > change
>>>> > > > > > > > > > > >>> > > > >> > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> directory
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > structure.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > Thanks,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> Jun
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> On
>>>> > Thu,
>>>> > > > Sep
>>>> > > > > > 17,
>>>> > > > > > > 2020
>>>> > > > > > > > > > at
>>>> > > > > > > > > > > >>> 2:01
>>>> > > > > > > > > > > >>> > PM
>>>> > > > > > > > > > > >>> > > > >> Justine
>>>> > > > > > > > > > > >>> > > > >> > > > > Olshan
>>>> > > > > > > > > > > >>> > > > >> > > > > > <
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>>>> > > > > jolshan@confluent.io
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> wrote:
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>>> > Hello
>>>> > > > all,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > I've
>>>> > > > > thought
>>>> > > > > > > some
>>>> > > > > > > > > > more
>>>> > > > > > > > > > > >>> about
>>>> > > > > > > > > > > >>> > > > >> removing
>>>> > > > > > > > > > > >>> > > > >> > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > topic
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> name
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > field
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > from
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > some
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > of
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>>> > > > requests.
>>>> > > > > On
>>>> > > > > > > > > closer
>>>> > > > > > > > > > > >>> > inspection
>>>> > > > > > > > > > > >>> > > > of
>>>> > > > > > > > > > > >>> > > > >> the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>>> requests/responses,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > it
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> seems
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>>> > > internal
>>>> > > > > > > changes
>>>> > > > > > > > > > would
>>>> > > > > > > > > > > >>> be
>>>> > > > > > > > > > > >>> > much
>>>> > > > > > > > > > > >>> > > > >> larger
>>>> > > > > > > > > > > >>> > > > >> > > > than
>>>> > > > > > > > > > > >>> > > > >> > > > > I
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> expected.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > Some
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>>>> protocols
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>>> > > involve
>>>> > > > > > > clients,
>>>> > > > > > > > > so
>>>> > > > > > > > > > > they
>>>> > > > > > > > > > > >>> > would
>>>> > > > > > > > > > > >>> > > > >> > require
>>>> > > > > > > > > > > >>> > > > >> > > > > > changes
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > too.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > I'm
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > thinking
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > that
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > for
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > now,
>>>> > > > > > removing
>>>> > > > > > > the
>>>> > > > > > > > > > > topic
>>>> > > > > > > > > > > >>> name
>>>> > > > > > > > > > > >>> > > > from
>>>> > > > > > > > > > > >>> > > > >> > these
>>>> > > > > > > > > > > >>> > > > >> > > > > > > requests
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> and
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > responses
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> are
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > out
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>>> > scope.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > I
>>>> > have
>>>> > > > > > > decided to
>>>> > > > > > > > > > just
>>>> > > > > > > > > > > >>> keep
>>>> > > > > > > > > > > >>> > > the
>>>> > > > > > > > > > > >>> > > > >> > change
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
>>>> LeaderAndIsrResponse
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > remove
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>>> > topic
>>>> > > > > name,
>>>> > > > > > > and
>>>> > > > > > > > > have
>>>> > > > > > > > > > > >>> updated
>>>> > > > > > > > > > > >>> > > the
>>>> > > > > > > > > > > >>> > > > >> KIP
>>>> > > > > > > > > > > >>> > > > >> > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > reflect
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> this
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > change. I
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> have
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> also
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>>> > > > mentioned
>>>> > > > > > the
>>>> > > > > > > > > other
>>>> > > > > > > > > > > >>> requests
>>>> > > > > > > > > > > >>> > > and
>>>> > > > > > > > > > > >>> > > > >> > > > responses
>>>> > > > > > > > > > > >>> > > > >> > > > > in
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> future
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > work.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > I'm
>>>> > > > hoping
>>>> > > > > > to
>>>> > > > > > > > > start
>>>> > > > > > > > > > > the
>>>> > > > > > > > > > > >>> > voting
>>>> > > > > > > > > > > >>> > > > >> > process
>>>> > > > > > > > > > > >>> > > > >> > > > > soon,
>>>> > > > > > > > > > > >>> > > > >> > > > > > so
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> let me
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > know
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > if
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > there
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > is
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>>> > > anything
>>>> > > > > > else
>>>> > > > > > > we
>>>> > > > > > > > > > > should
>>>> > > > > > > > > > > >>> > > discuss.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>>> > Thank
>>>> > > > you,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>>> > > Justine
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > On
>>>> > > Tue,
>>>> > > > > Sep
>>>> > > > > > > 15,
>>>> > > > > > > > > 2020
>>>> > > > > > > > > > > at
>>>> > > > > > > > > > > >>> 3:57
>>>> > > > > > > > > > > >>> > > PM
>>>> > > > > > > > > > > >>> > > > >> > Justine
>>>> > > > > > > > > > > >>> > > > >> > > > > > Olshan
>>>> > > > > > > > > > > >>> > > > >> > > > > > > <
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>>>> > > > > > jolshan@confluent.io
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>>> > wrote:
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >
>>>> > > Hello
>>>> > > > > > again,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > > To
>>>> > > > > follow
>>>> > > > > > > up on
>>>> > > > > > > > > > some
>>>> > > > > > > > > > > >>> of
>>>> > > > > > > > > > > >>> > the
>>>> > > > > > > > > > > >>> > > > >> other
>>>> > > > > > > > > > > >>> > > > >> > > > > comments:
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >
>>>> > > 10/11)
>>>> > > > > We
>>>> > > > > > > can
>>>> > > > > > > > > > remove
>>>> > > > > > > > > > > >>> the
>>>> > > > > > > > > > > >>> > > topic
>>>> > > > > > > > > > > >>> > > > >> name
>>>> > > > > > > > > > > >>> > > > >> > > > from
>>>> > > > > > > > > > > >>> > > > >> > > > > > > these
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>>>> > > > > requests/responses,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > and
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >
>>>> > that
>>>> > > > > means
>>>> > > > > > > we
>>>> > > > > > > > > will
>>>> > > > > > > > > > > >>> just
>>>> > > > > > > > > > > >>> > have
>>>> > > > > > > > > > > >>> > > > to
>>>> > > > > > > > > > > >>> > > > >> > make
>>>> > > > > > > > > > > >>> > > > >> > > a
>>>> > > > > > > > > > > >>> > > > >> > > > > few
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> internal
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > changes
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > make
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >
>>>> > > > > partitions
>>>> > > > > > > > > > > accessible
>>>> > > > > > > > > > > >>> by
>>>> > > > > > > > > > > >>> > > topic
>>>> > > > > > > > > > > >>> > > > >> id
>>>> > > > > > > > > > > >>> > > > >> > and
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > partition.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> I
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > can
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > update
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > KIP
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >
>>>> > > remove
>>>> > > > > > them
>>>> > > > > > > > > unless
>>>> > > > > > > > > > > >>> anyone
>>>> > > > > > > > > > > >>> > > > thinks
>>>> > > > > > > > > > > >>> > > > >> > they
>>>> > > > > > > > > > > >>> > > > >> > > > > > should
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> stay.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >
>>>> > 12)
>>>> > > > > > > Addressed in
>>>> > > > > > > > > > the
>>>> > > > > > > > > > > >>> > > previous
>>>> > > > > > > > > > > >>> > > > >> > email.
>>>> > > > > > > > > > > >>> > > > >> > > > I've
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > updated
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > KIP
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> include
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >
>>>> > > tagged
>>>> > > > > > > fields
>>>> > > > > > > > > for
>>>> > > > > > > > > > > the
>>>> > > > > > > > > > > >>> > > requests
>>>> > > > > > > > > > > >>> > > > >> and
>>>> > > > > > > > > > > >>> > > > >> > > > > > responses.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> (More
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > on
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > below)
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >
>>>> > 13)
>>>> > > I
>>>> > > > > > think
>>>> > > > > > > part
>>>> > > > > > > > > > of
>>>> > > > > > > > > > > >>> the
>>>> > > > > > > > > > > >>> > idea
>>>> > > > > > > > > > > >>> > > > for
>>>> > > > > > > > > > > >>> > > > >> > > > > including
>>>> > > > > > > > > > > >>> > > > >> > > > > > > this
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > information
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> is to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > prepare
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >
>>>> > for
>>>> > > > > future
>>>> > > > > > > > > > changes.
>>>> > > > > > > > > > > >>> > Perhaps
>>>> > > > > > > > > > > >>> > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > directory
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> structure
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > might
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> change
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > from
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >
>>>> > > > > > > > > > > >>> topicName_partitionNumber
>>>> > > > > > > > > > > >>> > to
>>>> > > > > > > > > > > >>> > > > >> > > something
>>>> > > > > > > > > > > >>> > > > >> > > > > like
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> > > > > > > topicID_partitionNumber.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> Then
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > > it
>>>> > > > would
>>>> > > > > > be
>>>> > > > > > > > > useful
>>>> > > > > > > > > > > to
>>>> > > > > > > > > > > >>> have
>>>> > > > > > > > > > > >>> > > the
>>>> > > > > > > > > > > >>> > > > >> > topic
>>>> > > > > > > > > > > >>> > > > >> > > > name
>>>> > > > > > > > > > > >>> > > > >> > > > > > in
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > file
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > since
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > it
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > would
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> not
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > be
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > > in
>>>> > > the
>>>> > > > > > > directory
>>>> > > > > > > > > > > >>> > structure.
>>>> > > > > > > > > > > >>> > > > >> > > Supporting
>>>> > > > > > > > > > > >>> > > > >> > > > > > topic
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> renames
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > might
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > be
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> easier
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > if
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >
>>>> > > other
>>>> > > > > > > fields are
>>>> > > > > > > > > > > >>> included.
>>>> > > > > > > > > > > >>> > > > Would
>>>> > > > > > > > > > > >>> > > > >> > > there
>>>> > > > > > > > > > > >>> > > > >> > > > be
>>>> > > > > > > > > > > >>> > > > >> > > > > > any
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > downsides
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> including
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> this
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >
>>>> > > > > > information?
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >
>>>> > 14)
>>>> > > > > Yes,
>>>> > > > > > we
>>>> > > > > > > > > would
>>>> > > > > > > > > > > >>> need to
>>>> > > > > > > > > > > >>> > > > copy
>>>> > > > > > > > > > > >>> > > > >> the
>>>> > > > > > > > > > > >>> > > > >> > > > > > partition
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > metadata
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > file
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > in
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > this
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >
>>>> > > > process.
>>>> > > > > > > I've
>>>> > > > > > > > > > > updated
>>>> > > > > > > > > > > >>> the
>>>> > > > > > > > > > > >>> > > KIP
>>>> > > > > > > > > > > >>> > > > to
>>>> > > > > > > > > > > >>> > > > >> > > > include
>>>> > > > > > > > > > > >>> > > > >> > > > > > > this.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >
>>>> > 15)
>>>> > > I
>>>> > > > > > > believe
>>>> > > > > > > > > > Lucas
>>>> > > > > > > > > > > >>> meant
>>>> > > > > > > > > > > >>> > v1
>>>> > > > > > > > > > > >>> > > > >> and v2
>>>> > > > > > > > > > > >>> > > > >> > > > here.
>>>> > > > > > > > > > > >>> > > > >> > > > > > He
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > was
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > referring
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> how
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >
>>>> > > > requests
>>>> > > > > > > would
>>>> > > > > > > > > > fall
>>>> > > > > > > > > > > >>> under
>>>> > > > > > > > > > > >>> > > > >> different
>>>> > > > > > > > > > > >>> > > > >> > > IBP
>>>> > > > > > > > > > > >>> > > > >> > > > > and
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > meant
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > that
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > older
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> brokers
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>>> > would
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >
>>>> > have
>>>> > > > to
>>>> > > > > > use
>>>> > > > > > > the
>>>> > > > > > > > > > > older
>>>> > > > > > > > > > > >>> > > version
>>>> > > > > > > > > > > >>> > > > of
>>>> > > > > > > > > > > >>> > > > >> > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > request
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > and
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > existing
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > topic
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >
>>>> > > > deletion
>>>> > > > > > > > > process.
>>>> > > > > > > > > > At
>>>> > > > > > > > > > > >>> > first,
>>>> > > > > > > > > > > >>> > > it
>>>> > > > > > > > > > > >>> > > > >> > seemed
>>>> > > > > > > > > > > >>> > > > >> > > > > like
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > tagged
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > fields
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > would
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> resolve
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >
>>>> > the
>>>> > > > IBP
>>>> > > > > > > issue.
>>>> > > > > > > > > > > >>> However, we
>>>> > > > > > > > > > > >>> > > may
>>>> > > > > > > > > > > >>> > > > >> need
>>>> > > > > > > > > > > >>> > > > >> > > IBP
>>>> > > > > > > > > > > >>> > > > >> > > > > for
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > this
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > request
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > after
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > all
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> since
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >
>>>> > the
>>>> > > > > > > controller
>>>> > > > > > > > > > > >>> handles the
>>>> > > > > > > > > > > >>> > > > topic
>>>> > > > > > > > > > > >>> > > > >> > > > deletion
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > differently
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> depending
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > on
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > IBP
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >
>>>> > > > version.
>>>> > > > > > In
>>>> > > > > > > an
>>>> > > > > > > > > > older
>>>> > > > > > > > > > > >>> > > version,
>>>> > > > > > > > > > > >>> > > > we
>>>> > > > > > > > > > > >>> > > > >> > > can't
>>>> > > > > > > > > > > >>> > > > >> > > > > just
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > send
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> a
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > StopReplica
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> delete
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >
>>>> > > topic
>>>> > > > > > > > > immediately
>>>> > > > > > > > > > > like
>>>> > > > > > > > > > > >>> > we'd
>>>> > > > > > > > > > > >>> > > > >> want to
>>>> > > > > > > > > > > >>> > > > >> > > for
>>>> > > > > > > > > > > >>> > > > >> > > > > > this
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > KIP.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >
>>>> > This
>>>> > > > > makes
>>>> > > > > > > me
>>>> > > > > > > > > > wonder
>>>> > > > > > > > > > > >>> if we
>>>> > > > > > > > > > > >>> > > > want
>>>> > > > > > > > > > > >>> > > > >> > > tagged
>>>> > > > > > > > > > > >>> > > > >> > > > > > fields
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > on
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> all
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> requests
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> after
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >
>>>> > all.
>>>> > > > Let
>>>> > > > > > me
>>>> > > > > > > know
>>>> > > > > > > > > > > your
>>>> > > > > > > > > > > >>> > > > thoughts!
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >
>>>> > > > Justine
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > > On
>>>> > > > Tue,
>>>> > > > > > Sep
>>>> > > > > > > 15,
>>>> > > > > > > > > > 2020
>>>> > > > > > > > > > > >>> at
>>>> > > > > > > > > > > >>> > 1:03
>>>> > > > > > > > > > > >>> > > > PM
>>>> > > > > > > > > > > >>> > > > >> > > Justine
>>>> > > > > > > > > > > >>> > > > >> > > > > > > Olshan
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > <
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> > > > > > > jolshan@confluent.io
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >
>>>> > > wrote:
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>
>>>> > Hi
>>>> > > > all,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>
>>>> > Jun
>>>> > > > > > > brought up
>>>> > > > > > > > > a
>>>> > > > > > > > > > > good
>>>> > > > > > > > > > > >>> > point
>>>> > > > > > > > > > > >>> > > > in
>>>> > > > > > > > > > > >>> > > > >> his
>>>> > > > > > > > > > > >>> > > > >> > > > last
>>>> > > > > > > > > > > >>> > > > >> > > > > > > email
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> about
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > tagged
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > fields,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > and
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>
>>>> > > I've
>>>> > > > > > > updated
>>>> > > > > > > > > the
>>>> > > > > > > > > > > KIP
>>>> > > > > > > > > > > >>> to
>>>> > > > > > > > > > > >>> > > > reflect
>>>> > > > > > > > > > > >>> > > > >> > that
>>>> > > > > > > > > > > >>> > > > >> > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> changes to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > requests
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>>> > > > responses
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>
>>>> > > will
>>>> > > > be
>>>> > > > > > in
>>>> > > > > > > the
>>>> > > > > > > > > > form
>>>> > > > > > > > > > > >>> of
>>>> > > > > > > > > > > >>> > > tagged
>>>> > > > > > > > > > > >>> > > > >> > fields
>>>> > > > > > > > > > > >>> > > > >> > > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > avoid
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > changing
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > IBP.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>
>>>> > > Jun:
>>>> > > > I
>>>> > > > > > > plan on
>>>> > > > > > > > > > > >>> sending a
>>>> > > > > > > > > > > >>> > > > >> followup
>>>> > > > > > > > > > > >>> > > > >> > > > email
>>>> > > > > > > > > > > >>> > > > >> > > > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> address
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > some
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > of
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > other
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>
>>>> > > > points.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>
>>>> > > > Thanks,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>
>>>> > > > Justine
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>
>>>> > On
>>>> > > > Mon,
>>>> > > > > > > Sep 14,
>>>> > > > > > > > > > > 2020
>>>> > > > > > > > > > > >>> at
>>>> > > > > > > > > > > >>> > > 4:25
>>>> > > > > > > > > > > >>> > > > PM
>>>> > > > > > > > > > > >>> > > > >> > Jun
>>>> > > > > > > > > > > >>> > > > >> > > > Rao
>>>> > > > > > > > > > > >>> > > > >> > > > > <
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
>>>> jun@confluent.io>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > wrote:
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > Hi,
>>>> > > > > > > Justine,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > Thanks
>>>> > > > > > > for the
>>>> > > > > > > > > > > >>> updated
>>>> > > > > > > > > > > >>> > > KIP.
>>>> > > > > > > > > > > >>> > > > A
>>>> > > > > > > > > > > >>> > > > >> few
>>>> > > > > > > > > > > >>> > > > >> > > > > > comments
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> below.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > 10.
>>>> > > > > > > > > LeaderAndIsr
>>>> > > > > > > > > > > >>> > Response:
>>>> > > > > > > > > > > >>> > > > Do
>>>> > > > > > > > > > > >>> > > > >> we
>>>> > > > > > > > > > > >>> > > > >> > > need
>>>> > > > > > > > > > > >>> > > > >> > > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > topic
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > name?
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > 11.
>>>> > > > > For
>>>> > > > > > > the
>>>> > > > > > > > > > > changed
>>>> > > > > > > > > > > >>> > > > >> > > request/response,
>>>> > > > > > > > > > > >>> > > > >> > > > > > other
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> than
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>>>> LeaderAndIsr,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > > > > > UpdateMetadata,
>>>> > > > > > > > > > > >>> > Metadata,
>>>> > > > > > > > > > > >>> > > do
>>>> > > > > > > > > > > >>> > > > >> we
>>>> > > > > > > > > > > >>> > > > >> > > need
>>>> > > > > > > > > > > >>> > > > >> > > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > include
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > topic
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> name?
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > 12.
>>>> > > > It
>>>> > > > > > > seems
>>>> > > > > > > > > > that
>>>> > > > > > > > > > > >>> > upgrades
>>>> > > > > > > > > > > >>> > > > >> don't
>>>> > > > > > > > > > > >>> > > > >> > > > > require
>>>> > > > > > > > > > > >>> > > > >> > > > > > > IBP.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> Does
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > that
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > mean
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > new
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > fields
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > in
>>>> > > > all
>>>> > > > > > the
>>>> > > > > > > > > > > >>> > > request/response
>>>> > > > > > > > > > > >>> > > > >> are
>>>> > > > > > > > > > > >>> > > > >> > > added
>>>> > > > > > > > > > > >>> > > > >> > > > > as
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > tagged
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > fields
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> without
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > bumping
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > up
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > the
>>>> > > > > > > request
>>>> > > > > > > > > > > >>> version? It
>>>> > > > > > > > > > > >>> > > > would
>>>> > > > > > > > > > > >>> > > > >> be
>>>> > > > > > > > > > > >>> > > > >> > > > useful
>>>> > > > > > > > > > > >>> > > > >> > > > > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > make
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > that
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > clear.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > 13.
>>>> > > > > > > Partition
>>>> > > > > > > > > > > >>> Metadata
>>>> > > > > > > > > > > >>> > > file:
>>>> > > > > > > > > > > >>> > > > >> Do
>>>> > > > > > > > > > > >>> > > > >> > we
>>>> > > > > > > > > > > >>> > > > >> > > > need
>>>> > > > > > > > > > > >>> > > > >> > > > > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> include
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > topic
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > name
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > and
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > > partition
>>>> > > > > > > id
>>>> > > > > > > > > > since
>>>> > > > > > > > > > > >>> they
>>>> > > > > > > > > > > >>> > > are
>>>> > > > > > > > > > > >>> > > > >> > implied
>>>> > > > > > > > > > > >>> > > > >> > > > in
>>>> > > > > > > > > > > >>> > > > >> > > > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > directory
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > name?
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > 14.
>>>> > > > In
>>>> > > > > > the
>>>> > > > > > > > > JBOD
>>>> > > > > > > > > > > >>> mode, we
>>>> > > > > > > > > > > >>> > > > >> support
>>>> > > > > > > > > > > >>> > > > >> > > > > moving a
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > partition's
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > data
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> from
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > one
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > disk
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > another.
>>>> > > > > > > Will
>>>> > > > > > > > > > the
>>>> > > > > > > > > > > >>> new
>>>> > > > > > > > > > > >>> > > > >> partition
>>>> > > > > > > > > > > >>> > > > >> > > > > metadata
>>>> > > > > > > > > > > >>> > > > >> > > > > > > file
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> be
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > copied
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> during
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > process?
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > 15.
>>>> > > > > The
>>>> > > > > > > KIP
>>>> > > > > > > > > says
>>>> > > > > > > > > > > >>> "Remove
>>>> > > > > > > > > > > >>> > > > >> deleted
>>>> > > > > > > > > > > >>> > > > >> > > > topics
>>>> > > > > > > > > > > >>> > > > >> > > > > > > from
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > replicas
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > by
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > sending
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > > > > > > StopReplicaRequest
>>>> > > > > > > > > > > >>> V2
>>>> > > > > > > > > > > >>> > for
>>>> > > > > > > > > > > >>> > > > any
>>>> > > > > > > > > > > >>> > > > >> > > topics
>>>> > > > > > > > > > > >>> > > > >> > > > > > which
>>>> > > > > > > > > > > >>> > > > >> > > > > > > do
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> not
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > contain
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > a
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > topic
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > ID,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > and
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > V3
>>>> > > > for
>>>> > > > > > any
>>>> > > > > > > > > > topics
>>>> > > > > > > > > > > >>> which
>>>> > > > > > > > > > > >>> > do
>>>> > > > > > > > > > > >>> > > > >> > contain
>>>> > > > > > > > > > > >>> > > > >> > > a
>>>> > > > > > > > > > > >>> > > > >> > > > > > topic
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> ID.".
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > However,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > it
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > seems
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > updated
>>>> > > > > > > > > > controller
>>>> > > > > > > > > > > >>> will
>>>> > > > > > > > > > > >>> > > > create
>>>> > > > > > > > > > > >>> > > > >> > all
>>>> > > > > > > > > > > >>> > > > >> > > > > > missing
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> topic
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > IDs
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > first
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > before
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> doing
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > other
>>>> > > > > > > actions.
>>>> > > > > > > > > > So,
>>>> > > > > > > > > > > >>> is
>>>> > > > > > > > > > > >>> > > > >> > > > > StopReplicaRequest
>>>> > > > > > > > > > > >>> > > > >> > > > > > V2
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > needed?
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > Jun
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > On
>>>> > > > > Fri,
>>>> > > > > > > Sep
>>>> > > > > > > > > 11,
>>>> > > > > > > > > > > >>> 2020 at
>>>> > > > > > > > > > > >>> > > > 10:31
>>>> > > > > > > > > > > >>> > > > >> AM
>>>> > > > > > > > > > > >>> > > > >> > > John
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > Roesler <
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> > > > > > vvcephei@apache.org
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > wrote:
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > > > Thanks,
>>>> > > > > > > > > > Justine!
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > Your
>>>> > > > > > > > > response
>>>> > > > > > > > > > > >>> seems
>>>> > > > > > > > > > > >>> > > > >> compelling
>>>> > > > > > > > > > > >>> > > > >> > to
>>>> > > > > > > > > > > >>> > > > >> > > > me.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > > -John
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > On
>>>> > > > > > Fri,
>>>> > > > > > > > > > > >>> 2020-09-11 at
>>>> > > > > > > > > > > >>> > > > 10:17
>>>> > > > > > > > > > > >>> > > > >> > > -0700,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > Justine
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > Olshan
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > wrote:
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > > > Hello
>>>> > > > > > > all,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > > > > Thanks for
>>>> > > > > > > > > > > >>> > continuing
>>>> > > > > > > > > > > >>> > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > discussion! I
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> have a
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > few
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>>>> responses
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > your
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > > > points.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > > Tom:
>>>> > > > > > > You
>>>> > > > > > > > > are
>>>> > > > > > > > > > > >>> correct
>>>> > > > > > > > > > > >>> > > in
>>>> > > > > > > > > > > >>> > > > >> that
>>>> > > > > > > > > > > >>> > > > >> > > this
>>>> > > > > > > > > > > >>> > > > >> > > > > KIP
>>>> > > > > > > > > > > >>> > > > >> > > > > > > has
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> not
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > mentioned
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > > > > > > > > >>> > DeleteTopicsRequest. I
>>>> > > > > > > > > > > >>> > > > >> think
>>>> > > > > > > > > > > >>> > > > >> > > that
>>>> > > > > > > > > > > >>> > > > >> > > > > > this
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> would
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > be
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > out
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > of
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > scope
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > for
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > now,
>>>> > > > > but
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > > may
>>>> > > > > > be
>>>> > > > > > > > > > > something
>>>> > > > > > > > > > > >>> > worth
>>>> > > > > > > > > > > >>> > > > >> adding
>>>> > > > > > > > > > > >>> > > > >> > > in
>>>> > > > > > > > > > > >>> > > > >> > > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> future.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > > > John:
>>>> > > > > > > We
>>>> > > > > > > > > did
>>>> > > > > > > > > > > >>> > consider
>>>> > > > > > > > > > > >>> > > > >> > sequence
>>>> > > > > > > > > > > >>> > > > >> > > > ids,
>>>> > > > > > > > > > > >>> > > > >> > > > > > but
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> there
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > are
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > a
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > few
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> reasons
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > favor
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > > > UUIDs.
>>>> > > > > > > > > There
>>>> > > > > > > > > > > are
>>>> > > > > > > > > > > >>> > > several
>>>> > > > > > > > > > > >>> > > > >> > cases
>>>> > > > > > > > > > > >>> > > > >> > > > > where
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > topics
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > from
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> different
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > clusters
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > may
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > > > > interact
>>>> > > > > > > > > now
>>>> > > > > > > > > > > >>> and in
>>>> > > > > > > > > > > >>> > > the
>>>> > > > > > > > > > > >>> > > > >> > future.
>>>> > > > > > > > > > > >>> > > > >> > > > For
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> example,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > Mirror
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Maker 2
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > may
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > benefit
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > > from
>>>> > > > > > > being
>>>> > > > > > > > > > > able
>>>> > > > > > > > > > > >>> to
>>>> > > > > > > > > > > >>> > > > detect
>>>> > > > > > > > > > > >>> > > > >> > when
>>>> > > > > > > > > > > >>> > > > >> > > a
>>>> > > > > > > > > > > >>> > > > >> > > > > > > cluster
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> being
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > mirrored
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> is
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > deleted
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > and
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > > > > recreated
>>>> > > > > > > > > > and
>>>> > > > > > > > > > > >>> > globally
>>>> > > > > > > > > > > >>> > > > >> unique
>>>> > > > > > > > > > > >>> > > > >> > > > > > > identifiers
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > would
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > make
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> resolving
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>>> > issues
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > > > easier
>>>> > > > > > > > > than
>>>> > > > > > > > > > > >>> sequence
>>>> > > > > > > > > > > >>> > > IDs
>>>> > > > > > > > > > > >>> > > > >> > which
>>>> > > > > > > > > > > >>> > > > >> > > > may
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > collide
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > between
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>>>> clusters.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>>> > > KIP-405
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > > > > (tiered
>>>> > > > > > > > > > > storage)
>>>> > > > > > > > > > > >>> > will
>>>> > > > > > > > > > > >>> > > > also
>>>> > > > > > > > > > > >>> > > > >> > > > benefit
>>>> > > > > > > > > > > >>> > > > >> > > > > > from
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > globally
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > unique
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> IDs
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > as
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>>> > shared
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > > > > buckets
>>>> > > > > > > > > may
>>>> > > > > > > > > > be
>>>> > > > > > > > > > > >>> used
>>>> > > > > > > > > > > >>> > > > >> between
>>>> > > > > > > > > > > >>> > > > >> > > > > clusters.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > > > > Globally
>>>> > > > > > > > > > > unique
>>>> > > > > > > > > > > >>> IDs
>>>> > > > > > > > > > > >>> > > > would
>>>> > > > > > > > > > > >>> > > > >> > also
>>>> > > > > > > > > > > >>> > > > >> > > > make
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > functionality
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > like
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > moving
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>>> > topics
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > > > > between
>>>> > > > > > > > > > > >>> disparate
>>>> > > > > > > > > > > >>> > > > clusters
>>>> > > > > > > > > > > >>> > > > >> > > easier
>>>> > > > > > > > > > > >>> > > > >> > > > > in
>>>> > > > > > > > > > > >>> > > > >> > > > > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > future,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> simplify
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > any
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>>> > future
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > > > > > > > > implementations
>>>> > > > > > > > > > > >>> of
>>>> > > > > > > > > > > >>> > > > backups
>>>> > > > > > > > > > > >>> > > > >> > and
>>>> > > > > > > > > > > >>> > > > >> > > > > > > restores,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> and
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > more.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > In
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> general,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > unique
>>>> > > > > > IDs
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > > > would
>>>> > > > > > > > > ensure
>>>> > > > > > > > > > > >>> that
>>>> > > > > > > > > > > >>> > the
>>>> > > > > > > > > > > >>> > > > >> source
>>>> > > > > > > > > > > >>> > > > >> > > > > cluster
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> topics do
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > not
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> conflict
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> with
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > > > > > > destination
>>>> > > > > > > > > > > >>> cluster
>>>> > > > > > > > > > > >>> > > > >> topics.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > If
>>>> > > > > > we
>>>> > > > > > > were
>>>> > > > > > > > > > to
>>>> > > > > > > > > > > >>> use
>>>> > > > > > > > > > > >>> > > > sequence
>>>> > > > > > > > > > > >>> > > > >> > ids,
>>>> > > > > > > > > > > >>> > > > >> > > > we
>>>> > > > > > > > > > > >>> > > > >> > > > > > > would
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> need
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>>>> sufficiently
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> large
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > cluster
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > > ids
>>>> > > > > > > to be
>>>> > > > > > > > > > > stored
>>>> > > > > > > > > > > >>> > with
>>>> > > > > > > > > > > >>> > > > the
>>>> > > > > > > > > > > >>> > > > >> > topic
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> identifiers or
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > we
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > run
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > risk
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > > > > > > collisions.
>>>> > > > > > > > > > > This
>>>> > > > > > > > > > > >>> > will
>>>> > > > > > > > > > > >>> > > > >> give up
>>>> > > > > > > > > > > >>> > > > >> > > any
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> advantage in
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>>>> compactness
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > sequence
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > > > > numbers
>>>> > > > > > > > > may
>>>> > > > > > > > > > > >>> bring.
>>>> > > > > > > > > > > >>> > > Given
>>>> > > > > > > > > > > >>> > > > >> > these
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > advantages I
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > think
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > it
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> makes
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> sense
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > use
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > > > UUIDs.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > > > Gokul:
>>>> > > > > > > > > This
>>>> > > > > > > > > > is
>>>> > > > > > > > > > > >>> an
>>>> > > > > > > > > > > >>> > > > >> interesting
>>>> > > > > > > > > > > >>> > > > >> > > > idea,
>>>> > > > > > > > > > > >>> > > > >> > > > > > but
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> this
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > is
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > a
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> breaking
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > change.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > Out
>>>> > > > of
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > > > scope
>>>> > > > > > > for
>>>> > > > > > > > > > now,
>>>> > > > > > > > > > > >>> but
>>>> > > > > > > > > > > >>> > > maybe
>>>> > > > > > > > > > > >>> > > > >> > worth
>>>> > > > > > > > > > > >>> > > > >> > > > > > > discussing
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> in
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > future.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > > Hope
>>>> > > > > > > this
>>>> > > > > > > > > > > >>> explains
>>>> > > > > > > > > > > >>> > > some
>>>> > > > > > > > > > > >>> > > > of
>>>> > > > > > > > > > > >>> > > > >> > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > decisions,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > > > > Justine
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > On
>>>> > > > > > > Fri,
>>>> > > > > > > > > Sep
>>>> > > > > > > > > > > 11,
>>>> > > > > > > > > > > >>> 2020
>>>> > > > > > > > > > > >>> > > at
>>>> > > > > > > > > > > >>> > > > >> 8:27
>>>> > > > > > > > > > > >>> > > > >> > AM
>>>> > > > > > > > > > > >>> > > > >> > > > > Gokul
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> Ramanan
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>>>> Subramanian <
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > > > > > > > > >>> > gokul2411s@gmail.com>
>>>> > > > > > > > > > > >>> > > > >> wrote:
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > > Hi.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > > > Thanks
>>>> > > > > > > > > for
>>>> > > > > > > > > > > the
>>>> > > > > > > > > > > >>> > KIP.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > > > Have you
>>>> > > > > > > > > > > >>> thought
>>>> > > > > > > > > > > >>> > > about
>>>> > > > > > > > > > > >>> > > > >> > > whether
>>>> > > > > > > > > > > >>> > > > >> > > > it
>>>> > > > > > > > > > > >>> > > > >> > > > > > > makes
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > sense
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> support
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > > > authorizing a
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > > > > > principal
>>>> > > > > > > > > > > for
>>>> > > > > > > > > > > >>> a
>>>> > > > > > > > > > > >>> > > topic
>>>> > > > > > > > > > > >>> > > > ID
>>>> > > > > > > > > > > >>> > > > >> > > rather
>>>> > > > > > > > > > > >>> > > > >> > > > > > than
>>>> > > > > > > > > > > >>> > > > >> > > > > > > a
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> topic
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > name
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> achieve
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > tighter
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > > > > > security?
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > Or
>>>> > > > > > > is
>>>> > > > > > > > > the
>>>> > > > > > > > > > > >>> topic ID
>>>> > > > > > > > > > > >>> > > > >> > > > fundamentally
>>>> > > > > > > > > > > >>> > > > >> > > > > an
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> internal
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > detail
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > similar
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > epochs
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > used
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > > in a
>>>> > > > > > > > > bunch
>>>> > > > > > > > > > > of
>>>> > > > > > > > > > > >>> > other
>>>> > > > > > > > > > > >>> > > > >> places
>>>> > > > > > > > > > > >>> > > > >> > in
>>>> > > > > > > > > > > >>> > > > >> > > > > > Kafka?
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > > > Thanks.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > On
>>>> > > > > > > Fri,
>>>> > > > > > > > > > Sep
>>>> > > > > > > > > > > >>> 11,
>>>> > > > > > > > > > > >>> > 2020
>>>> > > > > > > > > > > >>> > > > at
>>>> > > > > > > > > > > >>> > > > >> > 4:06
>>>> > > > > > > > > > > >>> > > > >> > > PM
>>>> > > > > > > > > > > >>> > > > >> > > > > > John
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > Roesler <
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>>> > > > > > > > > vvcephei@apache.org
>>>> > > > > > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > > wrote:
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > Hello
>>>> > > > > > > > > > > >>> Justine,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > Thanks
>>>> > > > > > > > > > for
>>>> > > > > > > > > > > >>> the
>>>> > > > > > > > > > > >>> > > KIP!
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > I
>>>> > > > > > > > > happen
>>>> > > > > > > > > > > to
>>>> > > > > > > > > > > >>> have
>>>> > > > > > > > > > > >>> > > > been
>>>> > > > > > > > > > > >>> > > > >> > > > > confronted
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> recently
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > with
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> need
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> keep
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > > track
>>>> > > > > > of
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > a
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > large
>>>> > > > > > > > > > > >>> number of
>>>> > > > > > > > > > > >>> > > > >> topics as
>>>> > > > > > > > > > > >>> > > > >> > > > > > compactly
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > as
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > possible. I
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> was
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> going
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > come
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > up
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > with
>>>> > > > > > > > > > some
>>>> > > > > > > > > > > >>> way to
>>>> > > > > > > > > > > >>> > > > >> > dictionary
>>>> > > > > > > > > > > >>> > > > >> > > > > > encode
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > topic
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > names
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> as
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > > integers,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > but
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > this
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > seems
>>>> > > > > > > > > > much
>>>> > > > > > > > > > > >>> > better!
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > > > > Apologies
>>>> > > > > > > > > > > if
>>>> > > > > > > > > > > >>> > this
>>>> > > > > > > > > > > >>> > > > has
>>>> > > > > > > > > > > >>> > > > >> > been
>>>> > > > > > > > > > > >>> > > > >> > > > > raised
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> before,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > but
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > I’m
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> wondering
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > about
>>>> > > > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > choice
>>>> > > > > > > > > > of
>>>> > > > > > > > > > > >>> UUID
>>>> > > > > > > > > > > >>> > vs
>>>> > > > > > > > > > > >>> > > > >> > sequence
>>>> > > > > > > > > > > >>> > > > >> > > > > number
>>>> > > > > > > > > > > >>> > > > >> > > > > > > for
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > ids.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>>>> Typically,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> I’ve
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > seen
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > > UUIDs
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > in two
>>>> > > > > > > > > > > >>> > situations:
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > 1.
>>>> > > > > > > > > When
>>>> > > > > > > > > > > >>> > processes
>>>> > > > > > > > > > > >>> > > > >> need to
>>>> > > > > > > > > > > >>> > > > >> > > > > > generate
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > non-colliding
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> > > identifiers
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > without
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > > > > > >>> coordination.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > 2.
>>>> > > > > > > > > When
>>>> > > > > > > > > > > the
>>>> > > > > > > > > > > >>> > > > identifier
>>>> > > > > > > > > > > >>> > > > >> > > needs
>>>> > > > > > > > > > > >>> > > > >> > > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > be
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > “universally
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>>>> unique”;
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> I.e.,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > > > > identifier
>>>> > > > > > > > > > > >>> needs
>>>> > > > > > > > > > > >>> > > to
>>>> > > > > > > > > > > >>> > > > >> > > > distinguish
>>>> > > > > > > > > > > >>> > > > >> > > > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> entity
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > from
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > all
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > other
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > entities
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > that
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > could
>>>> > > > > > > > > > ever
>>>> > > > > > > > > > > >>> > exist.
>>>> > > > > > > > > > > >>> > > > >> This is
>>>> > > > > > > > > > > >>> > > > >> > > > > useful
>>>> > > > > > > > > > > >>> > > > >> > > > > > in
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> cases
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > where
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>>>> entities
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> from
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > all
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > > kinds
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > of
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > > > systems
>>>> > > > > > > > > > > get
>>>> > > > > > > > > > > >>> > mixed
>>>> > > > > > > > > > > >>> > > > >> > together,
>>>> > > > > > > > > > > >>> > > > >> > > > > such
>>>> > > > > > > > > > > >>> > > > >> > > > > > as
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> when
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > dumping
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> logs
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > from
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> all
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > > > > processes
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > in
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > a
>>>> > > > > > > > > > company
>>>> > > > > > > > > > > >>> into a
>>>> > > > > > > > > > > >>> > > > >> common
>>>> > > > > > > > > > > >>> > > > >> > > > system.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > Maybe
>>>> > > > > > > > > > I’m
>>>> > > > > > > > > > > >>> being
>>>> > > > > > > > > > > >>> > > > >> > > > short-sighted,
>>>> > > > > > > > > > > >>> > > > >> > > > > > but
>>>> > > > > > > > > > > >>> > > > >> > > > > > > it
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > doesn’t
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > seem
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> like
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> either
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > really
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > > > applies
>>>> > > > > > > > > > > >>> here. It
>>>> > > > > > > > > > > >>> > > > seems
>>>> > > > > > > > > > > >>> > > > >> > like
>>>> > > > > > > > > > > >>> > > > >> > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > brokers
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > could
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > and
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> would
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > achieve
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > > > > consensus
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > when
>>>> > > > > > > > > > > >>> creating a
>>>> > > > > > > > > > > >>> > > > topic
>>>> > > > > > > > > > > >>> > > > >> > > anyway,
>>>> > > > > > > > > > > >>> > > > >> > > > > > which
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > is
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> all
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that’s
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> required
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > > > generate
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > > > > > >>> non-colliding
>>>> > > > > > > > > > > >>> > > > sequence
>>>> > > > > > > > > > > >>> > > > >> > ids.
>>>> > > > > > > > > > > >>> > > > >> > > > For
>>>> > > > > > > > > > > >>> > > > >> > > > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > second,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > as
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > you
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> mention,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> we
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > could
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > > > assign
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > a
>>>> > > > > > > UUID
>>>> > > > > > > > > > to
>>>> > > > > > > > > > > >>> the
>>>> > > > > > > > > > > >>> > > > cluster
>>>> > > > > > > > > > > >>> > > > >> as
>>>> > > > > > > > > > > >>> > > > >> > a
>>>> > > > > > > > > > > >>> > > > >> > > > > whole,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > which
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > would
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > render
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > any
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>>> > > resource
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > > scoped
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > the
>>>> > > > > > > > > > broker
>>>> > > > > > > > > > > >>> > > > universally
>>>> > > > > > > > > > > >>> > > > >> > > unique
>>>> > > > > > > > > > > >>> > > > >> > > > > as
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > well.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > The
>>>> > > > > > > > > > > reason I
>>>> > > > > > > > > > > >>> > > mention
>>>> > > > > > > > > > > >>> > > > >> this
>>>> > > > > > > > > > > >>> > > > >> > > is
>>>> > > > > > > > > > > >>> > > > >> > > > > > that,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > although
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > a
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > UUID
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> is
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > way
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> more
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > > > compact
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > than
>>>> > > > > > > > > > topic
>>>> > > > > > > > > > > >>> > names,
>>>> > > > > > > > > > > >>> > > > it’s
>>>> > > > > > > > > > > >>> > > > >> > > still
>>>> > > > > > > > > > > >>> > > > >> > > > 16
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > bytes.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> In
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > contrast,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> a
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> 4-byte
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > integer
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > > > sequence
>>>> > > > > > > > > > > id
>>>> > > > > > > > > > > >>> > would
>>>> > > > > > > > > > > >>> > > > give
>>>> > > > > > > > > > > >>> > > > >> > us 4
>>>> > > > > > > > > > > >>> > > > >> > > > > > billion
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> unique
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > topics
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> per
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > cluster,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > which
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > > > seems
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > like
>>>> > > > > > > > > > > enough
>>>> > > > > > > > > > > >>> ;)
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > > > > > Considering
>>>> > > > > > > > > > > >>> the
>>>> > > > > > > > > > > >>> > > > >> number of
>>>> > > > > > > > > > > >>> > > > >> > > > > > different
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> times
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > these
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> topic
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > > > identifiers
>>>> > > > > > > > > are
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > > sent
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > over
>>>> > > > > > > > > the
>>>> > > > > > > > > > > >>> wire or
>>>> > > > > > > > > > > >>> > > > >> stored
>>>> > > > > > > > > > > >>> > > > >> > in
>>>> > > > > > > > > > > >>> > > > >> > > > > > memory,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > it
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > seems
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > like
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > it
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > might
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > be
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > worth
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > > > > additional
>>>> > > > > > > > > > > >>> 4x
>>>> > > > > > > > > > > >>> > > space
>>>> > > > > > > > > > > >>> > > > >> > > savings.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > What
>>>> > > > > > > > > do
>>>> > > > > > > > > > > you
>>>> > > > > > > > > > > >>> > think
>>>> > > > > > > > > > > >>> > > > >> about
>>>> > > > > > > > > > > >>> > > > >> > > this?
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > > > Thanks,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > John
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > On
>>>> > > > > > > > > Fri,
>>>> > > > > > > > > > > Sep
>>>> > > > > > > > > > > >>> 11,
>>>> > > > > > > > > > > >>> > > > 2020,
>>>> > > > > > > > > > > >>> > > > >> at
>>>> > > > > > > > > > > >>> > > > >> > > > 03:20,
>>>> > > > > > > > > > > >>> > > > >> > > > > > Tom
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > Bentley
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > wrote:
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > Hi
>>>> > > > > > > > > > > >>> Justine,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > This
>>>> > > > > > > > > > > looks
>>>> > > > > > > > > > > >>> > like
>>>> > > > > > > > > > > >>> > > a
>>>> > > > > > > > > > > >>> > > > >> very
>>>> > > > > > > > > > > >>> > > > >> > > > > welcome
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > improvement.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Thanks!
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > > Maybe
>>>> > > > > > > > > > I
>>>> > > > > > > > > > > >>> missed
>>>> > > > > > > > > > > >>> > > it,
>>>> > > > > > > > > > > >>> > > > >> but
>>>> > > > > > > > > > > >>> > > > >> > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > KIP
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> doesn't
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > seem
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > mention
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>>> > > changing
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > > > > >>> > > > DeleteTopicsRequest
>>>> > > > > > > > > > > >>> > > > >> to
>>>> > > > > > > > > > > >>> > > > >> > > > > identify
>>>> > > > > > > > > > > >>> > > > >> > > > > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > topic
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > using
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > an
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > id.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> Maybe
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > > that's
>>>> > > > > > > out
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > of
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > > scope,
>>>> > > > > > > > > > > but
>>>> > > > > > > > > > > >>> > > > >> > > > > DeleteTopicsRequest
>>>> > > > > > > > > > > >>> > > > >> > > > > > is
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > not
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > listed
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > among
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> Future
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > Work
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > APIs
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > > > either.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > Kind
>>>> > > > > > > > > > > >>> regards,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > Tom
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > On
>>>> > > > > > > > > > Thu,
>>>> > > > > > > > > > > >>> Sep
>>>> > > > > > > > > > > >>> > 10,
>>>> > > > > > > > > > > >>> > > > >> 2020 at
>>>> > > > > > > > > > > >>> > > > >> > > > 3:59
>>>> > > > > > > > > > > >>> > > > >> > > > > PM
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> Satish
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > Duggana <
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > > > > > > > >>> > > > >> satish.duggana@gmail.com
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > > wrote:
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > Thanks
>>>> > > > > > > > > > > >>> > > > >> Lucas/Justine
>>>> > > > > > > > > > > >>> > > > >> > > for
>>>> > > > > > > > > > > >>> > > > >> > > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > nice
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> KIP.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > It
>>>> > > > > > > > > > has
>>>> > > > > > > > > > > >>> > several
>>>> > > > > > > > > > > >>> > > > >> > benefits
>>>> > > > > > > > > > > >>> > > > >> > > > > which
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > also
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > include
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> > simplifying
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > topic
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > > deletion
>>>> > > > > > > > > > > >>> > > process
>>>> > > > > > > > > > > >>> > > > >> by
>>>> > > > > > > > > > > >>> > > > >> > > > > > controller
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > and
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > logs
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > cleanup
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> by
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > brokers
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > in
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > > corner
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > cases.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > Best,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > > Satish.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > On
>>>> > > > > > > > > > > Wed,
>>>> > > > > > > > > > > >>> Sep
>>>> > > > > > > > > > > >>> > 9,
>>>> > > > > > > > > > > >>> > > > >> 2020
>>>> > > > > > > > > > > >>> > > > >> > at
>>>> > > > > > > > > > > >>> > > > >> > > > > 10:07
>>>> > > > > > > > > > > >>> > > > >> > > > > > PM
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > Justine
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Olshan <
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > > > > > > > > >>> jolshan@confluent.io
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > wrote:
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > > > Hello
>>>> > > > > > > > > > > >>> all,
>>>> > > > > > > > > > > >>> > > > it's
>>>> > > > > > > > > > > >>> > > > >> > been
>>>> > > > > > > > > > > >>> > > > >> > > > > > almost a
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> year!
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > I've
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > made
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > some
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>>> > > changes
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > this
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > KIP
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > and
>>>> > > > > > > > > > > >>> hope to
>>>> > > > > > > > > > > >>> > > > >> continue
>>>> > > > > > > > > > > >>> > > > >> > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> discussion.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > > One
>>>> > > > > > > > > > > >>> of the
>>>> > > > > > > > > > > >>> > > > main
>>>> > > > > > > > > > > >>> > > > >> > > changes
>>>> > > > > > > > > > > >>> > > > >> > > > > > I've
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> added
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > is
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > now
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> metadata
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > response
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > > will
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > > include
>>>> > > > > > > > > > > >>> the
>>>> > > > > > > > > > > >>> > > > topic
>>>> > > > > > > > > > > >>> > > > >> ID
>>>> > > > > > > > > > > >>> > > > >> > > (as
>>>> > > > > > > > > > > >>> > > > >> > > > > > Colin
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > suggested).
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Clients
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > can
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > obtain
>>>> > > > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > > > topicID
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > of a
>>>> > > > > > > > > > > >>> given
>>>> > > > > > > > > > > >>> > > topic
>>>> > > > > > > > > > > >>> > > > >> > > through
>>>> > > > > > > > > > > >>> > > > >> > > > a
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
>>>> TopicDescription.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > The
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> topicId
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > will
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > also
>>>> > > > > > be
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > > included
>>>> > > > > > > > > > > >>> > with
>>>> > > > > > > > > > > >>> > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > UpdateMetadata
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > request.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > > Let
>>>> > > > > > > > > > > me
>>>> > > > > > > > > > > >>> > know
>>>> > > > > > > > > > > >>> > > > what
>>>> > > > > > > > > > > >>> > > > >> > you
>>>> > > > > > > > > > > >>> > > > >> > > > all
>>>> > > > > > > > > > > >>> > > > >> > > > > > > think.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > > > Thank
>>>> > > > > > > > > > > >>> you,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > > > >>> Justine
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > On
>>>> > > > > > > > > > > >>> > > 2019/09/13
>>>> > > > > > > > > > > >>> > > > >> > > 16:38:26,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > "Colin
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > McCabe" <
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>>> > > > > > > > > cmccabe@apache.org
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > > wrote:
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > Hi
>>>> > > > > > > > > > > >>> > Lucas,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > > >>> Thanks
>>>> > > > > > > > > > > >>> > for
>>>> > > > > > > > > > > >>> > > > >> > tackling
>>>> > > > > > > > > > > >>> > > > >> > > > > this.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> Topic
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > IDs
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > are a
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > great
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> idea,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > and
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > this
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > is
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > a
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > really
>>>> > > > > > > > > > > >>> good
>>>> > > > > > > > > > > >>> > > > >> writeup.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > > For
>>>> > > > > > > > > > > >>> > > > >> > > > > > > /brokers/topics/[topic],
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > schema
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> version
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> should
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > be
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > > bumped
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > > version
>>>> > > > > > > > > > > >>> 3,
>>>> > > > > > > > > > > >>> > > > rather
>>>> > > > > > > > > > > >>> > > > >> > than
>>>> > > > > > > > > > > >>> > > > >> > > 2.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > KIP-455
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > bumped
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > version
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > this
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > > znode
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > 2
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > > already
>>>> > > > > > > > > > > >>> :)
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > > >>> Given
>>>> > > > > > > > > > > >>> > that
>>>> > > > > > > > > > > >>> > > > >> we're
>>>> > > > > > > > > > > >>> > > > >> > > > going
>>>> > > > > > > > > > > >>> > > > >> > > > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > be
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > seeing
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > these
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > things
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > as
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > strings
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > as
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > > lot
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > (in
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > logs,
>>>> > > > > > > > > > > in
>>>> > > > > > > > > > > >>> > > > >> ZooKeeper,
>>>> > > > > > > > > > > >>> > > > >> > on
>>>> > > > > > > > > > > >>> > > > >> > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > command-line,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > etc.),
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > does
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > it
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > make
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > > sense
>>>> > > > > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > use
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > base64
>>>> > > > > > > > > > > >>> when
>>>> > > > > > > > > > > >>> > > > >> > converting
>>>> > > > > > > > > > > >>> > > > >> > > > them
>>>> > > > > > > > > > > >>> > > > >> > > > > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > strings?
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > > >>> Here is
>>>> > > > > > > > > > > >>> > an
>>>> > > > > > > > > > > >>> > > > >> > example
>>>> > > > > > > > > > > >>> > > > >> > > of
>>>> > > > > > > > > > > >>> > > > >> > > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > hex
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>>>> > representation:
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>>>> > > > > > > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > > And
>>>> > > > > > > > > > > >>> here
>>>> > > > > > > > > > > >>> > > is
>>>> > > > > > > > > > > >>> > > > an
>>>> > > > > > > > > > > >>> > > > >> > > > example
>>>> > > > > > > > > > > >>> > > > >> > > > > in
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> base64.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > b8tRS7h4TJ2Vt43Dp85v2A
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > > The
>>>> > > > > > > > > > > >>> > base64
>>>> > > > > > > > > > > >>> > > > >> > version
>>>> > > > > > > > > > > >>> > > > >> > > > > saves
>>>> > > > > > > > > > > >>> > > > >> > > > > > 15
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > letters
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > (to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > be
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > fair,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > 4
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > those
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > were
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > dashes
>>>> > > > > > > > > > > >>> that
>>>> > > > > > > > > > > >>> > we
>>>> > > > > > > > > > > >>> > > > >> could
>>>> > > > > > > > > > > >>> > > > >> > > have
>>>> > > > > > > > > > > >>> > > > >> > > > > > > elided
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > in
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > hex
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>>> > > > > > > representation.)
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > > >>> Another
>>>> > > > > > > > > > > >>> > > > thing
>>>> > > > > > > > > > > >>> > > > >> to
>>>> > > > > > > > > > > >>> > > > >> > > > > consider
>>>> > > > > > > > > > > >>> > > > >> > > > > > > is
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> that
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > we
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > should
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> specify
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > that
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > > >>> all-zeroes
>>>> > > > > > > > > > > >>> > > UUID
>>>> > > > > > > > > > > >>> > > > is
>>>> > > > > > > > > > > >>> > > > >> > not
>>>> > > > > > > > > > > >>> > > > >> > > a
>>>> > > > > > > > > > > >>> > > > >> > > > > > valid
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> topic
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > UUID.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >  We
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > can't
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > use
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > null
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > for
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > > this
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > > because
>>>> > > > > > > > > > > >>> we
>>>> > > > > > > > > > > >>> > > can't
>>>> > > > > > > > > > > >>> > > > >> > pass a
>>>> > > > > > > > > > > >>> > > > >> > > > > null
>>>> > > > > > > > > > > >>> > > > >> > > > > > > UUID
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> over
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > RPC
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> protocol
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > (there
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > is
>>>> > > > > no
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > > special
>>>> > > > > > > > > > > >>> > > pattern
>>>> > > > > > > > > > > >>> > > > >> for
>>>> > > > > > > > > > > >>> > > > >> > > null,
>>>> > > > > > > > > > > >>> > > > >> > > > > nor
>>>> > > > > > > > > > > >>> > > > >> > > > > > > do
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > we
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > want
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> waste
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > space
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > > reserving
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > > such
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > a
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > > >>> pattern.)
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > > >>> Maybe I
>>>> > > > > > > > > > > >>> > > > missed
>>>> > > > > > > > > > > >>> > > > >> > it,
>>>> > > > > > > > > > > >>> > > > >> > > > but
>>>> > > > > > > > > > > >>> > > > >> > > > > > did
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > you
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > describe
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> "migration
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>>> > of...
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > > > existing
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > > topic[s]
>>>> > > > > > > > > > > >>> > > without
>>>> > > > > > > > > > > >>> > > > >> > topic
>>>> > > > > > > > > > > >>> > > > >> > > > IDs"
>>>> > > > > > > > > > > >>> > > > >> > > > > > in
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> detail
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > in
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > any
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>>>> section?
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > It
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > seems
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > like
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > when
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > the
>>>> > > > > > > > > > > new
>>>> > > > > > > > > > > >>> > > > controller
>>>> > > > > > > > > > > >>> > > > >> > > > becomes
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > active,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> it
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > should
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> just
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > generate
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > random
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > UUIDs
>>>> > > > > > > > > > for
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > these,
>>>> > > > > > > > > > > >>> and
>>>> > > > > > > > > > > >>> > > write
>>>> > > > > > > > > > > >>> > > > >> the
>>>> > > > > > > > > > > >>> > > > >> > > > random
>>>> > > > > > > > > > > >>> > > > >> > > > > > > UUIDs
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> back
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>>>> ZooKeeper.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > It
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > would
>>>> > > > > be
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > > good
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > spell
>>>> > > > > > > > > > > >>> that
>>>> > > > > > > > > > > >>> > > out.
>>>> > > > > > > > > > > >>> > > > >> We
>>>> > > > > > > > > > > >>> > > > >> > > > should
>>>> > > > > > > > > > > >>> > > > >> > > > > > make
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > it
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > clear
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> this
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > happens
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > > > > regardless
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > of
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > the
>>>> > > > > > > > > > > >>> > > inter-broker
>>>> > > > > > > > > > > >>> > > > >> > > protocol
>>>> > > > > > > > > > > >>> > > > >> > > > > > > version
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > (it's
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > a
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>>>> compatible
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>>> > > change).
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > "LeaderAndIsrRequests
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> including an
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> > > > > > is_every_partition
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > flag"
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > > > seems a
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > bit
>>>> > > > > > > > > > > >>> wordy.
>>>> > > > > > > > > > > >>> > > Can
>>>> > > > > > > > > > > >>> > > > we
>>>> > > > > > > > > > > >>> > > > >> > just
>>>> > > > > > > > > > > >>> > > > >> > > > > call
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > these
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > "full
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > > > > > > > >>> LeaderAndIsrRequests"?
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > Then
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > RPC
>>>> > > > > > > > > > > >>> field
>>>> > > > > > > > > > > >>> > > could
>>>> > > > > > > > > > > >>> > > > be
>>>> > > > > > > > > > > >>> > > > >> > > named
>>>> > > > > > > > > > > >>> > > > >> > > > > > > "full".
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > Also,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > it
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > would
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> probably
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > be
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > > better
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > for
>>>> > > > > > > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > RPC
>>>> > > > > > > > > > > >>> field to
>>>> > > > > > > > > > > >>> > > be
>>>> > > > > > > > > > > >>> > > > an
>>>> > > > > > > > > > > >>> > > > >> > enum
>>>> > > > > > > > > > > >>> > > > >> > > > of
>>>> > > > > > > > > > > >>> > > > >> > > > > {
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > UNSPECIFIED,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> > INCREMENTAL,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> FULL
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > },
>>>> > > > so
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > > that
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > we
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > can
>>>> > > > > > > > > > > >>> cleanly
>>>> > > > > > > > > > > >>> > > > handle
>>>> > > > > > > > > > > >>> > > > >> > old
>>>> > > > > > > > > > > >>> > > > >> > > > > > versions
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > (by
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > treating
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> them
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > as
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > > > UNSPECIFIED)
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > In
>>>> > > > > > > > > > > >>> the
>>>> > > > > > > > > > > >>> > > > >> > > > > > LeaderAndIsrRequest
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > section,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > you
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> write
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > "A
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> final
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > > > deletion
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > event
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > will
>>>> > > > > > > > > > > be
>>>> > > > > > > > > > > >>> > > > secheduled
>>>> > > > > > > > > > > >>> > > > >> > for
>>>> > > > > > > > > > > >>> > > > >> > > X
>>>> > > > > > > > > > > >>> > > > >> > > > ms
>>>> > > > > > > > > > > >>> > > > >> > > > > > > after
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> > > > > > > LeaderAndIsrRequest
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > was
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > > first
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > > >>> received..."
>>>> > > > > > > > > > > >>> > > I
>>>> > > > > > > > > > > >>> > > > >> guess
>>>> > > > > > > > > > > >>> > > > >> > > > the X
>>>> > > > > > > > > > > >>> > > > >> > > > > > > was a
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > placeholder
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> that
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > you
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > intended
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > > > replace
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > before
>>>> > > > > > > > > > > >>> > > posting?
>>>> > > > > > > > > > > >>> > > > :)
>>>> > > > > > > > > > > >>> > > > >> > In
>>>> > > > > > > > > > > >>> > > > >> > > > any
>>>> > > > > > > > > > > >>> > > > >> > > > > > > case,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> this
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > seems
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > like
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> kind
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > of
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > > thing
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > > we'd
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > want a
>>>> > > > > > > > > > > >>> > > > >> configuration
>>>> > > > > > > > > > > >>> > > > >> > > for.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > Let's
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > describe
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>>> > > > > > configuration
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > key
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > > > > somewhere
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > in
>>>> > > > > > > > > > > this
>>>> > > > > > > > > > > >>> KIP,
>>>> > > > > > > > > > > >>> > > > >> > including
>>>> > > > > > > > > > > >>> > > > >> > > > what
>>>> > > > > > > > > > > >>> > > > >> > > > > > its
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > default
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > value
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> is.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > We
>>>> > > > > > > > > > > >>> > should
>>>> > > > > > > > > > > >>> > > > >> > probably
>>>> > > > > > > > > > > >>> > > > >> > > > also
>>>> > > > > > > > > > > >>> > > > >> > > > > > > log a
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > bunch
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > of
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> messages
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > at
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> WARN
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > level
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > > when
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > > >>> something is
>>>> > > > > > > > > > > >>> > > > >> > scheduled
>>>> > > > > > > > > > > >>> > > > >> > > > for
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> deletion,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > as
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > well.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > (Maybe
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> this
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > was
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > > > assumed,
>>>> > > > > > > > > > > but
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > it
>>>> > > > > > > > > > > >>> would be
>>>> > > > > > > > > > > >>> > > good
>>>> > > > > > > > > > > >>> > > > >> to
>>>> > > > > > > > > > > >>> > > > >> > > > mention
>>>> > > > > > > > > > > >>> > > > >> > > > > > > it).
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > I
>>>> > > > > > > > > > > >>> feel
>>>> > > > > > > > > > > >>> > > like
>>>> > > > > > > > > > > >>> > > > >> there
>>>> > > > > > > > > > > >>> > > > >> > > > are a
>>>> > > > > > > > > > > >>> > > > >> > > > > > few
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > sections
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > should
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > be
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > moved
>>>> > > > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > > > > "rejected
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > > >>> > alternatives."
>>>> > > > > > > > > > > >>> > > > >> For
>>>> > > > > > > > > > > >>> > > > >> > > > > example,
>>>> > > > > > > > > > > >>> > > > >> > > > > > in
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > DeleteTopics
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> section,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > since
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > > we're
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > not
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > going
>>>> > > > > > > > > > > >>> to do
>>>> > > > > > > > > > > >>> > > > >> option 1
>>>> > > > > > > > > > > >>> > > > >> > or
>>>> > > > > > > > > > > >>> > > > >> > > > 2,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > these
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > should
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > be
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > moved
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > into
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > > "rejected
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > > >>> > alternatives,"
>>>> > > > > > > > > > > >>> > > > >> > rather
>>>> > > > > > > > > > > >>> > > > >> > > > than
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> appearing
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > inline.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> Another
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> case
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > is
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > > > "Should
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > we
>>>> > > > > > > > > > > >>> remove
>>>> > > > > > > > > > > >>> > > topic
>>>> > > > > > > > > > > >>> > > > >> name
>>>> > > > > > > > > > > >>> > > > >> > > from
>>>> > > > > > > > > > > >>> > > > >> > > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> protocol
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > where
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> possible"
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > section.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > This
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > is
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > > clearly
>>>> > > > > > > > > > > >>> > > > >> discussing a
>>>> > > > > > > > > > > >>> > > > >> > > > design
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > alternative
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > that
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> we're
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > not
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > > proposing
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > > >>> implement:
>>>> > > > > > > > > > > >>> > > > >> removing
>>>> > > > > > > > > > > >>> > > > >> > the
>>>> > > > > > > > > > > >>> > > > >> > > > > topic
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > name
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > from
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > those
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> > protocols.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > Is
>>>> > > > > > > > > > > >>> it
>>>> > > > > > > > > > > >>> > > really
>>>> > > > > > > > > > > >>> > > > >> > > > necessary
>>>> > > > > > > > > > > >>> > > > >> > > > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > have
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> a
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > new
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > > > > > > > > >>> > > /admin/delete_topics_by_id
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > path
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > in
>>>> > > > > > > > > > > >>> > ZooKeeper?
>>>> > > > > > > > > > > >>> > > > It
>>>> > > > > > > > > > > >>> > > > >> > seems
>>>> > > > > > > > > > > >>> > > > >> > > > > like
>>>> > > > > > > > > > > >>> > > > >> > > > > > we
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> don't
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > really
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> need
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > this.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > Whenever
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > there
>>>> > > > > > > > > is
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > a
>>>> > > > > > > > > > new
>>>> > > > > > > > > > > >>> > > > controller,
>>>> > > > > > > > > > > >>> > > > >> > we'll
>>>> > > > > > > > > > > >>> > > > >> > > > > send
>>>> > > > > > > > > > > >>> > > > >> > > > > > > out
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> full
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > > > > > > LeaderAndIsrRequests
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > which
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > will
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > > trigger
>>>> > > > > > > > > > > >>> the
>>>> > > > > > > > > > > >>> > > > stale
>>>> > > > > > > > > > > >>> > > > >> > > topics
>>>> > > > > > > > > > > >>> > > > >> > > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > be
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> cleaned
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > up.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >  The
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> active
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > > > > controller
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > > will
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > also
>>>> > > > > > > > > > > >>> send
>>>> > > > > > > > > > > >>> > the
>>>> > > > > > > > > > > >>> > > > full
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> LeaderAndIsrRequest
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> brokers
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> are
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > just
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > > > starting
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > up.
>>>> > > > > > > > > > > >>> So we
>>>> > > > > > > > > > > >>> > > > don't
>>>> > > > > > > > > > > >>> > > > >> > > really
>>>> > > > > > > > > > > >>> > > > >> > > > > > need
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > this
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > kind
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > of
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>>>> two-phase
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>>> > commit
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > > (send
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > > out
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > > >>> > > > >> StopReplicasRequest,
>>>> > > > > > > > > > > >>> > > > >> > > get
>>>> > > > > > > > > > > >>> > > > >> > > > > ACKs
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > from
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> all
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > nodes,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > commit
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > by
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > removing
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > > >>> > > > >> /admin/delete_topics
>>>> > > > > > > > > > > >>> > > > >> > > > node)
>>>> > > > > > > > > > > >>> > > > >> > > > > > any
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> more.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > > You
>>>> > > > > > > > > > > >>> > > mention
>>>> > > > > > > > > > > >>> > > > >> that
>>>> > > > > > > > > > > >>> > > > >> > > > > > > FetchRequest
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> will
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > now
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> include
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > UUID
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > avoid
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > > > issues
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > where
>>>> > > > > > > > > > > >>> > requests
>>>> > > > > > > > > > > >>> > > > are
>>>> > > > > > > > > > > >>> > > > >> > made
>>>> > > > > > > > > > > >>> > > > >> > > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > stale
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > partitions.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> However,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > adding a
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > UUID
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > > >>> > > MetadataRequest
>>>> > > > > > > > > > > >>> > > > is
>>>> > > > > > > > > > > >>> > > > >> > > listed
>>>> > > > > > > > > > > >>> > > > >> > > > > as
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > future
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > work,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > out
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > of
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > scope
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> for
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > this
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > KIP.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > How
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > will
>>>> > > > > > > > > > > the
>>>> > > > > > > > > > > >>> > > client
>>>> > > > > > > > > > > >>> > > > >> learn
>>>> > > > > > > > > > > >>> > > > >> > > > what
>>>> > > > > > > > > > > >>> > > > >> > > > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> topic
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > UUID
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > is,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > if
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>>> > > metadata
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > > > response
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > > doesn't
>>>> > > > > > > > > > > >>> > > include
>>>> > > > > > > > > > > >>> > > > >> that
>>>> > > > > > > > > > > >>> > > > >> > > > > > > information?
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> It
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > seems
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > like
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> adding
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > UUID
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > > >>> > > MetadataResponse
>>>> > > > > > > > > > > >>> > > > >> > would
>>>> > > > > > > > > > > >>> > > > >> > > be
>>>> > > > > > > > > > > >>> > > > >> > > > > an
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > improvement
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > here
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> that
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> might
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > not
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > be
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > too
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > hard
>>>> > > > > > > > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > make.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > > >>> best,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > > >>> Colin
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > On
>>>> > > > > > > > > > > >>> Mon,
>>>> > > > > > > > > > > >>> > > Sep
>>>> > > > > > > > > > > >>> > > > 9,
>>>> > > > > > > > > > > >>> > > > >> > > 2019,
>>>> > > > > > > > > > > >>> > > > >> > > > at
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > 17:48,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > Ryanne
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Dolan
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> wrote:
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > >
>>>> > > > > > > > > > > >>> Lucas,
>>>> > > > > > > > > > > >>> > > > this
>>>> > > > > > > > > > > >>> > > > >> > would
>>>> > > > > > > > > > > >>> > > > >> > > > be
>>>> > > > > > > > > > > >>> > > > >> > > > > > > great.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> I've
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > run
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > into
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> issues
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> with
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > > topics
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > being
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > >
>>>> > > > > > > > > > > >>> > > > resurrected
>>>> > > > > > > > > > > >>> > > > >> > > > > > accidentally,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> since
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > a
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > client
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > cannot
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>>> > easily
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > > > > > distinguish
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > > between
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > >
>>>> > > > > > > > > > > a
>>>> > > > > > > > > > > >>> > > deleted
>>>> > > > > > > > > > > >>> > > > >> topic
>>>> > > > > > > > > > > >>> > > > >> > > > and a
>>>> > > > > > > > > > > >>> > > > >> > > > > > new
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> topic
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > with
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> same
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> name.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > I'd
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > need
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > ID
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > >
>>>> > > > > > > > > > > >>> > > accessible
>>>> > > > > > > > > > > >>> > > > >> from
>>>> > > > > > > > > > > >>> > > > >> > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > client
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > solve
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > issue,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > but
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > this
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > is a
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > good
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > first
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > >
>>>> > > > > > > > > > > >>> step.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > >
>>>> > > > > > > > > > > >>> Ryanne
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > >
>>>> > > > > > > > > > > On
>>>> > > > > > > > > > > >>> > Wed,
>>>> > > > > > > > > > > >>> > > > Sep
>>>> > > > > > > > > > > >>> > > > >> 4,
>>>> > > > > > > > > > > >>> > > > >> > > 2019
>>>> > > > > > > > > > > >>> > > > >> > > > > at
>>>> > > > > > > > > > > >>> > > > >> > > > > > > 1:41
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> PM
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > Lucas
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> > Bradstreet <
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > > > > > >>> > > lucas@confluent.io>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > wrote:
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > >
>>>> > > > > > > > > > > >
>>>> > > > > > > > > > > >>> Hi
>>>> > > > > > > > > > > >>> > > all,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > >
>>>> > > > > > > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > >
>>>> > > > > > > > > > > >
>>>> > > > > > > > > > > >>> I
>>>> > > > > > > > > > > >>> > > would
>>>> > > > > > > > > > > >>> > > > >> like
>>>> > > > > > > > > > > >>> > > > >> > to
>>>> > > > > > > > > > > >>> > > > >> > > > > kick
>>>> > > > > > > > > > > >>> > > > >> > > > > > > off
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > discussion
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > of
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> KIP-516,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> an
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > > > > > >>> implementation
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > of
>>>> > > > > > > > > > > topic
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > >
>>>> > > > > > > > > > > >
>>>> > > > > > > > > > > >>> IDs
>>>> > > > > > > > > > > >>> > > for
>>>> > > > > > > > > > > >>> > > > >> > Kafka.
>>>> > > > > > > > > > > >>> > > > >> > > > > Topic
>>>> > > > > > > > > > > >>> > > > >> > > > > > > IDs
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> aim
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > solve
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> topic
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > > uniqueness
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > > > problems
>>>> > > > > > > > > > > in
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > Kafka,
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > >
>>>> > > > > > > > > > > >
>>>> > > > > > > > > > > >>> > where
>>>> > > > > > > > > > > >>> > > > >> > referring
>>>> > > > > > > > > > > >>> > > > >> > > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > a
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > topic
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> by
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > name
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> alone is
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > > > insufficient.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > > Such
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > cases
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > >
>>>> > > > > > > > > > > >
>>>> > > > > > > > > > > >>> > > include
>>>> > > > > > > > > > > >>> > > > >> when
>>>> > > > > > > > > > > >>> > > > >> > a
>>>> > > > > > > > > > > >>> > > > >> > > > > topic
>>>> > > > > > > > > > > >>> > > > >> > > > > > > has
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> been
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > deleted
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > > recreated
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > with
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > the
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > > same
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > name.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > >
>>>> > > > > > > > > > > >
>>>> > > > > > > > > > > >>> > Unique
>>>> > > > > > > > > > > >>> > > > >> > > > identifiers
>>>> > > > > > > > > > > >>> > > > >> > > > > > will
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> help
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > simplify
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> improve
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > Kafka's
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > > > topic
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > > deletion
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > >
>>>> > > > > > > > > > > >
>>>> > > > > > > > > > > >>> > > process,
>>>> > > > > > > > > > > >>> > > > >> as
>>>> > > > > > > > > > > >>> > > > >> > > well
>>>> > > > > > > > > > > >>> > > > >> > > > as
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > prevent
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > cases
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > where
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> brokers
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> may
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > > > > > > incorrectly
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > >
>>>> > > > > > > > > > > interact
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > >
>>>> > > > > > > > > > > >
>>>> > > > > > > > > > > >>> with
>>>> > > > > > > > > > > >>> > > > stale
>>>> > > > > > > > > > > >>> > > > >> > > > versions
>>>> > > > > > > > > > > >>> > > > >> > > > > > of
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> topics.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > >
>>>> > > > > > > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > >
>>>> > > > > > > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > >
>>>> > > > > > > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > >
>>>> > > > > > > > > > > >>> > > > >> > > >
>>>> > > > > > > > > > > >>> > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> >
>>>> > > > > > > > > > > >>> > > > >>
>>>> > > > > > > > > > > >>> > > >
>>>> > > > > > > > > > > >>> > >
>>>> > > > > > > > > > > >>> >
>>>> > > > > > > > > > > >>>
>>>> > > > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > >
>>>> > > > > > >
>>>> > > > > >
>>>> > > > >
>>>> > > >
>>>> > >
>>>> >
>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > >
>>>> > > > > > > > > > > >
>>>> > > > > > > > > > > >>> > > Looking
>>>> > > > > > > > > > > >>> > > > >> > forward
>>>> > > > > > > > > > > >>> > > > >> > > > to
>>>> > > > > > > > > > > >>> > > > >> > > > > > your
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > thoughts.
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > >
>>>> > > > > > > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > >
>>>> > > > > > > > > > > >
>>>> > > > > > > > > > > >>> > Lucas
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > >
>>>> > > > >
>>>> > > > > >
>>>> > > > > > >
>>>> > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > > >
>>>> > > > > > > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > >>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >>
>>>> > > > > > > > > > > >>> > > > >> > > > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > > >
>>>> > > > > > > > > > > >>> > > > >> > > > >
>>>> > > > > > > > > > > >>> > > > >> > > >
>>>> > > > > > > > > > > >>> > > > >> > >
>>>> > > > > > > > > > > >>> > > > >> >
>>>> > > > > > > > > > > >>> > > > >>
>>>> > > > > > > > > > > >>> > > > >
>>>> > > > > > > > > > > >>> > > >
>>>> > > > > > > > > > > >>> > >
>>>> > > > > > > > > > > >>> >
>>>> > > > > > > > > > > >>>
>>>> > > > > > > > > > > >>
>>>> > > > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > >
>>>> > > > > > >
>>>> > > > > > >
>>>> > > > > >
>>>> > > > >
>>>> > > >
>>>> > >
>>>> >
>>>>
>>>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Justine Olshan <jo...@confluent.io>]

Hi all,

I'm moving my response to the VOTE thread here as to not start filling the
vote thread with more discussion.

Here's what was said:




*1. It seems that we should remove the topic field from
theStopReplicaResponse below?StopReplica Response (Version: 4) =>
error_code [topics]  error_code => INT16    topics => topic topic_id*
[partitions]*

*Fixed*

*2. "*After controller* election, upon receiving the result, assign the*
*metadata topic its unique topic ID". Will the UUID for the metadata topic*
*be written to the metadata topic itself?*

* I believe so. The topic ID would be written to ZooKeeper, so when
ZooKeeper is replaced, it makes sense to write the topic ID for the
metadata topic in the metadata topic.*

*3. The vote request is designed to support multiple topics, each of them*
*may require a different sentinel ID. Should we reserve more than one*
*sentinel ID for future usage?*


*The plan was that only the metadata topic will use the sentinel ID before
the controller is elected. Once the controller is elected, the metadata
topic will simply use a unique id, so I believe these other topics can just
use a unique ID given by the controller.*
*4. UUID.randomUUID(): Could we clarify whether this method returns any*
*sentinel ID? Also, how do we expect the user to use it?*


*The method does not return the sentinel ID. I wasn't sure if this was
something worth mentioning since the sentinel ID is only used internally,
and it is only one out of many values that could be obtained. (In normal
use, the user would not notice this value is missing.) I thought this
method would be useful for testing, since a user might not want to
specify their own UUID values.*

Thanks,
Justine


On Fri, Oct 9, 2020 at 2:00 PM Justine Olshan <jo...@confluent.io> wrote:

> Hi all,
>
> I think it is almost time to vote on this KIP. Please let me know if there
> are any outstanding issues.
>
> Thank you,
> Justine
>
> On Fri, Oct 9, 2020 at 7:56 AM Justine Olshan <jo...@confluent.io>
> wrote:
>
>> Hi Jun,
>>
>> 50. Right. That's what I was thinking but wanted to make sure. Works for
>> me!
>>
>> Thanks again,
>> Justine
>>
>> On Thu, Oct 8, 2020 at 3:50 PM Jun Rao <ju...@confluent.io> wrote:
>>
>>> Hi, Justine,
>>>
>>> 50. We can't change the format of existing versions since it will break
>>> compatibility. Since we are bumping the version for MetadataRequest
>>> and DeleteTopicsRequest, we can make the topic name field nullable from
>>> the
>>> new version.
>>>
>>> Thanks,
>>>
>>> Jun
>>>
>>> On Thu, Oct 8, 2020 at 2:19 PM Justine Olshan <jo...@confluent.io>
>>> wrote:
>>>
>>> > Hi Jun,
>>> >
>>> > 50. Works for me. Is the idea that it will be nullable from this
>>> version
>>> > onwards, but not past versions?
>>> >
>>> > 51. Added some information and example methods for AdminClient future
>>> work
>>> >
>>> > Thanks,
>>> > Justine
>>> >
>>> >
>>> > On Thu, Oct 8, 2020 at 10:59 AM Jun Rao <ju...@confluent.io> wrote:
>>> >
>>> > > Hi, Justine,
>>> > >
>>> > > The changes sound reasonable to me.
>>> > >
>>> > > 50. Should we make the topic name in the new MetadataRequest
>>> > > and DeleteTopicsRequest nullable?
>>> > >
>>> > > 51. In the future work section, perhaps we could mention
>>> opportunities
>>> > for
>>> > > using topicId in other methods in AdminClient where topic name is
>>> used.
>>> > >
>>> > > Thanks,
>>> > >
>>> > > Jun
>>> > >
>>> > > On Wed, Oct 7, 2020 at 2:44 PM Justine Olshan <jo...@confluent.io>
>>> > > wrote:
>>> > >
>>> > > > Hi all,
>>> > > >
>>> > > > Another update on the KIP. I've added some information about how
>>> topic
>>> > > IDs
>>> > > > will be supported in AdminClient
>>> > > >
>>> > > >
>>> > > >
>>> > >
>>> >
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-AdminClientSupport
>>> > > >
>>> > > >
>>> > > > One specific decision I wanted to focus on is the inclusion of both
>>> > topic
>>> > > > names and topic Ids in metadata and deleteTopics requests.
>>> > > >
>>> > > > The idea is that when the request is received the topic id field is
>>> > > checked
>>> > > > first, but if it contains the default (zero) uuid, then the topic
>>> name
>>> > > > field will be used.
>>> > > >
>>> > > > In the case where ids are used, if the ids do not have a
>>> corresponding
>>> > > > topic, the request will fail. In the case where both ids and names
>>> are
>>> > > > included, the request will fail if the name does not match the id.
>>> > > >
>>> > > >
>>> > > > Initially the id field will be used to support the admin client use
>>> > cases
>>> > > > as described in the KIP, but over time, we can transfer to using
>>> topic
>>> > > ids
>>> > > > in all uses of the request.
>>> > > >
>>> > > >
>>> > > > There will also be some support for returning the topic Id on the
>>> > > > createtopics, deletetopics and metadata responses. I think it makes
>>> > sense
>>> > > > to include both the name and the id in the successful response for
>>> > these
>>> > > > cases.
>>> > > >
>>> > > >
>>> > > > Let me know if you have any comments or concerns regarding the
>>> Admin
>>> > > Client
>>> > > > changes.
>>> > > >
>>> > > > On Wed, Oct 7, 2020 at 11:05 AM Jun Rao <ju...@confluent.io> wrote:
>>> > > >
>>> > > > > Hi, Justine,
>>> > > > >
>>> > > > > I was just wondering if we expose topicId publicly, how would
>>> people
>>> > > use
>>> > > > > it. One possibility is to use topicId in all those public
>>> interfaces
>>> > > > where
>>> > > > > we use topic name. I am not sure if the need is very strong
>>> though.
>>> > So,
>>> > > > > maybe we can list this as future considerations.
>>> > > > >
>>> > > > > Thanks,
>>> > > > >
>>> > > > > Jun
>>> > > > >
>>> > > > > On Wed, Oct 7, 2020 at 8:57 AM Justine Olshan <
>>> jolshan@confluent.io>
>>> > > > > wrote:
>>> > > > >
>>> > > > > > Hi all,
>>> > > > > >
>>> > > > > > After some careful consideration, I've decided that the
>>> > > > TopicIdPartition
>>> > > > > > will be out of scope for this KIP. After investigating what it
>>> will
>>> > > > look
>>> > > > > > like using it in only the Fetch Request, I found that I would
>>> need
>>> > to
>>> > > > > > modify several public APIs in the consumer and I would need to
>>> take
>>> > > > > special
>>> > > > > > care in many situations to handle the request correctly based
>>> on
>>> > what
>>> > > > > > version of the request was sent. For these reasons, I've added
>>> > > > > > TopicIdPartition to potential future work along with an
>>> alternative
>>> > > to
>>> > > > > > modify TopicPartition to include a topic ID field.
>>> > > > > >
>>> > > > > > Jun- I was also wondering about some of the details for
>>> supporting
>>> > > > topic
>>> > > > > > IDs for delete and describe topics in the admin client. Is the
>>> idea
>>> > > > that
>>> > > > > > instead of supplying topic names to describe or delete, we
>>> would
>>> > > > supply a
>>> > > > > > collection of ids? If that is the case, I'm wondering if it
>>> makes
>>> > > sense
>>> > > > > to
>>> > > > > > add topic Ids to the metadata and delete topics requests.
>>> > > > > >
>>> > > > > > Thanks,
>>> > > > > > Justine
>>> > > > > >
>>> > > > > > On Mon, Oct 5, 2020 at 9:25 PM Satish Duggana <
>>> > > > satish.duggana@gmail.com>
>>> > > > > > wrote:
>>> > > > > >
>>> > > > > > > Hi Jun, Justine,
>>> > > > > > > +1 to have TopicIdPartition in public packages. This is
>>> useful
>>> > for
>>> > > > > > > KIP-405 tiered storage too. We currently use TopicPartition
>>> for
>>> > > > > > > pluggable log storage and metadata storage apis. We will be
>>> using
>>> > > > > > > TopicIdPartition once the proposed changes in this KIP are
>>> > merged.
>>> > > > > > >
>>> > > > > > > Thanks,
>>> > > > > > > Satish.
>>> > > > > > >
>>> > > > > > > On Tue, Oct 6, 2020 at 5:21 AM Jun Rao <ju...@confluent.io>
>>> wrote:
>>> > > > > > > >
>>> > > > > > > > Hi, Justine,
>>> > > > > > > >
>>> > > > > > > > I guess we have to maintain the mapping between topic name
>>> and
>>> > > > > topicId
>>> > > > > > in
>>> > > > > > > > both the client and the server. If TopicIdPartition is
>>> meant
>>> > for
>>> > > > > > internal
>>> > > > > > > > usage only, we can create it under a package
>>> > > > > > > > like org/apache/kafka/common/internal and we don't need to
>>> > > document
>>> > > > > it
>>> > > > > > in
>>> > > > > > > > the KIP. If it's meant for external public usage (and
>>> > potentially
>>> > > > > > > internal
>>> > > > > > > > usage too), we can create it in org/apache/kafka/common and
>>> > need
>>> > > to
>>> > > > > > > > document it in the KIP.
>>> > > > > > > >
>>> > > > > > > > Since topicId is now part of the topic metadata, perhaps
>>> it's
>>> > > > useful
>>> > > > > to
>>> > > > > > > > expose it in TopicMetadataAndConfig too.
>>> > > > > > > >
>>> > > > > > > > Thanks,
>>> > > > > > > >
>>> > > > > > > > Jun
>>> > > > > > > >
>>> > > > > > > > On Mon, Oct 5, 2020 at 3:35 PM Justine Olshan <
>>> > > > jolshan@confluent.io>
>>> > > > > > > wrote:
>>> > > > > > > >
>>> > > > > > > > > Hi Jun,
>>> > > > > > > > >
>>> > > > > > > > > This KIP removes the topic name from many requests. A
>>> lot of
>>> > > > > requests
>>> > > > > > > keep
>>> > > > > > > > > track of the partitions internally with a
>>> TopicPartition. So
>>> > > when
>>> > > > > we
>>> > > > > > > remove
>>> > > > > > > > > the topic name, we can not differentiate unless we can
>>> also
>>> > use
>>> > > > > topic
>>> > > > > > > ID.
>>> > > > > > > > > The Fetch request relies heavily on this, but ListOffset,
>>> > > > > > > > > OffsetsForLeaderEpoch, etc all use TopicPartition as a
>>> key
>>> > > value
>>> > > > > for
>>> > > > > > > > > internal maps. Digging deeper into Fetch, I've noticed
>>> that
>>> > the
>>> > > > > code
>>> > > > > > > path
>>> > > > > > > > > runs into some public APIs that use the TopicPartition.
>>> Some
>>> > > > > examples
>>> > > > > > > are
>>> > > > > > > > > Cluster.java or RecordTooLargeException.
>>> > > > > > > > >
>>> > > > > > > > > A related issue is that in some places like
>>> > > > > > > TopicAuthorizationException,
>>> > > > > > > > > the topic name is required and without the name in the
>>> > request,
>>> > > > we
>>> > > > > > > won't
>>> > > > > > > > > know it.
>>> > > > > > > > >
>>> > > > > > > > > I guess an alternative would be to have some mapping from
>>> > topic
>>> > > > > name
>>> > > > > > to
>>> > > > > > > > > topic ID, but I would have to think about that more.
>>> > > > > > > > >
>>> > > > > > > > > The KIP mentions exposing TopicId in TopicDescription.
>>> (The
>>> > > font
>>> > > > > was
>>> > > > > > > > > weirdly colored, so I've made it easier to read.)
>>> > > > > > > > >
>>> > > > > > > > > Would the idea for the TopicMetadataAndConfig be that the
>>> > topic
>>> > > > ID
>>> > > > > > > would be
>>> > > > > > > > > returned as a result?
>>> > > > > > > > > As for the other methods describe/deletetopics, it makes
>>> > sense
>>> > > to
>>> > > > > > > include
>>> > > > > > > > > them at some point, but I'm not sure how far we should
>>> go in
>>> > > > terms
>>> > > > > of
>>> > > > > > > > > supporting topic IDs everywhere.
>>> > > > > > > > >
>>> > > > > > > > > Thank you for continuing to look at the KIP!
>>> > > > > > > > > Justine
>>> > > > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > On Mon, Oct 5, 2020 at 2:53 PM Jun Rao <jun@confluent.io
>>> >
>>> > > wrote:
>>> > > > > > > > >
>>> > > > > > > > > > Hi, Justine,
>>> > > > > > > > > >
>>> > > > > > > > > > Adding a TopicIdPartition in the public interface may
>>> be
>>> > > > useful.
>>> > > > > > > Where
>>> > > > > > > > > > would you expect it to be used?
>>> > > > > > > > > >
>>> > > > > > > > > > Related to this, since we are exposing topicId in the
>>> tool,
>>> > > it
>>> > > > > > seems
>>> > > > > > > that
>>> > > > > > > > > > we may want to expose topicId in at least
>>> TopicDescription
>>> > > used
>>> > > > > by
>>> > > > > > > > > > AdminClient.describeTopics() and TopicMetadataAndConfig
>>> > used
>>> > > in
>>> > > > > > > > > > AdminClient.createTopics(Collection<NewTopic>
>>> newTopics). I
>>> > > am
>>> > > > > also
>>> > > > > > > not
>>> > > > > > > > > > sure if we plan to expose topicId in other AdminClient
>>> > > methods
>>> > > > in
>>> > > > > > > this
>>> > > > > > > > > KIP
>>> > > > > > > > > > or in some future KIP. For example, it might be useful
>>> to
>>> > > > support
>>> > > > > > > topicId
>>> > > > > > > > > > in the following methods.
>>> > > > > > > > > >
>>> > > > > > > > > >     DescribeTopicsResult
>>> describeTopics(Collection<String>
>>> > > > > > > topicNames,
>>> > > > > > > > > > DescribeTopicsOptions options);
>>> > > > > > > > > >     DeleteTopicsResult deleteTopics(Collection<String>
>>> > > topics,
>>> > > > > > > > > > DeleteTopicsOptions options);
>>> > > > > > > > > >
>>> > > > > > > > > > Thanks,
>>> > > > > > > > > >
>>> > > > > > > > > > Jun
>>> > > > > > > > > >
>>> > > > > > > > > > On Mon, Oct 5, 2020 at 11:03 AM Justine Olshan <
>>> > > > > > jolshan@confluent.io
>>> > > > > > > >
>>> > > > > > > > > > wrote:
>>> > > > > > > > > >
>>> > > > > > > > > > > Hi all,
>>> > > > > > > > > > >
>>> > > > > > > > > > > I was thinking about this KIP some more. We are
>>> removing
>>> > > the
>>> > > > > > topic
>>> > > > > > > name
>>> > > > > > > > > > > from many of these request types, and some of them
>>> (eg.
>>> > > > Fetch)
>>> > > > > > use
>>> > > > > > > > > > > TopicPartition as keys in a map when handling the
>>> > request.
>>> > > I
>>> > > > > was
>>> > > > > > > > > thinking
>>> > > > > > > > > > > it would make sense to make some sort of
>>> TopicIdPartition
>>> > > > with
>>> > > > > > > topic
>>> > > > > > > > > > name,
>>> > > > > > > > > > > topic id, and partition fields (name and id would
>>> > > potentially
>>> > > > > be
>>> > > > > > > > > optional
>>> > > > > > > > > > > types since different request versions will contain
>>> only
>>> > > one
>>> > > > of
>>> > > > > > the
>>> > > > > > > > > two)
>>> > > > > > > > > > I
>>> > > > > > > > > > > was wondering if it made sense to include this as a
>>> > public
>>> > > > > facing
>>> > > > > > > API
>>> > > > > > > > > > > similar to TopicPartition. If so, I will add this
>>> class
>>> > to
>>> > > > the
>>> > > > > > KIP
>>> > > > > > > as
>>> > > > > > > > > > well.
>>> > > > > > > > > > >
>>> > > > > > > > > > > Thanks,
>>> > > > > > > > > > > Justine
>>> > > > > > > > > > >
>>> > > > > > > > > > > On Mon, Oct 5, 2020 at 8:52 AM Justine Olshan <
>>> > > > > > > jolshan@confluent.io>
>>> > > > > > > > > > > wrote:
>>> > > > > > > > > > >
>>> > > > > > > > > > > > Hi all,
>>> > > > > > > > > > > > I've made a few minor changes to the UUID class.
>>> I've
>>> > > > added a
>>> > > > > > few
>>> > > > > > > > > more
>>> > > > > > > > > > > > methods to support UUIDs more generally, by
>>> including a
>>> > > > > > > constructor
>>> > > > > > > > > > and a
>>> > > > > > > > > > > > method to generate random UUIDs. Both of these
>>> will be
>>> > > > useful
>>> > > > > > for
>>> > > > > > > > > > > testing.
>>> > > > > > > > > > > >
>>> > > > > > > > > > > > Let me know if there are any other areas for
>>> > discussion.
>>> > > > > > > > > > > >
>>> > > > > > > > > > > > Thank you,
>>> > > > > > > > > > > > Justine
>>> > > > > > > > > > > >
>>> > > > > > > > > > > > On Thu, Oct 1, 2020 at 5:29 PM Justine Olshan <
>>> > > > > > > jolshan@confluent.io>
>>> > > > > > > > > > > > wrote:
>>> > > > > > > > > > > >
>>> > > > > > > > > > > >> Hi Jun,
>>> > > > > > > > > > > >> Thanks for looking at it again. Here's the new
>>> spec.
>>> > (I
>>> > > > > fixed
>>> > > > > > > the
>>> > > > > > > > > typo
>>> > > > > > > > > > > in
>>> > > > > > > > > > > >> it too.)
>>> > > > > > > > > > > >> {"name": "id", "type": "string", "doc":
>>> option[UUID]}
>>> > > > > > > > > > > >>
>>> > > > > > > > > > > >> Justine
>>> > > > > > > > > > > >>
>>> > > > > > > > > > > >>
>>> > > > > > > > > > > >> On Thu, Oct 1, 2020 at 5:03 PM Jun Rao <
>>> > > jun@confluent.io>
>>> > > > > > > wrote:
>>> > > > > > > > > > > >>
>>> > > > > > > > > > > >>> Hi, Justine,
>>> > > > > > > > > > > >>>
>>> > > > > > > > > > > >>> Thanks for the update. The KIP looks good to me
>>> now.
>>> > > > Just a
>>> > > > > > > minor
>>> > > > > > > > > > > comment
>>> > > > > > > > > > > >>> below.
>>> > > > > > > > > > > >>>
>>> > > > > > > > > > > >>> 30. Perhaps "option[UUID]" can be put in the doc.
>>> > > > > > > > > > > >>>
>>> > > > > > > > > > > >>> Jun
>>> > > > > > > > > > > >>>
>>> > > > > > > > > > > >>> On Thu, Oct 1, 2020 at 3:28 PM Justine Olshan <
>>> > > > > > > > > jolshan@confluent.io>
>>> > > > > > > > > > > >>> wrote:
>>> > > > > > > > > > > >>>
>>> > > > > > > > > > > >>> > Hi Jun,
>>> > > > > > > > > > > >>> > Thanks for the response!
>>> > > > > > > > > > > >>> >
>>> > > > > > > > > > > >>> > 30. I think I might have changed this in
>>> between.
>>> > The
>>> > > > > > current
>>> > > > > > > > > > version
>>> > > > > > > > > > > >>> > says:  {"name":
>>> > > > > > > > > > > >>> > "id", "type": "option[UUID]"}, "doc": topic id}
>>> > > > > > > > > > > >>> > I have switched to the option type to cover the
>>> > > > migration
>>> > > > > > > case
>>> > > > > > > > > > where
>>> > > > > > > > > > > a
>>> > > > > > > > > > > >>> > TopicZNode does not yet have a topic ID.
>>> > > > > > > > > > > >>> > I understand that due to json, this field is
>>> > written
>>> > > > as a
>>> > > > > > > string,
>>> > > > > > > > > > so
>>> > > > > > > > > > > >>> if I
>>> > > > > > > > > > > >>> > should move the "option[uuid]" to the doc
>>> field and
>>> > > the
>>> > > > > > type
>>> > > > > > > > > should
>>> > > > > > > > > > > be
>>> > > > > > > > > > > >>> > "string" please let me know.
>>> > > > > > > > > > > >>> >
>>> > > > > > > > > > > >>> > 40. I've added a definition for UUID.
>>> > > > > > > > > > > >>> > 41,42. Fixed
>>> > > > > > > > > > > >>> >
>>> > > > > > > > > > > >>> > Thank you,
>>> > > > > > > > > > > >>> > Justine
>>> > > > > > > > > > > >>> >
>>> > > > > > > > > > > >>> > On Wed, Sep 30, 2020 at 1:15 PM Jun Rao <
>>> > > > > jun@confluent.io>
>>> > > > > > > > > wrote:
>>> > > > > > > > > > > >>> >
>>> > > > > > > > > > > >>> > > Hi, Justine,
>>> > > > > > > > > > > >>> > >
>>> > > > > > > > > > > >>> > > Thanks for the summary. Just a few minor
>>> comments
>>> > > > blow.
>>> > > > > > > > > > > >>> > >
>>> > > > > > > > > > > >>> > > 30.  {"name": "id", "type": "string", "doc":
>>> > > "version
>>> > > > > > > id"}}:
>>> > > > > > > > > The
>>> > > > > > > > > > > doc
>>> > > > > > > > > > > >>> > should
>>> > > > > > > > > > > >>> > > say UUID. The issue seems unfixed.
>>> > > > > > > > > > > >>> > >
>>> > > > > > > > > > > >>> > > 40. Since UUID is public facing, could you
>>> > include
>>> > > > its
>>> > > > > > > > > > definition?
>>> > > > > > > > > > > >>> > >
>>> > > > > > > > > > > >>> > > 41. StopReplicaResponse still includes the
>>> topic
>>> > > > field.
>>> > > > > > > > > > > >>> > >
>>> > > > > > > > > > > >>> > > 42. "It is unnecessary to include the name
>>> of the
>>> > > > topic
>>> > > > > > in
>>> > > > > > > the
>>> > > > > > > > > > > >>> following
>>> > > > > > > > > > > >>> > > Request/Response calls" It would be useful to
>>> > > include
>>> > > > > all
>>> > > > > > > > > > modified
>>> > > > > > > > > > > >>> > requests
>>> > > > > > > > > > > >>> > > (e.g. produce) in the list below.
>>> > > > > > > > > > > >>> > >
>>> > > > > > > > > > > >>> > > Thanks,
>>> > > > > > > > > > > >>> > >
>>> > > > > > > > > > > >>> > > Jun
>>> > > > > > > > > > > >>> > >
>>> > > > > > > > > > > >>> > >
>>> > > > > > > > > > > >>> > > On Wed, Sep 30, 2020 at 10:17 AM Justine
>>> Olshan <
>>> > > > > > > > > > > >>> jolshan@confluent.io>
>>> > > > > > > > > > > >>> > > wrote:
>>> > > > > > > > > > > >>> > >
>>> > > > > > > > > > > >>> > > > Hello again,
>>> > > > > > > > > > > >>> > > >
>>> > > > > > > > > > > >>> > > > I've taken some time to discuss some of the
>>> > > > remaining
>>> > > > > > > points
>>> > > > > > > > > > > >>> brought up
>>> > > > > > > > > > > >>> > > by
>>> > > > > > > > > > > >>> > > > the previous emails offline. Here are some
>>> of
>>> > the
>>> > > > > > > > > conclusions.
>>> > > > > > > > > > > >>> > > >
>>> > > > > > > > > > > >>> > > > 1. Directory Structure:
>>> > > > > > > > > > > >>> > > > There was some talk about whether the
>>> directory
>>> > > > > > structure
>>> > > > > > > > > > should
>>> > > > > > > > > > > be
>>> > > > > > > > > > > >>> > > changed
>>> > > > > > > > > > > >>> > > > to replace all topic names with topic IDs.
>>> > > > > > > > > > > >>> > > > This will be a useful change, but will
>>> prevent
>>> > > > > > > downgrades. It
>>> > > > > > > > > > > will
>>> > > > > > > > > > > >>> be
>>> > > > > > > > > > > >>> > > best
>>> > > > > > > > > > > >>> > > > to wait until a major release, likely
>>> alongside
>>> > > > > KIP-500
>>> > > > > > > > > changes
>>> > > > > > > > > > > >>> that
>>> > > > > > > > > > > >>> > will
>>> > > > > > > > > > > >>> > > > prevent downgrades. I've updated the KIP to
>>> > > include
>>> > > > > > this
>>> > > > > > > > > change
>>> > > > > > > > > > > >>> with
>>> > > > > > > > > > > >>> > the
>>> > > > > > > > > > > >>> > > > note about migration and deprecation.
>>> > > > > > > > > > > >>> > > >
>>> > > > > > > > > > > >>> > > >
>>> > > > > > > > > > > >>> > >
>>> > > > > > > > > > > >>> >
>>> > > > > > > > > > > >>>
>>> > > > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-log.dirlayout
>>> > > > > > > > > > > >>> > > >
>>> > > > > > > > > > > >>> > > > 2. Partition Metadata file
>>> > > > > > > > > > > >>> > > > There was some disagreement about the
>>> inclusion
>>> > > of
>>> > > > > the
>>> > > > > > > > > > partition
>>> > > > > > > > > > > >>> > metadata
>>> > > > > > > > > > > >>> > > > file.
>>> > > > > > > > > > > >>> > > > This file will be needed to persist the
>>> topic
>>> > ID,
>>> > > > > > > especially
>>> > > > > > > > > > > while
>>> > > > > > > > > > > >>> we
>>> > > > > > > > > > > >>> > > still
>>> > > > > > > > > > > >>> > > > have the old directory structure. Even
>>> after
>>> > the
>>> > > > > > changes,
>>> > > > > > > > > this
>>> > > > > > > > > > > >>> file can
>>> > > > > > > > > > > >>> > > be
>>> > > > > > > > > > > >>> > > > useful for debugging and recovery.
>>> > > > > > > > > > > >>> > > > Creating a single mapping file from topic
>>> names
>>> > > to
>>> > > > > > topic
>>> > > > > > > IDs
>>> > > > > > > > > > was
>>> > > > > > > > > > > >>> > > > considered, but ultimately scrapped as it
>>> would
>>> > > not
>>> > > > > be
>>> > > > > > as
>>> > > > > > > > > easy
>>> > > > > > > > > > to
>>> > > > > > > > > > > >>> > > maintain.
>>> > > > > > > > > > > >>> > > >
>>> > > > > > > > > > > >>> > > >
>>> > > > > > > > > > > >>> > >
>>> > > > > > > > > > > >>> >
>>> > > > > > > > > > > >>>
>>> > > > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PartitionMetadatafile
>>> > > > > > > > > > > >>> > > >
>>> > > > > > > > > > > >>> > > >
>>> > > > > > > > > > > >>> > > >
>>> > > > > > > > > > > >>> > >
>>> > > > > > > > > > > >>> >
>>> > > > > > > > > > > >>>
>>> > > > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PersistingTopicIDs
>>> > > > > > > > > > > >>> > > >
>>> > > > > > > > > > > >>> > > > 3. Produce Protocols
>>> > > > > > > > > > > >>> > > > After some further discussion, this
>>> replacing
>>> > the
>>> > > > > topic
>>> > > > > > > name
>>> > > > > > > > > > with
>>> > > > > > > > > > > >>> topic
>>> > > > > > > > > > > >>> > > ID
>>> > > > > > > > > > > >>> > > > in these protocols has been added to the
>>> KIP.
>>> > > > > > > > > > > >>> > > > This will cut down on the size of the
>>> protocol.
>>> > > > Since
>>> > > > > > > changes
>>> > > > > > > > > > to
>>> > > > > > > > > > > >>> fetch
>>> > > > > > > > > > > >>> > > are
>>> > > > > > > > > > > >>> > > > included, it does make sense to update
>>> these
>>> > > > > protocols.
>>> > > > > > > > > > > >>> > > >
>>> > > > > > > > > > > >>> > > >
>>> > > > > > > > > > > >>> > >
>>> > > > > > > > > > > >>> >
>>> > > > > > > > > > > >>>
>>> > > > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-Produce
>>> > > > > > > > > > > >>> > > >
>>> > > > > > > > > > > >>> > > > 4. KIP-500 Compatibility
>>> > > > > > > > > > > >>> > > > After some discussion with those involved
>>> with
>>> > > > > KIP-500,
>>> > > > > > > it
>>> > > > > > > > > > seems
>>> > > > > > > > > > > >>> best
>>> > > > > > > > > > > >>> > to
>>> > > > > > > > > > > >>> > > > use a sentinel topic ID for the metadata
>>> topic
>>> > > that
>>> > > > > is
>>> > > > > > > used
>>> > > > > > > > > > > before
>>> > > > > > > > > > > >>> the
>>> > > > > > > > > > > >>> > > > first controller election.
>>> > > > > > > > > > > >>> > > > We had an issue where this metadata topic
>>> may
>>> > not
>>> > > > be
>>> > > > > > > assigned
>>> > > > > > > > > > an
>>> > > > > > > > > > > ID
>>> > > > > > > > > > > >>> > > before
>>> > > > > > > > > > > >>> > > > utilizing the Vote and Fetch protocols. It
>>> was
>>> > > > > decided
>>> > > > > > to
>>> > > > > > > > > > > reserve a
>>> > > > > > > > > > > >>> > > unique
>>> > > > > > > > > > > >>> > > > ID for this topic to be used until the
>>> > controller
>>> > > > > could
>>> > > > > > > give
>>> > > > > > > > > > the
>>> > > > > > > > > > > >>> topic
>>> > > > > > > > > > > >>> > a
>>> > > > > > > > > > > >>> > > > unique ID.
>>> > > > > > > > > > > >>> > > > Having the topic keep the sentinel ID (not
>>> be
>>> > > > > > reassigned
>>> > > > > > > to a
>>> > > > > > > > > > > >>> unique
>>> > > > > > > > > > > >>> > ID)
>>> > > > > > > > > > > >>> > > > was considered, but it seemed like a good
>>> idea
>>> > to
>>> > > > > leave
>>> > > > > > > the
>>> > > > > > > > > > > option
>>> > > > > > > > > > > >>> open
>>> > > > > > > > > > > >>> > > for
>>> > > > > > > > > > > >>> > > > the metadata topic to have a unique ID in
>>> cases
>>> > > > where
>>> > > > > > it
>>> > > > > > > > > would
>>> > > > > > > > > > > >>> need to
>>> > > > > > > > > > > >>> > be
>>> > > > > > > > > > > >>> > > > differentiated from other clusters'
>>> metadata
>>> > > > topics.
>>> > > > > > (ex.
>>> > > > > > > > > > tiered
>>> > > > > > > > > > > >>> > storage)
>>> > > > > > > > > > > >>> > > >
>>> > > > > > > > > > > >>> > > >
>>> > > > > > > > > > > >>> > >
>>> > > > > > > > > > > >>> >
>>> > > > > > > > > > > >>>
>>> > > > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-CompatibilitywithKIP-500
>>> > > > > > > > > > > >>> > > >
>>> > > > > > > > > > > >>> > > > I've also split up the KIP into sub-tasks
>>> on
>>> > the
>>> > > > > JIRA.
>>> > > > > > > > > > Hopefully
>>> > > > > > > > > > > >>> this
>>> > > > > > > > > > > >>> > > will
>>> > > > > > > > > > > >>> > > > provide a better idea about what tasks we
>>> have,
>>> > > and
>>> > > > > > > > > eventually
>>> > > > > > > > > > > >>> provide
>>> > > > > > > > > > > >>> > a
>>> > > > > > > > > > > >>> > > > place to see what's done and what is left.
>>> > > > > > > > > > > >>> > > > If there is a task I am missing, please
>>> let me
>>> > > > know!
>>> > > > > > > > > > > >>> > > >
>>> > https://issues.apache.org/jira/browse/KAFKA-8872
>>> > > > > > > > > > > >>> > > >
>>> > > > > > > > > > > >>> > > > Of course, these decisions are not set in
>>> > stone,
>>> > > > and
>>> > > > > I
>>> > > > > > > would
>>> > > > > > > > > > love
>>> > > > > > > > > > > >>> to
>>> > > > > > > > > > > >>> > hear
>>> > > > > > > > > > > >>> > > > any feedback.
>>> > > > > > > > > > > >>> > > >
>>> > > > > > > > > > > >>> > > > Thanks,
>>> > > > > > > > > > > >>> > > > Justine
>>> > > > > > > > > > > >>> > > >
>>> > > > > > > > > > > >>> > > > On Mon, Sep 28, 2020 at 11:38 AM Justine
>>> > Olshan <
>>> > > > > > > > > > > >>> jolshan@confluent.io>
>>> > > > > > > > > > > >>> > > > wrote:
>>> > > > > > > > > > > >>> > > >
>>> > > > > > > > > > > >>> > > > > Hello all,
>>> > > > > > > > > > > >>> > > > >
>>> > > > > > > > > > > >>> > > > > I just wanted to follow up on this
>>> > discussion.
>>> > > > Did
>>> > > > > we
>>> > > > > > > come
>>> > > > > > > > > to
>>> > > > > > > > > > > an
>>> > > > > > > > > > > >>> > > > > understanding about the directory
>>> structure?
>>> > > > > > > > > > > >>> > > > >
>>> > > > > > > > > > > >>> > > > > I think the biggest question here is
>>> what is
>>> > > > > > > acceptable to
>>> > > > > > > > > > > leave
>>> > > > > > > > > > > >>> out
>>> > > > > > > > > > > >>> > > due
>>> > > > > > > > > > > >>> > > > > to scope vs. what is considered to be too
>>> > much
>>> > > > tech
>>> > > > > > > debt.
>>> > > > > > > > > > > >>> > > > > This KIP is already pretty large with the
>>> > > number
>>> > > > of
>>> > > > > > > > > changes,
>>> > > > > > > > > > > but
>>> > > > > > > > > > > >>> it
>>> > > > > > > > > > > >>> > > also
>>> > > > > > > > > > > >>> > > > > makes sense to do things correctly, so
>>> I'd
>>> > love
>>> > > > to
>>> > > > > > hear
>>> > > > > > > > > > > >>> everyone's
>>> > > > > > > > > > > >>> > > > thoughts.
>>> > > > > > > > > > > >>> > > > >
>>> > > > > > > > > > > >>> > > > > Thanks,
>>> > > > > > > > > > > >>> > > > > Justine
>>> > > > > > > > > > > >>> > > > >
>>> > > > > > > > > > > >>> > > > > On Fri, Sep 25, 2020 at 8:19 AM Lucas
>>> > > Bradstreet
>>> > > > <
>>> > > > > > > > > > > >>> lucas@confluent.io
>>> > > > > > > > > > > >>> > >
>>> > > > > > > > > > > >>> > > > > wrote:
>>> > > > > > > > > > > >>> > > > >
>>> > > > > > > > > > > >>> > > > >> Hi Ismael,
>>> > > > > > > > > > > >>> > > > >>
>>> > > > > > > > > > > >>> > > > >> If you do not store it in a metadata
>>> file or
>>> > > in
>>> > > > > the
>>> > > > > > > > > > directory
>>> > > > > > > > > > > >>> > > structure
>>> > > > > > > > > > > >>> > > > >> would you then
>>> > > > > > > > > > > >>> > > > >> require the LeaderAndIsrRequest
>>> following
>>> > > > startup
>>> > > > > to
>>> > > > > > > give
>>> > > > > > > > > > you
>>> > > > > > > > > > > >>> some
>>> > > > > > > > > > > >>> > > > notion
>>> > > > > > > > > > > >>> > > > >> of
>>> > > > > > > > > > > >>> > > > >> topic name in memory? We would still
>>> need
>>> > this
>>> > > > > > > information
>>> > > > > > > > > > for
>>> > > > > > > > > > > >>> the
>>> > > > > > > > > > > >>> > > older
>>> > > > > > > > > > > >>> > > > >> protocols, but
>>> > > > > > > > > > > >>> > > > >> perhaps this is what's meant by tech
>>> debt.
>>> > > > > > > > > > > >>> > > > >>
>>> > > > > > > > > > > >>> > > > >> Once we're free of the old non-topicID
>>> > > requests
>>> > > > > > then I
>>> > > > > > > > > think
>>> > > > > > > > > > > you
>>> > > > > > > > > > > >>> > > > wouldn't
>>> > > > > > > > > > > >>> > > > >> need to retain the topic name.
>>> > > > > > > > > > > >>> > > > >> I think the ability to easily look up
>>> topic
>>> > > > names
>>> > > > > > > > > associated
>>> > > > > > > > > > > >>> with
>>> > > > > > > > > > > >>> > > > >> partition
>>> > > > > > > > > > > >>> > > > >> directories would still be missed
>>> > > > > > > > > > > >>> > > > >> when diagnosing issues, though maybe it
>>> > > wouldn't
>>> > > > > be
>>> > > > > > a
>>> > > > > > > deal
>>> > > > > > > > > > > >>> breaker
>>> > > > > > > > > > > >>> > > with
>>> > > > > > > > > > > >>> > > > >> the
>>> > > > > > > > > > > >>> > > > >> right tooling.
>>> > > > > > > > > > > >>> > > > >>
>>> > > > > > > > > > > >>> > > > >> Thanks,
>>> > > > > > > > > > > >>> > > > >>
>>> > > > > > > > > > > >>> > > > >> Lucas
>>> > > > > > > > > > > >>> > > > >>
>>> > > > > > > > > > > >>> > > > >> On Fri, Sep 25, 2020 at 7:55 AM Ismael
>>> Juma
>>> > <
>>> > > > > > > > > > > ismael@juma.me.uk>
>>> > > > > > > > > > > >>> > > wrote:
>>> > > > > > > > > > > >>> > > > >>
>>> > > > > > > > > > > >>> > > > >> > Hi Lucas,
>>> > > > > > > > > > > >>> > > > >> >
>>> > > > > > > > > > > >>> > > > >> > Why would you include the name and
>>> id? I
>>> > > think
>>> > > > > > you'd
>>> > > > > > > > > want
>>> > > > > > > > > > to
>>> > > > > > > > > > > >>> > remove
>>> > > > > > > > > > > >>> > > > the
>>> > > > > > > > > > > >>> > > > >> > name from the directory name right?
>>> > Jason's
>>> > > > > > > suggestion
>>> > > > > > > > > was
>>> > > > > > > > > > > >>> that if
>>> > > > > > > > > > > >>> > > you
>>> > > > > > > > > > > >>> > > > >> > remove the name from the directory,
>>> then
>>> > why
>>> > > > > would
>>> > > > > > > you
>>> > > > > > > > > > need
>>> > > > > > > > > > > >>> the id
>>> > > > > > > > > > > >>> > > > name
>>> > > > > > > > > > > >>> > > > >> > mapping file?
>>> > > > > > > > > > > >>> > > > >> >
>>> > > > > > > > > > > >>> > > > >> > Ismael
>>> > > > > > > > > > > >>> > > > >> >
>>> > > > > > > > > > > >>> > > > >> > On Thu, Sep 24, 2020 at 4:24 PM Lucas
>>> > > > > Bradstreet <
>>> > > > > > > > > > > >>> > > lucas@confluent.io>
>>> > > > > > > > > > > >>> > > > >> > wrote:
>>> > > > > > > > > > > >>> > > > >> >
>>> > > > > > > > > > > >>> > > > >> > > > 2. Part of the usage of the file
>>> is to
>>> > > > have
>>> > > > > > > > > persistent
>>> > > > > > > > > > > >>> storage
>>> > > > > > > > > > > >>> > > of
>>> > > > > > > > > > > >>> > > > >> the
>>> > > > > > > > > > > >>> > > > >> > > topic
>>> > > > > > > > > > > >>> > > > >> > > ID and use it to compare with the ID
>>> > > > supplied
>>> > > > > in
>>> > > > > > > the
>>> > > > > > > > > > > >>> > LeaderAndIsr
>>> > > > > > > > > > > >>> > > > >> > Request.
>>> > > > > > > > > > > >>> > > > >> > > There is some discussion in the KIP
>>> > about
>>> > > > > > changes
>>> > > > > > > to
>>> > > > > > > > > the
>>> > > > > > > > > > > >>> > directory
>>> > > > > > > > > > > >>> > > > >> > > structure, but I believe directory
>>> > changes
>>> > > > > were
>>> > > > > > > > > > considered
>>> > > > > > > > > > > >>> to be
>>> > > > > > > > > > > >>> > > out
>>> > > > > > > > > > > >>> > > > >> of
>>> > > > > > > > > > > >>> > > > >> > > scope when the KIP was written.
>>> > > > > > > > > > > >>> > > > >> > >
>>> > > > > > > > > > > >>> > > > >> > >
>>> > > > > > > > > > > >>> > > > >> > > Yeah, I was hoping to get a better
>>> > > > > understanding
>>> > > > > > > of
>>> > > > > > > > > why
>>> > > > > > > > > > it
>>> > > > > > > > > > > >>> was
>>> > > > > > > > > > > >>> > > taken
>>> > > > > > > > > > > >>> > > > >> out
>>> > > > > > > > > > > >>> > > > >> > of
>>> > > > > > > > > > > >>> > > > >> > > scope. Perhaps Lucas Bradstreet
>>> might
>>> > have
>>> > > > > more
>>> > > > > > > > > insight
>>> > > > > > > > > > > >>> about
>>> > > > > > > > > > > >>> > the
>>> > > > > > > > > > > >>> > > > >> > decision.
>>> > > > > > > > > > > >>> > > > >> > > Basically my point is that we have
>>> to
>>> > > create
>>> > > > > > > > > additional
>>> > > > > > > > > > > >>> > > > infrastructure
>>> > > > > > > > > > > >>> > > > >> > here
>>> > > > > > > > > > > >>> > > > >> > > to support the name/id mapping, so I
>>> > > wanted
>>> > > > to
>>> > > > > > > > > > understand
>>> > > > > > > > > > > >>> if we
>>> > > > > > > > > > > >>> > > just
>>> > > > > > > > > > > >>> > > > >> > > consider this a sort of tech debt.
>>> It
>>> > > would
>>> > > > be
>>> > > > > > > useful
>>> > > > > > > > > to
>>> > > > > > > > > > > >>> cover
>>> > > > > > > > > > > >>> > how
>>> > > > > > > > > > > >>> > > > we
>>> > > > > > > > > > > >>> > > > >> > > handle the case when this file gets
>>> > > > corrupted.
>>> > > > > > > Seems
>>> > > > > > > > > > like
>>> > > > > > > > > > > we
>>> > > > > > > > > > > >>> > just
>>> > > > > > > > > > > >>> > > > >> have to
>>> > > > > > > > > > > >>> > > > >> > > trust that it matches whatever the
>>> > > > controller
>>> > > > > > > tells us
>>> > > > > > > > > > and
>>> > > > > > > > > > > >>> > rewrite
>>> > > > > > > > > > > >>> > > > it?
>>> > > > > > > > > > > >>> > > > >> > >
>>> > > > > > > > > > > >>> > > > >> > >
>>> > > > > > > > > > > >>> > > > >> > > Hi Jason, Justine,
>>> > > > > > > > > > > >>> > > > >> > >
>>> > > > > > > > > > > >>> > > > >> > > My thought process is that we will
>>> > likely
>>> > > > need
>>> > > > > > the
>>> > > > > > > > > > > metadata
>>> > > > > > > > > > > >>> file
>>> > > > > > > > > > > >>> > > > >> whether
>>> > > > > > > > > > > >>> > > > >> > we
>>> > > > > > > > > > > >>> > > > >> > > rename the directories or not.
>>> > > > > > > > > > > >>> > > > >> > > Linux supports filenames of up to
>>> 255
>>> > > bytes
>>> > > > > and
>>> > > > > > > that
>>> > > > > > > > > > would
>>> > > > > > > > > > > >>> not
>>> > > > > > > > > > > >>> > be
>>> > > > > > > > > > > >>> > > > >> enough
>>> > > > > > > > > > > >>> > > > >> > to
>>> > > > > > > > > > > >>> > > > >> > > support a directory name
>>> > > > > > > > > > > >>> > > > >> > >  including both the name and topic
>>> ID.
>>> > > Given
>>> > > > > > that
>>> > > > > > > > > fact,
>>> > > > > > > > > > it
>>> > > > > > > > > > > >>> > seemed
>>> > > > > > > > > > > >>> > > > >> > > reasonable to add the metadata file
>>> > > > > > > > > > > >>> > > > >> > > and leave the directory rename until
>>> > some
>>> > > > time
>>> > > > > > in
>>> > > > > > > the
>>> > > > > > > > > > > future
>>> > > > > > > > > > > >>> > > > (possibly
>>> > > > > > > > > > > >>> > > > >> > > never).
>>> > > > > > > > > > > >>> > > > >> > >
>>> > > > > > > > > > > >>> > > > >> > > If the file does get corrupted I
>>> think
>>> > > > you're
>>> > > > > > > right.
>>> > > > > > > > > We
>>> > > > > > > > > > > >>> would
>>> > > > > > > > > > > >>> > > either
>>> > > > > > > > > > > >>> > > > >> have
>>> > > > > > > > > > > >>> > > > >> > > to trust it matches what the
>>> controller
>>> > > > tells
>>> > > > > us
>>> > > > > > > > > > > >>> > > > >> > >  or error out and let an
>>> administrator
>>> > > > resolve
>>> > > > > > it
>>> > > > > > > by
>>> > > > > > > > > > > >>> checking
>>> > > > > > > > > > > >>> > > across
>>> > > > > > > > > > > >>> > > > >> > > replicas for consistency.
>>> > > > > > > > > > > >>> > > > >> > >
>>> > > > > > > > > > > >>> > > > >> > > Lucas
>>> > > > > > > > > > > >>> > > > >> > >
>>> > > > > > > > > > > >>> > > > >> > >
>>> > > > > > > > > > > >>> > > > >> > > On Thu, Sep 24, 2020 at 3:41 PM
>>> Jason
>>> > > > > Gustafson
>>> > > > > > <
>>> > > > > > > > > > > >>> > > jason@confluent.io
>>> > > > > > > > > > > >>> > > > >
>>> > > > > > > > > > > >>> > > > >> > > wrote:
>>> > > > > > > > > > > >>> > > > >> > >
>>> > > > > > > > > > > >>> > > > >> > > > Thanks Justine. Responses below:
>>> > > > > > > > > > > >>> > > > >> > > >
>>> > > > > > > > > > > >>> > > > >> > > > > 1. Yes, the directory will
>>> still be
>>> > > > based
>>> > > > > on
>>> > > > > > > the
>>> > > > > > > > > > topic
>>> > > > > > > > > > > >>> > names.
>>> > > > > > > > > > > >>> > > > >> > > > LeaderAndIsrRequest is one of the
>>> few
>>> > > > > requests
>>> > > > > > > that
>>> > > > > > > > > > will
>>> > > > > > > > > > > >>> still
>>> > > > > > > > > > > >>> > > > >> contain
>>> > > > > > > > > > > >>> > > > >> > > the
>>> > > > > > > > > > > >>> > > > >> > > > topic name. So I think we have
>>> this
>>> > > > covered.
>>> > > > > > > Sorry
>>> > > > > > > > > for
>>> > > > > > > > > > > >>> > > confusion.
>>> > > > > > > > > > > >>> > > > >> > > >
>>> > > > > > > > > > > >>> > > > >> > > > Ah, you're right. My eyes passed
>>> right
>>> > > > over
>>> > > > > > the
>>> > > > > > > > > field.
>>> > > > > > > > > > > >>> > > > >> > > >
>>> > > > > > > > > > > >>> > > > >> > > > > 2. Part of the usage of the
>>> file is
>>> > to
>>> > > > > have
>>> > > > > > > > > > persistent
>>> > > > > > > > > > > >>> > storage
>>> > > > > > > > > > > >>> > > > of
>>> > > > > > > > > > > >>> > > > >> the
>>> > > > > > > > > > > >>> > > > >> > > > topic
>>> > > > > > > > > > > >>> > > > >> > > > ID and use it to compare with the
>>> ID
>>> > > > > supplied
>>> > > > > > > in the
>>> > > > > > > > > > > >>> > > LeaderAndIsr
>>> > > > > > > > > > > >>> > > > >> > > Request.
>>> > > > > > > > > > > >>> > > > >> > > > There is some discussion in the
>>> KIP
>>> > > about
>>> > > > > > > changes to
>>> > > > > > > > > > the
>>> > > > > > > > > > > >>> > > directory
>>> > > > > > > > > > > >>> > > > >> > > > structure, but I believe directory
>>> > > changes
>>> > > > > > were
>>> > > > > > > > > > > >>> considered to
>>> > > > > > > > > > > >>> > be
>>> > > > > > > > > > > >>> > > > >> out of
>>> > > > > > > > > > > >>> > > > >> > > > scope when the KIP was written.
>>> > > > > > > > > > > >>> > > > >> > > >
>>> > > > > > > > > > > >>> > > > >> > > > Yeah, I was hoping to get a better
>>> > > > > > > understanding of
>>> > > > > > > > > > why
>>> > > > > > > > > > > >>> it was
>>> > > > > > > > > > > >>> > > > taken
>>> > > > > > > > > > > >>> > > > >> > out
>>> > > > > > > > > > > >>> > > > >> > > of
>>> > > > > > > > > > > >>> > > > >> > > > scope. Perhaps Lucas Bradstreet
>>> might
>>> > > have
>>> > > > > > more
>>> > > > > > > > > > insight
>>> > > > > > > > > > > >>> about
>>> > > > > > > > > > > >>> > > the
>>> > > > > > > > > > > >>> > > > >> > > decision.
>>> > > > > > > > > > > >>> > > > >> > > > Basically my point is that we
>>> have to
>>> > > > create
>>> > > > > > > > > > additional
>>> > > > > > > > > > > >>> > > > >> infrastructure
>>> > > > > > > > > > > >>> > > > >> > > here
>>> > > > > > > > > > > >>> > > > >> > > > to support the name/id mapping,
>>> so I
>>> > > > wanted
>>> > > > > to
>>> > > > > > > > > > > understand
>>> > > > > > > > > > > >>> if
>>> > > > > > > > > > > >>> > we
>>> > > > > > > > > > > >>> > > > just
>>> > > > > > > > > > > >>> > > > >> > > > consider this a sort of tech
>>> debt. It
>>> > > > would
>>> > > > > be
>>> > > > > > > > > useful
>>> > > > > > > > > > to
>>> > > > > > > > > > > >>> cover
>>> > > > > > > > > > > >>> > > how
>>> > > > > > > > > > > >>> > > > >> we
>>> > > > > > > > > > > >>> > > > >> > > > handle the case when this file
>>> gets
>>> > > > > corrupted.
>>> > > > > > > Seems
>>> > > > > > > > > > > like
>>> > > > > > > > > > > >>> we
>>> > > > > > > > > > > >>> > > just
>>> > > > > > > > > > > >>> > > > >> have
>>> > > > > > > > > > > >>> > > > >> > to
>>> > > > > > > > > > > >>> > > > >> > > > trust that it matches whatever the
>>> > > > > controller
>>> > > > > > > tells
>>> > > > > > > > > us
>>> > > > > > > > > > > and
>>> > > > > > > > > > > >>> > > rewrite
>>> > > > > > > > > > > >>> > > > >> it?
>>> > > > > > > > > > > >>> > > > >> > > >
>>> > > > > > > > > > > >>> > > > >> > > > > 3. I think this is a good point,
>>> > but I
>>> > > > > > again I
>>> > > > > > > > > > wonder
>>> > > > > > > > > > > >>> about
>>> > > > > > > > > > > >>> > > the
>>> > > > > > > > > > > >>> > > > >> scope
>>> > > > > > > > > > > >>> > > > >> > > of
>>> > > > > > > > > > > >>> > > > >> > > > the KIP. Most of the changes
>>> mentioned
>>> > > in
>>> > > > > the
>>> > > > > > > KIP
>>> > > > > > > > > are
>>> > > > > > > > > > > for
>>> > > > > > > > > > > >>> > > > supporting
>>> > > > > > > > > > > >>> > > > >> > > topic
>>> > > > > > > > > > > >>> > > > >> > > > deletion and I believe that is
>>> why the
>>> > > > > produce
>>> > > > > > > > > request
>>> > > > > > > > > > > was
>>> > > > > > > > > > > >>> > > listed
>>> > > > > > > > > > > >>> > > > >> under
>>> > > > > > > > > > > >>> > > > >> > > > future work.
>>> > > > > > > > > > > >>> > > > >> > > >
>>> > > > > > > > > > > >>> > > > >> > > > That's fair. I brought it up since
>>> > > `Fetch`
>>> > > > > is
>>> > > > > > > > > already
>>> > > > > > > > > > > >>> > included.
>>> > > > > > > > > > > >>> > > If
>>> > > > > > > > > > > >>> > > > >> > we've
>>> > > > > > > > > > > >>> > > > >> > > > got `Metadata` and `Fetch`, seems
>>> we
>>> > may
>>> > > > as
>>> > > > > > > well do
>>> > > > > > > > > > > >>> `Produce`
>>> > > > > > > > > > > >>> > > and
>>> > > > > > > > > > > >>> > > > >> save
>>> > > > > > > > > > > >>> > > > >> > an
>>> > > > > > > > > > > >>> > > > >> > > > extra kip. No strong objection
>>> though
>>> > if
>>> > > > you
>>> > > > > > > want to
>>> > > > > > > > > > > >>> leave it
>>> > > > > > > > > > > >>> > > out.
>>> > > > > > > > > > > >>> > > > >> > > >
>>> > > > > > > > > > > >>> > > > >> > > >
>>> > > > > > > > > > > >>> > > > >> > > > -Jason
>>> > > > > > > > > > > >>> > > > >> > > >
>>> > > > > > > > > > > >>> > > > >> > > >
>>> > > > > > > > > > > >>> > > > >> > > > On Thu, Sep 24, 2020 at 3:26 PM
>>> > Justine
>>> > > > > > Olshan <
>>> > > > > > > > > > > >>> > > > >> jolshan@confluent.io>
>>> > > > > > > > > > > >>> > > > >> > > > wrote:
>>> > > > > > > > > > > >>> > > > >> > > >
>>> > > > > > > > > > > >>> > > > >> > > > > Hi Jason,
>>> > > > > > > > > > > >>> > > > >> > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > Thanks for your comments.
>>> > > > > > > > > > > >>> > > > >> > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > 1. Yes, the directory will
>>> still be
>>> > > > based
>>> > > > > on
>>> > > > > > > the
>>> > > > > > > > > > topic
>>> > > > > > > > > > > >>> > names.
>>> > > > > > > > > > > >>> > > > >> > > > > LeaderAndIsrRequest is one of
>>> the
>>> > few
>>> > > > > > requests
>>> > > > > > > > > that
>>> > > > > > > > > > > will
>>> > > > > > > > > > > >>> > still
>>> > > > > > > > > > > >>> > > > >> > contain
>>> > > > > > > > > > > >>> > > > >> > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > topic name. So I think we have
>>> this
>>> > > > > covered.
>>> > > > > > > Sorry
>>> > > > > > > > > > for
>>> > > > > > > > > > > >>> > > > confusion.
>>> > > > > > > > > > > >>> > > > >> > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > 2. Part of the usage of the
>>> file is
>>> > to
>>> > > > > have
>>> > > > > > > > > > persistent
>>> > > > > > > > > > > >>> > storage
>>> > > > > > > > > > > >>> > > > of
>>> > > > > > > > > > > >>> > > > >> the
>>> > > > > > > > > > > >>> > > > >> > > > topic
>>> > > > > > > > > > > >>> > > > >> > > > > ID and use it to compare with
>>> the ID
>>> > > > > > supplied
>>> > > > > > > in
>>> > > > > > > > > the
>>> > > > > > > > > > > >>> > > > LeaderAndIsr
>>> > > > > > > > > > > >>> > > > >> > > > Request.
>>> > > > > > > > > > > >>> > > > >> > > > > There is some discussion in the
>>> KIP
>>> > > > about
>>> > > > > > > changes
>>> > > > > > > > > to
>>> > > > > > > > > > > the
>>> > > > > > > > > > > >>> > > > directory
>>> > > > > > > > > > > >>> > > > >> > > > > structure, but I believe
>>> directory
>>> > > > changes
>>> > > > > > > were
>>> > > > > > > > > > > >>> considered
>>> > > > > > > > > > > >>> > to
>>> > > > > > > > > > > >>> > > be
>>> > > > > > > > > > > >>> > > > >> out
>>> > > > > > > > > > > >>> > > > >> > of
>>> > > > > > > > > > > >>> > > > >> > > > > scope when the KIP was written.
>>> > > > > > > > > > > >>> > > > >> > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > 3. I think this is a good point,
>>> > but I
>>> > > > > > again I
>>> > > > > > > > > > wonder
>>> > > > > > > > > > > >>> about
>>> > > > > > > > > > > >>> > > the
>>> > > > > > > > > > > >>> > > > >> scope
>>> > > > > > > > > > > >>> > > > >> > > of
>>> > > > > > > > > > > >>> > > > >> > > > > the KIP. Most of the changes
>>> > mentioned
>>> > > > in
>>> > > > > > the
>>> > > > > > > KIP
>>> > > > > > > > > > are
>>> > > > > > > > > > > >>> for
>>> > > > > > > > > > > >>> > > > >> supporting
>>> > > > > > > > > > > >>> > > > >> > > > topic
>>> > > > > > > > > > > >>> > > > >> > > > > deletion and I believe that is
>>> why
>>> > the
>>> > > > > > produce
>>> > > > > > > > > > request
>>> > > > > > > > > > > >>> was
>>> > > > > > > > > > > >>> > > > listed
>>> > > > > > > > > > > >>> > > > >> > under
>>> > > > > > > > > > > >>> > > > >> > > > > future work.
>>> > > > > > > > > > > >>> > > > >> > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > 4. This sounds like it might be
>>> a
>>> > good
>>> > > > > > > solution,
>>> > > > > > > > > > but I
>>> > > > > > > > > > > >>> will
>>> > > > > > > > > > > >>> > > need
>>> > > > > > > > > > > >>> > > > >> to
>>> > > > > > > > > > > >>> > > > >> > > > discuss
>>> > > > > > > > > > > >>> > > > >> > > > > more with KIP-500 folks to get
>>> the
>>> > > > details
>>> > > > > > > right.
>>> > > > > > > > > > > >>> > > > >> > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > Thanks,
>>> > > > > > > > > > > >>> > > > >> > > > > Justine
>>> > > > > > > > > > > >>> > > > >> > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > On Thu, Sep 24, 2020 at 12:30 PM
>>> > Jason
>>> > > > > > > Gustafson <
>>> > > > > > > > > > > >>> > > > >> jason@confluent.io
>>> > > > > > > > > > > >>> > > > >> > >
>>> > > > > > > > > > > >>> > > > >> > > > > wrote:
>>> > > > > > > > > > > >>> > > > >> > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > Hi Justine,
>>> > > > > > > > > > > >>> > > > >> > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > Thanks for picking up this
>>> work. I
>>> > > > have
>>> > > > > a
>>> > > > > > > few
>>> > > > > > > > > > > >>> > > > >> questions/comments:
>>> > > > > > > > > > > >>> > > > >> > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > 1. It sounds like the
>>> directory
>>> > > > > structure
>>> > > > > > is
>>> > > > > > > > > still
>>> > > > > > > > > > > >>> going
>>> > > > > > > > > > > >>> > to
>>> > > > > > > > > > > >>> > > be
>>> > > > > > > > > > > >>> > > > >> > based
>>> > > > > > > > > > > >>> > > > >> > > on
>>> > > > > > > > > > > >>> > > > >> > > > > > topic names. Do I have that
>>> right?
>>> > > One
>>> > > > > > > > > > complication
>>> > > > > > > > > > > is
>>> > > > > > > > > > > >>> > that
>>> > > > > > > > > > > >>> > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > LeaderAndIsr request does not
>>> > > include
>>> > > > > the
>>> > > > > > > topic
>>> > > > > > > > > > name
>>> > > > > > > > > > > >>> any
>>> > > > > > > > > > > >>> > > > longer.
>>> > > > > > > > > > > >>> > > > >> > This
>>> > > > > > > > > > > >>> > > > >> > > > > means
>>> > > > > > > > > > > >>> > > > >> > > > > > that a replica must wait for
>>> the
>>> > > > > > > UpdateMetadata
>>> > > > > > > > > > > >>> request to
>>> > > > > > > > > > > >>> > > > >> arrive
>>> > > > > > > > > > > >>> > > > >> > > with
>>> > > > > > > > > > > >>> > > > >> > > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > topic name mapping before it
>>> can
>>> > > > create
>>> > > > > > the
>>> > > > > > > > > > > >>> directory. I
>>> > > > > > > > > > > >>> > > > wonder
>>> > > > > > > > > > > >>> > > > >> if
>>> > > > > > > > > > > >>> > > > >> > it
>>> > > > > > > > > > > >>> > > > >> > > > > would
>>> > > > > > > > > > > >>> > > > >> > > > > > be simpler to avoid
>>> assumptions on
>>> > > the
>>> > > > > > > ordering
>>> > > > > > > > > of
>>> > > > > > > > > > > >>> > > > >> UpdateMetadata
>>> > > > > > > > > > > >>> > > > >> > and
>>> > > > > > > > > > > >>> > > > >> > > > let
>>> > > > > > > > > > > >>> > > > >> > > > > > LeaderAndIsr include the topic
>>> > name
>>> > > as
>>> > > > > > well.
>>> > > > > > > > > Feels
>>> > > > > > > > > > > >>> like we
>>> > > > > > > > > > > >>> > > are
>>> > > > > > > > > > > >>> > > > >> not
>>> > > > > > > > > > > >>> > > > >> > > > saving
>>> > > > > > > > > > > >>> > > > >> > > > > > that much by excluding it.
>>> > > > > > > > > > > >>> > > > >> > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > 2. On a related note, it seems
>>> > that
>>> > > > the
>>> > > > > > > reason
>>> > > > > > > > > we
>>> > > > > > > > > > > >>> have the
>>> > > > > > > > > > > >>> > > > >> > partition
>>> > > > > > > > > > > >>> > > > >> > > > > > metadata file is because we
>>> are
>>> > not
>>> > > > > > > changing the
>>> > > > > > > > > > > >>> directory
>>> > > > > > > > > > > >>> > > > >> > structure.
>>> > > > > > > > > > > >>> > > > >> > > > We
>>> > > > > > > > > > > >>> > > > >> > > > > > need it so that we remember
>>> which
>>> > > > > > > directories
>>> > > > > > > > > map
>>> > > > > > > > > > to
>>> > > > > > > > > > > >>> which
>>> > > > > > > > > > > >>> > > > topic
>>> > > > > > > > > > > >>> > > > >> > id.
>>> > > > > > > > > > > >>> > > > >> > > I
>>> > > > > > > > > > > >>> > > > >> > > > > > think it would be useful to
>>> add
>>> > some
>>> > > > > > detail
>>> > > > > > > > > about
>>> > > > > > > > > > > why
>>> > > > > > > > > > > >>> > > changing
>>> > > > > > > > > > > >>> > > > >> the
>>> > > > > > > > > > > >>> > > > >> > > > > > directory layout was rejected.
>>> > > > Basically
>>> > > > > > > trying
>>> > > > > > > > > to
>>> > > > > > > > > > > >>> > > understand
>>> > > > > > > > > > > >>> > > > >> how
>>> > > > > > > > > > > >>> > > > >> > > much
>>> > > > > > > > > > > >>> > > > >> > > > > > effort we are saving by
>>> skipping
>>> > > this
>>> > > > > step
>>> > > > > > > if we
>>> > > > > > > > > > > have
>>> > > > > > > > > > > >>> to
>>> > > > > > > > > > > >>> > > > >> implement
>>> > > > > > > > > > > >>> > > > >> > > this
>>> > > > > > > > > > > >>> > > > >> > > > > > additional bookkeeping. One
>>> > concern
>>> > > I
>>> > > > > have
>>> > > > > > > for
>>> > > > > > > > > > > >>> example is
>>> > > > > > > > > > > >>> > > that
>>> > > > > > > > > > > >>> > > > >> the
>>> > > > > > > > > > > >>> > > > >> > > > > > partition metadata file gets
>>> > corrupt
>>> > > > and
>>> > > > > > > then
>>> > > > > > > > > all
>>> > > > > > > > > > > >>> bets are
>>> > > > > > > > > > > >>> > > off
>>> > > > > > > > > > > >>> > > > >> as
>>> > > > > > > > > > > >>> > > > >> > far
>>> > > > > > > > > > > >>> > > > >> > > > as
>>> > > > > > > > > > > >>> > > > >> > > > > > topic consistency.
>>> > > > > > > > > > > >>> > > > >> > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > 3. Is it worth adding support
>>> now
>>> > > for
>>> > > > > > topic
>>> > > > > > > ids
>>> > > > > > > > > in
>>> > > > > > > > > > > the
>>> > > > > > > > > > > >>> > > > `Produce`
>>> > > > > > > > > > > >>> > > > >> > > > request
>>> > > > > > > > > > > >>> > > > >> > > > > > now? Overhead is mentioned as
>>> one
>>> > of
>>> > > > the
>>> > > > > > > > > > motivations
>>> > > > > > > > > > > >>> and
>>> > > > > > > > > > > >>> > the
>>> > > > > > > > > > > >>> > > > >> best
>>> > > > > > > > > > > >>> > > > >> > > APIs
>>> > > > > > > > > > > >>> > > > >> > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > get savings from are
>>> `Produce` and
>>> > > > > > `Fetch`.
>>> > > > > > > > > > > >>> > > > >> > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > 4. When it comes to
>>> bootstrapping
>>> > > the
>>> > > > > > > metadata
>>> > > > > > > > > > topic
>>> > > > > > > > > > > >>> with
>>> > > > > > > > > > > >>> > > > >> respect
>>> > > > > > > > > > > >>> > > > >> > to
>>> > > > > > > > > > > >>> > > > >> > > > > > KIP-500, one suggestion would
>>> be
>>> > to
>>> > > > > > reserve
>>> > > > > > > a
>>> > > > > > > > > > > sentinel
>>> > > > > > > > > > > >>> > topic
>>> > > > > > > > > > > >>> > > > ID
>>> > > > > > > > > > > >>> > > > >> > which
>>> > > > > > > > > > > >>> > > > >> > > > can
>>> > > > > > > > > > > >>> > > > >> > > > > > be used initially by new
>>> brokers.
>>> > > When
>>> > > > > the
>>> > > > > > > first
>>> > > > > > > > > > > >>> > controller
>>> > > > > > > > > > > >>> > > is
>>> > > > > > > > > > > >>> > > > >> > > elected,
>>> > > > > > > > > > > >>> > > > >> > > > > it
>>> > > > > > > > > > > >>> > > > >> > > > > > can assign a topicId to the
>>> > metadata
>>> > > > > > topic.
>>> > > > > > > This
>>> > > > > > > > > > is
>>> > > > > > > > > > > a
>>> > > > > > > > > > > >>> bit
>>> > > > > > > > > > > >>> > > > >> similar
>>> > > > > > > > > > > >>> > > > >> > to
>>> > > > > > > > > > > >>> > > > >> > > > how
>>> > > > > > > > > > > >>> > > > >> > > > > we
>>> > > > > > > > > > > >>> > > > >> > > > > > were planning to generate the
>>> > > > clusterId.
>>> > > > > > > > > > > >>> > > > >> > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > Thanks,
>>> > > > > > > > > > > >>> > > > >> > > > > > Jason
>>> > > > > > > > > > > >>> > > > >> > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > On Thu, Sep 24, 2020 at 11:10
>>> AM
>>> > Jun
>>> > > > > Rao <
>>> > > > > > > > > > > >>> > jun@confluent.io>
>>> > > > > > > > > > > >>> > > > >> wrote:
>>> > > > > > > > > > > >>> > > > >> > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > Hi, Justine,
>>> > > > > > > > > > > >>> > > > >> > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > Thanks for the updated KIP.
>>> A
>>> > few
>>> > > > more
>>> > > > > > > > > comments
>>> > > > > > > > > > > >>> below.
>>> > > > > > > > > > > >>> > > > >> > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > 30.  {"name": "id", "type":
>>> > > > "string",
>>> > > > > > > "doc":
>>> > > > > > > > > > > >>> "version
>>> > > > > > > > > > > >>> > > id"}}:
>>> > > > > > > > > > > >>> > > > >> The
>>> > > > > > > > > > > >>> > > > >> > > doc
>>> > > > > > > > > > > >>> > > > >> > > > > > should
>>> > > > > > > > > > > >>> > > > >> > > > > > > say UUID.
>>> > > > > > > > > > > >>> > > > >> > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > 31. LeaderAndIsrResponse v5
>>> and
>>> > > > > > > > > > > StopReplicaResponse
>>> > > > > > > > > > > >>> v4 :
>>> > > > > > > > > > > >>> > > It
>>> > > > > > > > > > > >>> > > > >> seems
>>> > > > > > > > > > > >>> > > > >> > > > there
>>> > > > > > > > > > > >>> > > > >> > > > > > is
>>> > > > > > > > > > > >>> > > > >> > > > > > > no need to add topic_id at
>>> > > > partitions
>>> > > > > > > level.
>>> > > > > > > > > > > >>> > > > >> > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > 32. Regarding partition
>>> metadata
>>> > > > file.
>>> > > > > > > Perhaps
>>> > > > > > > > > > the
>>> > > > > > > > > > > >>> key
>>> > > > > > > > > > > >>> > can
>>> > > > > > > > > > > >>> > > > be
>>> > > > > > > > > > > >>> > > > >> a
>>> > > > > > > > > > > >>> > > > >> > > > single
>>> > > > > > > > > > > >>> > > > >> > > > > > > word, sth like the
>>> following.
>>> > > > > > > > > > > >>> > > > >> > > > > > > version: 0
>>> > > > > > > > > > > >>> > > > >> > > > > > > topic_id:
>>> > > > > > > 46bdb63f-9e8d-4a38-bf7b-ee4eb2a794e4
>>> > > > > > > > > > > >>> > > > >> > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > 33. Another tricky thing
>>> that I
>>> > > > > realized
>>> > > > > > > is
>>> > > > > > > > > how
>>> > > > > > > > > > to
>>> > > > > > > > > > > >>> > support
>>> > > > > > > > > > > >>> > > > the
>>> > > > > > > > > > > >>> > > > >> > > > metadata
>>> > > > > > > > > > > >>> > > > >> > > > > > > topic introduced in KIP-595.
>>> > It's
>>> > > a
>>> > > > > bit
>>> > > > > > > tricky
>>> > > > > > > > > > to
>>> > > > > > > > > > > >>> > assign a
>>> > > > > > > > > > > >>> > > > >> UUID
>>> > > > > > > > > > > >>> > > > >> > to
>>> > > > > > > > > > > >>> > > > >> > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > metadata topic since we
>>> have a
>>> > > > > > > chicken-and-egg
>>> > > > > > > > > > > >>> problem.
>>> > > > > > > > > > > >>> > > The
>>> > > > > > > > > > > >>> > > > >> > > > controller
>>> > > > > > > > > > > >>> > > > >> > > > > > > needs to persist the UUID
>>> in the
>>> > > > > > metadata
>>> > > > > > > > > topic
>>> > > > > > > > > > in
>>> > > > > > > > > > > >>> order
>>> > > > > > > > > > > >>> > > to
>>> > > > > > > > > > > >>> > > > >> > assign
>>> > > > > > > > > > > >>> > > > >> > > > one
>>> > > > > > > > > > > >>> > > > >> > > > > > > successfully, but the
>>> metadata
>>> > > topic
>>> > > > > is
>>> > > > > > > needed
>>> > > > > > > > > > to
>>> > > > > > > > > > > >>> elect
>>> > > > > > > > > > > >>> > a
>>> > > > > > > > > > > >>> > > > >> > > controller
>>> > > > > > > > > > > >>> > > > >> > > > > > > (KIP-631). So, this probably
>>> > > needs a
>>> > > > > bit
>>> > > > > > > more
>>> > > > > > > > > > > >>> thought.
>>> > > > > > > > > > > >>> > > > >> > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > Jun
>>> > > > > > > > > > > >>> > > > >> > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > On Thu, Sep 24, 2020 at
>>> 3:04 AM
>>> > > > Ismael
>>> > > > > > > Juma <
>>> > > > > > > > > > > >>> > > > >> ismael@juma.me.uk>
>>> > > > > > > > > > > >>> > > > >> > > > wrote:
>>> > > > > > > > > > > >>> > > > >> > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > Also, can we provide more
>>> > > details
>>> > > > on
>>> > > > > > > how the
>>> > > > > > > > > > > >>> Partition
>>> > > > > > > > > > > >>> > > > >> Metadata
>>> > > > > > > > > > > >>> > > > >> > > > file
>>> > > > > > > > > > > >>> > > > >> > > > > > will
>>> > > > > > > > > > > >>> > > > >> > > > > > > > be used?
>>> > > > > > > > > > > >>> > > > >> > > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > Ismael
>>> > > > > > > > > > > >>> > > > >> > > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > On Thu, Sep 24, 2020 at
>>> 3:01
>>> > AM
>>> > > > > Ismael
>>> > > > > > > Juma
>>> > > > > > > > > <
>>> > > > > > > > > > > >>> > > > >> ismael@juma.me.uk
>>> > > > > > > > > > > >>> > > > >> > >
>>> > > > > > > > > > > >>> > > > >> > > > > wrote:
>>> > > > > > > > > > > >>> > > > >> > > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > > Hi Justine,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > > I think we need to
>>> update
>>> > the
>>> > > > > > > "Rejected
>>> > > > > > > > > > > >>> > Alternatives"
>>> > > > > > > > > > > >>> > > > >> section
>>> > > > > > > > > > > >>> > > > >> > > to
>>> > > > > > > > > > > >>> > > > >> > > > > take
>>> > > > > > > > > > > >>> > > > >> > > > > > > > into
>>> > > > > > > > > > > >>> > > > >> > > > > > > > > account that the
>>> proposal
>>> > now
>>> > > > > > removes
>>> > > > > > > the
>>> > > > > > > > > > > topic
>>> > > > > > > > > > > >>> name
>>> > > > > > > > > > > >>> > > > from
>>> > > > > > > > > > > >>> > > > >> the
>>> > > > > > > > > > > >>> > > > >> > > > fetch
>>> > > > > > > > > > > >>> > > > >> > > > > > > > > request. Also, if we are
>>> > > > removing
>>> > > > > it
>>> > > > > > > from
>>> > > > > > > > > > the
>>> > > > > > > > > > > >>> Fetch
>>> > > > > > > > > > > >>> > > > >> request,
>>> > > > > > > > > > > >>> > > > >> > > does
>>> > > > > > > > > > > >>> > > > >> > > > > it
>>> > > > > > > > > > > >>> > > > >> > > > > > > make
>>> > > > > > > > > > > >>> > > > >> > > > > > > > > sense not to remove it
>>> from
>>> > > > > similar
>>> > > > > > > > > requests
>>> > > > > > > > > > > >>> like
>>> > > > > > > > > > > >>> > > > >> > > > > ListOffsetRequest?
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > > Ismael
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > > On Thu, Sep 24, 2020 at
>>> 2:46
>>> > > AM
>>> > > > > > David
>>> > > > > > > > > Jacot
>>> > > > > > > > > > <
>>> > > > > > > > > > > >>> > > > >> > > djacot@confluent.io
>>> > > > > > > > > > > >>> > > > >> > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > wrote:
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> Hi Justine,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >>
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> Thanks for the KIP. I
>>> > finally
>>> > > > had
>>> > > > > > > time to
>>> > > > > > > > > > > read
>>> > > > > > > > > > > >>> it
>>> > > > > > > > > > > >>> > :).
>>> > > > > > > > > > > >>> > > > It
>>> > > > > > > > > > > >>> > > > >> is
>>> > > > > > > > > > > >>> > > > >> > a
>>> > > > > > > > > > > >>> > > > >> > > > > great
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> improvement.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >>
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> I have a few
>>> > > > comments/questions:
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >>
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> 1. It seems that the
>>> schema
>>> > > of
>>> > > > > the
>>> > > > > > > > > > > >>> > StopReplicaRequest
>>> > > > > > > > > > > >>> > > > is
>>> > > > > > > > > > > >>> > > > >> > > > slightly
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> outdated.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> We
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> did some changes as
>>> part of
>>> > > > > > KIP-570.
>>> > > > > > > V3
>>> > > > > > > > > is
>>> > > > > > > > > > > >>> already
>>> > > > > > > > > > > >>> > > > >> organized
>>> > > > > > > > > > > >>> > > > >> > > by
>>> > > > > > > > > > > >>> > > > >> > > > > > > topics.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >>
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> 2. I just want to make
>>> sure
>>> > > > that
>>> > > > > I
>>> > > > > > > > > > understand
>>> > > > > > > > > > > >>> the
>>> > > > > > > > > > > >>> > > > >> > > reconciliation
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> logic correctly. When
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> an "INCREMENTAL"
>>> > LeaderAndIsr
>>> > > > > > > Request is
>>> > > > > > > > > > > >>> received,
>>> > > > > > > > > > > >>> > > the
>>> > > > > > > > > > > >>> > > > >> > broker
>>> > > > > > > > > > > >>> > > > >> > > > will
>>> > > > > > > > > > > >>> > > > >> > > > > > > also
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> reconcile
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> when the local uuid
>>> does
>>> > not
>>> > > > > match
>>> > > > > > > the
>>> > > > > > > > > uuid
>>> > > > > > > > > > > in
>>> > > > > > > > > > > >>> the
>>> > > > > > > > > > > >>> > > > >> request,
>>> > > > > > > > > > > >>> > > > >> > > > right?
>>> > > > > > > > > > > >>> > > > >> > > > > > In
>>> > > > > > > > > > > >>> > > > >> > > > > > > > this
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> case, the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> local log is staged for
>>> > > > deletion.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >>
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> 3. In the
>>> documentation of
>>> > > the
>>> > > > `
>>> > > > > > > > > > > >>> > > > >> delete.stale.topic.delay.ms
>>> > > > > > > > > > > >>> > > > >> > `
>>> > > > > > > > > > > >>> > > > >> > > > > > config,
>>> > > > > > > > > > > >>> > > > >> > > > > > > it
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> says "When a
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> FULL
>>> LeaderAndIsrRequest is
>>> > > > > > > received..."
>>> > > > > > > > > > but
>>> > > > > > > > > > > I
>>> > > > > > > > > > > >>> > > suppose
>>> > > > > > > > > > > >>> > > > it
>>> > > > > > > > > > > >>> > > > >> > > > applies
>>> > > > > > > > > > > >>> > > > >> > > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > both
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> types.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >>
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> Best,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> David
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >>
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> On Thu, Sep 24, 2020 at
>>> > 1:40
>>> > > AM
>>> > > > > > > Justine
>>> > > > > > > > > > > Olshan
>>> > > > > > > > > > > >>> <
>>> > > > > > > > > > > >>> > > > >> > > > > > jolshan@confluent.io>
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> wrote:
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >>
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > Hi Jun,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > Thanks for the
>>> comments.
>>> > I
>>> > > > > > > apologize
>>> > > > > > > > > for
>>> > > > > > > > > > > >>> some of
>>> > > > > > > > > > > >>> > > the
>>> > > > > > > > > > > >>> > > > >> typos
>>> > > > > > > > > > > >>> > > > >> > > and
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> confusion.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > I’ve updated the KIP
>>> to
>>> > fix
>>> > > > > some
>>> > > > > > > of the
>>> > > > > > > > > > > >>> issues
>>> > > > > > > > > > > >>> > you
>>> > > > > > > > > > > >>> > > > >> > > mentioned.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > 20.2 I’ve changed the
>>> > type
>>> > > to
>>> > > > > > > String.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > 20.1/3 I’ve updated
>>> the
>>> > > > > > TopicZNode
>>> > > > > > > to
>>> > > > > > > > > fix
>>> > > > > > > > > > > >>> > > formatting
>>> > > > > > > > > > > >>> > > > >> and
>>> > > > > > > > > > > >>> > > > >> > > > reflect
>>> > > > > > > > > > > >>> > > > >> > > > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> latest
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > version before this
>>> > change.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > 21. You are correct
>>> and
>>> > > I’ve
>>> > > > > > > removed
>>> > > > > > > > > this
>>> > > > > > > > > > > >>> line.
>>> > > > > > > > > > > >>> > > I’ve
>>> > > > > > > > > > > >>> > > > >> also
>>> > > > > > > > > > > >>> > > > >> > > > added
>>> > > > > > > > > > > >>> > > > >> > > > > a
>>> > > > > > > > > > > >>> > > > >> > > > > > > line
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > mentioning an IBP
>>> bump is
>>> > > > > > > necessary for
>>> > > > > > > > > > > >>> migration
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > 22. I think the
>>> wording
>>> > was
>>> > > > > > > unclear but
>>> > > > > > > > > > > your
>>> > > > > > > > > > > >>> > > summary
>>> > > > > > > > > > > >>> > > > is
>>> > > > > > > > > > > >>> > > > >> > what
>>> > > > > > > > > > > >>> > > > >> > > > was
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> intended
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > by this line. I’ve
>>> > updated
>>> > > to
>>> > > > > > make
>>> > > > > > > this
>>> > > > > > > > > > > point
>>> > > > > > > > > > > >>> > more
>>> > > > > > > > > > > >>> > > > >> clear.
>>> > > > > > > > > > > >>> > > > >> > > > > “Remove
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> deleted
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > topics from replicas
>>> by
>>> > > > sending
>>> > > > > > > > > > > >>> > StopReplicaRequest
>>> > > > > > > > > > > >>> > > V3
>>> > > > > > > > > > > >>> > > > >> > before
>>> > > > > > > > > > > >>> > > > >> > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > IBP
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> bump
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > using the old logic,
>>> and
>>> > > > using
>>> > > > > V4
>>> > > > > > > and
>>> > > > > > > > > the
>>> > > > > > > > > > > new
>>> > > > > > > > > > > >>> > logic
>>> > > > > > > > > > > >>> > > > >> with
>>> > > > > > > > > > > >>> > > > >> > > topic
>>> > > > > > > > > > > >>> > > > >> > > > > IDs
>>> > > > > > > > > > > >>> > > > >> > > > > > > > after
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > the IBP bump.”
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > 23. I’ve removed the
>>> > > > > unspecified
>>> > > > > > > type
>>> > > > > > > > > > from
>>> > > > > > > > > > > >>> the
>>> > > > > > > > > > > >>> > KIP
>>> > > > > > > > > > > >>> > > > and
>>> > > > > > > > > > > >>> > > > >> > > mention
>>> > > > > > > > > > > >>> > > > >> > > > > > that
>>> > > > > > > > > > > >>> > > > >> > > > > > > > IBP
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > will be used to
>>> handle
>>> > this
>>> > > > > > > request.
>>> > > > > > > > > “IBP
>>> > > > > > > > > > > >>> will be
>>> > > > > > > > > > > >>> > > > used
>>> > > > > > > > > > > >>> > > > >> to
>>> > > > > > > > > > > >>> > > > >> > > > > > determine
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> whether
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > this new form of the
>>> > > request
>>> > > > > will
>>> > > > > > > be
>>> > > > > > > > > > used.
>>> > > > > > > > > > > >>> For
>>> > > > > > > > > > > >>> > > older
>>> > > > > > > > > > > >>> > > > >> > > requests,
>>> > > > > > > > > > > >>> > > > >> > > > > we
>>> > > > > > > > > > > >>> > > > >> > > > > > > will
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > ignore this field and
>>> > > default
>>> > > > > to
>>> > > > > > > > > previous
>>> > > > > > > > > > > >>> > > behavior.”
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > 24. I’ve fixed this
>>> typo.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > 25. I've added a
>>> topics
>>> > at
>>> > > a
>>> > > > > > higher
>>> > > > > > > > > level
>>> > > > > > > > > > > for
>>> > > > > > > > > > > >>> > > > >> > > > > LeaderAndIsrResponse
>>> > > > > > > > > > > >>> > > > >> > > > > > > v5,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > StopReplicaResponse
>>> v4.
>>> > > I've
>>> > > > > also
>>> > > > > > > > > changed
>>> > > > > > > > > > > >>> > > > >> > StopReplicaRequest
>>> > > > > > > > > > > >>> > > > >> > > > v4
>>> > > > > > > > > > > >>> > > > >> > > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > have
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > topics at a higher
>>> level.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > 26. I’ve updated
>>> > > > > > > > > > > >>> forgotten_topics_data--added the
>>> > > > > > > > > > > >>> > > > >> topic ID
>>> > > > > > > > > > > >>> > > > >> > > and
>>> > > > > > > > > > > >>> > > > >> > > > > > > removed
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > topic name
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > 27. I’ve decided on
>>> plain
>>> > > > text,
>>> > > > > > and
>>> > > > > > > > > I’ve
>>> > > > > > > > > > > >>> added an
>>> > > > > > > > > > > >>> > > > >> example.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > 28. I’ve added this
>>> idea
>>> > to
>>> > > > > > future
>>> > > > > > > > > work.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > Thanks again for
>>> taking a
>>> > > > look,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > Justine
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > On Wed, Sep 23, 2020
>>> at
>>> > > 10:28
>>> > > > > AM
>>> > > > > > > Jun
>>> > > > > > > > > Rao
>>> > > > > > > > > > <
>>> > > > > > > > > > > >>> > > > >> > jun@confluent.io>
>>> > > > > > > > > > > >>> > > > >> > > > > > wrote:
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > Hi, Justine,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > Thanks for the
>>> > response.
>>> > > > Made
>>> > > > > > > another
>>> > > > > > > > > > > >>> pass. A
>>> > > > > > > > > > > >>> > few
>>> > > > > > > > > > > >>> > > > >> more
>>> > > > > > > > > > > >>> > > > >> > > > > comments
>>> > > > > > > > > > > >>> > > > >> > > > > > > > below.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 20. znode schema:
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 20.1 It seems that
>>> > > {"name":
>>> > > > > > > > > "version",
>>> > > > > > > > > > > >>> "type":
>>> > > > > > > > > > > >>> > > > "int",
>>> > > > > > > > > > > >>> > > > >> > > "id":
>>> > > > > > > > > > > >>> > > > >> > > > > > > "UUID",
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > "doc":
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > "version id"}
>>> should be
>>> > > > > > {"name":
>>> > > > > > > > > > > "version",
>>> > > > > > > > > > > >>> > > "type":
>>> > > > > > > > > > > >>> > > > >> > > "int"},
>>> > > > > > > > > > > >>> > > > >> > > > > > > {"name":
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > "id",
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > "type": "UUID",
>>> "doc":
>>> > > > > "version
>>> > > > > > > id"}.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 20.2 The znode
>>> format
>>> > is
>>> > > > JSON
>>> > > > > > > which
>>> > > > > > > > > > > doesn't
>>> > > > > > > > > > > >>> > have
>>> > > > > > > > > > > >>> > > > UUID
>>> > > > > > > > > > > >>> > > > >> > > type.
>>> > > > > > > > > > > >>> > > > >> > > > So
>>> > > > > > > > > > > >>> > > > >> > > > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> type
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > probably should be
>>> > > string?
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 20.3 Also, the
>>> existing
>>> > > > > format
>>> > > > > > > used
>>> > > > > > > > > > seems
>>> > > > > > > > > > > >>> > > outdated.
>>> > > > > > > > > > > >>> > > > >> It
>>> > > > > > > > > > > >>> > > > >> > > > should
>>> > > > > > > > > > > >>> > > > >> > > > > > have
>>> > > > > > > > > > > >>> > > > >> > > > > > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > following format.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>> > >  Json.encodeAsBytes(Map(
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >       "version" ->
>>> 2,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >       "partitions"
>>> ->
>>> > > > > > > > > > > >>> > > replicaAssignmentJson.asJava,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>>  "adding_replicas"
>>> > > ->
>>> > > > > > > > > > > >>> > > > >> > > > >
>>> addingReplicasAssignmentJson.asJava,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>> >  "removing_replicas"
>>> > > > ->
>>> > > > > > > > > > > >>> > > > >> > > > > >
>>> > > removingReplicasAssignmentJson.asJava
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >     ).asJava)
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >   }
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 21. Migration: The
>>> KIP
>>> > > says
>>> > > > > > "The
>>> > > > > > > > > > > migration
>>> > > > > > > > > > > >>> > > process
>>> > > > > > > > > > > >>> > > > >> can
>>> > > > > > > > > > > >>> > > > >> > > take
>>> > > > > > > > > > > >>> > > > >> > > > > > place
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> without
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > an inter-broker
>>> > protocol
>>> > > > > bump,
>>> > > > > > > as the
>>> > > > > > > > > > > >>> format
>>> > > > > > > > > > > >>> > > stored
>>> > > > > > > > > > > >>> > > > >> in
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>> /brokers/topics/[topic]
>>> > > > will
>>> > > > > be
>>> > > > > > > > > > > compatible
>>> > > > > > > > > > > >>> with
>>> > > > > > > > > > > >>> > > > older
>>> > > > > > > > > > > >>> > > > >> > > broker
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> versions."
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > However, since we
>>> are
>>> > > > bumping
>>> > > > > > up
>>> > > > > > > the
>>> > > > > > > > > > > >>> version of
>>> > > > > > > > > > > >>> > > > >> > > inter-broker
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> requests, it
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > seems that we need
>>> to
>>> > use
>>> > > > IBP
>>> > > > > > for
>>> > > > > > > > > > > >>> migration.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 22. The KIP says
>>> > "Remove
>>> > > > > > deleted
>>> > > > > > > > > topics
>>> > > > > > > > > > > >>> from
>>> > > > > > > > > > > >>> > > > >> replicas by
>>> > > > > > > > > > > >>> > > > >> > > > > sending
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > StopReplicaRequest
>>> V3
>>> > for
>>> > > > any
>>> > > > > > > topics
>>> > > > > > > > > > > which
>>> > > > > > > > > > > >>> do
>>> > > > > > > > > > > >>> > not
>>> > > > > > > > > > > >>> > > > >> > contain
>>> > > > > > > > > > > >>> > > > >> > > a
>>> > > > > > > > > > > >>> > > > >> > > > > > topic
>>> > > > > > > > > > > >>> > > > >> > > > > > > > ID,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> and
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > V4 for any topics
>>> which
>>> > > do
>>> > > > > > > contain a
>>> > > > > > > > > > > topic
>>> > > > > > > > > > > >>> > ID.".
>>> > > > > > > > > > > >>> > > > >> > However,
>>> > > > > > > > > > > >>> > > > >> > > if
>>> > > > > > > > > > > >>> > > > >> > > > > we
>>> > > > > > > > > > > >>> > > > >> > > > > > > use
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> IBP,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > it
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > seems that the
>>> > controller
>>> > > > > will
>>> > > > > > > either
>>> > > > > > > > > > > send
>>> > > > > > > > > > > >>> > > > >> > > > StopReplicaRequest
>>> > > > > > > > > > > >>> > > > >> > > > > V3
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > or
>>> StopReplicaRequest
>>> > V4,
>>> > > > but
>>> > > > > > > never
>>> > > > > > > > > > mixed
>>> > > > > > > > > > > >>> V3
>>> > > > > > > > > > > >>> > and
>>> > > > > > > > > > > >>> > > V4
>>> > > > > > > > > > > >>> > > > >> for
>>> > > > > > > > > > > >>> > > > >> > > > > > different
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> topics.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > Basically, before
>>> the
>>> > IBP
>>> > > > > bump,
>>> > > > > > > V3
>>> > > > > > > > > will
>>> > > > > > > > > > > be
>>> > > > > > > > > > > >>> > used.
>>> > > > > > > > > > > >>> > > > >> After
>>> > > > > > > > > > > >>> > > > >> > the
>>> > > > > > > > > > > >>> > > > >> > > > IBP
>>> > > > > > > > > > > >>> > > > >> > > > > > > bump,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > topicId will be
>>> created
>>> > > and
>>> > > > > V4
>>> > > > > > > will
>>> > > > > > > > > be
>>> > > > > > > > > > > >>> used.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 23. Given that we
>>> > depend
>>> > > on
>>> > > > > > IBP,
>>> > > > > > > do
>>> > > > > > > > > we
>>> > > > > > > > > > > >>> still
>>> > > > > > > > > > > >>> > need
>>> > > > > > > > > > > >>> > > > "0
>>> > > > > > > > > > > >>> > > > >> > > > > > UNSPECIFIED"
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > in LeaderAndIsr?
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 24.
>>> > LeaderAndIsrResponse
>>> > > > v5 :
>>> > > > > > It
>>> > > > > > > > > still
>>> > > > > > > > > > > has
>>> > > > > > > > > > > >>> the
>>> > > > > > > > > > > >>> > > > topic
>>> > > > > > > > > > > >>> > > > >> > > field.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 25.
>>> > LeaderAndIsrResponse
>>> > > > v5,
>>> > > > > > > > > > > >>> > StopReplicaResponse
>>> > > > > > > > > > > >>> > > > v4:
>>> > > > > > > > > > > >>> > > > >> > Could
>>> > > > > > > > > > > >>> > > > >> > > > we
>>> > > > > > > > > > > >>> > > > >> > > > > > use
>>> > > > > > > > > > > >>> > > > >> > > > > > > > this
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > opportunity to
>>> organize
>>> > > the
>>> > > > > > > response
>>> > > > > > > > > > in 2
>>> > > > > > > > > > > >>> > levels,
>>> > > > > > > > > > > >>> > > > >> first
>>> > > > > > > > > > > >>> > > > >> > by
>>> > > > > > > > > > > >>> > > > >> > > > > > topic,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> then by
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > partition, as most
>>> > other
>>> > > > > > > > > > > >>> requests/responses?
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 26. FetchRequest
>>> v13 :
>>> > > > Should
>>> > > > > > > > > > > >>> > > forgotten_topics_data
>>> > > > > > > > > > > >>> > > > >> use
>>> > > > > > > > > > > >>> > > > >> > > > > topicId
>>> > > > > > > > > > > >>> > > > >> > > > > > > too?
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 27. "This file can
>>> > either
>>> > > > be
>>> > > > > > > plain
>>> > > > > > > > > text
>>> > > > > > > > > > > >>> > > (key/value
>>> > > > > > > > > > > >>> > > > >> > pairs)
>>> > > > > > > > > > > >>> > > > >> > > or
>>> > > > > > > > > > > >>> > > > >> > > > > > > JSON."
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> Have
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > we
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > decided which one
>>> to
>>> > use?
>>> > > > > Also,
>>> > > > > > > it
>>> > > > > > > > > > would
>>> > > > > > > > > > > be
>>> > > > > > > > > > > >>> > > helpful
>>> > > > > > > > > > > >>> > > > >> to
>>> > > > > > > > > > > >>> > > > >> > > > provide
>>> > > > > > > > > > > >>> > > > >> > > > > > an
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > example.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 28. Future
>>> improvement:
>>> > > > > Another
>>> > > > > > > > > future
>>> > > > > > > > > > > >>> > > improvement
>>> > > > > > > > > > > >>> > > > >> > > > opportunity
>>> > > > > > > > > > > >>> > > > >> > > > > > is
>>> > > > > > > > > > > >>> > > > >> > > > > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> use
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > topicId in
>>> > > > > > > > > > > >>> GroupMetadataManager.offsetCommitKey
>>> > > > > > > > > > > >>> > > in
>>> > > > > > > > > > > >>> > > > >> the
>>> > > > > > > > > > > >>> > > > >> > > > > > > offset_commit
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > topic.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > This may save some
>>> > space.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > Thanks,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > Jun
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > On Wed, Sep 23,
>>> 2020 at
>>> > > > 8:50
>>> > > > > AM
>>> > > > > > > > > Justine
>>> > > > > > > > > > > >>> Olshan
>>> > > > > > > > > > > >>> > <
>>> > > > > > > > > > > >>> > > > >> > > > > > > > jolshan@confluent.io>
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > wrote:
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > Hi Tom,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > Thanks for the
>>> > > comment. I
>>> > > > > > think
>>> > > > > > > > > this
>>> > > > > > > > > > > is a
>>> > > > > > > > > > > >>> > > really
>>> > > > > > > > > > > >>> > > > >> good
>>> > > > > > > > > > > >>> > > > >> > > idea
>>> > > > > > > > > > > >>> > > > >> > > > > and
>>> > > > > > > > > > > >>> > > > >> > > > > > > it
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> has
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > been
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > added to the KIP
>>> > under
>>> > > > the
>>> > > > > > > newly
>>> > > > > > > > > > added
>>> > > > > > > > > > > >>> > tooling
>>> > > > > > > > > > > >>> > > > >> > section.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > Thanks again,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > Justine
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > On Wed, Sep 23,
>>> 2020
>>> > at
>>> > > > > 3:17
>>> > > > > > > AM Tom
>>> > > > > > > > > > > >>> Bentley <
>>> > > > > > > > > > > >>> > > > >> > > > > > > tbentley@redhat.com>
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > wrote:
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > Hi Justine,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > I know you
>>> started
>>> > > the
>>> > > > > vote
>>> > > > > > > > > thread,
>>> > > > > > > > > > > >>> but on
>>> > > > > > > > > > > >>> > > > >> > re-reading
>>> > > > > > > > > > > >>> > > > >> > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > KIP
>>> > > > > > > > > > > >>> > > > >> > > > > > > I
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > noticed
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that although
>>> the
>>> > > topic
>>> > > > > id
>>> > > > > > is
>>> > > > > > > > > > > included
>>> > > > > > > > > > > >>> in
>>> > > > > > > > > > > >>> > the
>>> > > > > > > > > > > >>> > > > >> > > > > > MetadataResponse
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> it's
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > not
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > surfaced in the
>>> > > output
>>> > > > > from
>>> > > > > > > > > > > >>> > `kafka-topics.sh
>>> > > > > > > > > > > >>> > > > >> > > > --describe`.
>>> > > > > > > > > > > >>> > > > >> > > > > > > Maybe
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> that
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > was
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > intentional
>>> because
>>> > > ids
>>> > > > > are
>>> > > > > > > > > > > >>> intentionally
>>> > > > > > > > > > > >>> > not
>>> > > > > > > > > > > >>> > > > >> really
>>> > > > > > > > > > > >>> > > > >> > > > > > something
>>> > > > > > > > > > > >>> > > > >> > > > > > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > user
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > should care
>>> deeply
>>> > > > about,
>>> > > > > > > but it
>>> > > > > > > > > > > would
>>> > > > > > > > > > > >>> also
>>> > > > > > > > > > > >>> > > > make
>>> > > > > > > > > > > >>> > > > >> > life
>>> > > > > > > > > > > >>> > > > >> > > > > harder
>>> > > > > > > > > > > >>> > > > >> > > > > > > for
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > anyone
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > debugging
>>> Kafka and
>>> > > > this
>>> > > > > > > would
>>> > > > > > > > > > likely
>>> > > > > > > > > > > >>> get
>>> > > > > > > > > > > >>> > > worse
>>> > > > > > > > > > > >>> > > > >> the
>>> > > > > > > > > > > >>> > > > >> > > more
>>> > > > > > > > > > > >>> > > > >> > > > > > topic
>>> > > > > > > > > > > >>> > > > >> > > > > > > > ids
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > got
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > rolled out
>>> across
>>> > the
>>> > > > > > > protocols,
>>> > > > > > > > > > > >>> clients
>>> > > > > > > > > > > >>> > etc.
>>> > > > > > > > > > > >>> > > > It
>>> > > > > > > > > > > >>> > > > >> > seems
>>> > > > > > > > > > > >>> > > > >> > > > > > likely
>>> > > > > > > > > > > >>> > > > >> > > > > > > > that
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
>>> `kafka-topics.sh`
>>> > > will
>>> > > > > > > eventually
>>> > > > > > > > > > > need
>>> > > > > > > > > > > >>> the
>>> > > > > > > > > > > >>> > > > >> ability
>>> > > > > > > > > > > >>> > > > >> > to
>>> > > > > > > > > > > >>> > > > >> > > > show
>>> > > > > > > > > > > >>> > > > >> > > > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > id
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> of
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > a
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > topic and
>>> perhaps
>>> > > find
>>> > > > a
>>> > > > > > > topic
>>> > > > > > > > > name
>>> > > > > > > > > > > >>> given
>>> > > > > > > > > > > >>> > an
>>> > > > > > > > > > > >>> > > > id.
>>> > > > > > > > > > > >>> > > > >> Is
>>> > > > > > > > > > > >>> > > > >> > > > there
>>> > > > > > > > > > > >>> > > > >> > > > > > any
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> reason
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > not
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > implement that
>>> in
>>> > > this
>>> > > > > KIP?
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > Many thanks,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > Tom
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > On Mon, Sep 21,
>>> > 2020
>>> > > at
>>> > > > > > 9:54
>>> > > > > > > PM
>>> > > > > > > > > > > Justine
>>> > > > > > > > > > > >>> > > Olshan
>>> > > > > > > > > > > >>> > > > <
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> jolshan@confluent.io
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > wrote:
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Hi all,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > After
>>> thinking
>>> > > about
>>> > > > > it,
>>> > > > > > > I've
>>> > > > > > > > > > > >>> decided to
>>> > > > > > > > > > > >>> > > > remove
>>> > > > > > > > > > > >>> > > > >> > the
>>> > > > > > > > > > > >>> > > > >> > > > > topic
>>> > > > > > > > > > > >>> > > > >> > > > > > > name
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> from
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Fetch
>>> Request and
>>> > > > > > Response
>>> > > > > > > > > after
>>> > > > > > > > > > > all.
>>> > > > > > > > > > > >>> > Since
>>> > > > > > > > > > > >>> > > > >> there
>>> > > > > > > > > > > >>> > > > >> > > are
>>> > > > > > > > > > > >>> > > > >> > > > so
>>> > > > > > > > > > > >>> > > > >> > > > > > > many
>>> > > > > > > > > > > >>> > > > >> > > > > > > > of
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > these
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > requests per
>>> > > second,
>>> > > > it
>>> > > > > > is
>>> > > > > > > > > worth
>>> > > > > > > > > > > >>> removing
>>> > > > > > > > > > > >>> > > the
>>> > > > > > > > > > > >>> > > > >> > extra
>>> > > > > > > > > > > >>> > > > >> > > > > > > > information.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > I've
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > updated the
>>> KIP
>>> > to
>>> > > > > > reflect
>>> > > > > > > this
>>> > > > > > > > > > > >>> change.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Please let me
>>> > know
>>> > > if
>>> > > > > > > there is
>>> > > > > > > > > > > >>> anything
>>> > > > > > > > > > > >>> > > else
>>> > > > > > > > > > > >>> > > > we
>>> > > > > > > > > > > >>> > > > >> > > should
>>> > > > > > > > > > > >>> > > > >> > > > > > > discuss
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > before
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > voting.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Thank you,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Justine
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > On Fri, Sep
>>> 18,
>>> > > 2020
>>> > > > at
>>> > > > > > > 9:46 AM
>>> > > > > > > > > > > >>> Justine
>>> > > > > > > > > > > >>> > > > Olshan
>>> > > > > > > > > > > >>> > > > >> <
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > jolshan@confluent.io
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > wrote:
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > Hi Jun,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > I see what
>>> you
>>> > > are
>>> > > > > > > saying.
>>> > > > > > > > > For
>>> > > > > > > > > > > now
>>> > > > > > > > > > > >>> we
>>> > > > > > > > > > > >>> > can
>>> > > > > > > > > > > >>> > > > >> remove
>>> > > > > > > > > > > >>> > > > >> > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > extra
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > information.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > I'll leave
>>> the
>>> > > > option
>>> > > > > > to
>>> > > > > > > add
>>> > > > > > > > > > more
>>> > > > > > > > > > > >>> > fields
>>> > > > > > > > > > > >>> > > to
>>> > > > > > > > > > > >>> > > > >> the
>>> > > > > > > > > > > >>> > > > >> > > file
>>> > > > > > > > > > > >>> > > > >> > > > > in
>>> > > > > > > > > > > >>> > > > >> > > > > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > future.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > The
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > KIP
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > has been
>>> > updated
>>> > > to
>>> > > > > > > reflect
>>> > > > > > > > > > this
>>> > > > > > > > > > > >>> > change.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > Thanks,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > Justine
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > On Fri,
>>> Sep 18,
>>> > > > 2020
>>> > > > > at
>>> > > > > > > 8:46
>>> > > > > > > > > AM
>>> > > > > > > > > > > Jun
>>> > > > > > > > > > > >>> > Rao <
>>> > > > > > > > > > > >>> > > > >> > > > > > jun@confluent.io
>>> > > > > > > > > > > >>> > > > >> > > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > wrote:
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Hi,
>>> Justine,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Thanks
>>> for the
>>> > > > > reply.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> 13. If
>>> the log
>>> > > > > > > directory is
>>> > > > > > > > > > the
>>> > > > > > > > > > > >>> source
>>> > > > > > > > > > > >>> > > of
>>> > > > > > > > > > > >>> > > > >> > truth,
>>> > > > > > > > > > > >>> > > > >> > > it
>>> > > > > > > > > > > >>> > > > >> > > > > > means
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> that
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> redundant
>>> info
>>> > > in
>>> > > > > the
>>> > > > > > > > > metadata
>>> > > > > > > > > > > >>> file
>>> > > > > > > > > > > >>> > will
>>> > > > > > > > > > > >>> > > > be
>>> > > > > > > > > > > >>> > > > >> > > > ignored.
>>> > > > > > > > > > > >>> > > > >> > > > > > Then
>>> > > > > > > > > > > >>> > > > >> > > > > > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > question
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > is
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> why do we
>>> need
>>> > > to
>>> > > > > put
>>> > > > > > > the
>>> > > > > > > > > > > >>> redundant
>>> > > > > > > > > > > >>> > info
>>> > > > > > > > > > > >>> > > > in
>>> > > > > > > > > > > >>> > > > >> the
>>> > > > > > > > > > > >>> > > > >> > > > > > metadata
>>> > > > > > > > > > > >>> > > > >> > > > > > > > file
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > now?
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Thanks,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Jun
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> On Thu,
>>> Sep
>>> > 17,
>>> > > > 2020
>>> > > > > > at
>>> > > > > > > 5:07
>>> > > > > > > > > > PM
>>> > > > > > > > > > > >>> > Justine
>>> > > > > > > > > > > >>> > > > >> Olshan
>>> > > > > > > > > > > >>> > > > >> > <
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
>>> jolshan@confluent.io
>>> > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> wrote:
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Hi Jun,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Thanks
>>> for
>>> > the
>>> > > > > quick
>>> > > > > > > > > > response!
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > 12. I've
>>> > > decided
>>> > > > > to
>>> > > > > > > bump
>>> > > > > > > > > up
>>> > > > > > > > > > > the
>>> > > > > > > > > > > >>> > > versions
>>> > > > > > > > > > > >>> > > > >> on
>>> > > > > > > > > > > >>> > > > >> > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > requests
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> and
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > updated
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > KIP. I
>>> think
>>> > > > it's
>>> > > > > > > good we
>>> > > > > > > > > > > >>> thoroughly
>>> > > > > > > > > > > >>> > > > >> > discussed
>>> > > > > > > > > > > >>> > > > >> > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > options
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > here,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > so
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > we
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> know
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > we made
>>> a
>>> > good
>>> > > > > > > choice. :)
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > 13.
>>> This is
>>> > an
>>> > > > > > > interesting
>>> > > > > > > > > > > >>> > situation.
>>> > > > > > > > > > > >>> > > I
>>> > > > > > > > > > > >>> > > > >> think
>>> > > > > > > > > > > >>> > > > >> > > if
>>> > > > > > > > > > > >>> > > > >> > > > > this
>>> > > > > > > > > > > >>> > > > >> > > > > > > > does
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > occur
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > we
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> should
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > give a
>>> > > warning.
>>> > > > I
>>> > > > > > > agree
>>> > > > > > > > > that
>>> > > > > > > > > > > >>> it's
>>> > > > > > > > > > > >>> > hard
>>> > > > > > > > > > > >>> > > > to
>>> > > > > > > > > > > >>> > > > >> > know
>>> > > > > > > > > > > >>> > > > >> > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > source
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> of
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > truth
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > for
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> sure
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > since
>>> the
>>> > > > > directory
>>> > > > > > > or the
>>> > > > > > > > > > > file
>>> > > > > > > > > > > >>> > could
>>> > > > > > > > > > > >>> > > be
>>> > > > > > > > > > > >>> > > > >> > > manually
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> modified. I
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > guess
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>>> directory
>>> > > could
>>> > > > be
>>> > > > > > > used as
>>> > > > > > > > > > the
>>> > > > > > > > > > > >>> > source
>>> > > > > > > > > > > >>> > > of
>>> > > > > > > > > > > >>> > > > >> > truth.
>>> > > > > > > > > > > >>> > > > >> > > > To
>>> > > > > > > > > > > >>> > > > >> > > > > be
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> honest,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > I'm
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > not
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > really
>>> sure
>>> > > what
>>> > > > > > > happens
>>> > > > > > > > > in
>>> > > > > > > > > > > >>> kafka
>>> > > > > > > > > > > >>> > when
>>> > > > > > > > > > > >>> > > > the
>>> > > > > > > > > > > >>> > > > >> > log
>>> > > > > > > > > > > >>> > > > >> > > > > > > directory
>>> > > > > > > > > > > >>> > > > >> > > > > > > > is
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > renamed
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>>> manually in
>>> > > > such a
>>> > > > > > > way.
>>> > > > > > > > > I'm
>>> > > > > > > > > > > also
>>> > > > > > > > > > > >>> > > > >> wondering if
>>> > > > > > > > > > > >>> > > > >> > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> situation is
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>>> recoverable
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > in this
>>> > > > scenario.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Thanks,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Justine
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > On Thu,
>>> Sep
>>> > > 17,
>>> > > > > 2020
>>> > > > > > > at
>>> > > > > > > > > 4:28
>>> > > > > > > > > > > PM
>>> > > > > > > > > > > >>> Jun
>>> > > > > > > > > > > >>> > > Rao
>>> > > > > > > > > > > >>> > > > <
>>> > > > > > > > > > > >>> > > > >> > > > > > > > jun@confluent.io>
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > wrote:
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Hi,
>>> > Justine,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>> Thanks for
>>> > > the
>>> > > > > > > reply.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > 12. I
>>> > don't
>>> > > > > have a
>>> > > > > > > > > strong
>>> > > > > > > > > > > >>> > preference
>>> > > > > > > > > > > >>> > > > >> > either.
>>> > > > > > > > > > > >>> > > > >> > > > > > However,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > if
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> we
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > need
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > IBP
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>> anyway,
>>> > > maybe
>>> > > > > it's
>>> > > > > > > > > easier
>>> > > > > > > > > > to
>>> > > > > > > > > > > >>> just
>>> > > > > > > > > > > >>> > > bump
>>> > > > > > > > > > > >>> > > > >> up
>>> > > > > > > > > > > >>> > > > >> > the
>>> > > > > > > > > > > >>> > > > >> > > > > > version
>>> > > > > > > > > > > >>> > > > >> > > > > > > > for
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > all
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > inter
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > broker
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>> requests
>>> > and
>>> > > > add
>>> > > > > > the
>>> > > > > > > > > topic
>>> > > > > > > > > > > id
>>> > > > > > > > > > > >>> > field
>>> > > > > > > > > > > >>> > > > as a
>>> > > > > > > > > > > >>> > > > >> > > > regular
>>> > > > > > > > > > > >>> > > > >> > > > > > > > field. A
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > regular
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> field
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > is
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > a bit
>>> more
>>> > > > > concise
>>> > > > > > > in
>>> > > > > > > > > wire
>>> > > > > > > > > > > >>> > transfer
>>> > > > > > > > > > > >>> > > > >> than a
>>> > > > > > > > > > > >>> > > > >> > > > > flexible
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> field.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > 13.
>>> The
>>> > > > > confusion
>>> > > > > > > that I
>>> > > > > > > > > > was
>>> > > > > > > > > > > >>> > > referring
>>> > > > > > > > > > > >>> > > > >> to
>>> > > > > > > > > > > >>> > > > >> > is
>>> > > > > > > > > > > >>> > > > >> > > > > > between
>>> > > > > > > > > > > >>> > > > >> > > > > > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > topic
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > name
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>> partition
>>> > > > number
>>> > > > > > > between
>>> > > > > > > > > > the
>>> > > > > > > > > > > >>> log
>>> > > > > > > > > > > >>> > dir
>>> > > > > > > > > > > >>> > > > and
>>> > > > > > > > > > > >>> > > > >> > the
>>> > > > > > > > > > > >>> > > > >> > > > > > metadata
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> file.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > For
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> example,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > if
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > the
>>> log
>>> > dir
>>> > > is
>>> > > > > > > topicA-1
>>> > > > > > > > > > and
>>> > > > > > > > > > > >>> the
>>> > > > > > > > > > > >>> > > > metadata
>>> > > > > > > > > > > >>> > > > >> > file
>>> > > > > > > > > > > >>> > > > >> > > > in
>>> > > > > > > > > > > >>> > > > >> > > > > it
>>> > > > > > > > > > > >>> > > > >> > > > > > > has
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > topicB
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > and
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>> partition
>>> > 0
>>> > > > (say
>>> > > > > > > due to
>>> > > > > > > > > a
>>> > > > > > > > > > > bug
>>> > > > > > > > > > > >>> or
>>> > > > > > > > > > > >>> > > > manual
>>> > > > > > > > > > > >>> > > > >> > > > > > > modification),
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> which
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > one
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > do
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > we
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > use
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > as the
>>> > > source
>>> > > > of
>>> > > > > > > truth?
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Jun
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > On
>>> Thu,
>>> > Sep
>>> > > > 17,
>>> > > > > > > 2020 at
>>> > > > > > > > > > 3:43
>>> > > > > > > > > > > >>> PM
>>> > > > > > > > > > > >>> > > > Justine
>>> > > > > > > > > > > >>> > > > >> > > Olshan
>>> > > > > > > > > > > >>> > > > >> > > > <
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
>>> > > jolshan@confluent.io
>>> > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > wrote:
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > Hi
>>> Jun,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> Thanks
>>> > for
>>> > > > the
>>> > > > > > > > > comments.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > 12.
>>> I
>>> > > bumped
>>> > > > > the
>>> > > > > > > > > > > >>> > > LeaderAndIsrRequest
>>> > > > > > > > > > > >>> > > > >> > > because
>>> > > > > > > > > > > >>> > > > >> > > > I
>>> > > > > > > > > > > >>> > > > >> > > > > > > > removed
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > topic
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> name
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > field
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > in
>>> the
>>> > > > > response.
>>> > > > > > > It
>>> > > > > > > > > may
>>> > > > > > > > > > be
>>> > > > > > > > > > > >>> > > possible
>>> > > > > > > > > > > >>> > > > to
>>> > > > > > > > > > > >>> > > > >> > > avoid
>>> > > > > > > > > > > >>> > > > >> > > > > > > bumping
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > version
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > without
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > that
>>> > > change,
>>> > > > > but
>>> > > > > > > I may
>>> > > > > > > > > > be
>>> > > > > > > > > > > >>> > missing
>>> > > > > > > > > > > >>> > > > >> > > something.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > I
>>> > believe
>>> > > > > > > StopReplica
>>> > > > > > > > > is
>>> > > > > > > > > > > >>> > actually
>>> > > > > > > > > > > >>> > > on
>>> > > > > > > > > > > >>> > > > >> > > version
>>> > > > > > > > > > > >>> > > > >> > > > 3
>>> > > > > > > > > > > >>> > > > >> > > > > > now,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > but
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > because
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > version
>>> 2
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > is
>>> > > > flexible, I
>>> > > > > > > kept
>>> > > > > > > > > that
>>> > > > > > > > > > > >>> listed
>>> > > > > > > > > > > >>> > as
>>> > > > > > > > > > > >>> > > > >> > version
>>> > > > > > > > > > > >>> > > > >> > > 2
>>> > > > > > > > > > > >>> > > > >> > > > on
>>> > > > > > > > > > > >>> > > > >> > > > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > KIP
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > page.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> However,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > you
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > may
>>> be
>>> > > right
>>> > > > > in
>>> > > > > > > that
>>> > > > > > > > > we
>>> > > > > > > > > > > may
>>> > > > > > > > > > > >>> need
>>> > > > > > > > > > > >>> > > to
>>> > > > > > > > > > > >>> > > > >> bump
>>> > > > > > > > > > > >>> > > > >> > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > version
>>> > > > > > > > > > > >>> > > > >> > > > > > > > on
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > StopReplica
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > deal
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > with
>>> > > > deletion
>>> > > > > > > > > > differently
>>> > > > > > > > > > > as
>>> > > > > > > > > > > >>> > > > mentioned
>>> > > > > > > > > > > >>> > > > >> > > > above. I
>>> > > > > > > > > > > >>> > > > >> > > > > > > don't
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> know
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > if
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > I
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> have a
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > big
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> > preference
>>> > > > > over
>>> > > > > > > used
>>> > > > > > > > > > > tagged
>>> > > > > > > > > > > >>> > fields
>>> > > > > > > > > > > >>> > > > or
>>> > > > > > > > > > > >>> > > > >> > not.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > 13.
>>> I
>>> > was
>>> > > > > > thinking
>>> > > > > > > > > that
>>> > > > > > > > > > in
>>> > > > > > > > > > > >>> the
>>> > > > > > > > > > > >>> > > case
>>> > > > > > > > > > > >>> > > > >> where
>>> > > > > > > > > > > >>> > > > >> > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > file
>>> > > > > > > > > > > >>> > > > >> > > > > > > > and
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > request
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > topic
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > ids
>>> > don't
>>> > > > > match,
>>> > > > > > > it
>>> > > > > > > > > > means
>>> > > > > > > > > > > >>> that
>>> > > > > > > > > > > >>> > the
>>> > > > > > > > > > > >>> > > > >> > broker's
>>> > > > > > > > > > > >>> > > > >> > > > > > > topic/the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> one
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > in
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> file
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > has
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > been
>>> > > > deleted.
>>> > > > > In
>>> > > > > > > that
>>> > > > > > > > > > > case,
>>> > > > > > > > > > > >>> we
>>> > > > > > > > > > > >>> > > would
>>> > > > > > > > > > > >>> > > > >> need
>>> > > > > > > > > > > >>> > > > >> > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > delete
>>> > > > > > > > > > > >>> > > > >> > > > > > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > old
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > topic
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > start
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> > receiving
>>> > > > the
>>> > > > > > new
>>> > > > > > > > > > version.
>>> > > > > > > > > > > >>> If
>>> > > > > > > > > > > >>> > the
>>> > > > > > > > > > > >>> > > > >> topic
>>> > > > > > > > > > > >>> > > > >> > > name
>>> > > > > > > > > > > >>> > > > >> > > > > were
>>> > > > > > > > > > > >>> > > > >> > > > > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > change,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > but
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > ids
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> still
>>> > > > matched,
>>> > > > > > the
>>> > > > > > > > > file
>>> > > > > > > > > > > >>> would
>>> > > > > > > > > > > >>> > also
>>> > > > > > > > > > > >>> > > > >> need
>>> > > > > > > > > > > >>> > > > >> > to
>>> > > > > > > > > > > >>> > > > >> > > > > > update.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > Am I
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > missing
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > a
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> case
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> where
>>> > the
>>> > > > file
>>> > > > > > > would
>>> > > > > > > > > be
>>> > > > > > > > > > > >>> correct
>>> > > > > > > > > > > >>> > > and
>>> > > > > > > > > > > >>> > > > >> not
>>> > > > > > > > > > > >>> > > > >> > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > request?
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> Thanks,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> Justine
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > On
>>> Thu,
>>> > > Sep
>>> > > > > 17,
>>> > > > > > > 2020
>>> > > > > > > > > at
>>> > > > > > > > > > > >>> 3:18 PM
>>> > > > > > > > > > > >>> > > Jun
>>> > > > > > > > > > > >>> > > > >> Rao <
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> jun@confluent.io
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > wrote:
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> Hi,
>>> > > > Justine,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> Thanks
>>> > > for
>>> > > > > the
>>> > > > > > > > > reply.
>>> > > > > > > > > > A
>>> > > > > > > > > > > >>> couple
>>> > > > > > > > > > > >>> > > of
>>> > > > > > > > > > > >>> > > > >> more
>>> > > > > > > > > > > >>> > > > >> > > > > comments
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> below.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> 12.
>>> > > > > ListOffset
>>> > > > > > > and
>>> > > > > > > > > > > >>> > > OffsetForLeader
>>> > > > > > > > > > > >>> > > > >> > > > currently
>>> > > > > > > > > > > >>> > > > >> > > > > > > don't
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > support
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> flexible
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> fields.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> So, we
>>> > > > have
>>> > > > > to
>>> > > > > > > bump
>>> > > > > > > > > up
>>> > > > > > > > > > > the
>>> > > > > > > > > > > >>> > > version
>>> > > > > > > > > > > >>> > > > >> > number
>>> > > > > > > > > > > >>> > > > >> > > > and
>>> > > > > > > > > > > >>> > > > >> > > > > > use
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> IBP at
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > least
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > for
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > these
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> two
>>> > > > > requests.
>>> > > > > > > Note
>>> > > > > > > > > > that
>>> > > > > > > > > > > it
>>> > > > > > > > > > > >>> > seems
>>> > > > > > > > > > > >>> > > > >> 2.7.0
>>> > > > > > > > > > > >>> > > > >> > > will
>>> > > > > > > > > > > >>> > > > >> > > > > > > require
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> IBP
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > anyway
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > because
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > of
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> > changes
>>> > > in
>>> > > > > > > > > > KAFKA-10435.
>>> > > > > > > > > > > >>> Also,
>>> > > > > > > > > > > >>> > it
>>> > > > > > > > > > > >>> > > > >> seems
>>> > > > > > > > > > > >>> > > > >> > > that
>>> > > > > > > > > > > >>> > > > >> > > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> version
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > for
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> > > > > > > LeaderAndIsrRequest
>>> > > > > > > > > > and
>>> > > > > > > > > > > >>> > > > StopReplica
>>> > > > > > > > > > > >>> > > > >> are
>>> > > > > > > > > > > >>> > > > >> > > > > bumped
>>> > > > > > > > > > > >>> > > > >> > > > > > > even
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > though
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > we
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > only
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > added
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > a
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> tagged
>>> > > > > field.
>>> > > > > > > But
>>> > > > > > > > > > since
>>> > > > > > > > > > > >>> IBP is
>>> > > > > > > > > > > >>> > > > >> needed
>>> > > > > > > > > > > >>> > > > >> > > > anyway,
>>> > > > > > > > > > > >>> > > > >> > > > > > we
>>> > > > > > > > > > > >>> > > > >> > > > > > > > may
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > want
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> revisit
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> > overall
>>> > > > > tagged
>>> > > > > > > field
>>> > > > > > > > > > > >>> choice.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> 13.
>>> > The
>>> > > > only
>>> > > > > > > > > downside
>>> > > > > > > > > > is
>>> > > > > > > > > > > >>> the
>>> > > > > > > > > > > >>> > > > >> potential
>>> > > > > > > > > > > >>> > > > >> > > > > > confusion
>>> > > > > > > > > > > >>> > > > >> > > > > > > on
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > which
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > one
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > is
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> source
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > of
>>> > truth
>>> > > > if
>>> > > > > > they
>>> > > > > > > > > don't
>>> > > > > > > > > > > >>> match.
>>> > > > > > > > > > > >>> > > > >> Another
>>> > > > > > > > > > > >>> > > > >> > > > option
>>> > > > > > > > > > > >>> > > > >> > > > > is
>>> > > > > > > > > > > >>> > > > >> > > > > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > include
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > those
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > fields
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > in
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> the
>>> > > > metadata
>>> > > > > > > file
>>> > > > > > > > > when
>>> > > > > > > > > > > we
>>> > > > > > > > > > > >>> > > actually
>>> > > > > > > > > > > >>> > > > >> > change
>>> > > > > > > > > > > >>> > > > >> > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> directory
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > structure.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> > Thanks,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> Jun
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > On
>>> > Thu,
>>> > > > Sep
>>> > > > > > 17,
>>> > > > > > > 2020
>>> > > > > > > > > > at
>>> > > > > > > > > > > >>> 2:01
>>> > > > > > > > > > > >>> > PM
>>> > > > > > > > > > > >>> > > > >> Justine
>>> > > > > > > > > > > >>> > > > >> > > > > Olshan
>>> > > > > > > > > > > >>> > > > >> > > > > > <
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>>> > > > > jolshan@confluent.io
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> wrote:
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > Hello
>>> > > > all,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> I've
>>> > > > > thought
>>> > > > > > > some
>>> > > > > > > > > > more
>>> > > > > > > > > > > >>> about
>>> > > > > > > > > > > >>> > > > >> removing
>>> > > > > > > > > > > >>> > > > >> > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > topic
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> name
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > field
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > from
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > some
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > of
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > > > requests.
>>> > > > > On
>>> > > > > > > > > closer
>>> > > > > > > > > > > >>> > inspection
>>> > > > > > > > > > > >>> > > > of
>>> > > > > > > > > > > >>> > > > >> the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > requests/responses,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > it
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> seems
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > > internal
>>> > > > > > > changes
>>> > > > > > > > > > would
>>> > > > > > > > > > > >>> be
>>> > > > > > > > > > > >>> > much
>>> > > > > > > > > > > >>> > > > >> larger
>>> > > > > > > > > > > >>> > > > >> > > > than
>>> > > > > > > > > > > >>> > > > >> > > > > I
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> expected.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > Some
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>>> protocols
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > > involve
>>> > > > > > > clients,
>>> > > > > > > > > so
>>> > > > > > > > > > > they
>>> > > > > > > > > > > >>> > would
>>> > > > > > > > > > > >>> > > > >> > require
>>> > > > > > > > > > > >>> > > > >> > > > > > changes
>>> > > > > > > > > > > >>> > > > >> > > > > > > > too.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > I'm
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > thinking
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > that
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > for
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> now,
>>> > > > > > removing
>>> > > > > > > the
>>> > > > > > > > > > > topic
>>> > > > > > > > > > > >>> name
>>> > > > > > > > > > > >>> > > > from
>>> > > > > > > > > > > >>> > > > >> > these
>>> > > > > > > > > > > >>> > > > >> > > > > > > requests
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> and
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > responses
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> are
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > out
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > scope.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> I
>>> > have
>>> > > > > > > decided to
>>> > > > > > > > > > just
>>> > > > > > > > > > > >>> keep
>>> > > > > > > > > > > >>> > > the
>>> > > > > > > > > > > >>> > > > >> > change
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
>>> LeaderAndIsrResponse
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > remove
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > topic
>>> > > > > name,
>>> > > > > > > and
>>> > > > > > > > > have
>>> > > > > > > > > > > >>> updated
>>> > > > > > > > > > > >>> > > the
>>> > > > > > > > > > > >>> > > > >> KIP
>>> > > > > > > > > > > >>> > > > >> > to
>>> > > > > > > > > > > >>> > > > >> > > > > > reflect
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> this
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > change. I
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> have
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > also
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > > > mentioned
>>> > > > > > the
>>> > > > > > > > > other
>>> > > > > > > > > > > >>> requests
>>> > > > > > > > > > > >>> > > and
>>> > > > > > > > > > > >>> > > > >> > > > responses
>>> > > > > > > > > > > >>> > > > >> > > > > in
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> future
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > work.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> I'm
>>> > > > hoping
>>> > > > > > to
>>> > > > > > > > > start
>>> > > > > > > > > > > the
>>> > > > > > > > > > > >>> > voting
>>> > > > > > > > > > > >>> > > > >> > process
>>> > > > > > > > > > > >>> > > > >> > > > > soon,
>>> > > > > > > > > > > >>> > > > >> > > > > > so
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> let me
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > know
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > if
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > there
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > is
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > > anything
>>> > > > > > else
>>> > > > > > > we
>>> > > > > > > > > > > should
>>> > > > > > > > > > > >>> > > discuss.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > Thank
>>> > > > you,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > > Justine
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> On
>>> > > Tue,
>>> > > > > Sep
>>> > > > > > > 15,
>>> > > > > > > > > 2020
>>> > > > > > > > > > > at
>>> > > > > > > > > > > >>> 3:57
>>> > > > > > > > > > > >>> > > PM
>>> > > > > > > > > > > >>> > > > >> > Justine
>>> > > > > > > > > > > >>> > > > >> > > > > > Olshan
>>> > > > > > > > > > > >>> > > > >> > > > > > > <
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>>> > > > > > jolshan@confluent.io
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > wrote:
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >
>>> > > Hello
>>> > > > > > again,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > To
>>> > > > > follow
>>> > > > > > > up on
>>> > > > > > > > > > some
>>> > > > > > > > > > > >>> of
>>> > > > > > > > > > > >>> > the
>>> > > > > > > > > > > >>> > > > >> other
>>> > > > > > > > > > > >>> > > > >> > > > > comments:
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >
>>> > > 10/11)
>>> > > > > We
>>> > > > > > > can
>>> > > > > > > > > > remove
>>> > > > > > > > > > > >>> the
>>> > > > > > > > > > > >>> > > topic
>>> > > > > > > > > > > >>> > > > >> name
>>> > > > > > > > > > > >>> > > > >> > > > from
>>> > > > > > > > > > > >>> > > > >> > > > > > > these
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>>> > > > > requests/responses,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > and
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >
>>> > that
>>> > > > > means
>>> > > > > > > we
>>> > > > > > > > > will
>>> > > > > > > > > > > >>> just
>>> > > > > > > > > > > >>> > have
>>> > > > > > > > > > > >>> > > > to
>>> > > > > > > > > > > >>> > > > >> > make
>>> > > > > > > > > > > >>> > > > >> > > a
>>> > > > > > > > > > > >>> > > > >> > > > > few
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> internal
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > changes
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > make
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >
>>> > > > > partitions
>>> > > > > > > > > > > accessible
>>> > > > > > > > > > > >>> by
>>> > > > > > > > > > > >>> > > topic
>>> > > > > > > > > > > >>> > > > >> id
>>> > > > > > > > > > > >>> > > > >> > and
>>> > > > > > > > > > > >>> > > > >> > > > > > > > partition.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> I
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > can
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > update
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > KIP
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >
>>> > > remove
>>> > > > > > them
>>> > > > > > > > > unless
>>> > > > > > > > > > > >>> anyone
>>> > > > > > > > > > > >>> > > > thinks
>>> > > > > > > > > > > >>> > > > >> > they
>>> > > > > > > > > > > >>> > > > >> > > > > > should
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> stay.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >
>>> > 12)
>>> > > > > > > Addressed in
>>> > > > > > > > > > the
>>> > > > > > > > > > > >>> > > previous
>>> > > > > > > > > > > >>> > > > >> > email.
>>> > > > > > > > > > > >>> > > > >> > > > I've
>>> > > > > > > > > > > >>> > > > >> > > > > > > > updated
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > KIP
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>> include
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >
>>> > > tagged
>>> > > > > > > fields
>>> > > > > > > > > for
>>> > > > > > > > > > > the
>>> > > > > > > > > > > >>> > > requests
>>> > > > > > > > > > > >>> > > > >> and
>>> > > > > > > > > > > >>> > > > >> > > > > > responses.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> (More
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > on
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > below)
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >
>>> > 13)
>>> > > I
>>> > > > > > think
>>> > > > > > > part
>>> > > > > > > > > > of
>>> > > > > > > > > > > >>> the
>>> > > > > > > > > > > >>> > idea
>>> > > > > > > > > > > >>> > > > for
>>> > > > > > > > > > > >>> > > > >> > > > > including
>>> > > > > > > > > > > >>> > > > >> > > > > > > this
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > information
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> is to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> > prepare
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >
>>> > for
>>> > > > > future
>>> > > > > > > > > > changes.
>>> > > > > > > > > > > >>> > Perhaps
>>> > > > > > > > > > > >>> > > > the
>>> > > > > > > > > > > >>> > > > >> > > > directory
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> structure
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > might
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> change
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > from
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >
>>> > > > > > > > > > > >>> topicName_partitionNumber
>>> > > > > > > > > > > >>> > to
>>> > > > > > > > > > > >>> > > > >> > > something
>>> > > > > > > > > > > >>> > > > >> > > > > like
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>> > > > > > > topicID_partitionNumber.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> Then
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > it
>>> > > > would
>>> > > > > > be
>>> > > > > > > > > useful
>>> > > > > > > > > > > to
>>> > > > > > > > > > > >>> have
>>> > > > > > > > > > > >>> > > the
>>> > > > > > > > > > > >>> > > > >> > topic
>>> > > > > > > > > > > >>> > > > >> > > > name
>>> > > > > > > > > > > >>> > > > >> > > > > > in
>>> > > > > > > > > > > >>> > > > >> > > > > > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > file
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > since
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > it
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > would
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> not
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> be
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > in
>>> > > the
>>> > > > > > > directory
>>> > > > > > > > > > > >>> > structure.
>>> > > > > > > > > > > >>> > > > >> > > Supporting
>>> > > > > > > > > > > >>> > > > >> > > > > > topic
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> renames
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > might
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > be
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > easier
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > if
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >
>>> > > other
>>> > > > > > > fields are
>>> > > > > > > > > > > >>> included.
>>> > > > > > > > > > > >>> > > > Would
>>> > > > > > > > > > > >>> > > > >> > > there
>>> > > > > > > > > > > >>> > > > >> > > > be
>>> > > > > > > > > > > >>> > > > >> > > > > > any
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > downsides
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>> including
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> this
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >
>>> > > > > > information?
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >
>>> > 14)
>>> > > > > Yes,
>>> > > > > > we
>>> > > > > > > > > would
>>> > > > > > > > > > > >>> need to
>>> > > > > > > > > > > >>> > > > copy
>>> > > > > > > > > > > >>> > > > >> the
>>> > > > > > > > > > > >>> > > > >> > > > > > partition
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > metadata
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > file
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > in
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > this
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >
>>> > > > process.
>>> > > > > > > I've
>>> > > > > > > > > > > updated
>>> > > > > > > > > > > >>> the
>>> > > > > > > > > > > >>> > > KIP
>>> > > > > > > > > > > >>> > > > to
>>> > > > > > > > > > > >>> > > > >> > > > include
>>> > > > > > > > > > > >>> > > > >> > > > > > > this.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >
>>> > 15)
>>> > > I
>>> > > > > > > believe
>>> > > > > > > > > > Lucas
>>> > > > > > > > > > > >>> meant
>>> > > > > > > > > > > >>> > v1
>>> > > > > > > > > > > >>> > > > >> and v2
>>> > > > > > > > > > > >>> > > > >> > > > here.
>>> > > > > > > > > > > >>> > > > >> > > > > > He
>>> > > > > > > > > > > >>> > > > >> > > > > > > > was
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > referring
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> how
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >
>>> > > > requests
>>> > > > > > > would
>>> > > > > > > > > > fall
>>> > > > > > > > > > > >>> under
>>> > > > > > > > > > > >>> > > > >> different
>>> > > > > > > > > > > >>> > > > >> > > IBP
>>> > > > > > > > > > > >>> > > > >> > > > > and
>>> > > > > > > > > > > >>> > > > >> > > > > > > > meant
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > that
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > older
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>> brokers
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > would
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >
>>> > have
>>> > > > to
>>> > > > > > use
>>> > > > > > > the
>>> > > > > > > > > > > older
>>> > > > > > > > > > > >>> > > version
>>> > > > > > > > > > > >>> > > > of
>>> > > > > > > > > > > >>> > > > >> > the
>>> > > > > > > > > > > >>> > > > >> > > > > > request
>>> > > > > > > > > > > >>> > > > >> > > > > > > > and
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > existing
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > topic
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >
>>> > > > deletion
>>> > > > > > > > > process.
>>> > > > > > > > > > At
>>> > > > > > > > > > > >>> > first,
>>> > > > > > > > > > > >>> > > it
>>> > > > > > > > > > > >>> > > > >> > seemed
>>> > > > > > > > > > > >>> > > > >> > > > > like
>>> > > > > > > > > > > >>> > > > >> > > > > > > > tagged
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > fields
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > would
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> resolve
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >
>>> > the
>>> > > > IBP
>>> > > > > > > issue.
>>> > > > > > > > > > > >>> However, we
>>> > > > > > > > > > > >>> > > may
>>> > > > > > > > > > > >>> > > > >> need
>>> > > > > > > > > > > >>> > > > >> > > IBP
>>> > > > > > > > > > > >>> > > > >> > > > > for
>>> > > > > > > > > > > >>> > > > >> > > > > > > > this
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > request
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > after
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > all
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> since
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >
>>> > the
>>> > > > > > > controller
>>> > > > > > > > > > > >>> handles the
>>> > > > > > > > > > > >>> > > > topic
>>> > > > > > > > > > > >>> > > > >> > > > deletion
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > differently
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> depending
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > on
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> IBP
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >
>>> > > > version.
>>> > > > > > In
>>> > > > > > > an
>>> > > > > > > > > > older
>>> > > > > > > > > > > >>> > > version,
>>> > > > > > > > > > > >>> > > > we
>>> > > > > > > > > > > >>> > > > >> > > can't
>>> > > > > > > > > > > >>> > > > >> > > > > just
>>> > > > > > > > > > > >>> > > > >> > > > > > > > send
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> a
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > StopReplica
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > delete
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >
>>> > > topic
>>> > > > > > > > > immediately
>>> > > > > > > > > > > like
>>> > > > > > > > > > > >>> > we'd
>>> > > > > > > > > > > >>> > > > >> want to
>>> > > > > > > > > > > >>> > > > >> > > for
>>> > > > > > > > > > > >>> > > > >> > > > > > this
>>> > > > > > > > > > > >>> > > > >> > > > > > > > KIP.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >
>>> > This
>>> > > > > makes
>>> > > > > > > me
>>> > > > > > > > > > wonder
>>> > > > > > > > > > > >>> if we
>>> > > > > > > > > > > >>> > > > want
>>> > > > > > > > > > > >>> > > > >> > > tagged
>>> > > > > > > > > > > >>> > > > >> > > > > > fields
>>> > > > > > > > > > > >>> > > > >> > > > > > > > on
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> all
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> requests
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> after
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >
>>> > all.
>>> > > > Let
>>> > > > > > me
>>> > > > > > > know
>>> > > > > > > > > > > your
>>> > > > > > > > > > > >>> > > > thoughts!
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >
>>> > > > Justine
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > On
>>> > > > Tue,
>>> > > > > > Sep
>>> > > > > > > 15,
>>> > > > > > > > > > 2020
>>> > > > > > > > > > > >>> at
>>> > > > > > > > > > > >>> > 1:03
>>> > > > > > > > > > > >>> > > > PM
>>> > > > > > > > > > > >>> > > > >> > > Justine
>>> > > > > > > > > > > >>> > > > >> > > > > > > Olshan
>>> > > > > > > > > > > >>> > > > >> > > > > > > > <
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>> > > > > > > jolshan@confluent.io
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >
>>> > > wrote:
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>
>>> > Hi
>>> > > > all,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>
>>> > Jun
>>> > > > > > > brought up
>>> > > > > > > > > a
>>> > > > > > > > > > > good
>>> > > > > > > > > > > >>> > point
>>> > > > > > > > > > > >>> > > > in
>>> > > > > > > > > > > >>> > > > >> his
>>> > > > > > > > > > > >>> > > > >> > > > last
>>> > > > > > > > > > > >>> > > > >> > > > > > > email
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> about
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > tagged
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > fields,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > and
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>
>>> > > I've
>>> > > > > > > updated
>>> > > > > > > > > the
>>> > > > > > > > > > > KIP
>>> > > > > > > > > > > >>> to
>>> > > > > > > > > > > >>> > > > reflect
>>> > > > > > > > > > > >>> > > > >> > that
>>> > > > > > > > > > > >>> > > > >> > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> changes to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > requests
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > > > responses
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>
>>> > > will
>>> > > > be
>>> > > > > > in
>>> > > > > > > the
>>> > > > > > > > > > form
>>> > > > > > > > > > > >>> of
>>> > > > > > > > > > > >>> > > tagged
>>> > > > > > > > > > > >>> > > > >> > fields
>>> > > > > > > > > > > >>> > > > >> > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > avoid
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > changing
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > IBP.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>
>>> > > Jun:
>>> > > > I
>>> > > > > > > plan on
>>> > > > > > > > > > > >>> sending a
>>> > > > > > > > > > > >>> > > > >> followup
>>> > > > > > > > > > > >>> > > > >> > > > email
>>> > > > > > > > > > > >>> > > > >> > > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> address
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > some
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > of
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > other
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>
>>> > > > points.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>
>>> > > > Thanks,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>
>>> > > > Justine
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>
>>> > On
>>> > > > Mon,
>>> > > > > > > Sep 14,
>>> > > > > > > > > > > 2020
>>> > > > > > > > > > > >>> at
>>> > > > > > > > > > > >>> > > 4:25
>>> > > > > > > > > > > >>> > > > PM
>>> > > > > > > > > > > >>> > > > >> > Jun
>>> > > > > > > > > > > >>> > > > >> > > > Rao
>>> > > > > > > > > > > >>> > > > >> > > > > <
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
>>> jun@confluent.io>
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > wrote:
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > Hi,
>>> > > > > > > Justine,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > Thanks
>>> > > > > > > for the
>>> > > > > > > > > > > >>> updated
>>> > > > > > > > > > > >>> > > KIP.
>>> > > > > > > > > > > >>> > > > A
>>> > > > > > > > > > > >>> > > > >> few
>>> > > > > > > > > > > >>> > > > >> > > > > > comments
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> below.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > 10.
>>> > > > > > > > > LeaderAndIsr
>>> > > > > > > > > > > >>> > Response:
>>> > > > > > > > > > > >>> > > > Do
>>> > > > > > > > > > > >>> > > > >> we
>>> > > > > > > > > > > >>> > > > >> > > need
>>> > > > > > > > > > > >>> > > > >> > > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > topic
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > name?
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > 11.
>>> > > > > For
>>> > > > > > > the
>>> > > > > > > > > > > changed
>>> > > > > > > > > > > >>> > > > >> > > request/response,
>>> > > > > > > > > > > >>> > > > >> > > > > > other
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> than
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>>> LeaderAndIsr,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > > > > > UpdateMetadata,
>>> > > > > > > > > > > >>> > Metadata,
>>> > > > > > > > > > > >>> > > do
>>> > > > > > > > > > > >>> > > > >> we
>>> > > > > > > > > > > >>> > > > >> > > need
>>> > > > > > > > > > > >>> > > > >> > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > include
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > topic
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> name?
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > 12.
>>> > > > It
>>> > > > > > > seems
>>> > > > > > > > > > that
>>> > > > > > > > > > > >>> > upgrades
>>> > > > > > > > > > > >>> > > > >> don't
>>> > > > > > > > > > > >>> > > > >> > > > > require
>>> > > > > > > > > > > >>> > > > >> > > > > > > IBP.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> Does
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > that
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > mean
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > new
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > fields
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > in
>>> > > > all
>>> > > > > > the
>>> > > > > > > > > > > >>> > > request/response
>>> > > > > > > > > > > >>> > > > >> are
>>> > > > > > > > > > > >>> > > > >> > > added
>>> > > > > > > > > > > >>> > > > >> > > > > as
>>> > > > > > > > > > > >>> > > > >> > > > > > > > tagged
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > fields
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> without
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> > bumping
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> up
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > the
>>> > > > > > > request
>>> > > > > > > > > > > >>> version? It
>>> > > > > > > > > > > >>> > > > would
>>> > > > > > > > > > > >>> > > > >> be
>>> > > > > > > > > > > >>> > > > >> > > > useful
>>> > > > > > > > > > > >>> > > > >> > > > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > make
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > that
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > clear.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > 13.
>>> > > > > > > Partition
>>> > > > > > > > > > > >>> Metadata
>>> > > > > > > > > > > >>> > > file:
>>> > > > > > > > > > > >>> > > > >> Do
>>> > > > > > > > > > > >>> > > > >> > we
>>> > > > > > > > > > > >>> > > > >> > > > need
>>> > > > > > > > > > > >>> > > > >> > > > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> include
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > topic
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > name
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > and
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > > partition
>>> > > > > > > id
>>> > > > > > > > > > since
>>> > > > > > > > > > > >>> they
>>> > > > > > > > > > > >>> > > are
>>> > > > > > > > > > > >>> > > > >> > implied
>>> > > > > > > > > > > >>> > > > >> > > > in
>>> > > > > > > > > > > >>> > > > >> > > > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > directory
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > name?
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > 14.
>>> > > > In
>>> > > > > > the
>>> > > > > > > > > JBOD
>>> > > > > > > > > > > >>> mode, we
>>> > > > > > > > > > > >>> > > > >> support
>>> > > > > > > > > > > >>> > > > >> > > > > moving a
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > partition's
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > data
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> from
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > one
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> disk
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > another.
>>> > > > > > > Will
>>> > > > > > > > > > the
>>> > > > > > > > > > > >>> new
>>> > > > > > > > > > > >>> > > > >> partition
>>> > > > > > > > > > > >>> > > > >> > > > > metadata
>>> > > > > > > > > > > >>> > > > >> > > > > > > file
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> be
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > copied
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> during
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > process?
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > 15.
>>> > > > > The
>>> > > > > > > KIP
>>> > > > > > > > > says
>>> > > > > > > > > > > >>> "Remove
>>> > > > > > > > > > > >>> > > > >> deleted
>>> > > > > > > > > > > >>> > > > >> > > > topics
>>> > > > > > > > > > > >>> > > > >> > > > > > > from
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > replicas
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > by
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > sending
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > > > > > > StopReplicaRequest
>>> > > > > > > > > > > >>> V2
>>> > > > > > > > > > > >>> > for
>>> > > > > > > > > > > >>> > > > any
>>> > > > > > > > > > > >>> > > > >> > > topics
>>> > > > > > > > > > > >>> > > > >> > > > > > which
>>> > > > > > > > > > > >>> > > > >> > > > > > > do
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> not
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > contain
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > a
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > topic
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > ID,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> and
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > V3
>>> > > > for
>>> > > > > > any
>>> > > > > > > > > > topics
>>> > > > > > > > > > > >>> which
>>> > > > > > > > > > > >>> > do
>>> > > > > > > > > > > >>> > > > >> > contain
>>> > > > > > > > > > > >>> > > > >> > > a
>>> > > > > > > > > > > >>> > > > >> > > > > > topic
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> ID.".
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > However,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > it
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > seems
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > updated
>>> > > > > > > > > > controller
>>> > > > > > > > > > > >>> will
>>> > > > > > > > > > > >>> > > > create
>>> > > > > > > > > > > >>> > > > >> > all
>>> > > > > > > > > > > >>> > > > >> > > > > > missing
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> topic
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > IDs
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > first
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > before
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> doing
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > other
>>> > > > > > > actions.
>>> > > > > > > > > > So,
>>> > > > > > > > > > > >>> is
>>> > > > > > > > > > > >>> > > > >> > > > > StopReplicaRequest
>>> > > > > > > > > > > >>> > > > >> > > > > > V2
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > needed?
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > Jun
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > On
>>> > > > > Fri,
>>> > > > > > > Sep
>>> > > > > > > > > 11,
>>> > > > > > > > > > > >>> 2020 at
>>> > > > > > > > > > > >>> > > > 10:31
>>> > > > > > > > > > > >>> > > > >> AM
>>> > > > > > > > > > > >>> > > > >> > > John
>>> > > > > > > > > > > >>> > > > >> > > > > > > > Roesler <
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>> > > > > > vvcephei@apache.org
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > wrote:
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > > > Thanks,
>>> > > > > > > > > > Justine!
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > Your
>>> > > > > > > > > response
>>> > > > > > > > > > > >>> seems
>>> > > > > > > > > > > >>> > > > >> compelling
>>> > > > > > > > > > > >>> > > > >> > to
>>> > > > > > > > > > > >>> > > > >> > > > me.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > > -John
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > On
>>> > > > > > Fri,
>>> > > > > > > > > > > >>> 2020-09-11 at
>>> > > > > > > > > > > >>> > > > 10:17
>>> > > > > > > > > > > >>> > > > >> > > -0700,
>>> > > > > > > > > > > >>> > > > >> > > > > > > Justine
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > Olshan
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > wrote:
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > > > Hello
>>> > > > > > > all,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > > > > Thanks for
>>> > > > > > > > > > > >>> > continuing
>>> > > > > > > > > > > >>> > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > discussion! I
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> have a
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > few
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>>> responses
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> your
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > > > points.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > > Tom:
>>> > > > > > > You
>>> > > > > > > > > are
>>> > > > > > > > > > > >>> correct
>>> > > > > > > > > > > >>> > > in
>>> > > > > > > > > > > >>> > > > >> that
>>> > > > > > > > > > > >>> > > > >> > > this
>>> > > > > > > > > > > >>> > > > >> > > > > KIP
>>> > > > > > > > > > > >>> > > > >> > > > > > > has
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> not
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > mentioned
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > > > > > > > > >>> > DeleteTopicsRequest. I
>>> > > > > > > > > > > >>> > > > >> think
>>> > > > > > > > > > > >>> > > > >> > > that
>>> > > > > > > > > > > >>> > > > >> > > > > > this
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> would
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > be
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > out
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > of
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > scope
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > for
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > now,
>>> > > > > but
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > > may
>>> > > > > > be
>>> > > > > > > > > > > something
>>> > > > > > > > > > > >>> > worth
>>> > > > > > > > > > > >>> > > > >> adding
>>> > > > > > > > > > > >>> > > > >> > > in
>>> > > > > > > > > > > >>> > > > >> > > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> future.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > > > John:
>>> > > > > > > We
>>> > > > > > > > > did
>>> > > > > > > > > > > >>> > consider
>>> > > > > > > > > > > >>> > > > >> > sequence
>>> > > > > > > > > > > >>> > > > >> > > > ids,
>>> > > > > > > > > > > >>> > > > >> > > > > > but
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> there
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > are
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > a
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > few
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>> reasons
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > favor
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > > > UUIDs.
>>> > > > > > > > > There
>>> > > > > > > > > > > are
>>> > > > > > > > > > > >>> > > several
>>> > > > > > > > > > > >>> > > > >> > cases
>>> > > > > > > > > > > >>> > > > >> > > > > where
>>> > > > > > > > > > > >>> > > > >> > > > > > > > topics
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > from
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> different
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> > clusters
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > may
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > > > > interact
>>> > > > > > > > > now
>>> > > > > > > > > > > >>> and in
>>> > > > > > > > > > > >>> > > the
>>> > > > > > > > > > > >>> > > > >> > future.
>>> > > > > > > > > > > >>> > > > >> > > > For
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> example,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > Mirror
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Maker 2
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > may
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > benefit
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > > from
>>> > > > > > > being
>>> > > > > > > > > > > able
>>> > > > > > > > > > > >>> to
>>> > > > > > > > > > > >>> > > > detect
>>> > > > > > > > > > > >>> > > > >> > when
>>> > > > > > > > > > > >>> > > > >> > > a
>>> > > > > > > > > > > >>> > > > >> > > > > > > cluster
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> being
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > mirrored
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> is
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> > deleted
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > and
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > > > > recreated
>>> > > > > > > > > > and
>>> > > > > > > > > > > >>> > globally
>>> > > > > > > > > > > >>> > > > >> unique
>>> > > > > > > > > > > >>> > > > >> > > > > > > identifiers
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > would
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > make
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>> resolving
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > issues
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > > > easier
>>> > > > > > > > > than
>>> > > > > > > > > > > >>> sequence
>>> > > > > > > > > > > >>> > > IDs
>>> > > > > > > > > > > >>> > > > >> > which
>>> > > > > > > > > > > >>> > > > >> > > > may
>>> > > > > > > > > > > >>> > > > >> > > > > > > > collide
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > between
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>>> clusters.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > > KIP-405
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > > > > (tiered
>>> > > > > > > > > > > storage)
>>> > > > > > > > > > > >>> > will
>>> > > > > > > > > > > >>> > > > also
>>> > > > > > > > > > > >>> > > > >> > > > benefit
>>> > > > > > > > > > > >>> > > > >> > > > > > from
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > globally
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > unique
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> IDs
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > as
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > shared
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > > > > buckets
>>> > > > > > > > > may
>>> > > > > > > > > > be
>>> > > > > > > > > > > >>> used
>>> > > > > > > > > > > >>> > > > >> between
>>> > > > > > > > > > > >>> > > > >> > > > > clusters.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > > > > Globally
>>> > > > > > > > > > > unique
>>> > > > > > > > > > > >>> IDs
>>> > > > > > > > > > > >>> > > > would
>>> > > > > > > > > > > >>> > > > >> > also
>>> > > > > > > > > > > >>> > > > >> > > > make
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > functionality
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > like
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > moving
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > topics
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > > > > between
>>> > > > > > > > > > > >>> disparate
>>> > > > > > > > > > > >>> > > > clusters
>>> > > > > > > > > > > >>> > > > >> > > easier
>>> > > > > > > > > > > >>> > > > >> > > > > in
>>> > > > > > > > > > > >>> > > > >> > > > > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > future,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> simplify
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > any
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > future
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > > > > > > > > implementations
>>> > > > > > > > > > > >>> of
>>> > > > > > > > > > > >>> > > > backups
>>> > > > > > > > > > > >>> > > > >> > and
>>> > > > > > > > > > > >>> > > > >> > > > > > > restores,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> and
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > more.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > In
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>> general,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > unique
>>> > > > > > IDs
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > > > would
>>> > > > > > > > > ensure
>>> > > > > > > > > > > >>> that
>>> > > > > > > > > > > >>> > the
>>> > > > > > > > > > > >>> > > > >> source
>>> > > > > > > > > > > >>> > > > >> > > > > cluster
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> topics do
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > not
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> conflict
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > with
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > > > > > > destination
>>> > > > > > > > > > > >>> cluster
>>> > > > > > > > > > > >>> > > > >> topics.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > If
>>> > > > > > we
>>> > > > > > > were
>>> > > > > > > > > > to
>>> > > > > > > > > > > >>> use
>>> > > > > > > > > > > >>> > > > sequence
>>> > > > > > > > > > > >>> > > > >> > ids,
>>> > > > > > > > > > > >>> > > > >> > > > we
>>> > > > > > > > > > > >>> > > > >> > > > > > > would
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> need
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>>> sufficiently
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> large
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > cluster
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > > ids
>>> > > > > > > to be
>>> > > > > > > > > > > stored
>>> > > > > > > > > > > >>> > with
>>> > > > > > > > > > > >>> > > > the
>>> > > > > > > > > > > >>> > > > >> > topic
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> identifiers or
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > we
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > run
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > risk
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > > > > > > collisions.
>>> > > > > > > > > > > This
>>> > > > > > > > > > > >>> > will
>>> > > > > > > > > > > >>> > > > >> give up
>>> > > > > > > > > > > >>> > > > >> > > any
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> advantage in
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>>> compactness
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > sequence
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > > > > numbers
>>> > > > > > > > > may
>>> > > > > > > > > > > >>> bring.
>>> > > > > > > > > > > >>> > > Given
>>> > > > > > > > > > > >>> > > > >> > these
>>> > > > > > > > > > > >>> > > > >> > > > > > > > advantages I
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > think
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > it
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> makes
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> sense
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > use
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > > > UUIDs.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > > > Gokul:
>>> > > > > > > > > This
>>> > > > > > > > > > is
>>> > > > > > > > > > > >>> an
>>> > > > > > > > > > > >>> > > > >> interesting
>>> > > > > > > > > > > >>> > > > >> > > > idea,
>>> > > > > > > > > > > >>> > > > >> > > > > > but
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> this
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > is
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > a
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> breaking
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> > change.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > Out
>>> > > > of
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > > > scope
>>> > > > > > > for
>>> > > > > > > > > > now,
>>> > > > > > > > > > > >>> but
>>> > > > > > > > > > > >>> > > maybe
>>> > > > > > > > > > > >>> > > > >> > worth
>>> > > > > > > > > > > >>> > > > >> > > > > > > discussing
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> in
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > future.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > > Hope
>>> > > > > > > this
>>> > > > > > > > > > > >>> explains
>>> > > > > > > > > > > >>> > > some
>>> > > > > > > > > > > >>> > > > of
>>> > > > > > > > > > > >>> > > > >> > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > decisions,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > > > > Justine
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > On
>>> > > > > > > Fri,
>>> > > > > > > > > Sep
>>> > > > > > > > > > > 11,
>>> > > > > > > > > > > >>> 2020
>>> > > > > > > > > > > >>> > > at
>>> > > > > > > > > > > >>> > > > >> 8:27
>>> > > > > > > > > > > >>> > > > >> > AM
>>> > > > > > > > > > > >>> > > > >> > > > > Gokul
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> Ramanan
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>>> Subramanian <
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > > > > > > > > >>> > gokul2411s@gmail.com>
>>> > > > > > > > > > > >>> > > > >> wrote:
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > > Hi.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > > > Thanks
>>> > > > > > > > > for
>>> > > > > > > > > > > the
>>> > > > > > > > > > > >>> > KIP.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > > > Have you
>>> > > > > > > > > > > >>> thought
>>> > > > > > > > > > > >>> > > about
>>> > > > > > > > > > > >>> > > > >> > > whether
>>> > > > > > > > > > > >>> > > > >> > > > it
>>> > > > > > > > > > > >>> > > > >> > > > > > > makes
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > sense
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> support
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > > > authorizing a
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > > > > > principal
>>> > > > > > > > > > > for
>>> > > > > > > > > > > >>> a
>>> > > > > > > > > > > >>> > > topic
>>> > > > > > > > > > > >>> > > > ID
>>> > > > > > > > > > > >>> > > > >> > > rather
>>> > > > > > > > > > > >>> > > > >> > > > > > than
>>> > > > > > > > > > > >>> > > > >> > > > > > > a
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> topic
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > name
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>> achieve
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > tighter
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > > > > > security?
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > Or
>>> > > > > > > is
>>> > > > > > > > > the
>>> > > > > > > > > > > >>> topic ID
>>> > > > > > > > > > > >>> > > > >> > > > fundamentally
>>> > > > > > > > > > > >>> > > > >> > > > > an
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> internal
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > detail
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > similar
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > epochs
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > used
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > > in a
>>> > > > > > > > > bunch
>>> > > > > > > > > > > of
>>> > > > > > > > > > > >>> > other
>>> > > > > > > > > > > >>> > > > >> places
>>> > > > > > > > > > > >>> > > > >> > in
>>> > > > > > > > > > > >>> > > > >> > > > > > Kafka?
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > > > Thanks.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > On
>>> > > > > > > Fri,
>>> > > > > > > > > > Sep
>>> > > > > > > > > > > >>> 11,
>>> > > > > > > > > > > >>> > 2020
>>> > > > > > > > > > > >>> > > > at
>>> > > > > > > > > > > >>> > > > >> > 4:06
>>> > > > > > > > > > > >>> > > > >> > > PM
>>> > > > > > > > > > > >>> > > > >> > > > > > John
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > Roesler <
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > > > > > > > > vvcephei@apache.org
>>> > > > > > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > > wrote:
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > Hello
>>> > > > > > > > > > > >>> Justine,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > Thanks
>>> > > > > > > > > > for
>>> > > > > > > > > > > >>> the
>>> > > > > > > > > > > >>> > > KIP!
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > I
>>> > > > > > > > > happen
>>> > > > > > > > > > > to
>>> > > > > > > > > > > >>> have
>>> > > > > > > > > > > >>> > > > been
>>> > > > > > > > > > > >>> > > > >> > > > > confronted
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> recently
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > with
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> need
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> keep
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > > track
>>> > > > > > of
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > a
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > large
>>> > > > > > > > > > > >>> number of
>>> > > > > > > > > > > >>> > > > >> topics as
>>> > > > > > > > > > > >>> > > > >> > > > > > compactly
>>> > > > > > > > > > > >>> > > > >> > > > > > > > as
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > possible. I
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> was
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> going
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > come
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > up
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > with
>>> > > > > > > > > > some
>>> > > > > > > > > > > >>> way to
>>> > > > > > > > > > > >>> > > > >> > dictionary
>>> > > > > > > > > > > >>> > > > >> > > > > > encode
>>> > > > > > > > > > > >>> > > > >> > > > > > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > topic
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > names
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> as
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> > > integers,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > but
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > this
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > seems
>>> > > > > > > > > > much
>>> > > > > > > > > > > >>> > better!
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > > > > Apologies
>>> > > > > > > > > > > if
>>> > > > > > > > > > > >>> > this
>>> > > > > > > > > > > >>> > > > has
>>> > > > > > > > > > > >>> > > > >> > been
>>> > > > > > > > > > > >>> > > > >> > > > > raised
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> before,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > but
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > I’m
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>> wondering
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > about
>>> > > > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > choice
>>> > > > > > > > > > of
>>> > > > > > > > > > > >>> UUID
>>> > > > > > > > > > > >>> > vs
>>> > > > > > > > > > > >>> > > > >> > sequence
>>> > > > > > > > > > > >>> > > > >> > > > > number
>>> > > > > > > > > > > >>> > > > >> > > > > > > for
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > ids.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>>> Typically,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> I’ve
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > seen
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > > UUIDs
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > in two
>>> > > > > > > > > > > >>> > situations:
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > 1.
>>> > > > > > > > > When
>>> > > > > > > > > > > >>> > processes
>>> > > > > > > > > > > >>> > > > >> need to
>>> > > > > > > > > > > >>> > > > >> > > > > > generate
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > non-colliding
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> > > identifiers
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > without
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > > > > > >>> coordination.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > 2.
>>> > > > > > > > > When
>>> > > > > > > > > > > the
>>> > > > > > > > > > > >>> > > > identifier
>>> > > > > > > > > > > >>> > > > >> > > needs
>>> > > > > > > > > > > >>> > > > >> > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > be
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > “universally
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > unique”;
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> I.e.,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > > > > identifier
>>> > > > > > > > > > > >>> needs
>>> > > > > > > > > > > >>> > > to
>>> > > > > > > > > > > >>> > > > >> > > > distinguish
>>> > > > > > > > > > > >>> > > > >> > > > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> entity
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > from
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > all
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > other
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > entities
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > that
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > could
>>> > > > > > > > > > ever
>>> > > > > > > > > > > >>> > exist.
>>> > > > > > > > > > > >>> > > > >> This is
>>> > > > > > > > > > > >>> > > > >> > > > > useful
>>> > > > > > > > > > > >>> > > > >> > > > > > in
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> cases
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > where
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > entities
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > from
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> all
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > > kinds
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > of
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > > > systems
>>> > > > > > > > > > > get
>>> > > > > > > > > > > >>> > mixed
>>> > > > > > > > > > > >>> > > > >> > together,
>>> > > > > > > > > > > >>> > > > >> > > > > such
>>> > > > > > > > > > > >>> > > > >> > > > > > as
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> when
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > dumping
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> logs
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > from
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> all
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > > > > processes
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > in
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > a
>>> > > > > > > > > > company
>>> > > > > > > > > > > >>> into a
>>> > > > > > > > > > > >>> > > > >> common
>>> > > > > > > > > > > >>> > > > >> > > > system.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > Maybe
>>> > > > > > > > > > I’m
>>> > > > > > > > > > > >>> being
>>> > > > > > > > > > > >>> > > > >> > > > short-sighted,
>>> > > > > > > > > > > >>> > > > >> > > > > > but
>>> > > > > > > > > > > >>> > > > >> > > > > > > it
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > doesn’t
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > seem
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> like
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> either
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > really
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > > > applies
>>> > > > > > > > > > > >>> here. It
>>> > > > > > > > > > > >>> > > > seems
>>> > > > > > > > > > > >>> > > > >> > like
>>> > > > > > > > > > > >>> > > > >> > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > brokers
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > could
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > and
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> would
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> > achieve
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > > > > consensus
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > when
>>> > > > > > > > > > > >>> creating a
>>> > > > > > > > > > > >>> > > > topic
>>> > > > > > > > > > > >>> > > > >> > > anyway,
>>> > > > > > > > > > > >>> > > > >> > > > > > which
>>> > > > > > > > > > > >>> > > > >> > > > > > > > is
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> all
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that’s
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>> required
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > > > generate
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > > > > > >>> non-colliding
>>> > > > > > > > > > > >>> > > > sequence
>>> > > > > > > > > > > >>> > > > >> > ids.
>>> > > > > > > > > > > >>> > > > >> > > > For
>>> > > > > > > > > > > >>> > > > >> > > > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > second,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > as
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > you
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>> mention,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > we
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > could
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > > > assign
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > a
>>> > > > > > > UUID
>>> > > > > > > > > > to
>>> > > > > > > > > > > >>> the
>>> > > > > > > > > > > >>> > > > cluster
>>> > > > > > > > > > > >>> > > > >> as
>>> > > > > > > > > > > >>> > > > >> > a
>>> > > > > > > > > > > >>> > > > >> > > > > whole,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > which
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > would
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > render
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > any
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > > resource
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > > scoped
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > the
>>> > > > > > > > > > broker
>>> > > > > > > > > > > >>> > > > universally
>>> > > > > > > > > > > >>> > > > >> > > unique
>>> > > > > > > > > > > >>> > > > >> > > > > as
>>> > > > > > > > > > > >>> > > > >> > > > > > > > well.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > The
>>> > > > > > > > > > > reason I
>>> > > > > > > > > > > >>> > > mention
>>> > > > > > > > > > > >>> > > > >> this
>>> > > > > > > > > > > >>> > > > >> > > is
>>> > > > > > > > > > > >>> > > > >> > > > > > that,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > although
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > a
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > UUID
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> is
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > way
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> more
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > > > compact
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > than
>>> > > > > > > > > > topic
>>> > > > > > > > > > > >>> > names,
>>> > > > > > > > > > > >>> > > > it’s
>>> > > > > > > > > > > >>> > > > >> > > still
>>> > > > > > > > > > > >>> > > > >> > > > 16
>>> > > > > > > > > > > >>> > > > >> > > > > > > > bytes.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> In
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > contrast,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> a
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> 4-byte
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > integer
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > > > sequence
>>> > > > > > > > > > > id
>>> > > > > > > > > > > >>> > would
>>> > > > > > > > > > > >>> > > > give
>>> > > > > > > > > > > >>> > > > >> > us 4
>>> > > > > > > > > > > >>> > > > >> > > > > > billion
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> unique
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > topics
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> per
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> > cluster,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > which
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > > > seems
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > like
>>> > > > > > > > > > > enough
>>> > > > > > > > > > > >>> ;)
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > > > > > Considering
>>> > > > > > > > > > > >>> the
>>> > > > > > > > > > > >>> > > > >> number of
>>> > > > > > > > > > > >>> > > > >> > > > > > different
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> times
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > these
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> topic
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > > > identifiers
>>> > > > > > > > > are
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > > sent
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > over
>>> > > > > > > > > the
>>> > > > > > > > > > > >>> wire or
>>> > > > > > > > > > > >>> > > > >> stored
>>> > > > > > > > > > > >>> > > > >> > in
>>> > > > > > > > > > > >>> > > > >> > > > > > memory,
>>> > > > > > > > > > > >>> > > > >> > > > > > > it
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > seems
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > like
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > it
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > might
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > be
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > worth
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > > > > additional
>>> > > > > > > > > > > >>> 4x
>>> > > > > > > > > > > >>> > > space
>>> > > > > > > > > > > >>> > > > >> > > savings.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > What
>>> > > > > > > > > do
>>> > > > > > > > > > > you
>>> > > > > > > > > > > >>> > think
>>> > > > > > > > > > > >>> > > > >> about
>>> > > > > > > > > > > >>> > > > >> > > this?
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > > > Thanks,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > John
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > On
>>> > > > > > > > > Fri,
>>> > > > > > > > > > > Sep
>>> > > > > > > > > > > >>> 11,
>>> > > > > > > > > > > >>> > > > 2020,
>>> > > > > > > > > > > >>> > > > >> at
>>> > > > > > > > > > > >>> > > > >> > > > 03:20,
>>> > > > > > > > > > > >>> > > > >> > > > > > Tom
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > Bentley
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > wrote:
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > Hi
>>> > > > > > > > > > > >>> Justine,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > This
>>> > > > > > > > > > > looks
>>> > > > > > > > > > > >>> > like
>>> > > > > > > > > > > >>> > > a
>>> > > > > > > > > > > >>> > > > >> very
>>> > > > > > > > > > > >>> > > > >> > > > > welcome
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > improvement.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Thanks!
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > > Maybe
>>> > > > > > > > > > I
>>> > > > > > > > > > > >>> missed
>>> > > > > > > > > > > >>> > > it,
>>> > > > > > > > > > > >>> > > > >> but
>>> > > > > > > > > > > >>> > > > >> > > the
>>> > > > > > > > > > > >>> > > > >> > > > > KIP
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> doesn't
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > seem
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > mention
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > > changing
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > > > > >>> > > > DeleteTopicsRequest
>>> > > > > > > > > > > >>> > > > >> to
>>> > > > > > > > > > > >>> > > > >> > > > > identify
>>> > > > > > > > > > > >>> > > > >> > > > > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > topic
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > using
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > an
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > id.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> Maybe
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > > that's
>>> > > > > > > out
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > of
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > > scope,
>>> > > > > > > > > > > but
>>> > > > > > > > > > > >>> > > > >> > > > > DeleteTopicsRequest
>>> > > > > > > > > > > >>> > > > >> > > > > > is
>>> > > > > > > > > > > >>> > > > >> > > > > > > > not
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > listed
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > among
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> Future
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > Work
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > APIs
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > > > either.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > Kind
>>> > > > > > > > > > > >>> regards,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > Tom
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > On
>>> > > > > > > > > > Thu,
>>> > > > > > > > > > > >>> Sep
>>> > > > > > > > > > > >>> > 10,
>>> > > > > > > > > > > >>> > > > >> 2020 at
>>> > > > > > > > > > > >>> > > > >> > > > 3:59
>>> > > > > > > > > > > >>> > > > >> > > > > PM
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> Satish
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > Duggana <
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > > > > > > > >>> > > > >> satish.duggana@gmail.com
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > > wrote:
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > Thanks
>>> > > > > > > > > > > >>> > > > >> Lucas/Justine
>>> > > > > > > > > > > >>> > > > >> > > for
>>> > > > > > > > > > > >>> > > > >> > > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > nice
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> KIP.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > It
>>> > > > > > > > > > has
>>> > > > > > > > > > > >>> > several
>>> > > > > > > > > > > >>> > > > >> > benefits
>>> > > > > > > > > > > >>> > > > >> > > > > which
>>> > > > > > > > > > > >>> > > > >> > > > > > > > also
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > include
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>> > simplifying
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > topic
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > > deletion
>>> > > > > > > > > > > >>> > > process
>>> > > > > > > > > > > >>> > > > >> by
>>> > > > > > > > > > > >>> > > > >> > > > > > controller
>>> > > > > > > > > > > >>> > > > >> > > > > > > > and
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > logs
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > cleanup
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> by
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> > brokers
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> in
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > > corner
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > cases.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > Best,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > > Satish.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > On
>>> > > > > > > > > > > Wed,
>>> > > > > > > > > > > >>> Sep
>>> > > > > > > > > > > >>> > 9,
>>> > > > > > > > > > > >>> > > > >> 2020
>>> > > > > > > > > > > >>> > > > >> > at
>>> > > > > > > > > > > >>> > > > >> > > > > 10:07
>>> > > > > > > > > > > >>> > > > >> > > > > > PM
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > Justine
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Olshan <
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > > > > > > > > >>> jolshan@confluent.io
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > wrote:
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > > > Hello
>>> > > > > > > > > > > >>> all,
>>> > > > > > > > > > > >>> > > > it's
>>> > > > > > > > > > > >>> > > > >> > been
>>> > > > > > > > > > > >>> > > > >> > > > > > almost a
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> year!
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > I've
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > made
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > some
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > > changes
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > this
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > KIP
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > and
>>> > > > > > > > > > > >>> hope to
>>> > > > > > > > > > > >>> > > > >> continue
>>> > > > > > > > > > > >>> > > > >> > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> discussion.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > > One
>>> > > > > > > > > > > >>> of the
>>> > > > > > > > > > > >>> > > > main
>>> > > > > > > > > > > >>> > > > >> > > changes
>>> > > > > > > > > > > >>> > > > >> > > > > > I've
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> added
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > is
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > now
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> metadata
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > response
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > > will
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > > include
>>> > > > > > > > > > > >>> the
>>> > > > > > > > > > > >>> > > > topic
>>> > > > > > > > > > > >>> > > > >> ID
>>> > > > > > > > > > > >>> > > > >> > > (as
>>> > > > > > > > > > > >>> > > > >> > > > > > Colin
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > suggested).
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Clients
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > can
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > obtain
>>> > > > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > > > topicID
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > of a
>>> > > > > > > > > > > >>> given
>>> > > > > > > > > > > >>> > > topic
>>> > > > > > > > > > > >>> > > > >> > > through
>>> > > > > > > > > > > >>> > > > >> > > > a
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > TopicDescription.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > The
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> topicId
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> will
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > also
>>> > > > > > be
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > > included
>>> > > > > > > > > > > >>> > with
>>> > > > > > > > > > > >>> > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > UpdateMetadata
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > request.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > > Let
>>> > > > > > > > > > > me
>>> > > > > > > > > > > >>> > know
>>> > > > > > > > > > > >>> > > > what
>>> > > > > > > > > > > >>> > > > >> > you
>>> > > > > > > > > > > >>> > > > >> > > > all
>>> > > > > > > > > > > >>> > > > >> > > > > > > think.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > > > Thank
>>> > > > > > > > > > > >>> you,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > > > >>> Justine
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > On
>>> > > > > > > > > > > >>> > > 2019/09/13
>>> > > > > > > > > > > >>> > > > >> > > 16:38:26,
>>> > > > > > > > > > > >>> > > > >> > > > > > > "Colin
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > McCabe" <
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > > > > > > > > cmccabe@apache.org
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > > wrote:
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > Hi
>>> > > > > > > > > > > >>> > Lucas,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > > >>> Thanks
>>> > > > > > > > > > > >>> > for
>>> > > > > > > > > > > >>> > > > >> > tackling
>>> > > > > > > > > > > >>> > > > >> > > > > this.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> Topic
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > IDs
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > are a
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > great
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> idea,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > and
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > this
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > is
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > a
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > really
>>> > > > > > > > > > > >>> good
>>> > > > > > > > > > > >>> > > > >> writeup.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > > For
>>> > > > > > > > > > > >>> > > > >> > > > > > > /brokers/topics/[topic],
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > schema
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> version
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> should
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> be
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > > bumped
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > > version
>>> > > > > > > > > > > >>> 3,
>>> > > > > > > > > > > >>> > > > rather
>>> > > > > > > > > > > >>> > > > >> > than
>>> > > > > > > > > > > >>> > > > >> > > 2.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > KIP-455
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > bumped
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > version
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> this
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > > znode
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > 2
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > > already
>>> > > > > > > > > > > >>> :)
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > > >>> Given
>>> > > > > > > > > > > >>> > that
>>> > > > > > > > > > > >>> > > > >> we're
>>> > > > > > > > > > > >>> > > > >> > > > going
>>> > > > > > > > > > > >>> > > > >> > > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > be
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > seeing
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > these
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > things
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > as
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > strings
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > as
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > > lot
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > (in
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > logs,
>>> > > > > > > > > > > in
>>> > > > > > > > > > > >>> > > > >> ZooKeeper,
>>> > > > > > > > > > > >>> > > > >> > on
>>> > > > > > > > > > > >>> > > > >> > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > command-line,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > etc.),
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > does
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > it
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> make
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > > sense
>>> > > > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > use
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > base64
>>> > > > > > > > > > > >>> when
>>> > > > > > > > > > > >>> > > > >> > converting
>>> > > > > > > > > > > >>> > > > >> > > > them
>>> > > > > > > > > > > >>> > > > >> > > > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > strings?
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > > >>> Here is
>>> > > > > > > > > > > >>> > an
>>> > > > > > > > > > > >>> > > > >> > example
>>> > > > > > > > > > > >>> > > > >> > > of
>>> > > > > > > > > > > >>> > > > >> > > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > hex
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>>> > representation:
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>>> > > > > > > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > > And
>>> > > > > > > > > > > >>> here
>>> > > > > > > > > > > >>> > > is
>>> > > > > > > > > > > >>> > > > an
>>> > > > > > > > > > > >>> > > > >> > > > example
>>> > > > > > > > > > > >>> > > > >> > > > > in
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> base64.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > b8tRS7h4TJ2Vt43Dp85v2A
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > > The
>>> > > > > > > > > > > >>> > base64
>>> > > > > > > > > > > >>> > > > >> > version
>>> > > > > > > > > > > >>> > > > >> > > > > saves
>>> > > > > > > > > > > >>> > > > >> > > > > > 15
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > letters
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > (to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > be
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > fair,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > 4
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > those
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > were
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > dashes
>>> > > > > > > > > > > >>> that
>>> > > > > > > > > > > >>> > we
>>> > > > > > > > > > > >>> > > > >> could
>>> > > > > > > > > > > >>> > > > >> > > have
>>> > > > > > > > > > > >>> > > > >> > > > > > > elided
>>> > > > > > > > > > > >>> > > > >> > > > > > > > in
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > hex
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > > > > > > representation.)
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > > >>> Another
>>> > > > > > > > > > > >>> > > > thing
>>> > > > > > > > > > > >>> > > > >> to
>>> > > > > > > > > > > >>> > > > >> > > > > consider
>>> > > > > > > > > > > >>> > > > >> > > > > > > is
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> that
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > we
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > should
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>> specify
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> that
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > > >>> all-zeroes
>>> > > > > > > > > > > >>> > > UUID
>>> > > > > > > > > > > >>> > > > is
>>> > > > > > > > > > > >>> > > > >> > not
>>> > > > > > > > > > > >>> > > > >> > > a
>>> > > > > > > > > > > >>> > > > >> > > > > > valid
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> topic
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > UUID.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >  We
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > can't
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > use
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > null
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > for
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > > this
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > > because
>>> > > > > > > > > > > >>> we
>>> > > > > > > > > > > >>> > > can't
>>> > > > > > > > > > > >>> > > > >> > pass a
>>> > > > > > > > > > > >>> > > > >> > > > > null
>>> > > > > > > > > > > >>> > > > >> > > > > > > UUID
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> over
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > RPC
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>> protocol
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > (there
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > is
>>> > > > > no
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > > special
>>> > > > > > > > > > > >>> > > pattern
>>> > > > > > > > > > > >>> > > > >> for
>>> > > > > > > > > > > >>> > > > >> > > null,
>>> > > > > > > > > > > >>> > > > >> > > > > nor
>>> > > > > > > > > > > >>> > > > >> > > > > > > do
>>> > > > > > > > > > > >>> > > > >> > > > > > > > we
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > want
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> waste
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > space
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > > reserving
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > > such
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > a
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > > >>> pattern.)
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > > >>> Maybe I
>>> > > > > > > > > > > >>> > > > missed
>>> > > > > > > > > > > >>> > > > >> > it,
>>> > > > > > > > > > > >>> > > > >> > > > but
>>> > > > > > > > > > > >>> > > > >> > > > > > did
>>> > > > > > > > > > > >>> > > > >> > > > > > > > you
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > describe
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>> "migration
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > of...
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > > > existing
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > > topic[s]
>>> > > > > > > > > > > >>> > > without
>>> > > > > > > > > > > >>> > > > >> > topic
>>> > > > > > > > > > > >>> > > > >> > > > IDs"
>>> > > > > > > > > > > >>> > > > >> > > > > > in
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> detail
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > in
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > any
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > section?
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > It
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > seems
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > like
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > when
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > the
>>> > > > > > > > > > > new
>>> > > > > > > > > > > >>> > > > controller
>>> > > > > > > > > > > >>> > > > >> > > > becomes
>>> > > > > > > > > > > >>> > > > >> > > > > > > > active,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> it
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > should
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> just
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> > generate
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > random
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > UUIDs
>>> > > > > > > > > > for
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > these,
>>> > > > > > > > > > > >>> and
>>> > > > > > > > > > > >>> > > write
>>> > > > > > > > > > > >>> > > > >> the
>>> > > > > > > > > > > >>> > > > >> > > > random
>>> > > > > > > > > > > >>> > > > >> > > > > > > UUIDs
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> back
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>>> ZooKeeper.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > It
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > would
>>> > > > > be
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > > good
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > spell
>>> > > > > > > > > > > >>> that
>>> > > > > > > > > > > >>> > > out.
>>> > > > > > > > > > > >>> > > > >> We
>>> > > > > > > > > > > >>> > > > >> > > > should
>>> > > > > > > > > > > >>> > > > >> > > > > > make
>>> > > > > > > > > > > >>> > > > >> > > > > > > > it
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > clear
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> this
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> > happens
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > > > > regardless
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > of
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > the
>>> > > > > > > > > > > >>> > > inter-broker
>>> > > > > > > > > > > >>> > > > >> > > protocol
>>> > > > > > > > > > > >>> > > > >> > > > > > > version
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > (it's
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > a
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>>> compatible
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > > change).
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > "LeaderAndIsrRequests
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> including an
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> > > > > > is_every_partition
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > flag"
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > > > seems a
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > bit
>>> > > > > > > > > > > >>> wordy.
>>> > > > > > > > > > > >>> > > Can
>>> > > > > > > > > > > >>> > > > we
>>> > > > > > > > > > > >>> > > > >> > just
>>> > > > > > > > > > > >>> > > > >> > > > > call
>>> > > > > > > > > > > >>> > > > >> > > > > > > > these
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > "full
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > > > > > > > >>> LeaderAndIsrRequests"?
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > Then
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > RPC
>>> > > > > > > > > > > >>> field
>>> > > > > > > > > > > >>> > > could
>>> > > > > > > > > > > >>> > > > be
>>> > > > > > > > > > > >>> > > > >> > > named
>>> > > > > > > > > > > >>> > > > >> > > > > > > "full".
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > Also,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > it
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > would
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> probably
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> be
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > > better
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > for
>>> > > > > > > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > RPC
>>> > > > > > > > > > > >>> field to
>>> > > > > > > > > > > >>> > > be
>>> > > > > > > > > > > >>> > > > an
>>> > > > > > > > > > > >>> > > > >> > enum
>>> > > > > > > > > > > >>> > > > >> > > > of
>>> > > > > > > > > > > >>> > > > >> > > > > {
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > UNSPECIFIED,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>> > INCREMENTAL,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> FULL
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > },
>>> > > > so
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > > that
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > we
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > can
>>> > > > > > > > > > > >>> cleanly
>>> > > > > > > > > > > >>> > > > handle
>>> > > > > > > > > > > >>> > > > >> > old
>>> > > > > > > > > > > >>> > > > >> > > > > > versions
>>> > > > > > > > > > > >>> > > > >> > > > > > > > (by
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > treating
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> them
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > as
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > > > UNSPECIFIED)
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > In
>>> > > > > > > > > > > >>> the
>>> > > > > > > > > > > >>> > > > >> > > > > > LeaderAndIsrRequest
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > section,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > you
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> write
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > "A
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> final
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > > > deletion
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > event
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > will
>>> > > > > > > > > > > be
>>> > > > > > > > > > > >>> > > > secheduled
>>> > > > > > > > > > > >>> > > > >> > for
>>> > > > > > > > > > > >>> > > > >> > > X
>>> > > > > > > > > > > >>> > > > >> > > > ms
>>> > > > > > > > > > > >>> > > > >> > > > > > > after
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> > > > > > > LeaderAndIsrRequest
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> was
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > > first
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > > >>> received..."
>>> > > > > > > > > > > >>> > > I
>>> > > > > > > > > > > >>> > > > >> guess
>>> > > > > > > > > > > >>> > > > >> > > > the X
>>> > > > > > > > > > > >>> > > > >> > > > > > > was a
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > placeholder
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> that
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > you
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > intended
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > > > replace
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > before
>>> > > > > > > > > > > >>> > > posting?
>>> > > > > > > > > > > >>> > > > :)
>>> > > > > > > > > > > >>> > > > >> > In
>>> > > > > > > > > > > >>> > > > >> > > > any
>>> > > > > > > > > > > >>> > > > >> > > > > > > case,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> this
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > seems
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > like
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > kind
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> of
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > > thing
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > > we'd
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > want a
>>> > > > > > > > > > > >>> > > > >> configuration
>>> > > > > > > > > > > >>> > > > >> > > for.
>>> > > > > > > > > > > >>> > > > >> > > > > > > Let's
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > describe
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > > > > > configuration
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > key
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > > > > somewhere
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > in
>>> > > > > > > > > > > this
>>> > > > > > > > > > > >>> KIP,
>>> > > > > > > > > > > >>> > > > >> > including
>>> > > > > > > > > > > >>> > > > >> > > > what
>>> > > > > > > > > > > >>> > > > >> > > > > > its
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > default
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > value
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> is.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > We
>>> > > > > > > > > > > >>> > should
>>> > > > > > > > > > > >>> > > > >> > probably
>>> > > > > > > > > > > >>> > > > >> > > > also
>>> > > > > > > > > > > >>> > > > >> > > > > > > log a
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > bunch
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > of
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> messages
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > at
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> WARN
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > level
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > > when
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > > >>> something is
>>> > > > > > > > > > > >>> > > > >> > scheduled
>>> > > > > > > > > > > >>> > > > >> > > > for
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> deletion,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > as
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > well.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > (Maybe
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> this
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > was
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > > > assumed,
>>> > > > > > > > > > > but
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > it
>>> > > > > > > > > > > >>> would be
>>> > > > > > > > > > > >>> > > good
>>> > > > > > > > > > > >>> > > > >> to
>>> > > > > > > > > > > >>> > > > >> > > > mention
>>> > > > > > > > > > > >>> > > > >> > > > > > > it).
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > I
>>> > > > > > > > > > > >>> feel
>>> > > > > > > > > > > >>> > > like
>>> > > > > > > > > > > >>> > > > >> there
>>> > > > > > > > > > > >>> > > > >> > > > are a
>>> > > > > > > > > > > >>> > > > >> > > > > > few
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > sections
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > should
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > be
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > moved
>>> > > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > > > > "rejected
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > > >>> > alternatives."
>>> > > > > > > > > > > >>> > > > >> For
>>> > > > > > > > > > > >>> > > > >> > > > > example,
>>> > > > > > > > > > > >>> > > > >> > > > > > in
>>> > > > > > > > > > > >>> > > > >> > > > > > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > DeleteTopics
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> section,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > since
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > > we're
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > not
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > going
>>> > > > > > > > > > > >>> to do
>>> > > > > > > > > > > >>> > > > >> option 1
>>> > > > > > > > > > > >>> > > > >> > or
>>> > > > > > > > > > > >>> > > > >> > > > 2,
>>> > > > > > > > > > > >>> > > > >> > > > > > > these
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > should
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > be
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > moved
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > into
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > > "rejected
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > > >>> > alternatives,"
>>> > > > > > > > > > > >>> > > > >> > rather
>>> > > > > > > > > > > >>> > > > >> > > > than
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> appearing
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > inline.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>> Another
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> case
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > is
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > > > "Should
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > we
>>> > > > > > > > > > > >>> remove
>>> > > > > > > > > > > >>> > > topic
>>> > > > > > > > > > > >>> > > > >> name
>>> > > > > > > > > > > >>> > > > >> > > from
>>> > > > > > > > > > > >>> > > > >> > > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> protocol
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > where
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>> possible"
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > section.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > This
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > is
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > > clearly
>>> > > > > > > > > > > >>> > > > >> discussing a
>>> > > > > > > > > > > >>> > > > >> > > > design
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > alternative
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > that
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> we're
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > not
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > > proposing
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > > >>> implement:
>>> > > > > > > > > > > >>> > > > >> removing
>>> > > > > > > > > > > >>> > > > >> > the
>>> > > > > > > > > > > >>> > > > >> > > > > topic
>>> > > > > > > > > > > >>> > > > >> > > > > > > > name
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > from
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > those
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> > protocols.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > Is
>>> > > > > > > > > > > >>> it
>>> > > > > > > > > > > >>> > > really
>>> > > > > > > > > > > >>> > > > >> > > > necessary
>>> > > > > > > > > > > >>> > > > >> > > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > have
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> a
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > new
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > > > > > > > > >>> > > /admin/delete_topics_by_id
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > path
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > in
>>> > > > > > > > > > > >>> > ZooKeeper?
>>> > > > > > > > > > > >>> > > > It
>>> > > > > > > > > > > >>> > > > >> > seems
>>> > > > > > > > > > > >>> > > > >> > > > > like
>>> > > > > > > > > > > >>> > > > >> > > > > > we
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> don't
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > really
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> need
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > this.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > Whenever
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > there
>>> > > > > > > > > is
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > a
>>> > > > > > > > > > new
>>> > > > > > > > > > > >>> > > > controller,
>>> > > > > > > > > > > >>> > > > >> > we'll
>>> > > > > > > > > > > >>> > > > >> > > > > send
>>> > > > > > > > > > > >>> > > > >> > > > > > > out
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> full
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> > > > > > > LeaderAndIsrRequests
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > which
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > will
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > > trigger
>>> > > > > > > > > > > >>> the
>>> > > > > > > > > > > >>> > > > stale
>>> > > > > > > > > > > >>> > > > >> > > topics
>>> > > > > > > > > > > >>> > > > >> > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > be
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> cleaned
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > up.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >  The
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > active
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > > > > controller
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > > will
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > also
>>> > > > > > > > > > > >>> send
>>> > > > > > > > > > > >>> > the
>>> > > > > > > > > > > >>> > > > full
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> LeaderAndIsrRequest
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> brokers
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> are
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > just
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > > > starting
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > up.
>>> > > > > > > > > > > >>> So we
>>> > > > > > > > > > > >>> > > > don't
>>> > > > > > > > > > > >>> > > > >> > > really
>>> > > > > > > > > > > >>> > > > >> > > > > > need
>>> > > > > > > > > > > >>> > > > >> > > > > > > > this
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > kind
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > of
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>>> two-phase
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > commit
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > > (send
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > > out
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > > >>> > > > >> StopReplicasRequest,
>>> > > > > > > > > > > >>> > > > >> > > get
>>> > > > > > > > > > > >>> > > > >> > > > > ACKs
>>> > > > > > > > > > > >>> > > > >> > > > > > > > from
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> all
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > nodes,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > commit
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > by
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > removing
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > > >>> > > > >> /admin/delete_topics
>>> > > > > > > > > > > >>> > > > >> > > > node)
>>> > > > > > > > > > > >>> > > > >> > > > > > any
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> more.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > > You
>>> > > > > > > > > > > >>> > > mention
>>> > > > > > > > > > > >>> > > > >> that
>>> > > > > > > > > > > >>> > > > >> > > > > > > FetchRequest
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> will
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > now
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> include
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > UUID
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > avoid
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > > > issues
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > where
>>> > > > > > > > > > > >>> > requests
>>> > > > > > > > > > > >>> > > > are
>>> > > > > > > > > > > >>> > > > >> > made
>>> > > > > > > > > > > >>> > > > >> > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > stale
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > partitions.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>> However,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > adding a
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > UUID
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > > >>> > > MetadataRequest
>>> > > > > > > > > > > >>> > > > is
>>> > > > > > > > > > > >>> > > > >> > > listed
>>> > > > > > > > > > > >>> > > > >> > > > > as
>>> > > > > > > > > > > >>> > > > >> > > > > > > > future
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > work,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > out
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > of
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > scope
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> for
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > this
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > KIP.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > How
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > will
>>> > > > > > > > > > > the
>>> > > > > > > > > > > >>> > > client
>>> > > > > > > > > > > >>> > > > >> learn
>>> > > > > > > > > > > >>> > > > >> > > > what
>>> > > > > > > > > > > >>> > > > >> > > > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> topic
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > UUID
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > is,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > if
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > > metadata
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > > > response
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > > doesn't
>>> > > > > > > > > > > >>> > > include
>>> > > > > > > > > > > >>> > > > >> that
>>> > > > > > > > > > > >>> > > > >> > > > > > > information?
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> It
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > seems
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > like
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > adding
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > UUID
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > > >>> > > MetadataResponse
>>> > > > > > > > > > > >>> > > > >> > would
>>> > > > > > > > > > > >>> > > > >> > > be
>>> > > > > > > > > > > >>> > > > >> > > > > an
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > improvement
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > here
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> that
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> might
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> not
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > be
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > too
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > hard
>>> > > > > > > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > make.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > > >>> best,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > > >>> Colin
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > On
>>> > > > > > > > > > > >>> Mon,
>>> > > > > > > > > > > >>> > > Sep
>>> > > > > > > > > > > >>> > > > 9,
>>> > > > > > > > > > > >>> > > > >> > > 2019,
>>> > > > > > > > > > > >>> > > > >> > > > at
>>> > > > > > > > > > > >>> > > > >> > > > > > > > 17:48,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > Ryanne
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Dolan
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > wrote:
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > > > >>> Lucas,
>>> > > > > > > > > > > >>> > > > this
>>> > > > > > > > > > > >>> > > > >> > would
>>> > > > > > > > > > > >>> > > > >> > > > be
>>> > > > > > > > > > > >>> > > > >> > > > > > > great.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> I've
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > run
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > into
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > issues
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> with
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > > topics
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > being
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > > > >>> > > > resurrected
>>> > > > > > > > > > > >>> > > > >> > > > > > accidentally,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> since
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > a
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > client
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > cannot
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > easily
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > > > > > distinguish
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > > between
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > > > a
>>> > > > > > > > > > > >>> > > deleted
>>> > > > > > > > > > > >>> > > > >> topic
>>> > > > > > > > > > > >>> > > > >> > > > and a
>>> > > > > > > > > > > >>> > > > >> > > > > > new
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> topic
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > with
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> same
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> name.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> I'd
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > need
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > ID
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > > > >>> > > accessible
>>> > > > > > > > > > > >>> > > > >> from
>>> > > > > > > > > > > >>> > > > >> > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > client
>>> > > > > > > > > > > >>> > > > >> > > > > > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > solve
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > issue,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > but
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > this
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > is a
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > good
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > first
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > > > >>> step.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > > > >>> Ryanne
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > > > On
>>> > > > > > > > > > > >>> > Wed,
>>> > > > > > > > > > > >>> > > > Sep
>>> > > > > > > > > > > >>> > > > >> 4,
>>> > > > > > > > > > > >>> > > > >> > > 2019
>>> > > > > > > > > > > >>> > > > >> > > > > at
>>> > > > > > > > > > > >>> > > > >> > > > > > > 1:41
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> PM
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > Lucas
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>> > Bradstreet <
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > > > > > >>> > > lucas@confluent.io>
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > wrote:
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > > > >
>>> > > > > > > > > > > >>> Hi
>>> > > > > > > > > > > >>> > > all,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > > > >
>>> > > > > > > > > > > >>> I
>>> > > > > > > > > > > >>> > > would
>>> > > > > > > > > > > >>> > > > >> like
>>> > > > > > > > > > > >>> > > > >> > to
>>> > > > > > > > > > > >>> > > > >> > > > > kick
>>> > > > > > > > > > > >>> > > > >> > > > > > > off
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > discussion
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > of
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>> KIP-516,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > an
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > > > > > >>> implementation
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > of
>>> > > > > > > > > > > topic
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > > > >
>>> > > > > > > > > > > >>> IDs
>>> > > > > > > > > > > >>> > > for
>>> > > > > > > > > > > >>> > > > >> > Kafka.
>>> > > > > > > > > > > >>> > > > >> > > > > Topic
>>> > > > > > > > > > > >>> > > > >> > > > > > > IDs
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> aim
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > to
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > solve
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> topic
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > > uniqueness
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > > > problems
>>> > > > > > > > > > > in
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > Kafka,
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > > > >
>>> > > > > > > > > > > >>> > where
>>> > > > > > > > > > > >>> > > > >> > referring
>>> > > > > > > > > > > >>> > > > >> > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > a
>>> > > > > > > > > > > >>> > > > >> > > > > > > > topic
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> by
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > name
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> alone is
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > > > insufficient.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > > Such
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > cases
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > > > >
>>> > > > > > > > > > > >>> > > include
>>> > > > > > > > > > > >>> > > > >> when
>>> > > > > > > > > > > >>> > > > >> > a
>>> > > > > > > > > > > >>> > > > >> > > > > topic
>>> > > > > > > > > > > >>> > > > >> > > > > > > has
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> been
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > deleted
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> > > recreated
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > with
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > the
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > > same
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > name.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > > > >
>>> > > > > > > > > > > >>> > Unique
>>> > > > > > > > > > > >>> > > > >> > > > identifiers
>>> > > > > > > > > > > >>> > > > >> > > > > > will
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> help
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > simplify
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> improve
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > Kafka's
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > > > topic
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > > deletion
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > > > >
>>> > > > > > > > > > > >>> > > process,
>>> > > > > > > > > > > >>> > > > >> as
>>> > > > > > > > > > > >>> > > > >> > > well
>>> > > > > > > > > > > >>> > > > >> > > > as
>>> > > > > > > > > > > >>> > > > >> > > > > > > > prevent
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > cases
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > where
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>> brokers
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> may
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > > > > > > incorrectly
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > >
>>> > > > > > > > > > > interact
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > > > >
>>> > > > > > > > > > > >>> with
>>> > > > > > > > > > > >>> > > > stale
>>> > > > > > > > > > > >>> > > > >> > > > versions
>>> > > > > > > > > > > >>> > > > >> > > > > > of
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> topics.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >>
>>> > > > > > > > > > > >>> > > > >> > > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > >
>>> > > > > > > > > > > >>> > > > >> > > >
>>> > > > > > > > > > > >>> > > > >> > >
>>> > > > > > > > > > > >>> > > > >> >
>>> > > > > > > > > > > >>> > > > >>
>>> > > > > > > > > > > >>> > > >
>>> > > > > > > > > > > >>> > >
>>> > > > > > > > > > > >>> >
>>> > > > > > > > > > > >>>
>>> > > > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > > > >
>>> > > > > > > > > > > >>> > > Looking
>>> > > > > > > > > > > >>> > > > >> > forward
>>> > > > > > > > > > > >>> > > > >> > > > to
>>> > > > > > > > > > > >>> > > > >> > > > > > your
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > thoughts.
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > > > >
>>> > > > > > > > > > > >>> > Lucas
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > >
>>> > > > >
>>> > > > > >
>>> > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>>
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> >>
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >>
>>> > > > > > > > > > > >>> > > > >> > > > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > > >
>>> > > > > > > > > > > >>> > > > >> > > > >
>>> > > > > > > > > > > >>> > > > >> > > >
>>> > > > > > > > > > > >>> > > > >> > >
>>> > > > > > > > > > > >>> > > > >> >
>>> > > > > > > > > > > >>> > > > >>
>>> > > > > > > > > > > >>> > > > >
>>> > > > > > > > > > > >>> > > >
>>> > > > > > > > > > > >>> > >
>>> > > > > > > > > > > >>> >
>>> > > > > > > > > > > >>>
>>> > > > > > > > > > > >>
>>> > > > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > >
>>> > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>>
>>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Justine Olshan <jo...@confluent.io>]

Hi all,

I think it is almost time to vote on this KIP. Please let me know if there
are any outstanding issues.

Thank you,
Justine

On Fri, Oct 9, 2020 at 7:56 AM Justine Olshan <jo...@confluent.io> wrote:

> Hi Jun,
>
> 50. Right. That's what I was thinking but wanted to make sure. Works for
> me!
>
> Thanks again,
> Justine
>
> On Thu, Oct 8, 2020 at 3:50 PM Jun Rao <ju...@confluent.io> wrote:
>
>> Hi, Justine,
>>
>> 50. We can't change the format of existing versions since it will break
>> compatibility. Since we are bumping the version for MetadataRequest
>> and DeleteTopicsRequest, we can make the topic name field nullable from
>> the
>> new version.
>>
>> Thanks,
>>
>> Jun
>>
>> On Thu, Oct 8, 2020 at 2:19 PM Justine Olshan <jo...@confluent.io>
>> wrote:
>>
>> > Hi Jun,
>> >
>> > 50. Works for me. Is the idea that it will be nullable from this version
>> > onwards, but not past versions?
>> >
>> > 51. Added some information and example methods for AdminClient future
>> work
>> >
>> > Thanks,
>> > Justine
>> >
>> >
>> > On Thu, Oct 8, 2020 at 10:59 AM Jun Rao <ju...@confluent.io> wrote:
>> >
>> > > Hi, Justine,
>> > >
>> > > The changes sound reasonable to me.
>> > >
>> > > 50. Should we make the topic name in the new MetadataRequest
>> > > and DeleteTopicsRequest nullable?
>> > >
>> > > 51. In the future work section, perhaps we could mention opportunities
>> > for
>> > > using topicId in other methods in AdminClient where topic name is
>> used.
>> > >
>> > > Thanks,
>> > >
>> > > Jun
>> > >
>> > > On Wed, Oct 7, 2020 at 2:44 PM Justine Olshan <jo...@confluent.io>
>> > > wrote:
>> > >
>> > > > Hi all,
>> > > >
>> > > > Another update on the KIP. I've added some information about how
>> topic
>> > > IDs
>> > > > will be supported in AdminClient
>> > > >
>> > > >
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-AdminClientSupport
>> > > >
>> > > >
>> > > > One specific decision I wanted to focus on is the inclusion of both
>> > topic
>> > > > names and topic Ids in metadata and deleteTopics requests.
>> > > >
>> > > > The idea is that when the request is received the topic id field is
>> > > checked
>> > > > first, but if it contains the default (zero) uuid, then the topic
>> name
>> > > > field will be used.
>> > > >
>> > > > In the case where ids are used, if the ids do not have a
>> corresponding
>> > > > topic, the request will fail. In the case where both ids and names
>> are
>> > > > included, the request will fail if the name does not match the id.
>> > > >
>> > > >
>> > > > Initially the id field will be used to support the admin client use
>> > cases
>> > > > as described in the KIP, but over time, we can transfer to using
>> topic
>> > > ids
>> > > > in all uses of the request.
>> > > >
>> > > >
>> > > > There will also be some support for returning the topic Id on the
>> > > > createtopics, deletetopics and metadata responses. I think it makes
>> > sense
>> > > > to include both the name and the id in the successful response for
>> > these
>> > > > cases.
>> > > >
>> > > >
>> > > > Let me know if you have any comments or concerns regarding the Admin
>> > > Client
>> > > > changes.
>> > > >
>> > > > On Wed, Oct 7, 2020 at 11:05 AM Jun Rao <ju...@confluent.io> wrote:
>> > > >
>> > > > > Hi, Justine,
>> > > > >
>> > > > > I was just wondering if we expose topicId publicly, how would
>> people
>> > > use
>> > > > > it. One possibility is to use topicId in all those public
>> interfaces
>> > > > where
>> > > > > we use topic name. I am not sure if the need is very strong
>> though.
>> > So,
>> > > > > maybe we can list this as future considerations.
>> > > > >
>> > > > > Thanks,
>> > > > >
>> > > > > Jun
>> > > > >
>> > > > > On Wed, Oct 7, 2020 at 8:57 AM Justine Olshan <
>> jolshan@confluent.io>
>> > > > > wrote:
>> > > > >
>> > > > > > Hi all,
>> > > > > >
>> > > > > > After some careful consideration, I've decided that the
>> > > > TopicIdPartition
>> > > > > > will be out of scope for this KIP. After investigating what it
>> will
>> > > > look
>> > > > > > like using it in only the Fetch Request, I found that I would
>> need
>> > to
>> > > > > > modify several public APIs in the consumer and I would need to
>> take
>> > > > > special
>> > > > > > care in many situations to handle the request correctly based on
>> > what
>> > > > > > version of the request was sent. For these reasons, I've added
>> > > > > > TopicIdPartition to potential future work along with an
>> alternative
>> > > to
>> > > > > > modify TopicPartition to include a topic ID field.
>> > > > > >
>> > > > > > Jun- I was also wondering about some of the details for
>> supporting
>> > > > topic
>> > > > > > IDs for delete and describe topics in the admin client. Is the
>> idea
>> > > > that
>> > > > > > instead of supplying topic names to describe or delete, we would
>> > > > supply a
>> > > > > > collection of ids? If that is the case, I'm wondering if it
>> makes
>> > > sense
>> > > > > to
>> > > > > > add topic Ids to the metadata and delete topics requests.
>> > > > > >
>> > > > > > Thanks,
>> > > > > > Justine
>> > > > > >
>> > > > > > On Mon, Oct 5, 2020 at 9:25 PM Satish Duggana <
>> > > > satish.duggana@gmail.com>
>> > > > > > wrote:
>> > > > > >
>> > > > > > > Hi Jun, Justine,
>> > > > > > > +1 to have TopicIdPartition in public packages. This is useful
>> > for
>> > > > > > > KIP-405 tiered storage too. We currently use TopicPartition
>> for
>> > > > > > > pluggable log storage and metadata storage apis. We will be
>> using
>> > > > > > > TopicIdPartition once the proposed changes in this KIP are
>> > merged.
>> > > > > > >
>> > > > > > > Thanks,
>> > > > > > > Satish.
>> > > > > > >
>> > > > > > > On Tue, Oct 6, 2020 at 5:21 AM Jun Rao <ju...@confluent.io>
>> wrote:
>> > > > > > > >
>> > > > > > > > Hi, Justine,
>> > > > > > > >
>> > > > > > > > I guess we have to maintain the mapping between topic name
>> and
>> > > > > topicId
>> > > > > > in
>> > > > > > > > both the client and the server. If TopicIdPartition is meant
>> > for
>> > > > > > internal
>> > > > > > > > usage only, we can create it under a package
>> > > > > > > > like org/apache/kafka/common/internal and we don't need to
>> > > document
>> > > > > it
>> > > > > > in
>> > > > > > > > the KIP. If it's meant for external public usage (and
>> > potentially
>> > > > > > > internal
>> > > > > > > > usage too), we can create it in org/apache/kafka/common and
>> > need
>> > > to
>> > > > > > > > document it in the KIP.
>> > > > > > > >
>> > > > > > > > Since topicId is now part of the topic metadata, perhaps
>> it's
>> > > > useful
>> > > > > to
>> > > > > > > > expose it in TopicMetadataAndConfig too.
>> > > > > > > >
>> > > > > > > > Thanks,
>> > > > > > > >
>> > > > > > > > Jun
>> > > > > > > >
>> > > > > > > > On Mon, Oct 5, 2020 at 3:35 PM Justine Olshan <
>> > > > jolshan@confluent.io>
>> > > > > > > wrote:
>> > > > > > > >
>> > > > > > > > > Hi Jun,
>> > > > > > > > >
>> > > > > > > > > This KIP removes the topic name from many requests. A lot
>> of
>> > > > > requests
>> > > > > > > keep
>> > > > > > > > > track of the partitions internally with a TopicPartition.
>> So
>> > > when
>> > > > > we
>> > > > > > > remove
>> > > > > > > > > the topic name, we can not differentiate unless we can
>> also
>> > use
>> > > > > topic
>> > > > > > > ID.
>> > > > > > > > > The Fetch request relies heavily on this, but ListOffset,
>> > > > > > > > > OffsetsForLeaderEpoch, etc all use TopicPartition as a key
>> > > value
>> > > > > for
>> > > > > > > > > internal maps. Digging deeper into Fetch, I've noticed
>> that
>> > the
>> > > > > code
>> > > > > > > path
>> > > > > > > > > runs into some public APIs that use the TopicPartition.
>> Some
>> > > > > examples
>> > > > > > > are
>> > > > > > > > > Cluster.java or RecordTooLargeException.
>> > > > > > > > >
>> > > > > > > > > A related issue is that in some places like
>> > > > > > > TopicAuthorizationException,
>> > > > > > > > > the topic name is required and without the name in the
>> > request,
>> > > > we
>> > > > > > > won't
>> > > > > > > > > know it.
>> > > > > > > > >
>> > > > > > > > > I guess an alternative would be to have some mapping from
>> > topic
>> > > > > name
>> > > > > > to
>> > > > > > > > > topic ID, but I would have to think about that more.
>> > > > > > > > >
>> > > > > > > > > The KIP mentions exposing TopicId in TopicDescription.
>> (The
>> > > font
>> > > > > was
>> > > > > > > > > weirdly colored, so I've made it easier to read.)
>> > > > > > > > >
>> > > > > > > > > Would the idea for the TopicMetadataAndConfig be that the
>> > topic
>> > > > ID
>> > > > > > > would be
>> > > > > > > > > returned as a result?
>> > > > > > > > > As for the other methods describe/deletetopics, it makes
>> > sense
>> > > to
>> > > > > > > include
>> > > > > > > > > them at some point, but I'm not sure how far we should go
>> in
>> > > > terms
>> > > > > of
>> > > > > > > > > supporting topic IDs everywhere.
>> > > > > > > > >
>> > > > > > > > > Thank you for continuing to look at the KIP!
>> > > > > > > > > Justine
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > On Mon, Oct 5, 2020 at 2:53 PM Jun Rao <ju...@confluent.io>
>> > > wrote:
>> > > > > > > > >
>> > > > > > > > > > Hi, Justine,
>> > > > > > > > > >
>> > > > > > > > > > Adding a TopicIdPartition in the public interface may be
>> > > > useful.
>> > > > > > > Where
>> > > > > > > > > > would you expect it to be used?
>> > > > > > > > > >
>> > > > > > > > > > Related to this, since we are exposing topicId in the
>> tool,
>> > > it
>> > > > > > seems
>> > > > > > > that
>> > > > > > > > > > we may want to expose topicId in at least
>> TopicDescription
>> > > used
>> > > > > by
>> > > > > > > > > > AdminClient.describeTopics() and TopicMetadataAndConfig
>> > used
>> > > in
>> > > > > > > > > > AdminClient.createTopics(Collection<NewTopic>
>> newTopics). I
>> > > am
>> > > > > also
>> > > > > > > not
>> > > > > > > > > > sure if we plan to expose topicId in other AdminClient
>> > > methods
>> > > > in
>> > > > > > > this
>> > > > > > > > > KIP
>> > > > > > > > > > or in some future KIP. For example, it might be useful
>> to
>> > > > support
>> > > > > > > topicId
>> > > > > > > > > > in the following methods.
>> > > > > > > > > >
>> > > > > > > > > >     DescribeTopicsResult
>> describeTopics(Collection<String>
>> > > > > > > topicNames,
>> > > > > > > > > > DescribeTopicsOptions options);
>> > > > > > > > > >     DeleteTopicsResult deleteTopics(Collection<String>
>> > > topics,
>> > > > > > > > > > DeleteTopicsOptions options);
>> > > > > > > > > >
>> > > > > > > > > > Thanks,
>> > > > > > > > > >
>> > > > > > > > > > Jun
>> > > > > > > > > >
>> > > > > > > > > > On Mon, Oct 5, 2020 at 11:03 AM Justine Olshan <
>> > > > > > jolshan@confluent.io
>> > > > > > > >
>> > > > > > > > > > wrote:
>> > > > > > > > > >
>> > > > > > > > > > > Hi all,
>> > > > > > > > > > >
>> > > > > > > > > > > I was thinking about this KIP some more. We are
>> removing
>> > > the
>> > > > > > topic
>> > > > > > > name
>> > > > > > > > > > > from many of these request types, and some of them
>> (eg.
>> > > > Fetch)
>> > > > > > use
>> > > > > > > > > > > TopicPartition as keys in a map when handling the
>> > request.
>> > > I
>> > > > > was
>> > > > > > > > > thinking
>> > > > > > > > > > > it would make sense to make some sort of
>> TopicIdPartition
>> > > > with
>> > > > > > > topic
>> > > > > > > > > > name,
>> > > > > > > > > > > topic id, and partition fields (name and id would
>> > > potentially
>> > > > > be
>> > > > > > > > > optional
>> > > > > > > > > > > types since different request versions will contain
>> only
>> > > one
>> > > > of
>> > > > > > the
>> > > > > > > > > two)
>> > > > > > > > > > I
>> > > > > > > > > > > was wondering if it made sense to include this as a
>> > public
>> > > > > facing
>> > > > > > > API
>> > > > > > > > > > > similar to TopicPartition. If so, I will add this
>> class
>> > to
>> > > > the
>> > > > > > KIP
>> > > > > > > as
>> > > > > > > > > > well.
>> > > > > > > > > > >
>> > > > > > > > > > > Thanks,
>> > > > > > > > > > > Justine
>> > > > > > > > > > >
>> > > > > > > > > > > On Mon, Oct 5, 2020 at 8:52 AM Justine Olshan <
>> > > > > > > jolshan@confluent.io>
>> > > > > > > > > > > wrote:
>> > > > > > > > > > >
>> > > > > > > > > > > > Hi all,
>> > > > > > > > > > > > I've made a few minor changes to the UUID class.
>> I've
>> > > > added a
>> > > > > > few
>> > > > > > > > > more
>> > > > > > > > > > > > methods to support UUIDs more generally, by
>> including a
>> > > > > > > constructor
>> > > > > > > > > > and a
>> > > > > > > > > > > > method to generate random UUIDs. Both of these will
>> be
>> > > > useful
>> > > > > > for
>> > > > > > > > > > > testing.
>> > > > > > > > > > > >
>> > > > > > > > > > > > Let me know if there are any other areas for
>> > discussion.
>> > > > > > > > > > > >
>> > > > > > > > > > > > Thank you,
>> > > > > > > > > > > > Justine
>> > > > > > > > > > > >
>> > > > > > > > > > > > On Thu, Oct 1, 2020 at 5:29 PM Justine Olshan <
>> > > > > > > jolshan@confluent.io>
>> > > > > > > > > > > > wrote:
>> > > > > > > > > > > >
>> > > > > > > > > > > >> Hi Jun,
>> > > > > > > > > > > >> Thanks for looking at it again. Here's the new
>> spec.
>> > (I
>> > > > > fixed
>> > > > > > > the
>> > > > > > > > > typo
>> > > > > > > > > > > in
>> > > > > > > > > > > >> it too.)
>> > > > > > > > > > > >> {"name": "id", "type": "string", "doc":
>> option[UUID]}
>> > > > > > > > > > > >>
>> > > > > > > > > > > >> Justine
>> > > > > > > > > > > >>
>> > > > > > > > > > > >>
>> > > > > > > > > > > >> On Thu, Oct 1, 2020 at 5:03 PM Jun Rao <
>> > > jun@confluent.io>
>> > > > > > > wrote:
>> > > > > > > > > > > >>
>> > > > > > > > > > > >>> Hi, Justine,
>> > > > > > > > > > > >>>
>> > > > > > > > > > > >>> Thanks for the update. The KIP looks good to me
>> now.
>> > > > Just a
>> > > > > > > minor
>> > > > > > > > > > > comment
>> > > > > > > > > > > >>> below.
>> > > > > > > > > > > >>>
>> > > > > > > > > > > >>> 30. Perhaps "option[UUID]" can be put in the doc.
>> > > > > > > > > > > >>>
>> > > > > > > > > > > >>> Jun
>> > > > > > > > > > > >>>
>> > > > > > > > > > > >>> On Thu, Oct 1, 2020 at 3:28 PM Justine Olshan <
>> > > > > > > > > jolshan@confluent.io>
>> > > > > > > > > > > >>> wrote:
>> > > > > > > > > > > >>>
>> > > > > > > > > > > >>> > Hi Jun,
>> > > > > > > > > > > >>> > Thanks for the response!
>> > > > > > > > > > > >>> >
>> > > > > > > > > > > >>> > 30. I think I might have changed this in
>> between.
>> > The
>> > > > > > current
>> > > > > > > > > > version
>> > > > > > > > > > > >>> > says:  {"name":
>> > > > > > > > > > > >>> > "id", "type": "option[UUID]"}, "doc": topic id}
>> > > > > > > > > > > >>> > I have switched to the option type to cover the
>> > > > migration
>> > > > > > > case
>> > > > > > > > > > where
>> > > > > > > > > > > a
>> > > > > > > > > > > >>> > TopicZNode does not yet have a topic ID.
>> > > > > > > > > > > >>> > I understand that due to json, this field is
>> > written
>> > > > as a
>> > > > > > > string,
>> > > > > > > > > > so
>> > > > > > > > > > > >>> if I
>> > > > > > > > > > > >>> > should move the "option[uuid]" to the doc field
>> and
>> > > the
>> > > > > > type
>> > > > > > > > > should
>> > > > > > > > > > > be
>> > > > > > > > > > > >>> > "string" please let me know.
>> > > > > > > > > > > >>> >
>> > > > > > > > > > > >>> > 40. I've added a definition for UUID.
>> > > > > > > > > > > >>> > 41,42. Fixed
>> > > > > > > > > > > >>> >
>> > > > > > > > > > > >>> > Thank you,
>> > > > > > > > > > > >>> > Justine
>> > > > > > > > > > > >>> >
>> > > > > > > > > > > >>> > On Wed, Sep 30, 2020 at 1:15 PM Jun Rao <
>> > > > > jun@confluent.io>
>> > > > > > > > > wrote:
>> > > > > > > > > > > >>> >
>> > > > > > > > > > > >>> > > Hi, Justine,
>> > > > > > > > > > > >>> > >
>> > > > > > > > > > > >>> > > Thanks for the summary. Just a few minor
>> comments
>> > > > blow.
>> > > > > > > > > > > >>> > >
>> > > > > > > > > > > >>> > > 30.  {"name": "id", "type": "string", "doc":
>> > > "version
>> > > > > > > id"}}:
>> > > > > > > > > The
>> > > > > > > > > > > doc
>> > > > > > > > > > > >>> > should
>> > > > > > > > > > > >>> > > say UUID. The issue seems unfixed.
>> > > > > > > > > > > >>> > >
>> > > > > > > > > > > >>> > > 40. Since UUID is public facing, could you
>> > include
>> > > > its
>> > > > > > > > > > definition?
>> > > > > > > > > > > >>> > >
>> > > > > > > > > > > >>> > > 41. StopReplicaResponse still includes the
>> topic
>> > > > field.
>> > > > > > > > > > > >>> > >
>> > > > > > > > > > > >>> > > 42. "It is unnecessary to include the name of
>> the
>> > > > topic
>> > > > > > in
>> > > > > > > the
>> > > > > > > > > > > >>> following
>> > > > > > > > > > > >>> > > Request/Response calls" It would be useful to
>> > > include
>> > > > > all
>> > > > > > > > > > modified
>> > > > > > > > > > > >>> > requests
>> > > > > > > > > > > >>> > > (e.g. produce) in the list below.
>> > > > > > > > > > > >>> > >
>> > > > > > > > > > > >>> > > Thanks,
>> > > > > > > > > > > >>> > >
>> > > > > > > > > > > >>> > > Jun
>> > > > > > > > > > > >>> > >
>> > > > > > > > > > > >>> > >
>> > > > > > > > > > > >>> > > On Wed, Sep 30, 2020 at 10:17 AM Justine
>> Olshan <
>> > > > > > > > > > > >>> jolshan@confluent.io>
>> > > > > > > > > > > >>> > > wrote:
>> > > > > > > > > > > >>> > >
>> > > > > > > > > > > >>> > > > Hello again,
>> > > > > > > > > > > >>> > > >
>> > > > > > > > > > > >>> > > > I've taken some time to discuss some of the
>> > > > remaining
>> > > > > > > points
>> > > > > > > > > > > >>> brought up
>> > > > > > > > > > > >>> > > by
>> > > > > > > > > > > >>> > > > the previous emails offline. Here are some
>> of
>> > the
>> > > > > > > > > conclusions.
>> > > > > > > > > > > >>> > > >
>> > > > > > > > > > > >>> > > > 1. Directory Structure:
>> > > > > > > > > > > >>> > > > There was some talk about whether the
>> directory
>> > > > > > structure
>> > > > > > > > > > should
>> > > > > > > > > > > be
>> > > > > > > > > > > >>> > > changed
>> > > > > > > > > > > >>> > > > to replace all topic names with topic IDs.
>> > > > > > > > > > > >>> > > > This will be a useful change, but will
>> prevent
>> > > > > > > downgrades. It
>> > > > > > > > > > > will
>> > > > > > > > > > > >>> be
>> > > > > > > > > > > >>> > > best
>> > > > > > > > > > > >>> > > > to wait until a major release, likely
>> alongside
>> > > > > KIP-500
>> > > > > > > > > changes
>> > > > > > > > > > > >>> that
>> > > > > > > > > > > >>> > will
>> > > > > > > > > > > >>> > > > prevent downgrades. I've updated the KIP to
>> > > include
>> > > > > > this
>> > > > > > > > > change
>> > > > > > > > > > > >>> with
>> > > > > > > > > > > >>> > the
>> > > > > > > > > > > >>> > > > note about migration and deprecation.
>> > > > > > > > > > > >>> > > >
>> > > > > > > > > > > >>> > > >
>> > > > > > > > > > > >>> > >
>> > > > > > > > > > > >>> >
>> > > > > > > > > > > >>>
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-log.dirlayout
>> > > > > > > > > > > >>> > > >
>> > > > > > > > > > > >>> > > > 2. Partition Metadata file
>> > > > > > > > > > > >>> > > > There was some disagreement about the
>> inclusion
>> > > of
>> > > > > the
>> > > > > > > > > > partition
>> > > > > > > > > > > >>> > metadata
>> > > > > > > > > > > >>> > > > file.
>> > > > > > > > > > > >>> > > > This file will be needed to persist the
>> topic
>> > ID,
>> > > > > > > especially
>> > > > > > > > > > > while
>> > > > > > > > > > > >>> we
>> > > > > > > > > > > >>> > > still
>> > > > > > > > > > > >>> > > > have the old directory structure. Even after
>> > the
>> > > > > > changes,
>> > > > > > > > > this
>> > > > > > > > > > > >>> file can
>> > > > > > > > > > > >>> > > be
>> > > > > > > > > > > >>> > > > useful for debugging and recovery.
>> > > > > > > > > > > >>> > > > Creating a single mapping file from topic
>> names
>> > > to
>> > > > > > topic
>> > > > > > > IDs
>> > > > > > > > > > was
>> > > > > > > > > > > >>> > > > considered, but ultimately scrapped as it
>> would
>> > > not
>> > > > > be
>> > > > > > as
>> > > > > > > > > easy
>> > > > > > > > > > to
>> > > > > > > > > > > >>> > > maintain.
>> > > > > > > > > > > >>> > > >
>> > > > > > > > > > > >>> > > >
>> > > > > > > > > > > >>> > >
>> > > > > > > > > > > >>> >
>> > > > > > > > > > > >>>
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PartitionMetadatafile
>> > > > > > > > > > > >>> > > >
>> > > > > > > > > > > >>> > > >
>> > > > > > > > > > > >>> > > >
>> > > > > > > > > > > >>> > >
>> > > > > > > > > > > >>> >
>> > > > > > > > > > > >>>
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PersistingTopicIDs
>> > > > > > > > > > > >>> > > >
>> > > > > > > > > > > >>> > > > 3. Produce Protocols
>> > > > > > > > > > > >>> > > > After some further discussion, this
>> replacing
>> > the
>> > > > > topic
>> > > > > > > name
>> > > > > > > > > > with
>> > > > > > > > > > > >>> topic
>> > > > > > > > > > > >>> > > ID
>> > > > > > > > > > > >>> > > > in these protocols has been added to the
>> KIP.
>> > > > > > > > > > > >>> > > > This will cut down on the size of the
>> protocol.
>> > > > Since
>> > > > > > > changes
>> > > > > > > > > > to
>> > > > > > > > > > > >>> fetch
>> > > > > > > > > > > >>> > > are
>> > > > > > > > > > > >>> > > > included, it does make sense to update these
>> > > > > protocols.
>> > > > > > > > > > > >>> > > >
>> > > > > > > > > > > >>> > > >
>> > > > > > > > > > > >>> > >
>> > > > > > > > > > > >>> >
>> > > > > > > > > > > >>>
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-Produce
>> > > > > > > > > > > >>> > > >
>> > > > > > > > > > > >>> > > > 4. KIP-500 Compatibility
>> > > > > > > > > > > >>> > > > After some discussion with those involved
>> with
>> > > > > KIP-500,
>> > > > > > > it
>> > > > > > > > > > seems
>> > > > > > > > > > > >>> best
>> > > > > > > > > > > >>> > to
>> > > > > > > > > > > >>> > > > use a sentinel topic ID for the metadata
>> topic
>> > > that
>> > > > > is
>> > > > > > > used
>> > > > > > > > > > > before
>> > > > > > > > > > > >>> the
>> > > > > > > > > > > >>> > > > first controller election.
>> > > > > > > > > > > >>> > > > We had an issue where this metadata topic
>> may
>> > not
>> > > > be
>> > > > > > > assigned
>> > > > > > > > > > an
>> > > > > > > > > > > ID
>> > > > > > > > > > > >>> > > before
>> > > > > > > > > > > >>> > > > utilizing the Vote and Fetch protocols. It
>> was
>> > > > > decided
>> > > > > > to
>> > > > > > > > > > > reserve a
>> > > > > > > > > > > >>> > > unique
>> > > > > > > > > > > >>> > > > ID for this topic to be used until the
>> > controller
>> > > > > could
>> > > > > > > give
>> > > > > > > > > > the
>> > > > > > > > > > > >>> topic
>> > > > > > > > > > > >>> > a
>> > > > > > > > > > > >>> > > > unique ID.
>> > > > > > > > > > > >>> > > > Having the topic keep the sentinel ID (not
>> be
>> > > > > > reassigned
>> > > > > > > to a
>> > > > > > > > > > > >>> unique
>> > > > > > > > > > > >>> > ID)
>> > > > > > > > > > > >>> > > > was considered, but it seemed like a good
>> idea
>> > to
>> > > > > leave
>> > > > > > > the
>> > > > > > > > > > > option
>> > > > > > > > > > > >>> open
>> > > > > > > > > > > >>> > > for
>> > > > > > > > > > > >>> > > > the metadata topic to have a unique ID in
>> cases
>> > > > where
>> > > > > > it
>> > > > > > > > > would
>> > > > > > > > > > > >>> need to
>> > > > > > > > > > > >>> > be
>> > > > > > > > > > > >>> > > > differentiated from other clusters' metadata
>> > > > topics.
>> > > > > > (ex.
>> > > > > > > > > > tiered
>> > > > > > > > > > > >>> > storage)
>> > > > > > > > > > > >>> > > >
>> > > > > > > > > > > >>> > > >
>> > > > > > > > > > > >>> > >
>> > > > > > > > > > > >>> >
>> > > > > > > > > > > >>>
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-CompatibilitywithKIP-500
>> > > > > > > > > > > >>> > > >
>> > > > > > > > > > > >>> > > > I've also split up the KIP into sub-tasks on
>> > the
>> > > > > JIRA.
>> > > > > > > > > > Hopefully
>> > > > > > > > > > > >>> this
>> > > > > > > > > > > >>> > > will
>> > > > > > > > > > > >>> > > > provide a better idea about what tasks we
>> have,
>> > > and
>> > > > > > > > > eventually
>> > > > > > > > > > > >>> provide
>> > > > > > > > > > > >>> > a
>> > > > > > > > > > > >>> > > > place to see what's done and what is left.
>> > > > > > > > > > > >>> > > > If there is a task I am missing, please let
>> me
>> > > > know!
>> > > > > > > > > > > >>> > > >
>> > https://issues.apache.org/jira/browse/KAFKA-8872
>> > > > > > > > > > > >>> > > >
>> > > > > > > > > > > >>> > > > Of course, these decisions are not set in
>> > stone,
>> > > > and
>> > > > > I
>> > > > > > > would
>> > > > > > > > > > love
>> > > > > > > > > > > >>> to
>> > > > > > > > > > > >>> > hear
>> > > > > > > > > > > >>> > > > any feedback.
>> > > > > > > > > > > >>> > > >
>> > > > > > > > > > > >>> > > > Thanks,
>> > > > > > > > > > > >>> > > > Justine
>> > > > > > > > > > > >>> > > >
>> > > > > > > > > > > >>> > > > On Mon, Sep 28, 2020 at 11:38 AM Justine
>> > Olshan <
>> > > > > > > > > > > >>> jolshan@confluent.io>
>> > > > > > > > > > > >>> > > > wrote:
>> > > > > > > > > > > >>> > > >
>> > > > > > > > > > > >>> > > > > Hello all,
>> > > > > > > > > > > >>> > > > >
>> > > > > > > > > > > >>> > > > > I just wanted to follow up on this
>> > discussion.
>> > > > Did
>> > > > > we
>> > > > > > > come
>> > > > > > > > > to
>> > > > > > > > > > > an
>> > > > > > > > > > > >>> > > > > understanding about the directory
>> structure?
>> > > > > > > > > > > >>> > > > >
>> > > > > > > > > > > >>> > > > > I think the biggest question here is what
>> is
>> > > > > > > acceptable to
>> > > > > > > > > > > leave
>> > > > > > > > > > > >>> out
>> > > > > > > > > > > >>> > > due
>> > > > > > > > > > > >>> > > > > to scope vs. what is considered to be too
>> > much
>> > > > tech
>> > > > > > > debt.
>> > > > > > > > > > > >>> > > > > This KIP is already pretty large with the
>> > > number
>> > > > of
>> > > > > > > > > changes,
>> > > > > > > > > > > but
>> > > > > > > > > > > >>> it
>> > > > > > > > > > > >>> > > also
>> > > > > > > > > > > >>> > > > > makes sense to do things correctly, so I'd
>> > love
>> > > > to
>> > > > > > hear
>> > > > > > > > > > > >>> everyone's
>> > > > > > > > > > > >>> > > > thoughts.
>> > > > > > > > > > > >>> > > > >
>> > > > > > > > > > > >>> > > > > Thanks,
>> > > > > > > > > > > >>> > > > > Justine
>> > > > > > > > > > > >>> > > > >
>> > > > > > > > > > > >>> > > > > On Fri, Sep 25, 2020 at 8:19 AM Lucas
>> > > Bradstreet
>> > > > <
>> > > > > > > > > > > >>> lucas@confluent.io
>> > > > > > > > > > > >>> > >
>> > > > > > > > > > > >>> > > > > wrote:
>> > > > > > > > > > > >>> > > > >
>> > > > > > > > > > > >>> > > > >> Hi Ismael,
>> > > > > > > > > > > >>> > > > >>
>> > > > > > > > > > > >>> > > > >> If you do not store it in a metadata
>> file or
>> > > in
>> > > > > the
>> > > > > > > > > > directory
>> > > > > > > > > > > >>> > > structure
>> > > > > > > > > > > >>> > > > >> would you then
>> > > > > > > > > > > >>> > > > >> require the LeaderAndIsrRequest following
>> > > > startup
>> > > > > to
>> > > > > > > give
>> > > > > > > > > > you
>> > > > > > > > > > > >>> some
>> > > > > > > > > > > >>> > > > notion
>> > > > > > > > > > > >>> > > > >> of
>> > > > > > > > > > > >>> > > > >> topic name in memory? We would still need
>> > this
>> > > > > > > information
>> > > > > > > > > > for
>> > > > > > > > > > > >>> the
>> > > > > > > > > > > >>> > > older
>> > > > > > > > > > > >>> > > > >> protocols, but
>> > > > > > > > > > > >>> > > > >> perhaps this is what's meant by tech
>> debt.
>> > > > > > > > > > > >>> > > > >>
>> > > > > > > > > > > >>> > > > >> Once we're free of the old non-topicID
>> > > requests
>> > > > > > then I
>> > > > > > > > > think
>> > > > > > > > > > > you
>> > > > > > > > > > > >>> > > > wouldn't
>> > > > > > > > > > > >>> > > > >> need to retain the topic name.
>> > > > > > > > > > > >>> > > > >> I think the ability to easily look up
>> topic
>> > > > names
>> > > > > > > > > associated
>> > > > > > > > > > > >>> with
>> > > > > > > > > > > >>> > > > >> partition
>> > > > > > > > > > > >>> > > > >> directories would still be missed
>> > > > > > > > > > > >>> > > > >> when diagnosing issues, though maybe it
>> > > wouldn't
>> > > > > be
>> > > > > > a
>> > > > > > > deal
>> > > > > > > > > > > >>> breaker
>> > > > > > > > > > > >>> > > with
>> > > > > > > > > > > >>> > > > >> the
>> > > > > > > > > > > >>> > > > >> right tooling.
>> > > > > > > > > > > >>> > > > >>
>> > > > > > > > > > > >>> > > > >> Thanks,
>> > > > > > > > > > > >>> > > > >>
>> > > > > > > > > > > >>> > > > >> Lucas
>> > > > > > > > > > > >>> > > > >>
>> > > > > > > > > > > >>> > > > >> On Fri, Sep 25, 2020 at 7:55 AM Ismael
>> Juma
>> > <
>> > > > > > > > > > > ismael@juma.me.uk>
>> > > > > > > > > > > >>> > > wrote:
>> > > > > > > > > > > >>> > > > >>
>> > > > > > > > > > > >>> > > > >> > Hi Lucas,
>> > > > > > > > > > > >>> > > > >> >
>> > > > > > > > > > > >>> > > > >> > Why would you include the name and id?
>> I
>> > > think
>> > > > > > you'd
>> > > > > > > > > want
>> > > > > > > > > > to
>> > > > > > > > > > > >>> > remove
>> > > > > > > > > > > >>> > > > the
>> > > > > > > > > > > >>> > > > >> > name from the directory name right?
>> > Jason's
>> > > > > > > suggestion
>> > > > > > > > > was
>> > > > > > > > > > > >>> that if
>> > > > > > > > > > > >>> > > you
>> > > > > > > > > > > >>> > > > >> > remove the name from the directory,
>> then
>> > why
>> > > > > would
>> > > > > > > you
>> > > > > > > > > > need
>> > > > > > > > > > > >>> the id
>> > > > > > > > > > > >>> > > > name
>> > > > > > > > > > > >>> > > > >> > mapping file?
>> > > > > > > > > > > >>> > > > >> >
>> > > > > > > > > > > >>> > > > >> > Ismael
>> > > > > > > > > > > >>> > > > >> >
>> > > > > > > > > > > >>> > > > >> > On Thu, Sep 24, 2020 at 4:24 PM Lucas
>> > > > > Bradstreet <
>> > > > > > > > > > > >>> > > lucas@confluent.io>
>> > > > > > > > > > > >>> > > > >> > wrote:
>> > > > > > > > > > > >>> > > > >> >
>> > > > > > > > > > > >>> > > > >> > > > 2. Part of the usage of the file
>> is to
>> > > > have
>> > > > > > > > > persistent
>> > > > > > > > > > > >>> storage
>> > > > > > > > > > > >>> > > of
>> > > > > > > > > > > >>> > > > >> the
>> > > > > > > > > > > >>> > > > >> > > topic
>> > > > > > > > > > > >>> > > > >> > > ID and use it to compare with the ID
>> > > > supplied
>> > > > > in
>> > > > > > > the
>> > > > > > > > > > > >>> > LeaderAndIsr
>> > > > > > > > > > > >>> > > > >> > Request.
>> > > > > > > > > > > >>> > > > >> > > There is some discussion in the KIP
>> > about
>> > > > > > changes
>> > > > > > > to
>> > > > > > > > > the
>> > > > > > > > > > > >>> > directory
>> > > > > > > > > > > >>> > > > >> > > structure, but I believe directory
>> > changes
>> > > > > were
>> > > > > > > > > > considered
>> > > > > > > > > > > >>> to be
>> > > > > > > > > > > >>> > > out
>> > > > > > > > > > > >>> > > > >> of
>> > > > > > > > > > > >>> > > > >> > > scope when the KIP was written.
>> > > > > > > > > > > >>> > > > >> > >
>> > > > > > > > > > > >>> > > > >> > >
>> > > > > > > > > > > >>> > > > >> > > Yeah, I was hoping to get a better
>> > > > > understanding
>> > > > > > > of
>> > > > > > > > > why
>> > > > > > > > > > it
>> > > > > > > > > > > >>> was
>> > > > > > > > > > > >>> > > taken
>> > > > > > > > > > > >>> > > > >> out
>> > > > > > > > > > > >>> > > > >> > of
>> > > > > > > > > > > >>> > > > >> > > scope. Perhaps Lucas Bradstreet might
>> > have
>> > > > > more
>> > > > > > > > > insight
>> > > > > > > > > > > >>> about
>> > > > > > > > > > > >>> > the
>> > > > > > > > > > > >>> > > > >> > decision.
>> > > > > > > > > > > >>> > > > >> > > Basically my point is that we have to
>> > > create
>> > > > > > > > > additional
>> > > > > > > > > > > >>> > > > infrastructure
>> > > > > > > > > > > >>> > > > >> > here
>> > > > > > > > > > > >>> > > > >> > > to support the name/id mapping, so I
>> > > wanted
>> > > > to
>> > > > > > > > > > understand
>> > > > > > > > > > > >>> if we
>> > > > > > > > > > > >>> > > just
>> > > > > > > > > > > >>> > > > >> > > consider this a sort of tech debt. It
>> > > would
>> > > > be
>> > > > > > > useful
>> > > > > > > > > to
>> > > > > > > > > > > >>> cover
>> > > > > > > > > > > >>> > how
>> > > > > > > > > > > >>> > > > we
>> > > > > > > > > > > >>> > > > >> > > handle the case when this file gets
>> > > > corrupted.
>> > > > > > > Seems
>> > > > > > > > > > like
>> > > > > > > > > > > we
>> > > > > > > > > > > >>> > just
>> > > > > > > > > > > >>> > > > >> have to
>> > > > > > > > > > > >>> > > > >> > > trust that it matches whatever the
>> > > > controller
>> > > > > > > tells us
>> > > > > > > > > > and
>> > > > > > > > > > > >>> > rewrite
>> > > > > > > > > > > >>> > > > it?
>> > > > > > > > > > > >>> > > > >> > >
>> > > > > > > > > > > >>> > > > >> > >
>> > > > > > > > > > > >>> > > > >> > > Hi Jason, Justine,
>> > > > > > > > > > > >>> > > > >> > >
>> > > > > > > > > > > >>> > > > >> > > My thought process is that we will
>> > likely
>> > > > need
>> > > > > > the
>> > > > > > > > > > > metadata
>> > > > > > > > > > > >>> file
>> > > > > > > > > > > >>> > > > >> whether
>> > > > > > > > > > > >>> > > > >> > we
>> > > > > > > > > > > >>> > > > >> > > rename the directories or not.
>> > > > > > > > > > > >>> > > > >> > > Linux supports filenames of up to 255
>> > > bytes
>> > > > > and
>> > > > > > > that
>> > > > > > > > > > would
>> > > > > > > > > > > >>> not
>> > > > > > > > > > > >>> > be
>> > > > > > > > > > > >>> > > > >> enough
>> > > > > > > > > > > >>> > > > >> > to
>> > > > > > > > > > > >>> > > > >> > > support a directory name
>> > > > > > > > > > > >>> > > > >> > >  including both the name and topic
>> ID.
>> > > Given
>> > > > > > that
>> > > > > > > > > fact,
>> > > > > > > > > > it
>> > > > > > > > > > > >>> > seemed
>> > > > > > > > > > > >>> > > > >> > > reasonable to add the metadata file
>> > > > > > > > > > > >>> > > > >> > > and leave the directory rename until
>> > some
>> > > > time
>> > > > > > in
>> > > > > > > the
>> > > > > > > > > > > future
>> > > > > > > > > > > >>> > > > (possibly
>> > > > > > > > > > > >>> > > > >> > > never).
>> > > > > > > > > > > >>> > > > >> > >
>> > > > > > > > > > > >>> > > > >> > > If the file does get corrupted I
>> think
>> > > > you're
>> > > > > > > right.
>> > > > > > > > > We
>> > > > > > > > > > > >>> would
>> > > > > > > > > > > >>> > > either
>> > > > > > > > > > > >>> > > > >> have
>> > > > > > > > > > > >>> > > > >> > > to trust it matches what the
>> controller
>> > > > tells
>> > > > > us
>> > > > > > > > > > > >>> > > > >> > >  or error out and let an
>> administrator
>> > > > resolve
>> > > > > > it
>> > > > > > > by
>> > > > > > > > > > > >>> checking
>> > > > > > > > > > > >>> > > across
>> > > > > > > > > > > >>> > > > >> > > replicas for consistency.
>> > > > > > > > > > > >>> > > > >> > >
>> > > > > > > > > > > >>> > > > >> > > Lucas
>> > > > > > > > > > > >>> > > > >> > >
>> > > > > > > > > > > >>> > > > >> > >
>> > > > > > > > > > > >>> > > > >> > > On Thu, Sep 24, 2020 at 3:41 PM Jason
>> > > > > Gustafson
>> > > > > > <
>> > > > > > > > > > > >>> > > jason@confluent.io
>> > > > > > > > > > > >>> > > > >
>> > > > > > > > > > > >>> > > > >> > > wrote:
>> > > > > > > > > > > >>> > > > >> > >
>> > > > > > > > > > > >>> > > > >> > > > Thanks Justine. Responses below:
>> > > > > > > > > > > >>> > > > >> > > >
>> > > > > > > > > > > >>> > > > >> > > > > 1. Yes, the directory will still
>> be
>> > > > based
>> > > > > on
>> > > > > > > the
>> > > > > > > > > > topic
>> > > > > > > > > > > >>> > names.
>> > > > > > > > > > > >>> > > > >> > > > LeaderAndIsrRequest is one of the
>> few
>> > > > > requests
>> > > > > > > that
>> > > > > > > > > > will
>> > > > > > > > > > > >>> still
>> > > > > > > > > > > >>> > > > >> contain
>> > > > > > > > > > > >>> > > > >> > > the
>> > > > > > > > > > > >>> > > > >> > > > topic name. So I think we have this
>> > > > covered.
>> > > > > > > Sorry
>> > > > > > > > > for
>> > > > > > > > > > > >>> > > confusion.
>> > > > > > > > > > > >>> > > > >> > > >
>> > > > > > > > > > > >>> > > > >> > > > Ah, you're right. My eyes passed
>> right
>> > > > over
>> > > > > > the
>> > > > > > > > > field.
>> > > > > > > > > > > >>> > > > >> > > >
>> > > > > > > > > > > >>> > > > >> > > > > 2. Part of the usage of the file
>> is
>> > to
>> > > > > have
>> > > > > > > > > > persistent
>> > > > > > > > > > > >>> > storage
>> > > > > > > > > > > >>> > > > of
>> > > > > > > > > > > >>> > > > >> the
>> > > > > > > > > > > >>> > > > >> > > > topic
>> > > > > > > > > > > >>> > > > >> > > > ID and use it to compare with the
>> ID
>> > > > > supplied
>> > > > > > > in the
>> > > > > > > > > > > >>> > > LeaderAndIsr
>> > > > > > > > > > > >>> > > > >> > > Request.
>> > > > > > > > > > > >>> > > > >> > > > There is some discussion in the KIP
>> > > about
>> > > > > > > changes to
>> > > > > > > > > > the
>> > > > > > > > > > > >>> > > directory
>> > > > > > > > > > > >>> > > > >> > > > structure, but I believe directory
>> > > changes
>> > > > > > were
>> > > > > > > > > > > >>> considered to
>> > > > > > > > > > > >>> > be
>> > > > > > > > > > > >>> > > > >> out of
>> > > > > > > > > > > >>> > > > >> > > > scope when the KIP was written.
>> > > > > > > > > > > >>> > > > >> > > >
>> > > > > > > > > > > >>> > > > >> > > > Yeah, I was hoping to get a better
>> > > > > > > understanding of
>> > > > > > > > > > why
>> > > > > > > > > > > >>> it was
>> > > > > > > > > > > >>> > > > taken
>> > > > > > > > > > > >>> > > > >> > out
>> > > > > > > > > > > >>> > > > >> > > of
>> > > > > > > > > > > >>> > > > >> > > > scope. Perhaps Lucas Bradstreet
>> might
>> > > have
>> > > > > > more
>> > > > > > > > > > insight
>> > > > > > > > > > > >>> about
>> > > > > > > > > > > >>> > > the
>> > > > > > > > > > > >>> > > > >> > > decision.
>> > > > > > > > > > > >>> > > > >> > > > Basically my point is that we have
>> to
>> > > > create
>> > > > > > > > > > additional
>> > > > > > > > > > > >>> > > > >> infrastructure
>> > > > > > > > > > > >>> > > > >> > > here
>> > > > > > > > > > > >>> > > > >> > > > to support the name/id mapping, so
>> I
>> > > > wanted
>> > > > > to
>> > > > > > > > > > > understand
>> > > > > > > > > > > >>> if
>> > > > > > > > > > > >>> > we
>> > > > > > > > > > > >>> > > > just
>> > > > > > > > > > > >>> > > > >> > > > consider this a sort of tech debt.
>> It
>> > > > would
>> > > > > be
>> > > > > > > > > useful
>> > > > > > > > > > to
>> > > > > > > > > > > >>> cover
>> > > > > > > > > > > >>> > > how
>> > > > > > > > > > > >>> > > > >> we
>> > > > > > > > > > > >>> > > > >> > > > handle the case when this file gets
>> > > > > corrupted.
>> > > > > > > Seems
>> > > > > > > > > > > like
>> > > > > > > > > > > >>> we
>> > > > > > > > > > > >>> > > just
>> > > > > > > > > > > >>> > > > >> have
>> > > > > > > > > > > >>> > > > >> > to
>> > > > > > > > > > > >>> > > > >> > > > trust that it matches whatever the
>> > > > > controller
>> > > > > > > tells
>> > > > > > > > > us
>> > > > > > > > > > > and
>> > > > > > > > > > > >>> > > rewrite
>> > > > > > > > > > > >>> > > > >> it?
>> > > > > > > > > > > >>> > > > >> > > >
>> > > > > > > > > > > >>> > > > >> > > > > 3. I think this is a good point,
>> > but I
>> > > > > > again I
>> > > > > > > > > > wonder
>> > > > > > > > > > > >>> about
>> > > > > > > > > > > >>> > > the
>> > > > > > > > > > > >>> > > > >> scope
>> > > > > > > > > > > >>> > > > >> > > of
>> > > > > > > > > > > >>> > > > >> > > > the KIP. Most of the changes
>> mentioned
>> > > in
>> > > > > the
>> > > > > > > KIP
>> > > > > > > > > are
>> > > > > > > > > > > for
>> > > > > > > > > > > >>> > > > supporting
>> > > > > > > > > > > >>> > > > >> > > topic
>> > > > > > > > > > > >>> > > > >> > > > deletion and I believe that is why
>> the
>> > > > > produce
>> > > > > > > > > request
>> > > > > > > > > > > was
>> > > > > > > > > > > >>> > > listed
>> > > > > > > > > > > >>> > > > >> under
>> > > > > > > > > > > >>> > > > >> > > > future work.
>> > > > > > > > > > > >>> > > > >> > > >
>> > > > > > > > > > > >>> > > > >> > > > That's fair. I brought it up since
>> > > `Fetch`
>> > > > > is
>> > > > > > > > > already
>> > > > > > > > > > > >>> > included.
>> > > > > > > > > > > >>> > > If
>> > > > > > > > > > > >>> > > > >> > we've
>> > > > > > > > > > > >>> > > > >> > > > got `Metadata` and `Fetch`, seems
>> we
>> > may
>> > > > as
>> > > > > > > well do
>> > > > > > > > > > > >>> `Produce`
>> > > > > > > > > > > >>> > > and
>> > > > > > > > > > > >>> > > > >> save
>> > > > > > > > > > > >>> > > > >> > an
>> > > > > > > > > > > >>> > > > >> > > > extra kip. No strong objection
>> though
>> > if
>> > > > you
>> > > > > > > want to
>> > > > > > > > > > > >>> leave it
>> > > > > > > > > > > >>> > > out.
>> > > > > > > > > > > >>> > > > >> > > >
>> > > > > > > > > > > >>> > > > >> > > >
>> > > > > > > > > > > >>> > > > >> > > > -Jason
>> > > > > > > > > > > >>> > > > >> > > >
>> > > > > > > > > > > >>> > > > >> > > >
>> > > > > > > > > > > >>> > > > >> > > > On Thu, Sep 24, 2020 at 3:26 PM
>> > Justine
>> > > > > > Olshan <
>> > > > > > > > > > > >>> > > > >> jolshan@confluent.io>
>> > > > > > > > > > > >>> > > > >> > > > wrote:
>> > > > > > > > > > > >>> > > > >> > > >
>> > > > > > > > > > > >>> > > > >> > > > > Hi Jason,
>> > > > > > > > > > > >>> > > > >> > > > >
>> > > > > > > > > > > >>> > > > >> > > > > Thanks for your comments.
>> > > > > > > > > > > >>> > > > >> > > > >
>> > > > > > > > > > > >>> > > > >> > > > > 1. Yes, the directory will still
>> be
>> > > > based
>> > > > > on
>> > > > > > > the
>> > > > > > > > > > topic
>> > > > > > > > > > > >>> > names.
>> > > > > > > > > > > >>> > > > >> > > > > LeaderAndIsrRequest is one of the
>> > few
>> > > > > > requests
>> > > > > > > > > that
>> > > > > > > > > > > will
>> > > > > > > > > > > >>> > still
>> > > > > > > > > > > >>> > > > >> > contain
>> > > > > > > > > > > >>> > > > >> > > > the
>> > > > > > > > > > > >>> > > > >> > > > > topic name. So I think we have
>> this
>> > > > > covered.
>> > > > > > > Sorry
>> > > > > > > > > > for
>> > > > > > > > > > > >>> > > > confusion.
>> > > > > > > > > > > >>> > > > >> > > > >
>> > > > > > > > > > > >>> > > > >> > > > > 2. Part of the usage of the file
>> is
>> > to
>> > > > > have
>> > > > > > > > > > persistent
>> > > > > > > > > > > >>> > storage
>> > > > > > > > > > > >>> > > > of
>> > > > > > > > > > > >>> > > > >> the
>> > > > > > > > > > > >>> > > > >> > > > topic
>> > > > > > > > > > > >>> > > > >> > > > > ID and use it to compare with
>> the ID
>> > > > > > supplied
>> > > > > > > in
>> > > > > > > > > the
>> > > > > > > > > > > >>> > > > LeaderAndIsr
>> > > > > > > > > > > >>> > > > >> > > > Request.
>> > > > > > > > > > > >>> > > > >> > > > > There is some discussion in the
>> KIP
>> > > > about
>> > > > > > > changes
>> > > > > > > > > to
>> > > > > > > > > > > the
>> > > > > > > > > > > >>> > > > directory
>> > > > > > > > > > > >>> > > > >> > > > > structure, but I believe
>> directory
>> > > > changes
>> > > > > > > were
>> > > > > > > > > > > >>> considered
>> > > > > > > > > > > >>> > to
>> > > > > > > > > > > >>> > > be
>> > > > > > > > > > > >>> > > > >> out
>> > > > > > > > > > > >>> > > > >> > of
>> > > > > > > > > > > >>> > > > >> > > > > scope when the KIP was written.
>> > > > > > > > > > > >>> > > > >> > > > >
>> > > > > > > > > > > >>> > > > >> > > > > 3. I think this is a good point,
>> > but I
>> > > > > > again I
>> > > > > > > > > > wonder
>> > > > > > > > > > > >>> about
>> > > > > > > > > > > >>> > > the
>> > > > > > > > > > > >>> > > > >> scope
>> > > > > > > > > > > >>> > > > >> > > of
>> > > > > > > > > > > >>> > > > >> > > > > the KIP. Most of the changes
>> > mentioned
>> > > > in
>> > > > > > the
>> > > > > > > KIP
>> > > > > > > > > > are
>> > > > > > > > > > > >>> for
>> > > > > > > > > > > >>> > > > >> supporting
>> > > > > > > > > > > >>> > > > >> > > > topic
>> > > > > > > > > > > >>> > > > >> > > > > deletion and I believe that is
>> why
>> > the
>> > > > > > produce
>> > > > > > > > > > request
>> > > > > > > > > > > >>> was
>> > > > > > > > > > > >>> > > > listed
>> > > > > > > > > > > >>> > > > >> > under
>> > > > > > > > > > > >>> > > > >> > > > > future work.
>> > > > > > > > > > > >>> > > > >> > > > >
>> > > > > > > > > > > >>> > > > >> > > > > 4. This sounds like it might be a
>> > good
>> > > > > > > solution,
>> > > > > > > > > > but I
>> > > > > > > > > > > >>> will
>> > > > > > > > > > > >>> > > need
>> > > > > > > > > > > >>> > > > >> to
>> > > > > > > > > > > >>> > > > >> > > > discuss
>> > > > > > > > > > > >>> > > > >> > > > > more with KIP-500 folks to get
>> the
>> > > > details
>> > > > > > > right.
>> > > > > > > > > > > >>> > > > >> > > > >
>> > > > > > > > > > > >>> > > > >> > > > > Thanks,
>> > > > > > > > > > > >>> > > > >> > > > > Justine
>> > > > > > > > > > > >>> > > > >> > > > >
>> > > > > > > > > > > >>> > > > >> > > > > On Thu, Sep 24, 2020 at 12:30 PM
>> > Jason
>> > > > > > > Gustafson <
>> > > > > > > > > > > >>> > > > >> jason@confluent.io
>> > > > > > > > > > > >>> > > > >> > >
>> > > > > > > > > > > >>> > > > >> > > > > wrote:
>> > > > > > > > > > > >>> > > > >> > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > Hi Justine,
>> > > > > > > > > > > >>> > > > >> > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > Thanks for picking up this
>> work. I
>> > > > have
>> > > > > a
>> > > > > > > few
>> > > > > > > > > > > >>> > > > >> questions/comments:
>> > > > > > > > > > > >>> > > > >> > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > 1. It sounds like the directory
>> > > > > structure
>> > > > > > is
>> > > > > > > > > still
>> > > > > > > > > > > >>> going
>> > > > > > > > > > > >>> > to
>> > > > > > > > > > > >>> > > be
>> > > > > > > > > > > >>> > > > >> > based
>> > > > > > > > > > > >>> > > > >> > > on
>> > > > > > > > > > > >>> > > > >> > > > > > topic names. Do I have that
>> right?
>> > > One
>> > > > > > > > > > complication
>> > > > > > > > > > > is
>> > > > > > > > > > > >>> > that
>> > > > > > > > > > > >>> > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > LeaderAndIsr request does not
>> > > include
>> > > > > the
>> > > > > > > topic
>> > > > > > > > > > name
>> > > > > > > > > > > >>> any
>> > > > > > > > > > > >>> > > > longer.
>> > > > > > > > > > > >>> > > > >> > This
>> > > > > > > > > > > >>> > > > >> > > > > means
>> > > > > > > > > > > >>> > > > >> > > > > > that a replica must wait for
>> the
>> > > > > > > UpdateMetadata
>> > > > > > > > > > > >>> request to
>> > > > > > > > > > > >>> > > > >> arrive
>> > > > > > > > > > > >>> > > > >> > > with
>> > > > > > > > > > > >>> > > > >> > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > topic name mapping before it
>> can
>> > > > create
>> > > > > > the
>> > > > > > > > > > > >>> directory. I
>> > > > > > > > > > > >>> > > > wonder
>> > > > > > > > > > > >>> > > > >> if
>> > > > > > > > > > > >>> > > > >> > it
>> > > > > > > > > > > >>> > > > >> > > > > would
>> > > > > > > > > > > >>> > > > >> > > > > > be simpler to avoid
>> assumptions on
>> > > the
>> > > > > > > ordering
>> > > > > > > > > of
>> > > > > > > > > > > >>> > > > >> UpdateMetadata
>> > > > > > > > > > > >>> > > > >> > and
>> > > > > > > > > > > >>> > > > >> > > > let
>> > > > > > > > > > > >>> > > > >> > > > > > LeaderAndIsr include the topic
>> > name
>> > > as
>> > > > > > well.
>> > > > > > > > > Feels
>> > > > > > > > > > > >>> like we
>> > > > > > > > > > > >>> > > are
>> > > > > > > > > > > >>> > > > >> not
>> > > > > > > > > > > >>> > > > >> > > > saving
>> > > > > > > > > > > >>> > > > >> > > > > > that much by excluding it.
>> > > > > > > > > > > >>> > > > >> > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > 2. On a related note, it seems
>> > that
>> > > > the
>> > > > > > > reason
>> > > > > > > > > we
>> > > > > > > > > > > >>> have the
>> > > > > > > > > > > >>> > > > >> > partition
>> > > > > > > > > > > >>> > > > >> > > > > > metadata file is because we are
>> > not
>> > > > > > > changing the
>> > > > > > > > > > > >>> directory
>> > > > > > > > > > > >>> > > > >> > structure.
>> > > > > > > > > > > >>> > > > >> > > > We
>> > > > > > > > > > > >>> > > > >> > > > > > need it so that we remember
>> which
>> > > > > > > directories
>> > > > > > > > > map
>> > > > > > > > > > to
>> > > > > > > > > > > >>> which
>> > > > > > > > > > > >>> > > > topic
>> > > > > > > > > > > >>> > > > >> > id.
>> > > > > > > > > > > >>> > > > >> > > I
>> > > > > > > > > > > >>> > > > >> > > > > > think it would be useful to add
>> > some
>> > > > > > detail
>> > > > > > > > > about
>> > > > > > > > > > > why
>> > > > > > > > > > > >>> > > changing
>> > > > > > > > > > > >>> > > > >> the
>> > > > > > > > > > > >>> > > > >> > > > > > directory layout was rejected.
>> > > > Basically
>> > > > > > > trying
>> > > > > > > > > to
>> > > > > > > > > > > >>> > > understand
>> > > > > > > > > > > >>> > > > >> how
>> > > > > > > > > > > >>> > > > >> > > much
>> > > > > > > > > > > >>> > > > >> > > > > > effort we are saving by
>> skipping
>> > > this
>> > > > > step
>> > > > > > > if we
>> > > > > > > > > > > have
>> > > > > > > > > > > >>> to
>> > > > > > > > > > > >>> > > > >> implement
>> > > > > > > > > > > >>> > > > >> > > this
>> > > > > > > > > > > >>> > > > >> > > > > > additional bookkeeping. One
>> > concern
>> > > I
>> > > > > have
>> > > > > > > for
>> > > > > > > > > > > >>> example is
>> > > > > > > > > > > >>> > > that
>> > > > > > > > > > > >>> > > > >> the
>> > > > > > > > > > > >>> > > > >> > > > > > partition metadata file gets
>> > corrupt
>> > > > and
>> > > > > > > then
>> > > > > > > > > all
>> > > > > > > > > > > >>> bets are
>> > > > > > > > > > > >>> > > off
>> > > > > > > > > > > >>> > > > >> as
>> > > > > > > > > > > >>> > > > >> > far
>> > > > > > > > > > > >>> > > > >> > > > as
>> > > > > > > > > > > >>> > > > >> > > > > > topic consistency.
>> > > > > > > > > > > >>> > > > >> > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > 3. Is it worth adding support
>> now
>> > > for
>> > > > > > topic
>> > > > > > > ids
>> > > > > > > > > in
>> > > > > > > > > > > the
>> > > > > > > > > > > >>> > > > `Produce`
>> > > > > > > > > > > >>> > > > >> > > > request
>> > > > > > > > > > > >>> > > > >> > > > > > now? Overhead is mentioned as
>> one
>> > of
>> > > > the
>> > > > > > > > > > motivations
>> > > > > > > > > > > >>> and
>> > > > > > > > > > > >>> > the
>> > > > > > > > > > > >>> > > > >> best
>> > > > > > > > > > > >>> > > > >> > > APIs
>> > > > > > > > > > > >>> > > > >> > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > get savings from are `Produce`
>> and
>> > > > > > `Fetch`.
>> > > > > > > > > > > >>> > > > >> > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > 4. When it comes to
>> bootstrapping
>> > > the
>> > > > > > > metadata
>> > > > > > > > > > topic
>> > > > > > > > > > > >>> with
>> > > > > > > > > > > >>> > > > >> respect
>> > > > > > > > > > > >>> > > > >> > to
>> > > > > > > > > > > >>> > > > >> > > > > > KIP-500, one suggestion would
>> be
>> > to
>> > > > > > reserve
>> > > > > > > a
>> > > > > > > > > > > sentinel
>> > > > > > > > > > > >>> > topic
>> > > > > > > > > > > >>> > > > ID
>> > > > > > > > > > > >>> > > > >> > which
>> > > > > > > > > > > >>> > > > >> > > > can
>> > > > > > > > > > > >>> > > > >> > > > > > be used initially by new
>> brokers.
>> > > When
>> > > > > the
>> > > > > > > first
>> > > > > > > > > > > >>> > controller
>> > > > > > > > > > > >>> > > is
>> > > > > > > > > > > >>> > > > >> > > elected,
>> > > > > > > > > > > >>> > > > >> > > > > it
>> > > > > > > > > > > >>> > > > >> > > > > > can assign a topicId to the
>> > metadata
>> > > > > > topic.
>> > > > > > > This
>> > > > > > > > > > is
>> > > > > > > > > > > a
>> > > > > > > > > > > >>> bit
>> > > > > > > > > > > >>> > > > >> similar
>> > > > > > > > > > > >>> > > > >> > to
>> > > > > > > > > > > >>> > > > >> > > > how
>> > > > > > > > > > > >>> > > > >> > > > > we
>> > > > > > > > > > > >>> > > > >> > > > > > were planning to generate the
>> > > > clusterId.
>> > > > > > > > > > > >>> > > > >> > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > Thanks,
>> > > > > > > > > > > >>> > > > >> > > > > > Jason
>> > > > > > > > > > > >>> > > > >> > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > On Thu, Sep 24, 2020 at 11:10
>> AM
>> > Jun
>> > > > > Rao <
>> > > > > > > > > > > >>> > jun@confluent.io>
>> > > > > > > > > > > >>> > > > >> wrote:
>> > > > > > > > > > > >>> > > > >> > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > Hi, Justine,
>> > > > > > > > > > > >>> > > > >> > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > Thanks for the updated KIP. A
>> > few
>> > > > more
>> > > > > > > > > comments
>> > > > > > > > > > > >>> below.
>> > > > > > > > > > > >>> > > > >> > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > 30.  {"name": "id", "type":
>> > > > "string",
>> > > > > > > "doc":
>> > > > > > > > > > > >>> "version
>> > > > > > > > > > > >>> > > id"}}:
>> > > > > > > > > > > >>> > > > >> The
>> > > > > > > > > > > >>> > > > >> > > doc
>> > > > > > > > > > > >>> > > > >> > > > > > should
>> > > > > > > > > > > >>> > > > >> > > > > > > say UUID.
>> > > > > > > > > > > >>> > > > >> > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > 31. LeaderAndIsrResponse v5
>> and
>> > > > > > > > > > > StopReplicaResponse
>> > > > > > > > > > > >>> v4 :
>> > > > > > > > > > > >>> > > It
>> > > > > > > > > > > >>> > > > >> seems
>> > > > > > > > > > > >>> > > > >> > > > there
>> > > > > > > > > > > >>> > > > >> > > > > > is
>> > > > > > > > > > > >>> > > > >> > > > > > > no need to add topic_id at
>> > > > partitions
>> > > > > > > level.
>> > > > > > > > > > > >>> > > > >> > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > 32. Regarding partition
>> metadata
>> > > > file.
>> > > > > > > Perhaps
>> > > > > > > > > > the
>> > > > > > > > > > > >>> key
>> > > > > > > > > > > >>> > can
>> > > > > > > > > > > >>> > > > be
>> > > > > > > > > > > >>> > > > >> a
>> > > > > > > > > > > >>> > > > >> > > > single
>> > > > > > > > > > > >>> > > > >> > > > > > > word, sth like the following.
>> > > > > > > > > > > >>> > > > >> > > > > > > version: 0
>> > > > > > > > > > > >>> > > > >> > > > > > > topic_id:
>> > > > > > > 46bdb63f-9e8d-4a38-bf7b-ee4eb2a794e4
>> > > > > > > > > > > >>> > > > >> > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > 33. Another tricky thing
>> that I
>> > > > > realized
>> > > > > > > is
>> > > > > > > > > how
>> > > > > > > > > > to
>> > > > > > > > > > > >>> > support
>> > > > > > > > > > > >>> > > > the
>> > > > > > > > > > > >>> > > > >> > > > metadata
>> > > > > > > > > > > >>> > > > >> > > > > > > topic introduced in KIP-595.
>> > It's
>> > > a
>> > > > > bit
>> > > > > > > tricky
>> > > > > > > > > > to
>> > > > > > > > > > > >>> > assign a
>> > > > > > > > > > > >>> > > > >> UUID
>> > > > > > > > > > > >>> > > > >> > to
>> > > > > > > > > > > >>> > > > >> > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > metadata topic since we have
>> a
>> > > > > > > chicken-and-egg
>> > > > > > > > > > > >>> problem.
>> > > > > > > > > > > >>> > > The
>> > > > > > > > > > > >>> > > > >> > > > controller
>> > > > > > > > > > > >>> > > > >> > > > > > > needs to persist the UUID in
>> the
>> > > > > > metadata
>> > > > > > > > > topic
>> > > > > > > > > > in
>> > > > > > > > > > > >>> order
>> > > > > > > > > > > >>> > > to
>> > > > > > > > > > > >>> > > > >> > assign
>> > > > > > > > > > > >>> > > > >> > > > one
>> > > > > > > > > > > >>> > > > >> > > > > > > successfully, but the
>> metadata
>> > > topic
>> > > > > is
>> > > > > > > needed
>> > > > > > > > > > to
>> > > > > > > > > > > >>> elect
>> > > > > > > > > > > >>> > a
>> > > > > > > > > > > >>> > > > >> > > controller
>> > > > > > > > > > > >>> > > > >> > > > > > > (KIP-631). So, this probably
>> > > needs a
>> > > > > bit
>> > > > > > > more
>> > > > > > > > > > > >>> thought.
>> > > > > > > > > > > >>> > > > >> > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > Jun
>> > > > > > > > > > > >>> > > > >> > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > On Thu, Sep 24, 2020 at 3:04
>> AM
>> > > > Ismael
>> > > > > > > Juma <
>> > > > > > > > > > > >>> > > > >> ismael@juma.me.uk>
>> > > > > > > > > > > >>> > > > >> > > > wrote:
>> > > > > > > > > > > >>> > > > >> > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > Also, can we provide more
>> > > details
>> > > > on
>> > > > > > > how the
>> > > > > > > > > > > >>> Partition
>> > > > > > > > > > > >>> > > > >> Metadata
>> > > > > > > > > > > >>> > > > >> > > > file
>> > > > > > > > > > > >>> > > > >> > > > > > will
>> > > > > > > > > > > >>> > > > >> > > > > > > > be used?
>> > > > > > > > > > > >>> > > > >> > > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > Ismael
>> > > > > > > > > > > >>> > > > >> > > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > On Thu, Sep 24, 2020 at
>> 3:01
>> > AM
>> > > > > Ismael
>> > > > > > > Juma
>> > > > > > > > > <
>> > > > > > > > > > > >>> > > > >> ismael@juma.me.uk
>> > > > > > > > > > > >>> > > > >> > >
>> > > > > > > > > > > >>> > > > >> > > > > wrote:
>> > > > > > > > > > > >>> > > > >> > > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > > Hi Justine,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > > I think we need to update
>> > the
>> > > > > > > "Rejected
>> > > > > > > > > > > >>> > Alternatives"
>> > > > > > > > > > > >>> > > > >> section
>> > > > > > > > > > > >>> > > > >> > > to
>> > > > > > > > > > > >>> > > > >> > > > > take
>> > > > > > > > > > > >>> > > > >> > > > > > > > into
>> > > > > > > > > > > >>> > > > >> > > > > > > > > account that the proposal
>> > now
>> > > > > > removes
>> > > > > > > the
>> > > > > > > > > > > topic
>> > > > > > > > > > > >>> name
>> > > > > > > > > > > >>> > > > from
>> > > > > > > > > > > >>> > > > >> the
>> > > > > > > > > > > >>> > > > >> > > > fetch
>> > > > > > > > > > > >>> > > > >> > > > > > > > > request. Also, if we are
>> > > > removing
>> > > > > it
>> > > > > > > from
>> > > > > > > > > > the
>> > > > > > > > > > > >>> Fetch
>> > > > > > > > > > > >>> > > > >> request,
>> > > > > > > > > > > >>> > > > >> > > does
>> > > > > > > > > > > >>> > > > >> > > > > it
>> > > > > > > > > > > >>> > > > >> > > > > > > make
>> > > > > > > > > > > >>> > > > >> > > > > > > > > sense not to remove it
>> from
>> > > > > similar
>> > > > > > > > > requests
>> > > > > > > > > > > >>> like
>> > > > > > > > > > > >>> > > > >> > > > > ListOffsetRequest?
>> > > > > > > > > > > >>> > > > >> > > > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > > Ismael
>> > > > > > > > > > > >>> > > > >> > > > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > > On Thu, Sep 24, 2020 at
>> 2:46
>> > > AM
>> > > > > > David
>> > > > > > > > > Jacot
>> > > > > > > > > > <
>> > > > > > > > > > > >>> > > > >> > > djacot@confluent.io
>> > > > > > > > > > > >>> > > > >> > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > wrote:
>> > > > > > > > > > > >>> > > > >> > > > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> Hi Justine,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >>
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> Thanks for the KIP. I
>> > finally
>> > > > had
>> > > > > > > time to
>> > > > > > > > > > > read
>> > > > > > > > > > > >>> it
>> > > > > > > > > > > >>> > :).
>> > > > > > > > > > > >>> > > > It
>> > > > > > > > > > > >>> > > > >> is
>> > > > > > > > > > > >>> > > > >> > a
>> > > > > > > > > > > >>> > > > >> > > > > great
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> improvement.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >>
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> I have a few
>> > > > comments/questions:
>> > > > > > > > > > > >>> > > > >> > > > > > > > >>
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> 1. It seems that the
>> schema
>> > > of
>> > > > > the
>> > > > > > > > > > > >>> > StopReplicaRequest
>> > > > > > > > > > > >>> > > > is
>> > > > > > > > > > > >>> > > > >> > > > slightly
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> outdated.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> We
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> did some changes as
>> part of
>> > > > > > KIP-570.
>> > > > > > > V3
>> > > > > > > > > is
>> > > > > > > > > > > >>> already
>> > > > > > > > > > > >>> > > > >> organized
>> > > > > > > > > > > >>> > > > >> > > by
>> > > > > > > > > > > >>> > > > >> > > > > > > topics.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >>
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> 2. I just want to make
>> sure
>> > > > that
>> > > > > I
>> > > > > > > > > > understand
>> > > > > > > > > > > >>> the
>> > > > > > > > > > > >>> > > > >> > > reconciliation
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> logic correctly. When
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> an "INCREMENTAL"
>> > LeaderAndIsr
>> > > > > > > Request is
>> > > > > > > > > > > >>> received,
>> > > > > > > > > > > >>> > > the
>> > > > > > > > > > > >>> > > > >> > broker
>> > > > > > > > > > > >>> > > > >> > > > will
>> > > > > > > > > > > >>> > > > >> > > > > > > also
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> reconcile
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> when the local uuid does
>> > not
>> > > > > match
>> > > > > > > the
>> > > > > > > > > uuid
>> > > > > > > > > > > in
>> > > > > > > > > > > >>> the
>> > > > > > > > > > > >>> > > > >> request,
>> > > > > > > > > > > >>> > > > >> > > > right?
>> > > > > > > > > > > >>> > > > >> > > > > > In
>> > > > > > > > > > > >>> > > > >> > > > > > > > this
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> case, the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> local log is staged for
>> > > > deletion.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >>
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> 3. In the documentation
>> of
>> > > the
>> > > > `
>> > > > > > > > > > > >>> > > > >> delete.stale.topic.delay.ms
>> > > > > > > > > > > >>> > > > >> > `
>> > > > > > > > > > > >>> > > > >> > > > > > config,
>> > > > > > > > > > > >>> > > > >> > > > > > > it
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> says "When a
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> FULL
>> LeaderAndIsrRequest is
>> > > > > > > received..."
>> > > > > > > > > > but
>> > > > > > > > > > > I
>> > > > > > > > > > > >>> > > suppose
>> > > > > > > > > > > >>> > > > it
>> > > > > > > > > > > >>> > > > >> > > > applies
>> > > > > > > > > > > >>> > > > >> > > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > both
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> types.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >>
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> Best,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> David
>> > > > > > > > > > > >>> > > > >> > > > > > > > >>
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> On Thu, Sep 24, 2020 at
>> > 1:40
>> > > AM
>> > > > > > > Justine
>> > > > > > > > > > > Olshan
>> > > > > > > > > > > >>> <
>> > > > > > > > > > > >>> > > > >> > > > > > jolshan@confluent.io>
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> wrote:
>> > > > > > > > > > > >>> > > > >> > > > > > > > >>
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > Hi Jun,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > Thanks for the
>> comments.
>> > I
>> > > > > > > apologize
>> > > > > > > > > for
>> > > > > > > > > > > >>> some of
>> > > > > > > > > > > >>> > > the
>> > > > > > > > > > > >>> > > > >> typos
>> > > > > > > > > > > >>> > > > >> > > and
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> confusion.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > I’ve updated the KIP
>> to
>> > fix
>> > > > > some
>> > > > > > > of the
>> > > > > > > > > > > >>> issues
>> > > > > > > > > > > >>> > you
>> > > > > > > > > > > >>> > > > >> > > mentioned.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > 20.2 I’ve changed the
>> > type
>> > > to
>> > > > > > > String.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > 20.1/3 I’ve updated
>> the
>> > > > > > TopicZNode
>> > > > > > > to
>> > > > > > > > > fix
>> > > > > > > > > > > >>> > > formatting
>> > > > > > > > > > > >>> > > > >> and
>> > > > > > > > > > > >>> > > > >> > > > reflect
>> > > > > > > > > > > >>> > > > >> > > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> latest
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > version before this
>> > change.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > 21. You are correct
>> and
>> > > I’ve
>> > > > > > > removed
>> > > > > > > > > this
>> > > > > > > > > > > >>> line.
>> > > > > > > > > > > >>> > > I’ve
>> > > > > > > > > > > >>> > > > >> also
>> > > > > > > > > > > >>> > > > >> > > > added
>> > > > > > > > > > > >>> > > > >> > > > > a
>> > > > > > > > > > > >>> > > > >> > > > > > > line
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > mentioning an IBP
>> bump is
>> > > > > > > necessary for
>> > > > > > > > > > > >>> migration
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > 22. I think the
>> wording
>> > was
>> > > > > > > unclear but
>> > > > > > > > > > > your
>> > > > > > > > > > > >>> > > summary
>> > > > > > > > > > > >>> > > > is
>> > > > > > > > > > > >>> > > > >> > what
>> > > > > > > > > > > >>> > > > >> > > > was
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> intended
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > by this line. I’ve
>> > updated
>> > > to
>> > > > > > make
>> > > > > > > this
>> > > > > > > > > > > point
>> > > > > > > > > > > >>> > more
>> > > > > > > > > > > >>> > > > >> clear.
>> > > > > > > > > > > >>> > > > >> > > > > “Remove
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> deleted
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > topics from replicas
>> by
>> > > > sending
>> > > > > > > > > > > >>> > StopReplicaRequest
>> > > > > > > > > > > >>> > > V3
>> > > > > > > > > > > >>> > > > >> > before
>> > > > > > > > > > > >>> > > > >> > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > IBP
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> bump
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > using the old logic,
>> and
>> > > > using
>> > > > > V4
>> > > > > > > and
>> > > > > > > > > the
>> > > > > > > > > > > new
>> > > > > > > > > > > >>> > logic
>> > > > > > > > > > > >>> > > > >> with
>> > > > > > > > > > > >>> > > > >> > > topic
>> > > > > > > > > > > >>> > > > >> > > > > IDs
>> > > > > > > > > > > >>> > > > >> > > > > > > > after
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > the IBP bump.”
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > 23. I’ve removed the
>> > > > > unspecified
>> > > > > > > type
>> > > > > > > > > > from
>> > > > > > > > > > > >>> the
>> > > > > > > > > > > >>> > KIP
>> > > > > > > > > > > >>> > > > and
>> > > > > > > > > > > >>> > > > >> > > mention
>> > > > > > > > > > > >>> > > > >> > > > > > that
>> > > > > > > > > > > >>> > > > >> > > > > > > > IBP
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > will be used to handle
>> > this
>> > > > > > > request.
>> > > > > > > > > “IBP
>> > > > > > > > > > > >>> will be
>> > > > > > > > > > > >>> > > > used
>> > > > > > > > > > > >>> > > > >> to
>> > > > > > > > > > > >>> > > > >> > > > > > determine
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> whether
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > this new form of the
>> > > request
>> > > > > will
>> > > > > > > be
>> > > > > > > > > > used.
>> > > > > > > > > > > >>> For
>> > > > > > > > > > > >>> > > older
>> > > > > > > > > > > >>> > > > >> > > requests,
>> > > > > > > > > > > >>> > > > >> > > > > we
>> > > > > > > > > > > >>> > > > >> > > > > > > will
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > ignore this field and
>> > > default
>> > > > > to
>> > > > > > > > > previous
>> > > > > > > > > > > >>> > > behavior.”
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > 24. I’ve fixed this
>> typo.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > 25. I've added a
>> topics
>> > at
>> > > a
>> > > > > > higher
>> > > > > > > > > level
>> > > > > > > > > > > for
>> > > > > > > > > > > >>> > > > >> > > > > LeaderAndIsrResponse
>> > > > > > > > > > > >>> > > > >> > > > > > > v5,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > StopReplicaResponse
>> v4.
>> > > I've
>> > > > > also
>> > > > > > > > > changed
>> > > > > > > > > > > >>> > > > >> > StopReplicaRequest
>> > > > > > > > > > > >>> > > > >> > > > v4
>> > > > > > > > > > > >>> > > > >> > > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > have
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > topics at a higher
>> level.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > 26. I’ve updated
>> > > > > > > > > > > >>> forgotten_topics_data--added the
>> > > > > > > > > > > >>> > > > >> topic ID
>> > > > > > > > > > > >>> > > > >> > > and
>> > > > > > > > > > > >>> > > > >> > > > > > > removed
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > topic name
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > 27. I’ve decided on
>> plain
>> > > > text,
>> > > > > > and
>> > > > > > > > > I’ve
>> > > > > > > > > > > >>> added an
>> > > > > > > > > > > >>> > > > >> example.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > 28. I’ve added this
>> idea
>> > to
>> > > > > > future
>> > > > > > > > > work.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > Thanks again for
>> taking a
>> > > > look,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > Justine
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > On Wed, Sep 23, 2020
>> at
>> > > 10:28
>> > > > > AM
>> > > > > > > Jun
>> > > > > > > > > Rao
>> > > > > > > > > > <
>> > > > > > > > > > > >>> > > > >> > jun@confluent.io>
>> > > > > > > > > > > >>> > > > >> > > > > > wrote:
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > Hi, Justine,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > Thanks for the
>> > response.
>> > > > Made
>> > > > > > > another
>> > > > > > > > > > > >>> pass. A
>> > > > > > > > > > > >>> > few
>> > > > > > > > > > > >>> > > > >> more
>> > > > > > > > > > > >>> > > > >> > > > > comments
>> > > > > > > > > > > >>> > > > >> > > > > > > > below.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 20. znode schema:
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 20.1 It seems that
>> > > {"name":
>> > > > > > > > > "version",
>> > > > > > > > > > > >>> "type":
>> > > > > > > > > > > >>> > > > "int",
>> > > > > > > > > > > >>> > > > >> > > "id":
>> > > > > > > > > > > >>> > > > >> > > > > > > "UUID",
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > "doc":
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > "version id"}
>> should be
>> > > > > > {"name":
>> > > > > > > > > > > "version",
>> > > > > > > > > > > >>> > > "type":
>> > > > > > > > > > > >>> > > > >> > > "int"},
>> > > > > > > > > > > >>> > > > >> > > > > > > {"name":
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > "id",
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > "type": "UUID",
>> "doc":
>> > > > > "version
>> > > > > > > id"}.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 20.2 The znode
>> format
>> > is
>> > > > JSON
>> > > > > > > which
>> > > > > > > > > > > doesn't
>> > > > > > > > > > > >>> > have
>> > > > > > > > > > > >>> > > > UUID
>> > > > > > > > > > > >>> > > > >> > > type.
>> > > > > > > > > > > >>> > > > >> > > > So
>> > > > > > > > > > > >>> > > > >> > > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> type
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > probably should be
>> > > string?
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 20.3 Also, the
>> existing
>> > > > > format
>> > > > > > > used
>> > > > > > > > > > seems
>> > > > > > > > > > > >>> > > outdated.
>> > > > > > > > > > > >>> > > > >> It
>> > > > > > > > > > > >>> > > > >> > > > should
>> > > > > > > > > > > >>> > > > >> > > > > > have
>> > > > > > > > > > > >>> > > > >> > > > > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > following format.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>> > >  Json.encodeAsBytes(Map(
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >       "version" ->
>> 2,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >       "partitions"
>> ->
>> > > > > > > > > > > >>> > > replicaAssignmentJson.asJava,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>>  "adding_replicas"
>> > > ->
>> > > > > > > > > > > >>> > > > >> > > > >
>> addingReplicasAssignmentJson.asJava,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>> >  "removing_replicas"
>> > > > ->
>> > > > > > > > > > > >>> > > > >> > > > > >
>> > > removingReplicasAssignmentJson.asJava
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >     ).asJava)
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >   }
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 21. Migration: The
>> KIP
>> > > says
>> > > > > > "The
>> > > > > > > > > > > migration
>> > > > > > > > > > > >>> > > process
>> > > > > > > > > > > >>> > > > >> can
>> > > > > > > > > > > >>> > > > >> > > take
>> > > > > > > > > > > >>> > > > >> > > > > > place
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> without
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > an inter-broker
>> > protocol
>> > > > > bump,
>> > > > > > > as the
>> > > > > > > > > > > >>> format
>> > > > > > > > > > > >>> > > stored
>> > > > > > > > > > > >>> > > > >> in
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>> /brokers/topics/[topic]
>> > > > will
>> > > > > be
>> > > > > > > > > > > compatible
>> > > > > > > > > > > >>> with
>> > > > > > > > > > > >>> > > > older
>> > > > > > > > > > > >>> > > > >> > > broker
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> versions."
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > However, since we
>> are
>> > > > bumping
>> > > > > > up
>> > > > > > > the
>> > > > > > > > > > > >>> version of
>> > > > > > > > > > > >>> > > > >> > > inter-broker
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> requests, it
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > seems that we need
>> to
>> > use
>> > > > IBP
>> > > > > > for
>> > > > > > > > > > > >>> migration.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 22. The KIP says
>> > "Remove
>> > > > > > deleted
>> > > > > > > > > topics
>> > > > > > > > > > > >>> from
>> > > > > > > > > > > >>> > > > >> replicas by
>> > > > > > > > > > > >>> > > > >> > > > > sending
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > StopReplicaRequest
>> V3
>> > for
>> > > > any
>> > > > > > > topics
>> > > > > > > > > > > which
>> > > > > > > > > > > >>> do
>> > > > > > > > > > > >>> > not
>> > > > > > > > > > > >>> > > > >> > contain
>> > > > > > > > > > > >>> > > > >> > > a
>> > > > > > > > > > > >>> > > > >> > > > > > topic
>> > > > > > > > > > > >>> > > > >> > > > > > > > ID,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> and
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > V4 for any topics
>> which
>> > > do
>> > > > > > > contain a
>> > > > > > > > > > > topic
>> > > > > > > > > > > >>> > ID.".
>> > > > > > > > > > > >>> > > > >> > However,
>> > > > > > > > > > > >>> > > > >> > > if
>> > > > > > > > > > > >>> > > > >> > > > > we
>> > > > > > > > > > > >>> > > > >> > > > > > > use
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> IBP,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > it
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > seems that the
>> > controller
>> > > > > will
>> > > > > > > either
>> > > > > > > > > > > send
>> > > > > > > > > > > >>> > > > >> > > > StopReplicaRequest
>> > > > > > > > > > > >>> > > > >> > > > > V3
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > or
>> StopReplicaRequest
>> > V4,
>> > > > but
>> > > > > > > never
>> > > > > > > > > > mixed
>> > > > > > > > > > > >>> V3
>> > > > > > > > > > > >>> > and
>> > > > > > > > > > > >>> > > V4
>> > > > > > > > > > > >>> > > > >> for
>> > > > > > > > > > > >>> > > > >> > > > > > different
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> topics.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > Basically, before
>> the
>> > IBP
>> > > > > bump,
>> > > > > > > V3
>> > > > > > > > > will
>> > > > > > > > > > > be
>> > > > > > > > > > > >>> > used.
>> > > > > > > > > > > >>> > > > >> After
>> > > > > > > > > > > >>> > > > >> > the
>> > > > > > > > > > > >>> > > > >> > > > IBP
>> > > > > > > > > > > >>> > > > >> > > > > > > bump,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > topicId will be
>> created
>> > > and
>> > > > > V4
>> > > > > > > will
>> > > > > > > > > be
>> > > > > > > > > > > >>> used.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 23. Given that we
>> > depend
>> > > on
>> > > > > > IBP,
>> > > > > > > do
>> > > > > > > > > we
>> > > > > > > > > > > >>> still
>> > > > > > > > > > > >>> > need
>> > > > > > > > > > > >>> > > > "0
>> > > > > > > > > > > >>> > > > >> > > > > > UNSPECIFIED"
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > in LeaderAndIsr?
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 24.
>> > LeaderAndIsrResponse
>> > > > v5 :
>> > > > > > It
>> > > > > > > > > still
>> > > > > > > > > > > has
>> > > > > > > > > > > >>> the
>> > > > > > > > > > > >>> > > > topic
>> > > > > > > > > > > >>> > > > >> > > field.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 25.
>> > LeaderAndIsrResponse
>> > > > v5,
>> > > > > > > > > > > >>> > StopReplicaResponse
>> > > > > > > > > > > >>> > > > v4:
>> > > > > > > > > > > >>> > > > >> > Could
>> > > > > > > > > > > >>> > > > >> > > > we
>> > > > > > > > > > > >>> > > > >> > > > > > use
>> > > > > > > > > > > >>> > > > >> > > > > > > > this
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > opportunity to
>> organize
>> > > the
>> > > > > > > response
>> > > > > > > > > > in 2
>> > > > > > > > > > > >>> > levels,
>> > > > > > > > > > > >>> > > > >> first
>> > > > > > > > > > > >>> > > > >> > by
>> > > > > > > > > > > >>> > > > >> > > > > > topic,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> then by
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > partition, as most
>> > other
>> > > > > > > > > > > >>> requests/responses?
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 26. FetchRequest
>> v13 :
>> > > > Should
>> > > > > > > > > > > >>> > > forgotten_topics_data
>> > > > > > > > > > > >>> > > > >> use
>> > > > > > > > > > > >>> > > > >> > > > > topicId
>> > > > > > > > > > > >>> > > > >> > > > > > > too?
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 27. "This file can
>> > either
>> > > > be
>> > > > > > > plain
>> > > > > > > > > text
>> > > > > > > > > > > >>> > > (key/value
>> > > > > > > > > > > >>> > > > >> > pairs)
>> > > > > > > > > > > >>> > > > >> > > or
>> > > > > > > > > > > >>> > > > >> > > > > > > JSON."
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> Have
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > we
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > decided which one to
>> > use?
>> > > > > Also,
>> > > > > > > it
>> > > > > > > > > > would
>> > > > > > > > > > > be
>> > > > > > > > > > > >>> > > helpful
>> > > > > > > > > > > >>> > > > >> to
>> > > > > > > > > > > >>> > > > >> > > > provide
>> > > > > > > > > > > >>> > > > >> > > > > > an
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > example.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 28. Future
>> improvement:
>> > > > > Another
>> > > > > > > > > future
>> > > > > > > > > > > >>> > > improvement
>> > > > > > > > > > > >>> > > > >> > > > opportunity
>> > > > > > > > > > > >>> > > > >> > > > > > is
>> > > > > > > > > > > >>> > > > >> > > > > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> use
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > topicId in
>> > > > > > > > > > > >>> GroupMetadataManager.offsetCommitKey
>> > > > > > > > > > > >>> > > in
>> > > > > > > > > > > >>> > > > >> the
>> > > > > > > > > > > >>> > > > >> > > > > > > offset_commit
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > topic.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > This may save some
>> > space.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > Thanks,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > Jun
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > On Wed, Sep 23,
>> 2020 at
>> > > > 8:50
>> > > > > AM
>> > > > > > > > > Justine
>> > > > > > > > > > > >>> Olshan
>> > > > > > > > > > > >>> > <
>> > > > > > > > > > > >>> > > > >> > > > > > > > jolshan@confluent.io>
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > wrote:
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > Hi Tom,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > Thanks for the
>> > > comment. I
>> > > > > > think
>> > > > > > > > > this
>> > > > > > > > > > > is a
>> > > > > > > > > > > >>> > > really
>> > > > > > > > > > > >>> > > > >> good
>> > > > > > > > > > > >>> > > > >> > > idea
>> > > > > > > > > > > >>> > > > >> > > > > and
>> > > > > > > > > > > >>> > > > >> > > > > > > it
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> has
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > been
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > added to the KIP
>> > under
>> > > > the
>> > > > > > > newly
>> > > > > > > > > > added
>> > > > > > > > > > > >>> > tooling
>> > > > > > > > > > > >>> > > > >> > section.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > Thanks again,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > Justine
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > On Wed, Sep 23,
>> 2020
>> > at
>> > > > > 3:17
>> > > > > > > AM Tom
>> > > > > > > > > > > >>> Bentley <
>> > > > > > > > > > > >>> > > > >> > > > > > > tbentley@redhat.com>
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > wrote:
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > Hi Justine,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > I know you
>> started
>> > > the
>> > > > > vote
>> > > > > > > > > thread,
>> > > > > > > > > > > >>> but on
>> > > > > > > > > > > >>> > > > >> > re-reading
>> > > > > > > > > > > >>> > > > >> > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > KIP
>> > > > > > > > > > > >>> > > > >> > > > > > > I
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > noticed
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that although
>> the
>> > > topic
>> > > > > id
>> > > > > > is
>> > > > > > > > > > > included
>> > > > > > > > > > > >>> in
>> > > > > > > > > > > >>> > the
>> > > > > > > > > > > >>> > > > >> > > > > > MetadataResponse
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> it's
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > not
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > surfaced in the
>> > > output
>> > > > > from
>> > > > > > > > > > > >>> > `kafka-topics.sh
>> > > > > > > > > > > >>> > > > >> > > > --describe`.
>> > > > > > > > > > > >>> > > > >> > > > > > > Maybe
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> that
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > was
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > intentional
>> because
>> > > ids
>> > > > > are
>> > > > > > > > > > > >>> intentionally
>> > > > > > > > > > > >>> > not
>> > > > > > > > > > > >>> > > > >> really
>> > > > > > > > > > > >>> > > > >> > > > > > something
>> > > > > > > > > > > >>> > > > >> > > > > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > user
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > should care
>> deeply
>> > > > about,
>> > > > > > > but it
>> > > > > > > > > > > would
>> > > > > > > > > > > >>> also
>> > > > > > > > > > > >>> > > > make
>> > > > > > > > > > > >>> > > > >> > life
>> > > > > > > > > > > >>> > > > >> > > > > harder
>> > > > > > > > > > > >>> > > > >> > > > > > > for
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > anyone
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > debugging Kafka
>> and
>> > > > this
>> > > > > > > would
>> > > > > > > > > > likely
>> > > > > > > > > > > >>> get
>> > > > > > > > > > > >>> > > worse
>> > > > > > > > > > > >>> > > > >> the
>> > > > > > > > > > > >>> > > > >> > > more
>> > > > > > > > > > > >>> > > > >> > > > > > topic
>> > > > > > > > > > > >>> > > > >> > > > > > > > ids
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > got
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > rolled out
>> across
>> > the
>> > > > > > > protocols,
>> > > > > > > > > > > >>> clients
>> > > > > > > > > > > >>> > etc.
>> > > > > > > > > > > >>> > > > It
>> > > > > > > > > > > >>> > > > >> > seems
>> > > > > > > > > > > >>> > > > >> > > > > > likely
>> > > > > > > > > > > >>> > > > >> > > > > > > > that
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
>> `kafka-topics.sh`
>> > > will
>> > > > > > > eventually
>> > > > > > > > > > > need
>> > > > > > > > > > > >>> the
>> > > > > > > > > > > >>> > > > >> ability
>> > > > > > > > > > > >>> > > > >> > to
>> > > > > > > > > > > >>> > > > >> > > > show
>> > > > > > > > > > > >>> > > > >> > > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > id
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> of
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > a
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > topic and
>> perhaps
>> > > find
>> > > > a
>> > > > > > > topic
>> > > > > > > > > name
>> > > > > > > > > > > >>> given
>> > > > > > > > > > > >>> > an
>> > > > > > > > > > > >>> > > > id.
>> > > > > > > > > > > >>> > > > >> Is
>> > > > > > > > > > > >>> > > > >> > > > there
>> > > > > > > > > > > >>> > > > >> > > > > > any
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> reason
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > not
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > implement that
>> in
>> > > this
>> > > > > KIP?
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > Many thanks,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > Tom
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > On Mon, Sep 21,
>> > 2020
>> > > at
>> > > > > > 9:54
>> > > > > > > PM
>> > > > > > > > > > > Justine
>> > > > > > > > > > > >>> > > Olshan
>> > > > > > > > > > > >>> > > > <
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> jolshan@confluent.io
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > wrote:
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Hi all,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > After thinking
>> > > about
>> > > > > it,
>> > > > > > > I've
>> > > > > > > > > > > >>> decided to
>> > > > > > > > > > > >>> > > > remove
>> > > > > > > > > > > >>> > > > >> > the
>> > > > > > > > > > > >>> > > > >> > > > > topic
>> > > > > > > > > > > >>> > > > >> > > > > > > name
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> from
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Fetch Request
>> and
>> > > > > > Response
>> > > > > > > > > after
>> > > > > > > > > > > all.
>> > > > > > > > > > > >>> > Since
>> > > > > > > > > > > >>> > > > >> there
>> > > > > > > > > > > >>> > > > >> > > are
>> > > > > > > > > > > >>> > > > >> > > > so
>> > > > > > > > > > > >>> > > > >> > > > > > > many
>> > > > > > > > > > > >>> > > > >> > > > > > > > of
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > these
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > requests per
>> > > second,
>> > > > it
>> > > > > > is
>> > > > > > > > > worth
>> > > > > > > > > > > >>> removing
>> > > > > > > > > > > >>> > > the
>> > > > > > > > > > > >>> > > > >> > extra
>> > > > > > > > > > > >>> > > > >> > > > > > > > information.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > I've
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > updated the
>> KIP
>> > to
>> > > > > > reflect
>> > > > > > > this
>> > > > > > > > > > > >>> change.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Please let me
>> > know
>> > > if
>> > > > > > > there is
>> > > > > > > > > > > >>> anything
>> > > > > > > > > > > >>> > > else
>> > > > > > > > > > > >>> > > > we
>> > > > > > > > > > > >>> > > > >> > > should
>> > > > > > > > > > > >>> > > > >> > > > > > > discuss
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > before
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > voting.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Thank you,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Justine
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > On Fri, Sep
>> 18,
>> > > 2020
>> > > > at
>> > > > > > > 9:46 AM
>> > > > > > > > > > > >>> Justine
>> > > > > > > > > > > >>> > > > Olshan
>> > > > > > > > > > > >>> > > > >> <
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > jolshan@confluent.io
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > wrote:
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > Hi Jun,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > I see what
>> you
>> > > are
>> > > > > > > saying.
>> > > > > > > > > For
>> > > > > > > > > > > now
>> > > > > > > > > > > >>> we
>> > > > > > > > > > > >>> > can
>> > > > > > > > > > > >>> > > > >> remove
>> > > > > > > > > > > >>> > > > >> > > the
>> > > > > > > > > > > >>> > > > >> > > > > > extra
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > information.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > I'll leave
>> the
>> > > > option
>> > > > > > to
>> > > > > > > add
>> > > > > > > > > > more
>> > > > > > > > > > > >>> > fields
>> > > > > > > > > > > >>> > > to
>> > > > > > > > > > > >>> > > > >> the
>> > > > > > > > > > > >>> > > > >> > > file
>> > > > > > > > > > > >>> > > > >> > > > > in
>> > > > > > > > > > > >>> > > > >> > > > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > future.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > The
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > KIP
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > has been
>> > updated
>> > > to
>> > > > > > > reflect
>> > > > > > > > > > this
>> > > > > > > > > > > >>> > change.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > Thanks,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > Justine
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > On Fri, Sep
>> 18,
>> > > > 2020
>> > > > > at
>> > > > > > > 8:46
>> > > > > > > > > AM
>> > > > > > > > > > > Jun
>> > > > > > > > > > > >>> > Rao <
>> > > > > > > > > > > >>> > > > >> > > > > > jun@confluent.io
>> > > > > > > > > > > >>> > > > >> > > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > wrote:
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Hi,
>> Justine,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Thanks for
>> the
>> > > > > reply.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> 13. If the
>> log
>> > > > > > > directory is
>> > > > > > > > > > the
>> > > > > > > > > > > >>> source
>> > > > > > > > > > > >>> > > of
>> > > > > > > > > > > >>> > > > >> > truth,
>> > > > > > > > > > > >>> > > > >> > > it
>> > > > > > > > > > > >>> > > > >> > > > > > means
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> that
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> redundant
>> info
>> > > in
>> > > > > the
>> > > > > > > > > metadata
>> > > > > > > > > > > >>> file
>> > > > > > > > > > > >>> > will
>> > > > > > > > > > > >>> > > > be
>> > > > > > > > > > > >>> > > > >> > > > ignored.
>> > > > > > > > > > > >>> > > > >> > > > > > Then
>> > > > > > > > > > > >>> > > > >> > > > > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > question
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > is
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> why do we
>> need
>> > > to
>> > > > > put
>> > > > > > > the
>> > > > > > > > > > > >>> redundant
>> > > > > > > > > > > >>> > info
>> > > > > > > > > > > >>> > > > in
>> > > > > > > > > > > >>> > > > >> the
>> > > > > > > > > > > >>> > > > >> > > > > > metadata
>> > > > > > > > > > > >>> > > > >> > > > > > > > file
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > now?
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Thanks,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Jun
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> On Thu, Sep
>> > 17,
>> > > > 2020
>> > > > > > at
>> > > > > > > 5:07
>> > > > > > > > > > PM
>> > > > > > > > > > > >>> > Justine
>> > > > > > > > > > > >>> > > > >> Olshan
>> > > > > > > > > > > >>> > > > >> > <
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
>> jolshan@confluent.io
>> > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> wrote:
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Hi Jun,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Thanks
>> for
>> > the
>> > > > > quick
>> > > > > > > > > > response!
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > 12. I've
>> > > decided
>> > > > > to
>> > > > > > > bump
>> > > > > > > > > up
>> > > > > > > > > > > the
>> > > > > > > > > > > >>> > > versions
>> > > > > > > > > > > >>> > > > >> on
>> > > > > > > > > > > >>> > > > >> > the
>> > > > > > > > > > > >>> > > > >> > > > > > > requests
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> and
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > updated
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > KIP. I
>> think
>> > > > it's
>> > > > > > > good we
>> > > > > > > > > > > >>> thoroughly
>> > > > > > > > > > > >>> > > > >> > discussed
>> > > > > > > > > > > >>> > > > >> > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > options
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > here,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > so
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > we
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> know
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > we made a
>> > good
>> > > > > > > choice. :)
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > 13. This
>> is
>> > an
>> > > > > > > interesting
>> > > > > > > > > > > >>> > situation.
>> > > > > > > > > > > >>> > > I
>> > > > > > > > > > > >>> > > > >> think
>> > > > > > > > > > > >>> > > > >> > > if
>> > > > > > > > > > > >>> > > > >> > > > > this
>> > > > > > > > > > > >>> > > > >> > > > > > > > does
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > occur
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > we
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> should
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > give a
>> > > warning.
>> > > > I
>> > > > > > > agree
>> > > > > > > > > that
>> > > > > > > > > > > >>> it's
>> > > > > > > > > > > >>> > hard
>> > > > > > > > > > > >>> > > > to
>> > > > > > > > > > > >>> > > > >> > know
>> > > > > > > > > > > >>> > > > >> > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > source
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> of
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > truth
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > for
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> sure
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > since the
>> > > > > directory
>> > > > > > > or the
>> > > > > > > > > > > file
>> > > > > > > > > > > >>> > could
>> > > > > > > > > > > >>> > > be
>> > > > > > > > > > > >>> > > > >> > > manually
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> modified. I
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > guess
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > directory
>> > > could
>> > > > be
>> > > > > > > used as
>> > > > > > > > > > the
>> > > > > > > > > > > >>> > source
>> > > > > > > > > > > >>> > > of
>> > > > > > > > > > > >>> > > > >> > truth.
>> > > > > > > > > > > >>> > > > >> > > > To
>> > > > > > > > > > > >>> > > > >> > > > > be
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> honest,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > I'm
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > not
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > really
>> sure
>> > > what
>> > > > > > > happens
>> > > > > > > > > in
>> > > > > > > > > > > >>> kafka
>> > > > > > > > > > > >>> > when
>> > > > > > > > > > > >>> > > > the
>> > > > > > > > > > > >>> > > > >> > log
>> > > > > > > > > > > >>> > > > >> > > > > > > directory
>> > > > > > > > > > > >>> > > > >> > > > > > > > is
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > renamed
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > manually
>> in
>> > > > such a
>> > > > > > > way.
>> > > > > > > > > I'm
>> > > > > > > > > > > also
>> > > > > > > > > > > >>> > > > >> wondering if
>> > > > > > > > > > > >>> > > > >> > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> situation is
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> recoverable
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > in this
>> > > > scenario.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Thanks,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Justine
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > On Thu,
>> Sep
>> > > 17,
>> > > > > 2020
>> > > > > > > at
>> > > > > > > > > 4:28
>> > > > > > > > > > > PM
>> > > > > > > > > > > >>> Jun
>> > > > > > > > > > > >>> > > Rao
>> > > > > > > > > > > >>> > > > <
>> > > > > > > > > > > >>> > > > >> > > > > > > > jun@confluent.io>
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > wrote:
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Hi,
>> > Justine,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Thanks
>> for
>> > > the
>> > > > > > > reply.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > 12. I
>> > don't
>> > > > > have a
>> > > > > > > > > strong
>> > > > > > > > > > > >>> > preference
>> > > > > > > > > > > >>> > > > >> > either.
>> > > > > > > > > > > >>> > > > >> > > > > > However,
>> > > > > > > > > > > >>> > > > >> > > > > > > > if
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> we
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > need
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > IBP
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > anyway,
>> > > maybe
>> > > > > it's
>> > > > > > > > > easier
>> > > > > > > > > > to
>> > > > > > > > > > > >>> just
>> > > > > > > > > > > >>> > > bump
>> > > > > > > > > > > >>> > > > >> up
>> > > > > > > > > > > >>> > > > >> > the
>> > > > > > > > > > > >>> > > > >> > > > > > version
>> > > > > > > > > > > >>> > > > >> > > > > > > > for
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > all
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > inter
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > broker
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>> requests
>> > and
>> > > > add
>> > > > > > the
>> > > > > > > > > topic
>> > > > > > > > > > > id
>> > > > > > > > > > > >>> > field
>> > > > > > > > > > > >>> > > > as a
>> > > > > > > > > > > >>> > > > >> > > > regular
>> > > > > > > > > > > >>> > > > >> > > > > > > > field. A
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > regular
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> field
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > is
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > a bit
>> more
>> > > > > concise
>> > > > > > > in
>> > > > > > > > > wire
>> > > > > > > > > > > >>> > transfer
>> > > > > > > > > > > >>> > > > >> than a
>> > > > > > > > > > > >>> > > > >> > > > > flexible
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> field.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > 13. The
>> > > > > confusion
>> > > > > > > that I
>> > > > > > > > > > was
>> > > > > > > > > > > >>> > > referring
>> > > > > > > > > > > >>> > > > >> to
>> > > > > > > > > > > >>> > > > >> > is
>> > > > > > > > > > > >>> > > > >> > > > > > between
>> > > > > > > > > > > >>> > > > >> > > > > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > topic
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > name
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>> partition
>> > > > number
>> > > > > > > between
>> > > > > > > > > > the
>> > > > > > > > > > > >>> log
>> > > > > > > > > > > >>> > dir
>> > > > > > > > > > > >>> > > > and
>> > > > > > > > > > > >>> > > > >> > the
>> > > > > > > > > > > >>> > > > >> > > > > > metadata
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> file.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > For
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> example,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > if
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > the log
>> > dir
>> > > is
>> > > > > > > topicA-1
>> > > > > > > > > > and
>> > > > > > > > > > > >>> the
>> > > > > > > > > > > >>> > > > metadata
>> > > > > > > > > > > >>> > > > >> > file
>> > > > > > > > > > > >>> > > > >> > > > in
>> > > > > > > > > > > >>> > > > >> > > > > it
>> > > > > > > > > > > >>> > > > >> > > > > > > has
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > topicB
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > and
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>> partition
>> > 0
>> > > > (say
>> > > > > > > due to
>> > > > > > > > > a
>> > > > > > > > > > > bug
>> > > > > > > > > > > >>> or
>> > > > > > > > > > > >>> > > > manual
>> > > > > > > > > > > >>> > > > >> > > > > > > modification),
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> which
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > one
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > do
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > we
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > use
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > as the
>> > > source
>> > > > of
>> > > > > > > truth?
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Jun
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > On Thu,
>> > Sep
>> > > > 17,
>> > > > > > > 2020 at
>> > > > > > > > > > 3:43
>> > > > > > > > > > > >>> PM
>> > > > > > > > > > > >>> > > > Justine
>> > > > > > > > > > > >>> > > > >> > > Olshan
>> > > > > > > > > > > >>> > > > >> > > > <
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
>> > > jolshan@confluent.io
>> > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > wrote:
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > Hi
>> Jun,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>> Thanks
>> > for
>> > > > the
>> > > > > > > > > comments.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > 12. I
>> > > bumped
>> > > > > the
>> > > > > > > > > > > >>> > > LeaderAndIsrRequest
>> > > > > > > > > > > >>> > > > >> > > because
>> > > > > > > > > > > >>> > > > >> > > > I
>> > > > > > > > > > > >>> > > > >> > > > > > > > removed
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > topic
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> name
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > field
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > in
>> the
>> > > > > response.
>> > > > > > > It
>> > > > > > > > > may
>> > > > > > > > > > be
>> > > > > > > > > > > >>> > > possible
>> > > > > > > > > > > >>> > > > to
>> > > > > > > > > > > >>> > > > >> > > avoid
>> > > > > > > > > > > >>> > > > >> > > > > > > bumping
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > version
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > without
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > that
>> > > change,
>> > > > > but
>> > > > > > > I may
>> > > > > > > > > > be
>> > > > > > > > > > > >>> > missing
>> > > > > > > > > > > >>> > > > >> > > something.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > I
>> > believe
>> > > > > > > StopReplica
>> > > > > > > > > is
>> > > > > > > > > > > >>> > actually
>> > > > > > > > > > > >>> > > on
>> > > > > > > > > > > >>> > > > >> > > version
>> > > > > > > > > > > >>> > > > >> > > > 3
>> > > > > > > > > > > >>> > > > >> > > > > > now,
>> > > > > > > > > > > >>> > > > >> > > > > > > > but
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > because
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > version 2
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > is
>> > > > flexible, I
>> > > > > > > kept
>> > > > > > > > > that
>> > > > > > > > > > > >>> listed
>> > > > > > > > > > > >>> > as
>> > > > > > > > > > > >>> > > > >> > version
>> > > > > > > > > > > >>> > > > >> > > 2
>> > > > > > > > > > > >>> > > > >> > > > on
>> > > > > > > > > > > >>> > > > >> > > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > KIP
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > page.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> However,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > you
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > may
>> be
>> > > right
>> > > > > in
>> > > > > > > that
>> > > > > > > > > we
>> > > > > > > > > > > may
>> > > > > > > > > > > >>> need
>> > > > > > > > > > > >>> > > to
>> > > > > > > > > > > >>> > > > >> bump
>> > > > > > > > > > > >>> > > > >> > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > version
>> > > > > > > > > > > >>> > > > >> > > > > > > > on
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > StopReplica
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > deal
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > with
>> > > > deletion
>> > > > > > > > > > differently
>> > > > > > > > > > > as
>> > > > > > > > > > > >>> > > > mentioned
>> > > > > > > > > > > >>> > > > >> > > > above. I
>> > > > > > > > > > > >>> > > > >> > > > > > > don't
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> know
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > if
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > I
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> have a
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > big
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>> > preference
>> > > > > over
>> > > > > > > used
>> > > > > > > > > > > tagged
>> > > > > > > > > > > >>> > fields
>> > > > > > > > > > > >>> > > > or
>> > > > > > > > > > > >>> > > > >> > not.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > 13. I
>> > was
>> > > > > > thinking
>> > > > > > > > > that
>> > > > > > > > > > in
>> > > > > > > > > > > >>> the
>> > > > > > > > > > > >>> > > case
>> > > > > > > > > > > >>> > > > >> where
>> > > > > > > > > > > >>> > > > >> > > the
>> > > > > > > > > > > >>> > > > >> > > > > > file
>> > > > > > > > > > > >>> > > > >> > > > > > > > and
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > request
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > topic
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > ids
>> > don't
>> > > > > match,
>> > > > > > > it
>> > > > > > > > > > means
>> > > > > > > > > > > >>> that
>> > > > > > > > > > > >>> > the
>> > > > > > > > > > > >>> > > > >> > broker's
>> > > > > > > > > > > >>> > > > >> > > > > > > topic/the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> one
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > in
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> file
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > has
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > been
>> > > > deleted.
>> > > > > In
>> > > > > > > that
>> > > > > > > > > > > case,
>> > > > > > > > > > > >>> we
>> > > > > > > > > > > >>> > > would
>> > > > > > > > > > > >>> > > > >> need
>> > > > > > > > > > > >>> > > > >> > > to
>> > > > > > > > > > > >>> > > > >> > > > > > delete
>> > > > > > > > > > > >>> > > > >> > > > > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > old
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > topic
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > start
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>> > receiving
>> > > > the
>> > > > > > new
>> > > > > > > > > > version.
>> > > > > > > > > > > >>> If
>> > > > > > > > > > > >>> > the
>> > > > > > > > > > > >>> > > > >> topic
>> > > > > > > > > > > >>> > > > >> > > name
>> > > > > > > > > > > >>> > > > >> > > > > were
>> > > > > > > > > > > >>> > > > >> > > > > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > change,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > but
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > ids
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > still
>> > > > matched,
>> > > > > > the
>> > > > > > > > > file
>> > > > > > > > > > > >>> would
>> > > > > > > > > > > >>> > also
>> > > > > > > > > > > >>> > > > >> need
>> > > > > > > > > > > >>> > > > >> > to
>> > > > > > > > > > > >>> > > > >> > > > > > update.
>> > > > > > > > > > > >>> > > > >> > > > > > > > Am I
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > missing
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > a
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> case
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > where
>> > the
>> > > > file
>> > > > > > > would
>> > > > > > > > > be
>> > > > > > > > > > > >>> correct
>> > > > > > > > > > > >>> > > and
>> > > > > > > > > > > >>> > > > >> not
>> > > > > > > > > > > >>> > > > >> > the
>> > > > > > > > > > > >>> > > > >> > > > > > > request?
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>> Thanks,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>> Justine
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > On
>> Thu,
>> > > Sep
>> > > > > 17,
>> > > > > > > 2020
>> > > > > > > > > at
>> > > > > > > > > > > >>> 3:18 PM
>> > > > > > > > > > > >>> > > Jun
>> > > > > > > > > > > >>> > > > >> Rao <
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> jun@confluent.io
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > wrote:
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Hi,
>> > > > Justine,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> Thanks
>> > > for
>> > > > > the
>> > > > > > > > > reply.
>> > > > > > > > > > A
>> > > > > > > > > > > >>> couple
>> > > > > > > > > > > >>> > > of
>> > > > > > > > > > > >>> > > > >> more
>> > > > > > > > > > > >>> > > > >> > > > > comments
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> below.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > 12.
>> > > > > ListOffset
>> > > > > > > and
>> > > > > > > > > > > >>> > > OffsetForLeader
>> > > > > > > > > > > >>> > > > >> > > > currently
>> > > > > > > > > > > >>> > > > >> > > > > > > don't
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > support
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> flexible
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>> fields.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> So, we
>> > > > have
>> > > > > to
>> > > > > > > bump
>> > > > > > > > > up
>> > > > > > > > > > > the
>> > > > > > > > > > > >>> > > version
>> > > > > > > > > > > >>> > > > >> > number
>> > > > > > > > > > > >>> > > > >> > > > and
>> > > > > > > > > > > >>> > > > >> > > > > > use
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> IBP at
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > least
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > for
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > these
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > two
>> > > > > requests.
>> > > > > > > Note
>> > > > > > > > > > that
>> > > > > > > > > > > it
>> > > > > > > > > > > >>> > seems
>> > > > > > > > > > > >>> > > > >> 2.7.0
>> > > > > > > > > > > >>> > > > >> > > will
>> > > > > > > > > > > >>> > > > >> > > > > > > require
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> IBP
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > anyway
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > because
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > of
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> > changes
>> > > in
>> > > > > > > > > > KAFKA-10435.
>> > > > > > > > > > > >>> Also,
>> > > > > > > > > > > >>> > it
>> > > > > > > > > > > >>> > > > >> seems
>> > > > > > > > > > > >>> > > > >> > > that
>> > > > > > > > > > > >>> > > > >> > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> version
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > for
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> > > > > > > LeaderAndIsrRequest
>> > > > > > > > > > and
>> > > > > > > > > > > >>> > > > StopReplica
>> > > > > > > > > > > >>> > > > >> are
>> > > > > > > > > > > >>> > > > >> > > > > bumped
>> > > > > > > > > > > >>> > > > >> > > > > > > even
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > though
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > we
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > only
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > added
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > a
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> tagged
>> > > > > field.
>> > > > > > > But
>> > > > > > > > > > since
>> > > > > > > > > > > >>> IBP is
>> > > > > > > > > > > >>> > > > >> needed
>> > > > > > > > > > > >>> > > > >> > > > anyway,
>> > > > > > > > > > > >>> > > > >> > > > > > we
>> > > > > > > > > > > >>> > > > >> > > > > > > > may
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > want
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> revisit
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> > overall
>> > > > > tagged
>> > > > > > > field
>> > > > > > > > > > > >>> choice.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > 13.
>> > The
>> > > > only
>> > > > > > > > > downside
>> > > > > > > > > > is
>> > > > > > > > > > > >>> the
>> > > > > > > > > > > >>> > > > >> potential
>> > > > > > > > > > > >>> > > > >> > > > > > confusion
>> > > > > > > > > > > >>> > > > >> > > > > > > on
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > which
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > one
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > is
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>> source
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > of
>> > truth
>> > > > if
>> > > > > > they
>> > > > > > > > > don't
>> > > > > > > > > > > >>> match.
>> > > > > > > > > > > >>> > > > >> Another
>> > > > > > > > > > > >>> > > > >> > > > option
>> > > > > > > > > > > >>> > > > >> > > > > is
>> > > > > > > > > > > >>> > > > >> > > > > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > include
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > those
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > fields
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > in
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
>> > > > metadata
>> > > > > > > file
>> > > > > > > > > when
>> > > > > > > > > > > we
>> > > > > > > > > > > >>> > > actually
>> > > > > > > > > > > >>> > > > >> > change
>> > > > > > > > > > > >>> > > > >> > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> directory
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > structure.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> > Thanks,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Jun
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > On
>> > Thu,
>> > > > Sep
>> > > > > > 17,
>> > > > > > > 2020
>> > > > > > > > > > at
>> > > > > > > > > > > >>> 2:01
>> > > > > > > > > > > >>> > PM
>> > > > > > > > > > > >>> > > > >> Justine
>> > > > > > > > > > > >>> > > > >> > > > > Olshan
>> > > > > > > > > > > >>> > > > >> > > > > > <
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>> > > > > jolshan@confluent.io
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> wrote:
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > Hello
>> > > > all,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> I've
>> > > > > thought
>> > > > > > > some
>> > > > > > > > > > more
>> > > > > > > > > > > >>> about
>> > > > > > > > > > > >>> > > > >> removing
>> > > > > > > > > > > >>> > > > >> > > the
>> > > > > > > > > > > >>> > > > >> > > > > > topic
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> name
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > field
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > from
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > some
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > of
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > > > requests.
>> > > > > On
>> > > > > > > > > closer
>> > > > > > > > > > > >>> > inspection
>> > > > > > > > > > > >>> > > > of
>> > > > > > > > > > > >>> > > > >> the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > requests/responses,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > it
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> seems
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > > internal
>> > > > > > > changes
>> > > > > > > > > > would
>> > > > > > > > > > > >>> be
>> > > > > > > > > > > >>> > much
>> > > > > > > > > > > >>> > > > >> larger
>> > > > > > > > > > > >>> > > > >> > > > than
>> > > > > > > > > > > >>> > > > >> > > > > I
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> expected.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > Some
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > protocols
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > > involve
>> > > > > > > clients,
>> > > > > > > > > so
>> > > > > > > > > > > they
>> > > > > > > > > > > >>> > would
>> > > > > > > > > > > >>> > > > >> > require
>> > > > > > > > > > > >>> > > > >> > > > > > changes
>> > > > > > > > > > > >>> > > > >> > > > > > > > too.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > I'm
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > thinking
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > that
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > for
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> now,
>> > > > > > removing
>> > > > > > > the
>> > > > > > > > > > > topic
>> > > > > > > > > > > >>> name
>> > > > > > > > > > > >>> > > > from
>> > > > > > > > > > > >>> > > > >> > these
>> > > > > > > > > > > >>> > > > >> > > > > > > requests
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> and
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > responses
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> are
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > out
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > scope.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I
>> > have
>> > > > > > > decided to
>> > > > > > > > > > just
>> > > > > > > > > > > >>> keep
>> > > > > > > > > > > >>> > > the
>> > > > > > > > > > > >>> > > > >> > change
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
>> LeaderAndIsrResponse
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > remove
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > topic
>> > > > > name,
>> > > > > > > and
>> > > > > > > > > have
>> > > > > > > > > > > >>> updated
>> > > > > > > > > > > >>> > > the
>> > > > > > > > > > > >>> > > > >> KIP
>> > > > > > > > > > > >>> > > > >> > to
>> > > > > > > > > > > >>> > > > >> > > > > > reflect
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> this
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > change. I
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> have
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > also
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > > > mentioned
>> > > > > > the
>> > > > > > > > > other
>> > > > > > > > > > > >>> requests
>> > > > > > > > > > > >>> > > and
>> > > > > > > > > > > >>> > > > >> > > > responses
>> > > > > > > > > > > >>> > > > >> > > > > in
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> future
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > work.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> I'm
>> > > > hoping
>> > > > > > to
>> > > > > > > > > start
>> > > > > > > > > > > the
>> > > > > > > > > > > >>> > voting
>> > > > > > > > > > > >>> > > > >> > process
>> > > > > > > > > > > >>> > > > >> > > > > soon,
>> > > > > > > > > > > >>> > > > >> > > > > > so
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> let me
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > know
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > if
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > there
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > is
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > > anything
>> > > > > > else
>> > > > > > > we
>> > > > > > > > > > > should
>> > > > > > > > > > > >>> > > discuss.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > Thank
>> > > > you,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > > Justine
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> On
>> > > Tue,
>> > > > > Sep
>> > > > > > > 15,
>> > > > > > > > > 2020
>> > > > > > > > > > > at
>> > > > > > > > > > > >>> 3:57
>> > > > > > > > > > > >>> > > PM
>> > > > > > > > > > > >>> > > > >> > Justine
>> > > > > > > > > > > >>> > > > >> > > > > > Olshan
>> > > > > > > > > > > >>> > > > >> > > > > > > <
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>> > > > > > jolshan@confluent.io
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > wrote:
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > > Hello
>> > > > > > again,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > To
>> > > > > follow
>> > > > > > > up on
>> > > > > > > > > > some
>> > > > > > > > > > > >>> of
>> > > > > > > > > > > >>> > the
>> > > > > > > > > > > >>> > > > >> other
>> > > > > > > > > > > >>> > > > >> > > > > comments:
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > > 10/11)
>> > > > > We
>> > > > > > > can
>> > > > > > > > > > remove
>> > > > > > > > > > > >>> the
>> > > > > > > > > > > >>> > > topic
>> > > > > > > > > > > >>> > > > >> name
>> > > > > > > > > > > >>> > > > >> > > > from
>> > > > > > > > > > > >>> > > > >> > > > > > > these
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>> > > > > requests/responses,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > and
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > that
>> > > > > means
>> > > > > > > we
>> > > > > > > > > will
>> > > > > > > > > > > >>> just
>> > > > > > > > > > > >>> > have
>> > > > > > > > > > > >>> > > > to
>> > > > > > > > > > > >>> > > > >> > make
>> > > > > > > > > > > >>> > > > >> > > a
>> > > > > > > > > > > >>> > > > >> > > > > few
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> internal
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > changes
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > make
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > > > > partitions
>> > > > > > > > > > > accessible
>> > > > > > > > > > > >>> by
>> > > > > > > > > > > >>> > > topic
>> > > > > > > > > > > >>> > > > >> id
>> > > > > > > > > > > >>> > > > >> > and
>> > > > > > > > > > > >>> > > > >> > > > > > > > partition.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> I
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > can
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > update
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > KIP
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > > remove
>> > > > > > them
>> > > > > > > > > unless
>> > > > > > > > > > > >>> anyone
>> > > > > > > > > > > >>> > > > thinks
>> > > > > > > > > > > >>> > > > >> > they
>> > > > > > > > > > > >>> > > > >> > > > > > should
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> stay.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > 12)
>> > > > > > > Addressed in
>> > > > > > > > > > the
>> > > > > > > > > > > >>> > > previous
>> > > > > > > > > > > >>> > > > >> > email.
>> > > > > > > > > > > >>> > > > >> > > > I've
>> > > > > > > > > > > >>> > > > >> > > > > > > > updated
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > KIP
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > include
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > > tagged
>> > > > > > > fields
>> > > > > > > > > for
>> > > > > > > > > > > the
>> > > > > > > > > > > >>> > > requests
>> > > > > > > > > > > >>> > > > >> and
>> > > > > > > > > > > >>> > > > >> > > > > > responses.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> (More
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > on
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > below)
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > 13)
>> > > I
>> > > > > > think
>> > > > > > > part
>> > > > > > > > > > of
>> > > > > > > > > > > >>> the
>> > > > > > > > > > > >>> > idea
>> > > > > > > > > > > >>> > > > for
>> > > > > > > > > > > >>> > > > >> > > > > including
>> > > > > > > > > > > >>> > > > >> > > > > > > this
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > information
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> is to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> > prepare
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > for
>> > > > > future
>> > > > > > > > > > changes.
>> > > > > > > > > > > >>> > Perhaps
>> > > > > > > > > > > >>> > > > the
>> > > > > > > > > > > >>> > > > >> > > > directory
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> structure
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > might
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> change
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > from
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > > > > > > > > > > >>> topicName_partitionNumber
>> > > > > > > > > > > >>> > to
>> > > > > > > > > > > >>> > > > >> > > something
>> > > > > > > > > > > >>> > > > >> > > > > like
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>> > > > > > > topicID_partitionNumber.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> Then
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > it
>> > > > would
>> > > > > > be
>> > > > > > > > > useful
>> > > > > > > > > > > to
>> > > > > > > > > > > >>> have
>> > > > > > > > > > > >>> > > the
>> > > > > > > > > > > >>> > > > >> > topic
>> > > > > > > > > > > >>> > > > >> > > > name
>> > > > > > > > > > > >>> > > > >> > > > > > in
>> > > > > > > > > > > >>> > > > >> > > > > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > file
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > since
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > it
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > would
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > not
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> be
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > in
>> > > the
>> > > > > > > directory
>> > > > > > > > > > > >>> > structure.
>> > > > > > > > > > > >>> > > > >> > > Supporting
>> > > > > > > > > > > >>> > > > >> > > > > > topic
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> renames
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > might
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > be
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > easier
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > if
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > > other
>> > > > > > > fields are
>> > > > > > > > > > > >>> included.
>> > > > > > > > > > > >>> > > > Would
>> > > > > > > > > > > >>> > > > >> > > there
>> > > > > > > > > > > >>> > > > >> > > > be
>> > > > > > > > > > > >>> > > > >> > > > > > any
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > downsides
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>> including
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> this
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > > > > > information?
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > 14)
>> > > > > Yes,
>> > > > > > we
>> > > > > > > > > would
>> > > > > > > > > > > >>> need to
>> > > > > > > > > > > >>> > > > copy
>> > > > > > > > > > > >>> > > > >> the
>> > > > > > > > > > > >>> > > > >> > > > > > partition
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > metadata
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > file
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > in
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > this
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > > > process.
>> > > > > > > I've
>> > > > > > > > > > > updated
>> > > > > > > > > > > >>> the
>> > > > > > > > > > > >>> > > KIP
>> > > > > > > > > > > >>> > > > to
>> > > > > > > > > > > >>> > > > >> > > > include
>> > > > > > > > > > > >>> > > > >> > > > > > > this.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > 15)
>> > > I
>> > > > > > > believe
>> > > > > > > > > > Lucas
>> > > > > > > > > > > >>> meant
>> > > > > > > > > > > >>> > v1
>> > > > > > > > > > > >>> > > > >> and v2
>> > > > > > > > > > > >>> > > > >> > > > here.
>> > > > > > > > > > > >>> > > > >> > > > > > He
>> > > > > > > > > > > >>> > > > >> > > > > > > > was
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > referring
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> how
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > > > requests
>> > > > > > > would
>> > > > > > > > > > fall
>> > > > > > > > > > > >>> under
>> > > > > > > > > > > >>> > > > >> different
>> > > > > > > > > > > >>> > > > >> > > IBP
>> > > > > > > > > > > >>> > > > >> > > > > and
>> > > > > > > > > > > >>> > > > >> > > > > > > > meant
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > that
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > older
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > brokers
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > would
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > have
>> > > > to
>> > > > > > use
>> > > > > > > the
>> > > > > > > > > > > older
>> > > > > > > > > > > >>> > > version
>> > > > > > > > > > > >>> > > > of
>> > > > > > > > > > > >>> > > > >> > the
>> > > > > > > > > > > >>> > > > >> > > > > > request
>> > > > > > > > > > > >>> > > > >> > > > > > > > and
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > existing
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > topic
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > > > deletion
>> > > > > > > > > process.
>> > > > > > > > > > At
>> > > > > > > > > > > >>> > first,
>> > > > > > > > > > > >>> > > it
>> > > > > > > > > > > >>> > > > >> > seemed
>> > > > > > > > > > > >>> > > > >> > > > > like
>> > > > > > > > > > > >>> > > > >> > > > > > > > tagged
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > fields
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > would
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>> resolve
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > the
>> > > > IBP
>> > > > > > > issue.
>> > > > > > > > > > > >>> However, we
>> > > > > > > > > > > >>> > > may
>> > > > > > > > > > > >>> > > > >> need
>> > > > > > > > > > > >>> > > > >> > > IBP
>> > > > > > > > > > > >>> > > > >> > > > > for
>> > > > > > > > > > > >>> > > > >> > > > > > > > this
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > request
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > after
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > all
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> since
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > the
>> > > > > > > controller
>> > > > > > > > > > > >>> handles the
>> > > > > > > > > > > >>> > > > topic
>> > > > > > > > > > > >>> > > > >> > > > deletion
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > differently
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> depending
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > on
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> IBP
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > > > version.
>> > > > > > In
>> > > > > > > an
>> > > > > > > > > > older
>> > > > > > > > > > > >>> > > version,
>> > > > > > > > > > > >>> > > > we
>> > > > > > > > > > > >>> > > > >> > > can't
>> > > > > > > > > > > >>> > > > >> > > > > just
>> > > > > > > > > > > >>> > > > >> > > > > > > > send
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> a
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > StopReplica
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > delete
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > > topic
>> > > > > > > > > immediately
>> > > > > > > > > > > like
>> > > > > > > > > > > >>> > we'd
>> > > > > > > > > > > >>> > > > >> want to
>> > > > > > > > > > > >>> > > > >> > > for
>> > > > > > > > > > > >>> > > > >> > > > > > this
>> > > > > > > > > > > >>> > > > >> > > > > > > > KIP.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > This
>> > > > > makes
>> > > > > > > me
>> > > > > > > > > > wonder
>> > > > > > > > > > > >>> if we
>> > > > > > > > > > > >>> > > > want
>> > > > > > > > > > > >>> > > > >> > > tagged
>> > > > > > > > > > > >>> > > > >> > > > > > fields
>> > > > > > > > > > > >>> > > > >> > > > > > > > on
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> all
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> requests
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > after
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > all.
>> > > > Let
>> > > > > > me
>> > > > > > > know
>> > > > > > > > > > > your
>> > > > > > > > > > > >>> > > > thoughts!
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > > > Justine
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > On
>> > > > Tue,
>> > > > > > Sep
>> > > > > > > 15,
>> > > > > > > > > > 2020
>> > > > > > > > > > > >>> at
>> > > > > > > > > > > >>> > 1:03
>> > > > > > > > > > > >>> > > > PM
>> > > > > > > > > > > >>> > > > >> > > Justine
>> > > > > > > > > > > >>> > > > >> > > > > > > Olshan
>> > > > > > > > > > > >>> > > > >> > > > > > > > <
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>> > > > > > > jolshan@confluent.io
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > > wrote:
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>
>> > Hi
>> > > > all,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>
>> > Jun
>> > > > > > > brought up
>> > > > > > > > > a
>> > > > > > > > > > > good
>> > > > > > > > > > > >>> > point
>> > > > > > > > > > > >>> > > > in
>> > > > > > > > > > > >>> > > > >> his
>> > > > > > > > > > > >>> > > > >> > > > last
>> > > > > > > > > > > >>> > > > >> > > > > > > email
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> about
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > tagged
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > fields,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > and
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>
>> > > I've
>> > > > > > > updated
>> > > > > > > > > the
>> > > > > > > > > > > KIP
>> > > > > > > > > > > >>> to
>> > > > > > > > > > > >>> > > > reflect
>> > > > > > > > > > > >>> > > > >> > that
>> > > > > > > > > > > >>> > > > >> > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> changes to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > requests
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > > > responses
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>
>> > > will
>> > > > be
>> > > > > > in
>> > > > > > > the
>> > > > > > > > > > form
>> > > > > > > > > > > >>> of
>> > > > > > > > > > > >>> > > tagged
>> > > > > > > > > > > >>> > > > >> > fields
>> > > > > > > > > > > >>> > > > >> > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > avoid
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > changing
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > IBP.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>
>> > > Jun:
>> > > > I
>> > > > > > > plan on
>> > > > > > > > > > > >>> sending a
>> > > > > > > > > > > >>> > > > >> followup
>> > > > > > > > > > > >>> > > > >> > > > email
>> > > > > > > > > > > >>> > > > >> > > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> address
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > some
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > of
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > other
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>
>> > > > points.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>
>> > > > Thanks,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>
>> > > > Justine
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>
>> > On
>> > > > Mon,
>> > > > > > > Sep 14,
>> > > > > > > > > > > 2020
>> > > > > > > > > > > >>> at
>> > > > > > > > > > > >>> > > 4:25
>> > > > > > > > > > > >>> > > > PM
>> > > > > > > > > > > >>> > > > >> > Jun
>> > > > > > > > > > > >>> > > > >> > > > Rao
>> > > > > > > > > > > >>> > > > >> > > > > <
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
>> jun@confluent.io>
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > wrote:
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > Hi,
>> > > > > > > Justine,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > Thanks
>> > > > > > > for the
>> > > > > > > > > > > >>> updated
>> > > > > > > > > > > >>> > > KIP.
>> > > > > > > > > > > >>> > > > A
>> > > > > > > > > > > >>> > > > >> few
>> > > > > > > > > > > >>> > > > >> > > > > > comments
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> below.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > 10.
>> > > > > > > > > LeaderAndIsr
>> > > > > > > > > > > >>> > Response:
>> > > > > > > > > > > >>> > > > Do
>> > > > > > > > > > > >>> > > > >> we
>> > > > > > > > > > > >>> > > > >> > > need
>> > > > > > > > > > > >>> > > > >> > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > topic
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > name?
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > 11.
>> > > > > For
>> > > > > > > the
>> > > > > > > > > > > changed
>> > > > > > > > > > > >>> > > > >> > > request/response,
>> > > > > > > > > > > >>> > > > >> > > > > > other
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> than
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>> LeaderAndIsr,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > > > > > UpdateMetadata,
>> > > > > > > > > > > >>> > Metadata,
>> > > > > > > > > > > >>> > > do
>> > > > > > > > > > > >>> > > > >> we
>> > > > > > > > > > > >>> > > > >> > > need
>> > > > > > > > > > > >>> > > > >> > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > include
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > topic
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> name?
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > 12.
>> > > > It
>> > > > > > > seems
>> > > > > > > > > > that
>> > > > > > > > > > > >>> > upgrades
>> > > > > > > > > > > >>> > > > >> don't
>> > > > > > > > > > > >>> > > > >> > > > > require
>> > > > > > > > > > > >>> > > > >> > > > > > > IBP.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> Does
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > that
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > mean
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > new
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > fields
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > in
>> > > > all
>> > > > > > the
>> > > > > > > > > > > >>> > > request/response
>> > > > > > > > > > > >>> > > > >> are
>> > > > > > > > > > > >>> > > > >> > > added
>> > > > > > > > > > > >>> > > > >> > > > > as
>> > > > > > > > > > > >>> > > > >> > > > > > > > tagged
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > fields
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> without
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> > bumping
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> up
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > the
>> > > > > > > request
>> > > > > > > > > > > >>> version? It
>> > > > > > > > > > > >>> > > > would
>> > > > > > > > > > > >>> > > > >> be
>> > > > > > > > > > > >>> > > > >> > > > useful
>> > > > > > > > > > > >>> > > > >> > > > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > make
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > that
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > clear.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > 13.
>> > > > > > > Partition
>> > > > > > > > > > > >>> Metadata
>> > > > > > > > > > > >>> > > file:
>> > > > > > > > > > > >>> > > > >> Do
>> > > > > > > > > > > >>> > > > >> > we
>> > > > > > > > > > > >>> > > > >> > > > need
>> > > > > > > > > > > >>> > > > >> > > > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> include
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > topic
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > name
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > and
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > > partition
>> > > > > > > id
>> > > > > > > > > > since
>> > > > > > > > > > > >>> they
>> > > > > > > > > > > >>> > > are
>> > > > > > > > > > > >>> > > > >> > implied
>> > > > > > > > > > > >>> > > > >> > > > in
>> > > > > > > > > > > >>> > > > >> > > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > directory
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > name?
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > 14.
>> > > > In
>> > > > > > the
>> > > > > > > > > JBOD
>> > > > > > > > > > > >>> mode, we
>> > > > > > > > > > > >>> > > > >> support
>> > > > > > > > > > > >>> > > > >> > > > > moving a
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > partition's
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > data
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> from
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > one
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> disk
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > another.
>> > > > > > > Will
>> > > > > > > > > > the
>> > > > > > > > > > > >>> new
>> > > > > > > > > > > >>> > > > >> partition
>> > > > > > > > > > > >>> > > > >> > > > > metadata
>> > > > > > > > > > > >>> > > > >> > > > > > > file
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> be
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > copied
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> during
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > process?
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > 15.
>> > > > > The
>> > > > > > > KIP
>> > > > > > > > > says
>> > > > > > > > > > > >>> "Remove
>> > > > > > > > > > > >>> > > > >> deleted
>> > > > > > > > > > > >>> > > > >> > > > topics
>> > > > > > > > > > > >>> > > > >> > > > > > > from
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > replicas
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > by
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > sending
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > > > > > > StopReplicaRequest
>> > > > > > > > > > > >>> V2
>> > > > > > > > > > > >>> > for
>> > > > > > > > > > > >>> > > > any
>> > > > > > > > > > > >>> > > > >> > > topics
>> > > > > > > > > > > >>> > > > >> > > > > > which
>> > > > > > > > > > > >>> > > > >> > > > > > > do
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> not
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > contain
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > a
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > topic
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > ID,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> and
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > V3
>> > > > for
>> > > > > > any
>> > > > > > > > > > topics
>> > > > > > > > > > > >>> which
>> > > > > > > > > > > >>> > do
>> > > > > > > > > > > >>> > > > >> > contain
>> > > > > > > > > > > >>> > > > >> > > a
>> > > > > > > > > > > >>> > > > >> > > > > > topic
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> ID.".
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > However,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > it
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > seems
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > updated
>> > > > > > > > > > controller
>> > > > > > > > > > > >>> will
>> > > > > > > > > > > >>> > > > create
>> > > > > > > > > > > >>> > > > >> > all
>> > > > > > > > > > > >>> > > > >> > > > > > missing
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> topic
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > IDs
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > first
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > before
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> doing
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > other
>> > > > > > > actions.
>> > > > > > > > > > So,
>> > > > > > > > > > > >>> is
>> > > > > > > > > > > >>> > > > >> > > > > StopReplicaRequest
>> > > > > > > > > > > >>> > > > >> > > > > > V2
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > needed?
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > Jun
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > On
>> > > > > Fri,
>> > > > > > > Sep
>> > > > > > > > > 11,
>> > > > > > > > > > > >>> 2020 at
>> > > > > > > > > > > >>> > > > 10:31
>> > > > > > > > > > > >>> > > > >> AM
>> > > > > > > > > > > >>> > > > >> > > John
>> > > > > > > > > > > >>> > > > >> > > > > > > > Roesler <
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>> > > > > > vvcephei@apache.org
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > wrote:
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > > > Thanks,
>> > > > > > > > > > Justine!
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > Your
>> > > > > > > > > response
>> > > > > > > > > > > >>> seems
>> > > > > > > > > > > >>> > > > >> compelling
>> > > > > > > > > > > >>> > > > >> > to
>> > > > > > > > > > > >>> > > > >> > > > me.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > > -John
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > On
>> > > > > > Fri,
>> > > > > > > > > > > >>> 2020-09-11 at
>> > > > > > > > > > > >>> > > > 10:17
>> > > > > > > > > > > >>> > > > >> > > -0700,
>> > > > > > > > > > > >>> > > > >> > > > > > > Justine
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > Olshan
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > wrote:
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > > > Hello
>> > > > > > > all,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > > > > Thanks for
>> > > > > > > > > > > >>> > continuing
>> > > > > > > > > > > >>> > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > discussion! I
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> have a
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > few
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > responses
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> your
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > > > points.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > > Tom:
>> > > > > > > You
>> > > > > > > > > are
>> > > > > > > > > > > >>> correct
>> > > > > > > > > > > >>> > > in
>> > > > > > > > > > > >>> > > > >> that
>> > > > > > > > > > > >>> > > > >> > > this
>> > > > > > > > > > > >>> > > > >> > > > > KIP
>> > > > > > > > > > > >>> > > > >> > > > > > > has
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> not
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > mentioned
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > > > > > > > > >>> > DeleteTopicsRequest. I
>> > > > > > > > > > > >>> > > > >> think
>> > > > > > > > > > > >>> > > > >> > > that
>> > > > > > > > > > > >>> > > > >> > > > > > this
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> would
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > be
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > out
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > of
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > scope
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > for
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > now,
>> > > > > but
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > > may
>> > > > > > be
>> > > > > > > > > > > something
>> > > > > > > > > > > >>> > worth
>> > > > > > > > > > > >>> > > > >> adding
>> > > > > > > > > > > >>> > > > >> > > in
>> > > > > > > > > > > >>> > > > >> > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> future.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > > > John:
>> > > > > > > We
>> > > > > > > > > did
>> > > > > > > > > > > >>> > consider
>> > > > > > > > > > > >>> > > > >> > sequence
>> > > > > > > > > > > >>> > > > >> > > > ids,
>> > > > > > > > > > > >>> > > > >> > > > > > but
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> there
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > are
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > a
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > few
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > reasons
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > favor
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > > > UUIDs.
>> > > > > > > > > There
>> > > > > > > > > > > are
>> > > > > > > > > > > >>> > > several
>> > > > > > > > > > > >>> > > > >> > cases
>> > > > > > > > > > > >>> > > > >> > > > > where
>> > > > > > > > > > > >>> > > > >> > > > > > > > topics
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > from
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> different
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> > clusters
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > may
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > > > > interact
>> > > > > > > > > now
>> > > > > > > > > > > >>> and in
>> > > > > > > > > > > >>> > > the
>> > > > > > > > > > > >>> > > > >> > future.
>> > > > > > > > > > > >>> > > > >> > > > For
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> example,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > Mirror
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Maker 2
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > may
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > benefit
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > > from
>> > > > > > > being
>> > > > > > > > > > > able
>> > > > > > > > > > > >>> to
>> > > > > > > > > > > >>> > > > detect
>> > > > > > > > > > > >>> > > > >> > when
>> > > > > > > > > > > >>> > > > >> > > a
>> > > > > > > > > > > >>> > > > >> > > > > > > cluster
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> being
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > mirrored
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> is
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> > deleted
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > and
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > > > > recreated
>> > > > > > > > > > and
>> > > > > > > > > > > >>> > globally
>> > > > > > > > > > > >>> > > > >> unique
>> > > > > > > > > > > >>> > > > >> > > > > > > identifiers
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > would
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > make
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>> resolving
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > issues
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > > > easier
>> > > > > > > > > than
>> > > > > > > > > > > >>> sequence
>> > > > > > > > > > > >>> > > IDs
>> > > > > > > > > > > >>> > > > >> > which
>> > > > > > > > > > > >>> > > > >> > > > may
>> > > > > > > > > > > >>> > > > >> > > > > > > > collide
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > between
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > clusters.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > > KIP-405
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > > > > (tiered
>> > > > > > > > > > > storage)
>> > > > > > > > > > > >>> > will
>> > > > > > > > > > > >>> > > > also
>> > > > > > > > > > > >>> > > > >> > > > benefit
>> > > > > > > > > > > >>> > > > >> > > > > > from
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > globally
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > unique
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> IDs
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > as
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > shared
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > > > > buckets
>> > > > > > > > > may
>> > > > > > > > > > be
>> > > > > > > > > > > >>> used
>> > > > > > > > > > > >>> > > > >> between
>> > > > > > > > > > > >>> > > > >> > > > > clusters.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > > > > Globally
>> > > > > > > > > > > unique
>> > > > > > > > > > > >>> IDs
>> > > > > > > > > > > >>> > > > would
>> > > > > > > > > > > >>> > > > >> > also
>> > > > > > > > > > > >>> > > > >> > > > make
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > functionality
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > like
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > moving
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > topics
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > > > > between
>> > > > > > > > > > > >>> disparate
>> > > > > > > > > > > >>> > > > clusters
>> > > > > > > > > > > >>> > > > >> > > easier
>> > > > > > > > > > > >>> > > > >> > > > > in
>> > > > > > > > > > > >>> > > > >> > > > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > future,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> simplify
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > any
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > future
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > > > > > > > > implementations
>> > > > > > > > > > > >>> of
>> > > > > > > > > > > >>> > > > backups
>> > > > > > > > > > > >>> > > > >> > and
>> > > > > > > > > > > >>> > > > >> > > > > > > restores,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> and
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > more.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > In
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>> general,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > unique
>> > > > > > IDs
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > > > would
>> > > > > > > > > ensure
>> > > > > > > > > > > >>> that
>> > > > > > > > > > > >>> > the
>> > > > > > > > > > > >>> > > > >> source
>> > > > > > > > > > > >>> > > > >> > > > > cluster
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> topics do
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > not
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> conflict
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > with
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > > > > > > destination
>> > > > > > > > > > > >>> cluster
>> > > > > > > > > > > >>> > > > >> topics.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > If
>> > > > > > we
>> > > > > > > were
>> > > > > > > > > > to
>> > > > > > > > > > > >>> use
>> > > > > > > > > > > >>> > > > sequence
>> > > > > > > > > > > >>> > > > >> > ids,
>> > > > > > > > > > > >>> > > > >> > > > we
>> > > > > > > > > > > >>> > > > >> > > > > > > would
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> need
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>> sufficiently
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > large
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > cluster
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > > ids
>> > > > > > > to be
>> > > > > > > > > > > stored
>> > > > > > > > > > > >>> > with
>> > > > > > > > > > > >>> > > > the
>> > > > > > > > > > > >>> > > > >> > topic
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> identifiers or
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > we
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > run
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > risk
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > > > > > > collisions.
>> > > > > > > > > > > This
>> > > > > > > > > > > >>> > will
>> > > > > > > > > > > >>> > > > >> give up
>> > > > > > > > > > > >>> > > > >> > > any
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> advantage in
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> compactness
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > sequence
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > > > > numbers
>> > > > > > > > > may
>> > > > > > > > > > > >>> bring.
>> > > > > > > > > > > >>> > > Given
>> > > > > > > > > > > >>> > > > >> > these
>> > > > > > > > > > > >>> > > > >> > > > > > > > advantages I
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > think
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > it
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> makes
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > sense
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > use
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > > > UUIDs.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > > > Gokul:
>> > > > > > > > > This
>> > > > > > > > > > is
>> > > > > > > > > > > >>> an
>> > > > > > > > > > > >>> > > > >> interesting
>> > > > > > > > > > > >>> > > > >> > > > idea,
>> > > > > > > > > > > >>> > > > >> > > > > > but
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> this
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > is
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > a
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> breaking
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> > change.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > Out
>> > > > of
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > > > scope
>> > > > > > > for
>> > > > > > > > > > now,
>> > > > > > > > > > > >>> but
>> > > > > > > > > > > >>> > > maybe
>> > > > > > > > > > > >>> > > > >> > worth
>> > > > > > > > > > > >>> > > > >> > > > > > > discussing
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> in
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > future.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > > Hope
>> > > > > > > this
>> > > > > > > > > > > >>> explains
>> > > > > > > > > > > >>> > > some
>> > > > > > > > > > > >>> > > > of
>> > > > > > > > > > > >>> > > > >> > the
>> > > > > > > > > > > >>> > > > >> > > > > > > decisions,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > > > > Justine
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > On
>> > > > > > > Fri,
>> > > > > > > > > Sep
>> > > > > > > > > > > 11,
>> > > > > > > > > > > >>> 2020
>> > > > > > > > > > > >>> > > at
>> > > > > > > > > > > >>> > > > >> 8:27
>> > > > > > > > > > > >>> > > > >> > AM
>> > > > > > > > > > > >>> > > > >> > > > > Gokul
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> Ramanan
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>> Subramanian <
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > > > > > > > > >>> > gokul2411s@gmail.com>
>> > > > > > > > > > > >>> > > > >> wrote:
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > > Hi.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > > > Thanks
>> > > > > > > > > for
>> > > > > > > > > > > the
>> > > > > > > > > > > >>> > KIP.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > > > Have you
>> > > > > > > > > > > >>> thought
>> > > > > > > > > > > >>> > > about
>> > > > > > > > > > > >>> > > > >> > > whether
>> > > > > > > > > > > >>> > > > >> > > > it
>> > > > > > > > > > > >>> > > > >> > > > > > > makes
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > sense
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> support
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > > > authorizing a
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > > > > > principal
>> > > > > > > > > > > for
>> > > > > > > > > > > >>> a
>> > > > > > > > > > > >>> > > topic
>> > > > > > > > > > > >>> > > > ID
>> > > > > > > > > > > >>> > > > >> > > rather
>> > > > > > > > > > > >>> > > > >> > > > > > than
>> > > > > > > > > > > >>> > > > >> > > > > > > a
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> topic
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > name
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > achieve
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > tighter
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > > > > > security?
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > Or
>> > > > > > > is
>> > > > > > > > > the
>> > > > > > > > > > > >>> topic ID
>> > > > > > > > > > > >>> > > > >> > > > fundamentally
>> > > > > > > > > > > >>> > > > >> > > > > an
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> internal
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > detail
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > similar
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > epochs
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > used
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > > in a
>> > > > > > > > > bunch
>> > > > > > > > > > > of
>> > > > > > > > > > > >>> > other
>> > > > > > > > > > > >>> > > > >> places
>> > > > > > > > > > > >>> > > > >> > in
>> > > > > > > > > > > >>> > > > >> > > > > > Kafka?
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > > > Thanks.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > On
>> > > > > > > Fri,
>> > > > > > > > > > Sep
>> > > > > > > > > > > >>> 11,
>> > > > > > > > > > > >>> > 2020
>> > > > > > > > > > > >>> > > > at
>> > > > > > > > > > > >>> > > > >> > 4:06
>> > > > > > > > > > > >>> > > > >> > > PM
>> > > > > > > > > > > >>> > > > >> > > > > > John
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > Roesler <
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > > > > > > > > vvcephei@apache.org
>> > > > > > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > > wrote:
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > Hello
>> > > > > > > > > > > >>> Justine,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > Thanks
>> > > > > > > > > > for
>> > > > > > > > > > > >>> the
>> > > > > > > > > > > >>> > > KIP!
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > I
>> > > > > > > > > happen
>> > > > > > > > > > > to
>> > > > > > > > > > > >>> have
>> > > > > > > > > > > >>> > > > been
>> > > > > > > > > > > >>> > > > >> > > > > confronted
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> recently
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > with
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> need
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> keep
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > > track
>> > > > > > of
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > a
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > large
>> > > > > > > > > > > >>> number of
>> > > > > > > > > > > >>> > > > >> topics as
>> > > > > > > > > > > >>> > > > >> > > > > > compactly
>> > > > > > > > > > > >>> > > > >> > > > > > > > as
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > possible. I
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> was
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > going
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > come
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > up
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > with
>> > > > > > > > > > some
>> > > > > > > > > > > >>> way to
>> > > > > > > > > > > >>> > > > >> > dictionary
>> > > > > > > > > > > >>> > > > >> > > > > > encode
>> > > > > > > > > > > >>> > > > >> > > > > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > topic
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > names
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> as
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> > > integers,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > but
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > this
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > seems
>> > > > > > > > > > much
>> > > > > > > > > > > >>> > better!
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > > > > Apologies
>> > > > > > > > > > > if
>> > > > > > > > > > > >>> > this
>> > > > > > > > > > > >>> > > > has
>> > > > > > > > > > > >>> > > > >> > been
>> > > > > > > > > > > >>> > > > >> > > > > raised
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> before,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > but
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > I’m
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>> wondering
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > about
>> > > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > choice
>> > > > > > > > > > of
>> > > > > > > > > > > >>> UUID
>> > > > > > > > > > > >>> > vs
>> > > > > > > > > > > >>> > > > >> > sequence
>> > > > > > > > > > > >>> > > > >> > > > > number
>> > > > > > > > > > > >>> > > > >> > > > > > > for
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > ids.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>> Typically,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> I’ve
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > seen
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > > UUIDs
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > in two
>> > > > > > > > > > > >>> > situations:
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > 1.
>> > > > > > > > > When
>> > > > > > > > > > > >>> > processes
>> > > > > > > > > > > >>> > > > >> need to
>> > > > > > > > > > > >>> > > > >> > > > > > generate
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > non-colliding
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>> > > identifiers
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > without
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > > > > > >>> coordination.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > 2.
>> > > > > > > > > When
>> > > > > > > > > > > the
>> > > > > > > > > > > >>> > > > identifier
>> > > > > > > > > > > >>> > > > >> > > needs
>> > > > > > > > > > > >>> > > > >> > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > be
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > “universally
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > unique”;
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> I.e.,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > > > > identifier
>> > > > > > > > > > > >>> needs
>> > > > > > > > > > > >>> > > to
>> > > > > > > > > > > >>> > > > >> > > > distinguish
>> > > > > > > > > > > >>> > > > >> > > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> entity
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > from
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > all
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > other
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > entities
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > that
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > could
>> > > > > > > > > > ever
>> > > > > > > > > > > >>> > exist.
>> > > > > > > > > > > >>> > > > >> This is
>> > > > > > > > > > > >>> > > > >> > > > > useful
>> > > > > > > > > > > >>> > > > >> > > > > > in
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> cases
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > where
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > entities
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > from
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> all
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > > kinds
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > of
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > > > systems
>> > > > > > > > > > > get
>> > > > > > > > > > > >>> > mixed
>> > > > > > > > > > > >>> > > > >> > together,
>> > > > > > > > > > > >>> > > > >> > > > > such
>> > > > > > > > > > > >>> > > > >> > > > > > as
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> when
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > dumping
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> logs
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > from
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > all
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > > > > processes
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > in
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > a
>> > > > > > > > > > company
>> > > > > > > > > > > >>> into a
>> > > > > > > > > > > >>> > > > >> common
>> > > > > > > > > > > >>> > > > >> > > > system.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > Maybe
>> > > > > > > > > > I’m
>> > > > > > > > > > > >>> being
>> > > > > > > > > > > >>> > > > >> > > > short-sighted,
>> > > > > > > > > > > >>> > > > >> > > > > > but
>> > > > > > > > > > > >>> > > > >> > > > > > > it
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > doesn’t
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > seem
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> like
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> either
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > really
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > > > applies
>> > > > > > > > > > > >>> here. It
>> > > > > > > > > > > >>> > > > seems
>> > > > > > > > > > > >>> > > > >> > like
>> > > > > > > > > > > >>> > > > >> > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > brokers
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > could
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > and
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> would
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> > achieve
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > > > > consensus
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > when
>> > > > > > > > > > > >>> creating a
>> > > > > > > > > > > >>> > > > topic
>> > > > > > > > > > > >>> > > > >> > > anyway,
>> > > > > > > > > > > >>> > > > >> > > > > > which
>> > > > > > > > > > > >>> > > > >> > > > > > > > is
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> all
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that’s
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>> required
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > > > generate
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > > > > > >>> non-colliding
>> > > > > > > > > > > >>> > > > sequence
>> > > > > > > > > > > >>> > > > >> > ids.
>> > > > > > > > > > > >>> > > > >> > > > For
>> > > > > > > > > > > >>> > > > >> > > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > second,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > as
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > you
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>> mention,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > we
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > could
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > > > assign
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > a
>> > > > > > > UUID
>> > > > > > > > > > to
>> > > > > > > > > > > >>> the
>> > > > > > > > > > > >>> > > > cluster
>> > > > > > > > > > > >>> > > > >> as
>> > > > > > > > > > > >>> > > > >> > a
>> > > > > > > > > > > >>> > > > >> > > > > whole,
>> > > > > > > > > > > >>> > > > >> > > > > > > > which
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > would
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > render
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > any
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > > resource
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > > scoped
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > the
>> > > > > > > > > > broker
>> > > > > > > > > > > >>> > > > universally
>> > > > > > > > > > > >>> > > > >> > > unique
>> > > > > > > > > > > >>> > > > >> > > > > as
>> > > > > > > > > > > >>> > > > >> > > > > > > > well.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > The
>> > > > > > > > > > > reason I
>> > > > > > > > > > > >>> > > mention
>> > > > > > > > > > > >>> > > > >> this
>> > > > > > > > > > > >>> > > > >> > > is
>> > > > > > > > > > > >>> > > > >> > > > > > that,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > although
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > a
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > UUID
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> is
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > way
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> more
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > > > compact
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > than
>> > > > > > > > > > topic
>> > > > > > > > > > > >>> > names,
>> > > > > > > > > > > >>> > > > it’s
>> > > > > > > > > > > >>> > > > >> > > still
>> > > > > > > > > > > >>> > > > >> > > > 16
>> > > > > > > > > > > >>> > > > >> > > > > > > > bytes.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> In
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > contrast,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> a
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>> 4-byte
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > integer
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > > > sequence
>> > > > > > > > > > > id
>> > > > > > > > > > > >>> > would
>> > > > > > > > > > > >>> > > > give
>> > > > > > > > > > > >>> > > > >> > us 4
>> > > > > > > > > > > >>> > > > >> > > > > > billion
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> unique
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > topics
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> per
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> > cluster,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > which
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > > > seems
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > like
>> > > > > > > > > > > enough
>> > > > > > > > > > > >>> ;)
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > > > > > Considering
>> > > > > > > > > > > >>> the
>> > > > > > > > > > > >>> > > > >> number of
>> > > > > > > > > > > >>> > > > >> > > > > > different
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> times
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > these
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> topic
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > > > identifiers
>> > > > > > > > > are
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > > sent
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > over
>> > > > > > > > > the
>> > > > > > > > > > > >>> wire or
>> > > > > > > > > > > >>> > > > >> stored
>> > > > > > > > > > > >>> > > > >> > in
>> > > > > > > > > > > >>> > > > >> > > > > > memory,
>> > > > > > > > > > > >>> > > > >> > > > > > > it
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > seems
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > like
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > it
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > might
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > be
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > worth
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > > > > additional
>> > > > > > > > > > > >>> 4x
>> > > > > > > > > > > >>> > > space
>> > > > > > > > > > > >>> > > > >> > > savings.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > What
>> > > > > > > > > do
>> > > > > > > > > > > you
>> > > > > > > > > > > >>> > think
>> > > > > > > > > > > >>> > > > >> about
>> > > > > > > > > > > >>> > > > >> > > this?
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > > > Thanks,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > John
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > On
>> > > > > > > > > Fri,
>> > > > > > > > > > > Sep
>> > > > > > > > > > > >>> 11,
>> > > > > > > > > > > >>> > > > 2020,
>> > > > > > > > > > > >>> > > > >> at
>> > > > > > > > > > > >>> > > > >> > > > 03:20,
>> > > > > > > > > > > >>> > > > >> > > > > > Tom
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > Bentley
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > wrote:
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > Hi
>> > > > > > > > > > > >>> Justine,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > This
>> > > > > > > > > > > looks
>> > > > > > > > > > > >>> > like
>> > > > > > > > > > > >>> > > a
>> > > > > > > > > > > >>> > > > >> very
>> > > > > > > > > > > >>> > > > >> > > > > welcome
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > improvement.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Thanks!
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > > Maybe
>> > > > > > > > > > I
>> > > > > > > > > > > >>> missed
>> > > > > > > > > > > >>> > > it,
>> > > > > > > > > > > >>> > > > >> but
>> > > > > > > > > > > >>> > > > >> > > the
>> > > > > > > > > > > >>> > > > >> > > > > KIP
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> doesn't
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > seem
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > mention
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > > changing
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > > > > >>> > > > DeleteTopicsRequest
>> > > > > > > > > > > >>> > > > >> to
>> > > > > > > > > > > >>> > > > >> > > > > identify
>> > > > > > > > > > > >>> > > > >> > > > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > topic
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > using
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > an
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > id.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> Maybe
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > > that's
>> > > > > > > out
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > of
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > > scope,
>> > > > > > > > > > > but
>> > > > > > > > > > > >>> > > > >> > > > > DeleteTopicsRequest
>> > > > > > > > > > > >>> > > > >> > > > > > is
>> > > > > > > > > > > >>> > > > >> > > > > > > > not
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > listed
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > among
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> Future
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > Work
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > APIs
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > > > either.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > Kind
>> > > > > > > > > > > >>> regards,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > Tom
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > On
>> > > > > > > > > > Thu,
>> > > > > > > > > > > >>> Sep
>> > > > > > > > > > > >>> > 10,
>> > > > > > > > > > > >>> > > > >> 2020 at
>> > > > > > > > > > > >>> > > > >> > > > 3:59
>> > > > > > > > > > > >>> > > > >> > > > > PM
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> Satish
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > Duggana <
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > > > > > > > >>> > > > >> satish.duggana@gmail.com
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > > wrote:
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > Thanks
>> > > > > > > > > > > >>> > > > >> Lucas/Justine
>> > > > > > > > > > > >>> > > > >> > > for
>> > > > > > > > > > > >>> > > > >> > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > nice
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> KIP.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > It
>> > > > > > > > > > has
>> > > > > > > > > > > >>> > several
>> > > > > > > > > > > >>> > > > >> > benefits
>> > > > > > > > > > > >>> > > > >> > > > > which
>> > > > > > > > > > > >>> > > > >> > > > > > > > also
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > include
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>> > simplifying
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > topic
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > > deletion
>> > > > > > > > > > > >>> > > process
>> > > > > > > > > > > >>> > > > >> by
>> > > > > > > > > > > >>> > > > >> > > > > > controller
>> > > > > > > > > > > >>> > > > >> > > > > > > > and
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > logs
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > cleanup
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> by
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> > brokers
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> in
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > > corner
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > cases.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > Best,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > > Satish.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > On
>> > > > > > > > > > > Wed,
>> > > > > > > > > > > >>> Sep
>> > > > > > > > > > > >>> > 9,
>> > > > > > > > > > > >>> > > > >> 2020
>> > > > > > > > > > > >>> > > > >> > at
>> > > > > > > > > > > >>> > > > >> > > > > 10:07
>> > > > > > > > > > > >>> > > > >> > > > > > PM
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > Justine
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Olshan <
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > > > > > > > > >>> jolshan@confluent.io
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > wrote:
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > > > Hello
>> > > > > > > > > > > >>> all,
>> > > > > > > > > > > >>> > > > it's
>> > > > > > > > > > > >>> > > > >> > been
>> > > > > > > > > > > >>> > > > >> > > > > > almost a
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> year!
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > I've
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > made
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > some
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > > changes
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > this
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > KIP
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > and
>> > > > > > > > > > > >>> hope to
>> > > > > > > > > > > >>> > > > >> continue
>> > > > > > > > > > > >>> > > > >> > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> discussion.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > > One
>> > > > > > > > > > > >>> of the
>> > > > > > > > > > > >>> > > > main
>> > > > > > > > > > > >>> > > > >> > > changes
>> > > > > > > > > > > >>> > > > >> > > > > > I've
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> added
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > is
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > now
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>> metadata
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > response
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > > will
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > > include
>> > > > > > > > > > > >>> the
>> > > > > > > > > > > >>> > > > topic
>> > > > > > > > > > > >>> > > > >> ID
>> > > > > > > > > > > >>> > > > >> > > (as
>> > > > > > > > > > > >>> > > > >> > > > > > Colin
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > suggested).
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Clients
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > can
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > obtain
>> > > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > > > topicID
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > of a
>> > > > > > > > > > > >>> given
>> > > > > > > > > > > >>> > > topic
>> > > > > > > > > > > >>> > > > >> > > through
>> > > > > > > > > > > >>> > > > >> > > > a
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > TopicDescription.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > The
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>> topicId
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> will
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > also
>> > > > > > be
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > > included
>> > > > > > > > > > > >>> > with
>> > > > > > > > > > > >>> > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > UpdateMetadata
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > request.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > > Let
>> > > > > > > > > > > me
>> > > > > > > > > > > >>> > know
>> > > > > > > > > > > >>> > > > what
>> > > > > > > > > > > >>> > > > >> > you
>> > > > > > > > > > > >>> > > > >> > > > all
>> > > > > > > > > > > >>> > > > >> > > > > > > think.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > > > Thank
>> > > > > > > > > > > >>> you,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > > > >>> Justine
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > On
>> > > > > > > > > > > >>> > > 2019/09/13
>> > > > > > > > > > > >>> > > > >> > > 16:38:26,
>> > > > > > > > > > > >>> > > > >> > > > > > > "Colin
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > McCabe" <
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > > > > > > > > cmccabe@apache.org
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > > wrote:
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > Hi
>> > > > > > > > > > > >>> > Lucas,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > >>> Thanks
>> > > > > > > > > > > >>> > for
>> > > > > > > > > > > >>> > > > >> > tackling
>> > > > > > > > > > > >>> > > > >> > > > > this.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> Topic
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > IDs
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > are a
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > great
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> idea,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > and
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > this
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > is
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > a
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > really
>> > > > > > > > > > > >>> good
>> > > > > > > > > > > >>> > > > >> writeup.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > For
>> > > > > > > > > > > >>> > > > >> > > > > > > /brokers/topics/[topic],
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > schema
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> version
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> should
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> be
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > > bumped
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > > version
>> > > > > > > > > > > >>> 3,
>> > > > > > > > > > > >>> > > > rather
>> > > > > > > > > > > >>> > > > >> > than
>> > > > > > > > > > > >>> > > > >> > > 2.
>> > > > > > > > > > > >>> > > > >> > > > > > > > KIP-455
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > bumped
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > version
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> this
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > > znode
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > 2
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > > already
>> > > > > > > > > > > >>> :)
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > >>> Given
>> > > > > > > > > > > >>> > that
>> > > > > > > > > > > >>> > > > >> we're
>> > > > > > > > > > > >>> > > > >> > > > going
>> > > > > > > > > > > >>> > > > >> > > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > be
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > seeing
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > these
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > things
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > as
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > strings
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > as
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > > lot
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > (in
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > logs,
>> > > > > > > > > > > in
>> > > > > > > > > > > >>> > > > >> ZooKeeper,
>> > > > > > > > > > > >>> > > > >> > on
>> > > > > > > > > > > >>> > > > >> > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > command-line,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > etc.),
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > does
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > it
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> make
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > > sense
>> > > > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > use
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > base64
>> > > > > > > > > > > >>> when
>> > > > > > > > > > > >>> > > > >> > converting
>> > > > > > > > > > > >>> > > > >> > > > them
>> > > > > > > > > > > >>> > > > >> > > > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > strings?
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > >>> Here is
>> > > > > > > > > > > >>> > an
>> > > > > > > > > > > >>> > > > >> > example
>> > > > > > > > > > > >>> > > > >> > > of
>> > > > > > > > > > > >>> > > > >> > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > hex
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>> > representation:
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>> > > > > > > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > And
>> > > > > > > > > > > >>> here
>> > > > > > > > > > > >>> > > is
>> > > > > > > > > > > >>> > > > an
>> > > > > > > > > > > >>> > > > >> > > > example
>> > > > > > > > > > > >>> > > > >> > > > > in
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> base64.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > b8tRS7h4TJ2Vt43Dp85v2A
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > The
>> > > > > > > > > > > >>> > base64
>> > > > > > > > > > > >>> > > > >> > version
>> > > > > > > > > > > >>> > > > >> > > > > saves
>> > > > > > > > > > > >>> > > > >> > > > > > 15
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > letters
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > (to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > be
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > fair,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > 4
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > those
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > were
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > dashes
>> > > > > > > > > > > >>> that
>> > > > > > > > > > > >>> > we
>> > > > > > > > > > > >>> > > > >> could
>> > > > > > > > > > > >>> > > > >> > > have
>> > > > > > > > > > > >>> > > > >> > > > > > > elided
>> > > > > > > > > > > >>> > > > >> > > > > > > > in
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > hex
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > > > > > > representation.)
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > >>> Another
>> > > > > > > > > > > >>> > > > thing
>> > > > > > > > > > > >>> > > > >> to
>> > > > > > > > > > > >>> > > > >> > > > > consider
>> > > > > > > > > > > >>> > > > >> > > > > > > is
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> that
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > we
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > should
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > specify
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> that
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > > >>> all-zeroes
>> > > > > > > > > > > >>> > > UUID
>> > > > > > > > > > > >>> > > > is
>> > > > > > > > > > > >>> > > > >> > not
>> > > > > > > > > > > >>> > > > >> > > a
>> > > > > > > > > > > >>> > > > >> > > > > > valid
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> topic
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > UUID.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >  We
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > can't
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > use
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > null
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > for
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > > this
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > > because
>> > > > > > > > > > > >>> we
>> > > > > > > > > > > >>> > > can't
>> > > > > > > > > > > >>> > > > >> > pass a
>> > > > > > > > > > > >>> > > > >> > > > > null
>> > > > > > > > > > > >>> > > > >> > > > > > > UUID
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> over
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > RPC
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>> protocol
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > (there
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > is
>> > > > > no
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > > special
>> > > > > > > > > > > >>> > > pattern
>> > > > > > > > > > > >>> > > > >> for
>> > > > > > > > > > > >>> > > > >> > > null,
>> > > > > > > > > > > >>> > > > >> > > > > nor
>> > > > > > > > > > > >>> > > > >> > > > > > > do
>> > > > > > > > > > > >>> > > > >> > > > > > > > we
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > want
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> waste
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > space
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > > reserving
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > > such
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > a
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > > >>> pattern.)
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > >>> Maybe I
>> > > > > > > > > > > >>> > > > missed
>> > > > > > > > > > > >>> > > > >> > it,
>> > > > > > > > > > > >>> > > > >> > > > but
>> > > > > > > > > > > >>> > > > >> > > > > > did
>> > > > > > > > > > > >>> > > > >> > > > > > > > you
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > describe
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>> "migration
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > of...
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > > > existing
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > > topic[s]
>> > > > > > > > > > > >>> > > without
>> > > > > > > > > > > >>> > > > >> > topic
>> > > > > > > > > > > >>> > > > >> > > > IDs"
>> > > > > > > > > > > >>> > > > >> > > > > > in
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> detail
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > in
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > any
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > section?
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > It
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > seems
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > like
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > when
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > the
>> > > > > > > > > > > new
>> > > > > > > > > > > >>> > > > controller
>> > > > > > > > > > > >>> > > > >> > > > becomes
>> > > > > > > > > > > >>> > > > >> > > > > > > > active,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> it
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > should
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> just
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> > generate
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > random
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > UUIDs
>> > > > > > > > > > for
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > these,
>> > > > > > > > > > > >>> and
>> > > > > > > > > > > >>> > > write
>> > > > > > > > > > > >>> > > > >> the
>> > > > > > > > > > > >>> > > > >> > > > random
>> > > > > > > > > > > >>> > > > >> > > > > > > UUIDs
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> back
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>> ZooKeeper.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > It
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > would
>> > > > > be
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > > good
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > spell
>> > > > > > > > > > > >>> that
>> > > > > > > > > > > >>> > > out.
>> > > > > > > > > > > >>> > > > >> We
>> > > > > > > > > > > >>> > > > >> > > > should
>> > > > > > > > > > > >>> > > > >> > > > > > make
>> > > > > > > > > > > >>> > > > >> > > > > > > > it
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > clear
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> this
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> > happens
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > > > > regardless
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > of
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > the
>> > > > > > > > > > > >>> > > inter-broker
>> > > > > > > > > > > >>> > > > >> > > protocol
>> > > > > > > > > > > >>> > > > >> > > > > > > version
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > (it's
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > a
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>> compatible
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > > change).
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > >>> > > > >> > > "LeaderAndIsrRequests
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> including an
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>> > > > > > is_every_partition
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > flag"
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > > > seems a
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > bit
>> > > > > > > > > > > >>> wordy.
>> > > > > > > > > > > >>> > > Can
>> > > > > > > > > > > >>> > > > we
>> > > > > > > > > > > >>> > > > >> > just
>> > > > > > > > > > > >>> > > > >> > > > > call
>> > > > > > > > > > > >>> > > > >> > > > > > > > these
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > "full
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > > > > > > > >>> LeaderAndIsrRequests"?
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > Then
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > RPC
>> > > > > > > > > > > >>> field
>> > > > > > > > > > > >>> > > could
>> > > > > > > > > > > >>> > > > be
>> > > > > > > > > > > >>> > > > >> > > named
>> > > > > > > > > > > >>> > > > >> > > > > > > "full".
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > Also,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > it
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > would
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>> probably
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> be
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > > better
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > for
>> > > > > > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > RPC
>> > > > > > > > > > > >>> field to
>> > > > > > > > > > > >>> > > be
>> > > > > > > > > > > >>> > > > an
>> > > > > > > > > > > >>> > > > >> > enum
>> > > > > > > > > > > >>> > > > >> > > > of
>> > > > > > > > > > > >>> > > > >> > > > > {
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > UNSPECIFIED,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>> > INCREMENTAL,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> FULL
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > },
>> > > > so
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > > that
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > we
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > can
>> > > > > > > > > > > >>> cleanly
>> > > > > > > > > > > >>> > > > handle
>> > > > > > > > > > > >>> > > > >> > old
>> > > > > > > > > > > >>> > > > >> > > > > > versions
>> > > > > > > > > > > >>> > > > >> > > > > > > > (by
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > treating
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> them
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > as
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > > > UNSPECIFIED)
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > In
>> > > > > > > > > > > >>> the
>> > > > > > > > > > > >>> > > > >> > > > > > LeaderAndIsrRequest
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > section,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > you
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> write
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > "A
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> final
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > > > deletion
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > event
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > will
>> > > > > > > > > > > be
>> > > > > > > > > > > >>> > > > secheduled
>> > > > > > > > > > > >>> > > > >> > for
>> > > > > > > > > > > >>> > > > >> > > X
>> > > > > > > > > > > >>> > > > >> > > > ms
>> > > > > > > > > > > >>> > > > >> > > > > > > after
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>> > > > > > > LeaderAndIsrRequest
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> was
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > > first
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > > >>> received..."
>> > > > > > > > > > > >>> > > I
>> > > > > > > > > > > >>> > > > >> guess
>> > > > > > > > > > > >>> > > > >> > > > the X
>> > > > > > > > > > > >>> > > > >> > > > > > > was a
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > placeholder
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> that
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > you
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > intended
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > > > replace
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > before
>> > > > > > > > > > > >>> > > posting?
>> > > > > > > > > > > >>> > > > :)
>> > > > > > > > > > > >>> > > > >> > In
>> > > > > > > > > > > >>> > > > >> > > > any
>> > > > > > > > > > > >>> > > > >> > > > > > > case,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> this
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > seems
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > like
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > kind
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> of
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > > thing
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > > we'd
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > want a
>> > > > > > > > > > > >>> > > > >> configuration
>> > > > > > > > > > > >>> > > > >> > > for.
>> > > > > > > > > > > >>> > > > >> > > > > > > Let's
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > describe
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > > > > > configuration
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > key
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > > > > somewhere
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > in
>> > > > > > > > > > > this
>> > > > > > > > > > > >>> KIP,
>> > > > > > > > > > > >>> > > > >> > including
>> > > > > > > > > > > >>> > > > >> > > > what
>> > > > > > > > > > > >>> > > > >> > > > > > its
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > default
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > value
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> is.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > We
>> > > > > > > > > > > >>> > should
>> > > > > > > > > > > >>> > > > >> > probably
>> > > > > > > > > > > >>> > > > >> > > > also
>> > > > > > > > > > > >>> > > > >> > > > > > > log a
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > bunch
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > of
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> messages
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > at
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> WARN
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > level
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > > when
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > > >>> something is
>> > > > > > > > > > > >>> > > > >> > scheduled
>> > > > > > > > > > > >>> > > > >> > > > for
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> deletion,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > as
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > well.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > (Maybe
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> this
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > was
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > > > assumed,
>> > > > > > > > > > > but
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > it
>> > > > > > > > > > > >>> would be
>> > > > > > > > > > > >>> > > good
>> > > > > > > > > > > >>> > > > >> to
>> > > > > > > > > > > >>> > > > >> > > > mention
>> > > > > > > > > > > >>> > > > >> > > > > > > it).
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > I
>> > > > > > > > > > > >>> feel
>> > > > > > > > > > > >>> > > like
>> > > > > > > > > > > >>> > > > >> there
>> > > > > > > > > > > >>> > > > >> > > > are a
>> > > > > > > > > > > >>> > > > >> > > > > > few
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > sections
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > should
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > be
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > moved
>> > > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > > > > "rejected
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > > >>> > alternatives."
>> > > > > > > > > > > >>> > > > >> For
>> > > > > > > > > > > >>> > > > >> > > > > example,
>> > > > > > > > > > > >>> > > > >> > > > > > in
>> > > > > > > > > > > >>> > > > >> > > > > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > DeleteTopics
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>> section,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > since
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > > we're
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > not
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > going
>> > > > > > > > > > > >>> to do
>> > > > > > > > > > > >>> > > > >> option 1
>> > > > > > > > > > > >>> > > > >> > or
>> > > > > > > > > > > >>> > > > >> > > > 2,
>> > > > > > > > > > > >>> > > > >> > > > > > > these
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > should
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > be
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > moved
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > into
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > > "rejected
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > > >>> > alternatives,"
>> > > > > > > > > > > >>> > > > >> > rather
>> > > > > > > > > > > >>> > > > >> > > > than
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> appearing
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > inline.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Another
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> case
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > is
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > > > "Should
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > we
>> > > > > > > > > > > >>> remove
>> > > > > > > > > > > >>> > > topic
>> > > > > > > > > > > >>> > > > >> name
>> > > > > > > > > > > >>> > > > >> > > from
>> > > > > > > > > > > >>> > > > >> > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> protocol
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > where
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>> possible"
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > section.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > This
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > is
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > > clearly
>> > > > > > > > > > > >>> > > > >> discussing a
>> > > > > > > > > > > >>> > > > >> > > > design
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > alternative
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > that
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> we're
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > not
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > > proposing
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > > >>> implement:
>> > > > > > > > > > > >>> > > > >> removing
>> > > > > > > > > > > >>> > > > >> > the
>> > > > > > > > > > > >>> > > > >> > > > > topic
>> > > > > > > > > > > >>> > > > >> > > > > > > > name
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > from
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > those
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>> > protocols.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > Is
>> > > > > > > > > > > >>> it
>> > > > > > > > > > > >>> > > really
>> > > > > > > > > > > >>> > > > >> > > > necessary
>> > > > > > > > > > > >>> > > > >> > > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > have
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> a
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > new
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > > > > > > > > >>> > > /admin/delete_topics_by_id
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > path
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > in
>> > > > > > > > > > > >>> > ZooKeeper?
>> > > > > > > > > > > >>> > > > It
>> > > > > > > > > > > >>> > > > >> > seems
>> > > > > > > > > > > >>> > > > >> > > > > like
>> > > > > > > > > > > >>> > > > >> > > > > > we
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> don't
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > really
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> need
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > this.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > Whenever
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > there
>> > > > > > > > > is
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > a
>> > > > > > > > > > new
>> > > > > > > > > > > >>> > > > controller,
>> > > > > > > > > > > >>> > > > >> > we'll
>> > > > > > > > > > > >>> > > > >> > > > > send
>> > > > > > > > > > > >>> > > > >> > > > > > > out
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> full
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> > > > > > > LeaderAndIsrRequests
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > which
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > will
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > > trigger
>> > > > > > > > > > > >>> the
>> > > > > > > > > > > >>> > > > stale
>> > > > > > > > > > > >>> > > > >> > > topics
>> > > > > > > > > > > >>> > > > >> > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > be
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> cleaned
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > up.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >  The
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > active
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > > > > controller
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > > will
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > also
>> > > > > > > > > > > >>> send
>> > > > > > > > > > > >>> > the
>> > > > > > > > > > > >>> > > > full
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> LeaderAndIsrRequest
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> brokers
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > are
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > just
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > > > starting
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > up.
>> > > > > > > > > > > >>> So we
>> > > > > > > > > > > >>> > > > don't
>> > > > > > > > > > > >>> > > > >> > > really
>> > > > > > > > > > > >>> > > > >> > > > > > need
>> > > > > > > > > > > >>> > > > >> > > > > > > > this
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > kind
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > of
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > two-phase
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > commit
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > > (send
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > > out
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > > >>> > > > >> StopReplicasRequest,
>> > > > > > > > > > > >>> > > > >> > > get
>> > > > > > > > > > > >>> > > > >> > > > > ACKs
>> > > > > > > > > > > >>> > > > >> > > > > > > > from
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> all
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > nodes,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > commit
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > by
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > removing
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > > >>> > > > >> /admin/delete_topics
>> > > > > > > > > > > >>> > > > >> > > > node)
>> > > > > > > > > > > >>> > > > >> > > > > > any
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> more.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > You
>> > > > > > > > > > > >>> > > mention
>> > > > > > > > > > > >>> > > > >> that
>> > > > > > > > > > > >>> > > > >> > > > > > > FetchRequest
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> will
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > now
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> include
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > UUID
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > avoid
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > > > issues
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > where
>> > > > > > > > > > > >>> > requests
>> > > > > > > > > > > >>> > > > are
>> > > > > > > > > > > >>> > > > >> > made
>> > > > > > > > > > > >>> > > > >> > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > stale
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > partitions.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>> However,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > adding a
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > UUID
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > > >>> > > MetadataRequest
>> > > > > > > > > > > >>> > > > is
>> > > > > > > > > > > >>> > > > >> > > listed
>> > > > > > > > > > > >>> > > > >> > > > > as
>> > > > > > > > > > > >>> > > > >> > > > > > > > future
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > work,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > out
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > of
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > scope
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > for
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > this
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > KIP.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > How
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > will
>> > > > > > > > > > > the
>> > > > > > > > > > > >>> > > client
>> > > > > > > > > > > >>> > > > >> learn
>> > > > > > > > > > > >>> > > > >> > > > what
>> > > > > > > > > > > >>> > > > >> > > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> topic
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > UUID
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > is,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > if
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > > metadata
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > > > response
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > > doesn't
>> > > > > > > > > > > >>> > > include
>> > > > > > > > > > > >>> > > > >> that
>> > > > > > > > > > > >>> > > > >> > > > > > > information?
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> It
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > seems
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > like
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > adding
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > UUID
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > > >>> > > MetadataResponse
>> > > > > > > > > > > >>> > > > >> > would
>> > > > > > > > > > > >>> > > > >> > > be
>> > > > > > > > > > > >>> > > > >> > > > > an
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > improvement
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > here
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> that
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > might
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> not
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > be
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > too
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > hard
>> > > > > > > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > make.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > >>> best,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > >>> Colin
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > On
>> > > > > > > > > > > >>> Mon,
>> > > > > > > > > > > >>> > > Sep
>> > > > > > > > > > > >>> > > > 9,
>> > > > > > > > > > > >>> > > > >> > > 2019,
>> > > > > > > > > > > >>> > > > >> > > > at
>> > > > > > > > > > > >>> > > > >> > > > > > > > 17:48,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > Ryanne
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Dolan
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > wrote:
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > >>> Lucas,
>> > > > > > > > > > > >>> > > > this
>> > > > > > > > > > > >>> > > > >> > would
>> > > > > > > > > > > >>> > > > >> > > > be
>> > > > > > > > > > > >>> > > > >> > > > > > > great.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> I've
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > run
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > into
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > issues
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> with
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > > topics
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > being
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > >>> > > > resurrected
>> > > > > > > > > > > >>> > > > >> > > > > > accidentally,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> since
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > a
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > client
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > cannot
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > easily
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > > > > > distinguish
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > > between
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > a
>> > > > > > > > > > > >>> > > deleted
>> > > > > > > > > > > >>> > > > >> topic
>> > > > > > > > > > > >>> > > > >> > > > and a
>> > > > > > > > > > > >>> > > > >> > > > > > new
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> topic
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > with
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> same
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > name.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> I'd
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > need
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > ID
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > >>> > > accessible
>> > > > > > > > > > > >>> > > > >> from
>> > > > > > > > > > > >>> > > > >> > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > client
>> > > > > > > > > > > >>> > > > >> > > > > > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > solve
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > issue,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > but
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > this
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > is a
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > good
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > first
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > >>> step.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > >>> Ryanne
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > On
>> > > > > > > > > > > >>> > Wed,
>> > > > > > > > > > > >>> > > > Sep
>> > > > > > > > > > > >>> > > > >> 4,
>> > > > > > > > > > > >>> > > > >> > > 2019
>> > > > > > > > > > > >>> > > > >> > > > > at
>> > > > > > > > > > > >>> > > > >> > > > > > > 1:41
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> PM
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > Lucas
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>> > Bradstreet <
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > > > > > >>> > > lucas@confluent.io>
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > wrote:
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > >>> Hi
>> > > > > > > > > > > >>> > > all,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > >>> I
>> > > > > > > > > > > >>> > > would
>> > > > > > > > > > > >>> > > > >> like
>> > > > > > > > > > > >>> > > > >> > to
>> > > > > > > > > > > >>> > > > >> > > > > kick
>> > > > > > > > > > > >>> > > > >> > > > > > > off
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > discussion
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > of
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>> KIP-516,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > an
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > > > > > >>> implementation
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > of
>> > > > > > > > > > > topic
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > >>> IDs
>> > > > > > > > > > > >>> > > for
>> > > > > > > > > > > >>> > > > >> > Kafka.
>> > > > > > > > > > > >>> > > > >> > > > > Topic
>> > > > > > > > > > > >>> > > > >> > > > > > > IDs
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> aim
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > to
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > solve
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> topic
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > > uniqueness
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > > > problems
>> > > > > > > > > > > in
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > Kafka,
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > >>> > where
>> > > > > > > > > > > >>> > > > >> > referring
>> > > > > > > > > > > >>> > > > >> > > > to
>> > > > > > > > > > > >>> > > > >> > > > > a
>> > > > > > > > > > > >>> > > > >> > > > > > > > topic
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> by
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > name
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> alone is
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > > > insufficient.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > > Such
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > cases
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > >>> > > include
>> > > > > > > > > > > >>> > > > >> when
>> > > > > > > > > > > >>> > > > >> > a
>> > > > > > > > > > > >>> > > > >> > > > > topic
>> > > > > > > > > > > >>> > > > >> > > > > > > has
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> been
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > deleted
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> > > recreated
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > with
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > the
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > > same
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > name.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > >>> > Unique
>> > > > > > > > > > > >>> > > > >> > > > identifiers
>> > > > > > > > > > > >>> > > > >> > > > > > will
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> help
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > simplify
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>> improve
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > Kafka's
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > > > topic
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > > deletion
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > >>> > > process,
>> > > > > > > > > > > >>> > > > >> as
>> > > > > > > > > > > >>> > > > >> > > well
>> > > > > > > > > > > >>> > > > >> > > > as
>> > > > > > > > > > > >>> > > > >> > > > > > > > prevent
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > cases
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > where
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > brokers
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > may
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > > > > > > incorrectly
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > > > > interact
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > >>> with
>> > > > > > > > > > > >>> > > > stale
>> > > > > > > > > > > >>> > > > >> > > > versions
>> > > > > > > > > > > >>> > > > >> > > > > > of
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> topics.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >>
>> > > > > > > > > > > >>> > > > >> > > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > >
>> > > > > > > > > > > >>> > > > >> > > > >
>> > > > > > > > > > > >>> > > > >> > > >
>> > > > > > > > > > > >>> > > > >> > >
>> > > > > > > > > > > >>> > > > >> >
>> > > > > > > > > > > >>> > > > >>
>> > > > > > > > > > > >>> > > >
>> > > > > > > > > > > >>> > >
>> > > > > > > > > > > >>> >
>> > > > > > > > > > > >>>
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > >>> > > Looking
>> > > > > > > > > > > >>> > > > >> > forward
>> > > > > > > > > > > >>> > > > >> > > > to
>> > > > > > > > > > > >>> > > > >> > > > > > your
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > thoughts.
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > >>> > Lucas
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > >
>> > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>>
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> >>
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
>> > > > > > > > > > > >>> > > > >> > > > > > > > >>
>> > > > > > > > > > > >>> > > > >> > > > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > > >
>> > > > > > > > > > > >>> > > > >> > > > > >
>> > > > > > > > > > > >>> > > > >> > > > >
>> > > > > > > > > > > >>> > > > >> > > >
>> > > > > > > > > > > >>> > > > >> > >
>> > > > > > > > > > > >>> > > > >> >
>> > > > > > > > > > > >>> > > > >>
>> > > > > > > > > > > >>> > > > >
>> > > > > > > > > > > >>> > > >
>> > > > > > > > > > > >>> > >
>> > > > > > > > > > > >>> >
>> > > > > > > > > > > >>>
>> > > > > > > > > > > >>
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Justine Olshan <jo...@confluent.io>]

Hi Jun,

50. Right. That's what I was thinking but wanted to make sure. Works for me!

Thanks again,
Justine

On Thu, Oct 8, 2020 at 3:50 PM Jun Rao <ju...@confluent.io> wrote:

> Hi, Justine,
>
> 50. We can't change the format of existing versions since it will break
> compatibility. Since we are bumping the version for MetadataRequest
> and DeleteTopicsRequest, we can make the topic name field nullable from the
> new version.
>
> Thanks,
>
> Jun
>
> On Thu, Oct 8, 2020 at 2:19 PM Justine Olshan <jo...@confluent.io>
> wrote:
>
> > Hi Jun,
> >
> > 50. Works for me. Is the idea that it will be nullable from this version
> > onwards, but not past versions?
> >
> > 51. Added some information and example methods for AdminClient future
> work
> >
> > Thanks,
> > Justine
> >
> >
> > On Thu, Oct 8, 2020 at 10:59 AM Jun Rao <ju...@confluent.io> wrote:
> >
> > > Hi, Justine,
> > >
> > > The changes sound reasonable to me.
> > >
> > > 50. Should we make the topic name in the new MetadataRequest
> > > and DeleteTopicsRequest nullable?
> > >
> > > 51. In the future work section, perhaps we could mention opportunities
> > for
> > > using topicId in other methods in AdminClient where topic name is used.
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Wed, Oct 7, 2020 at 2:44 PM Justine Olshan <jo...@confluent.io>
> > > wrote:
> > >
> > > > Hi all,
> > > >
> > > > Another update on the KIP. I've added some information about how
> topic
> > > IDs
> > > > will be supported in AdminClient
> > > >
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-AdminClientSupport
> > > >
> > > >
> > > > One specific decision I wanted to focus on is the inclusion of both
> > topic
> > > > names and topic Ids in metadata and deleteTopics requests.
> > > >
> > > > The idea is that when the request is received the topic id field is
> > > checked
> > > > first, but if it contains the default (zero) uuid, then the topic
> name
> > > > field will be used.
> > > >
> > > > In the case where ids are used, if the ids do not have a
> corresponding
> > > > topic, the request will fail. In the case where both ids and names
> are
> > > > included, the request will fail if the name does not match the id.
> > > >
> > > >
> > > > Initially the id field will be used to support the admin client use
> > cases
> > > > as described in the KIP, but over time, we can transfer to using
> topic
> > > ids
> > > > in all uses of the request.
> > > >
> > > >
> > > > There will also be some support for returning the topic Id on the
> > > > createtopics, deletetopics and metadata responses. I think it makes
> > sense
> > > > to include both the name and the id in the successful response for
> > these
> > > > cases.
> > > >
> > > >
> > > > Let me know if you have any comments or concerns regarding the Admin
> > > Client
> > > > changes.
> > > >
> > > > On Wed, Oct 7, 2020 at 11:05 AM Jun Rao <ju...@confluent.io> wrote:
> > > >
> > > > > Hi, Justine,
> > > > >
> > > > > I was just wondering if we expose topicId publicly, how would
> people
> > > use
> > > > > it. One possibility is to use topicId in all those public
> interfaces
> > > > where
> > > > > we use topic name. I am not sure if the need is very strong though.
> > So,
> > > > > maybe we can list this as future considerations.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Jun
> > > > >
> > > > > On Wed, Oct 7, 2020 at 8:57 AM Justine Olshan <
> jolshan@confluent.io>
> > > > > wrote:
> > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > After some careful consideration, I've decided that the
> > > > TopicIdPartition
> > > > > > will be out of scope for this KIP. After investigating what it
> will
> > > > look
> > > > > > like using it in only the Fetch Request, I found that I would
> need
> > to
> > > > > > modify several public APIs in the consumer and I would need to
> take
> > > > > special
> > > > > > care in many situations to handle the request correctly based on
> > what
> > > > > > version of the request was sent. For these reasons, I've added
> > > > > > TopicIdPartition to potential future work along with an
> alternative
> > > to
> > > > > > modify TopicPartition to include a topic ID field.
> > > > > >
> > > > > > Jun- I was also wondering about some of the details for
> supporting
> > > > topic
> > > > > > IDs for delete and describe topics in the admin client. Is the
> idea
> > > > that
> > > > > > instead of supplying topic names to describe or delete, we would
> > > > supply a
> > > > > > collection of ids? If that is the case, I'm wondering if it makes
> > > sense
> > > > > to
> > > > > > add topic Ids to the metadata and delete topics requests.
> > > > > >
> > > > > > Thanks,
> > > > > > Justine
> > > > > >
> > > > > > On Mon, Oct 5, 2020 at 9:25 PM Satish Duggana <
> > > > satish.duggana@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Jun, Justine,
> > > > > > > +1 to have TopicIdPartition in public packages. This is useful
> > for
> > > > > > > KIP-405 tiered storage too. We currently use TopicPartition for
> > > > > > > pluggable log storage and metadata storage apis. We will be
> using
> > > > > > > TopicIdPartition once the proposed changes in this KIP are
> > merged.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Satish.
> > > > > > >
> > > > > > > On Tue, Oct 6, 2020 at 5:21 AM Jun Rao <ju...@confluent.io>
> wrote:
> > > > > > > >
> > > > > > > > Hi, Justine,
> > > > > > > >
> > > > > > > > I guess we have to maintain the mapping between topic name
> and
> > > > > topicId
> > > > > > in
> > > > > > > > both the client and the server. If TopicIdPartition is meant
> > for
> > > > > > internal
> > > > > > > > usage only, we can create it under a package
> > > > > > > > like org/apache/kafka/common/internal and we don't need to
> > > document
> > > > > it
> > > > > > in
> > > > > > > > the KIP. If it's meant for external public usage (and
> > potentially
> > > > > > > internal
> > > > > > > > usage too), we can create it in org/apache/kafka/common and
> > need
> > > to
> > > > > > > > document it in the KIP.
> > > > > > > >
> > > > > > > > Since topicId is now part of the topic metadata, perhaps it's
> > > > useful
> > > > > to
> > > > > > > > expose it in TopicMetadataAndConfig too.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > >
> > > > > > > > Jun
> > > > > > > >
> > > > > > > > On Mon, Oct 5, 2020 at 3:35 PM Justine Olshan <
> > > > jolshan@confluent.io>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi Jun,
> > > > > > > > >
> > > > > > > > > This KIP removes the topic name from many requests. A lot
> of
> > > > > requests
> > > > > > > keep
> > > > > > > > > track of the partitions internally with a TopicPartition.
> So
> > > when
> > > > > we
> > > > > > > remove
> > > > > > > > > the topic name, we can not differentiate unless we can also
> > use
> > > > > topic
> > > > > > > ID.
> > > > > > > > > The Fetch request relies heavily on this, but ListOffset,
> > > > > > > > > OffsetsForLeaderEpoch, etc all use TopicPartition as a key
> > > value
> > > > > for
> > > > > > > > > internal maps. Digging deeper into Fetch, I've noticed that
> > the
> > > > > code
> > > > > > > path
> > > > > > > > > runs into some public APIs that use the TopicPartition.
> Some
> > > > > examples
> > > > > > > are
> > > > > > > > > Cluster.java or RecordTooLargeException.
> > > > > > > > >
> > > > > > > > > A related issue is that in some places like
> > > > > > > TopicAuthorizationException,
> > > > > > > > > the topic name is required and without the name in the
> > request,
> > > > we
> > > > > > > won't
> > > > > > > > > know it.
> > > > > > > > >
> > > > > > > > > I guess an alternative would be to have some mapping from
> > topic
> > > > > name
> > > > > > to
> > > > > > > > > topic ID, but I would have to think about that more.
> > > > > > > > >
> > > > > > > > > The KIP mentions exposing TopicId in TopicDescription. (The
> > > font
> > > > > was
> > > > > > > > > weirdly colored, so I've made it easier to read.)
> > > > > > > > >
> > > > > > > > > Would the idea for the TopicMetadataAndConfig be that the
> > topic
> > > > ID
> > > > > > > would be
> > > > > > > > > returned as a result?
> > > > > > > > > As for the other methods describe/deletetopics, it makes
> > sense
> > > to
> > > > > > > include
> > > > > > > > > them at some point, but I'm not sure how far we should go
> in
> > > > terms
> > > > > of
> > > > > > > > > supporting topic IDs everywhere.
> > > > > > > > >
> > > > > > > > > Thank you for continuing to look at the KIP!
> > > > > > > > > Justine
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Mon, Oct 5, 2020 at 2:53 PM Jun Rao <ju...@confluent.io>
> > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi, Justine,
> > > > > > > > > >
> > > > > > > > > > Adding a TopicIdPartition in the public interface may be
> > > > useful.
> > > > > > > Where
> > > > > > > > > > would you expect it to be used?
> > > > > > > > > >
> > > > > > > > > > Related to this, since we are exposing topicId in the
> tool,
> > > it
> > > > > > seems
> > > > > > > that
> > > > > > > > > > we may want to expose topicId in at least
> TopicDescription
> > > used
> > > > > by
> > > > > > > > > > AdminClient.describeTopics() and TopicMetadataAndConfig
> > used
> > > in
> > > > > > > > > > AdminClient.createTopics(Collection<NewTopic>
> newTopics). I
> > > am
> > > > > also
> > > > > > > not
> > > > > > > > > > sure if we plan to expose topicId in other AdminClient
> > > methods
> > > > in
> > > > > > > this
> > > > > > > > > KIP
> > > > > > > > > > or in some future KIP. For example, it might be useful to
> > > > support
> > > > > > > topicId
> > > > > > > > > > in the following methods.
> > > > > > > > > >
> > > > > > > > > >     DescribeTopicsResult
> describeTopics(Collection<String>
> > > > > > > topicNames,
> > > > > > > > > > DescribeTopicsOptions options);
> > > > > > > > > >     DeleteTopicsResult deleteTopics(Collection<String>
> > > topics,
> > > > > > > > > > DeleteTopicsOptions options);
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > >
> > > > > > > > > > Jun
> > > > > > > > > >
> > > > > > > > > > On Mon, Oct 5, 2020 at 11:03 AM Justine Olshan <
> > > > > > jolshan@confluent.io
> > > > > > > >
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Hi all,
> > > > > > > > > > >
> > > > > > > > > > > I was thinking about this KIP some more. We are
> removing
> > > the
> > > > > > topic
> > > > > > > name
> > > > > > > > > > > from many of these request types, and some of them (eg.
> > > > Fetch)
> > > > > > use
> > > > > > > > > > > TopicPartition as keys in a map when handling the
> > request.
> > > I
> > > > > was
> > > > > > > > > thinking
> > > > > > > > > > > it would make sense to make some sort of
> TopicIdPartition
> > > > with
> > > > > > > topic
> > > > > > > > > > name,
> > > > > > > > > > > topic id, and partition fields (name and id would
> > > potentially
> > > > > be
> > > > > > > > > optional
> > > > > > > > > > > types since different request versions will contain
> only
> > > one
> > > > of
> > > > > > the
> > > > > > > > > two)
> > > > > > > > > > I
> > > > > > > > > > > was wondering if it made sense to include this as a
> > public
> > > > > facing
> > > > > > > API
> > > > > > > > > > > similar to TopicPartition. If so, I will add this class
> > to
> > > > the
> > > > > > KIP
> > > > > > > as
> > > > > > > > > > well.
> > > > > > > > > > >
> > > > > > > > > > > Thanks,
> > > > > > > > > > > Justine
> > > > > > > > > > >
> > > > > > > > > > > On Mon, Oct 5, 2020 at 8:52 AM Justine Olshan <
> > > > > > > jolshan@confluent.io>
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Hi all,
> > > > > > > > > > > > I've made a few minor changes to the UUID class. I've
> > > > added a
> > > > > > few
> > > > > > > > > more
> > > > > > > > > > > > methods to support UUIDs more generally, by
> including a
> > > > > > > constructor
> > > > > > > > > > and a
> > > > > > > > > > > > method to generate random UUIDs. Both of these will
> be
> > > > useful
> > > > > > for
> > > > > > > > > > > testing.
> > > > > > > > > > > >
> > > > > > > > > > > > Let me know if there are any other areas for
> > discussion.
> > > > > > > > > > > >
> > > > > > > > > > > > Thank you,
> > > > > > > > > > > > Justine
> > > > > > > > > > > >
> > > > > > > > > > > > On Thu, Oct 1, 2020 at 5:29 PM Justine Olshan <
> > > > > > > jolshan@confluent.io>
> > > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > >> Hi Jun,
> > > > > > > > > > > >> Thanks for looking at it again. Here's the new spec.
> > (I
> > > > > fixed
> > > > > > > the
> > > > > > > > > typo
> > > > > > > > > > > in
> > > > > > > > > > > >> it too.)
> > > > > > > > > > > >> {"name": "id", "type": "string", "doc":
> option[UUID]}
> > > > > > > > > > > >>
> > > > > > > > > > > >> Justine
> > > > > > > > > > > >>
> > > > > > > > > > > >>
> > > > > > > > > > > >> On Thu, Oct 1, 2020 at 5:03 PM Jun Rao <
> > > jun@confluent.io>
> > > > > > > wrote:
> > > > > > > > > > > >>
> > > > > > > > > > > >>> Hi, Justine,
> > > > > > > > > > > >>>
> > > > > > > > > > > >>> Thanks for the update. The KIP looks good to me
> now.
> > > > Just a
> > > > > > > minor
> > > > > > > > > > > comment
> > > > > > > > > > > >>> below.
> > > > > > > > > > > >>>
> > > > > > > > > > > >>> 30. Perhaps "option[UUID]" can be put in the doc.
> > > > > > > > > > > >>>
> > > > > > > > > > > >>> Jun
> > > > > > > > > > > >>>
> > > > > > > > > > > >>> On Thu, Oct 1, 2020 at 3:28 PM Justine Olshan <
> > > > > > > > > jolshan@confluent.io>
> > > > > > > > > > > >>> wrote:
> > > > > > > > > > > >>>
> > > > > > > > > > > >>> > Hi Jun,
> > > > > > > > > > > >>> > Thanks for the response!
> > > > > > > > > > > >>> >
> > > > > > > > > > > >>> > 30. I think I might have changed this in between.
> > The
> > > > > > current
> > > > > > > > > > version
> > > > > > > > > > > >>> > says:  {"name":
> > > > > > > > > > > >>> > "id", "type": "option[UUID]"}, "doc": topic id}
> > > > > > > > > > > >>> > I have switched to the option type to cover the
> > > > migration
> > > > > > > case
> > > > > > > > > > where
> > > > > > > > > > > a
> > > > > > > > > > > >>> > TopicZNode does not yet have a topic ID.
> > > > > > > > > > > >>> > I understand that due to json, this field is
> > written
> > > > as a
> > > > > > > string,
> > > > > > > > > > so
> > > > > > > > > > > >>> if I
> > > > > > > > > > > >>> > should move the "option[uuid]" to the doc field
> and
> > > the
> > > > > > type
> > > > > > > > > should
> > > > > > > > > > > be
> > > > > > > > > > > >>> > "string" please let me know.
> > > > > > > > > > > >>> >
> > > > > > > > > > > >>> > 40. I've added a definition for UUID.
> > > > > > > > > > > >>> > 41,42. Fixed
> > > > > > > > > > > >>> >
> > > > > > > > > > > >>> > Thank you,
> > > > > > > > > > > >>> > Justine
> > > > > > > > > > > >>> >
> > > > > > > > > > > >>> > On Wed, Sep 30, 2020 at 1:15 PM Jun Rao <
> > > > > jun@confluent.io>
> > > > > > > > > wrote:
> > > > > > > > > > > >>> >
> > > > > > > > > > > >>> > > Hi, Justine,
> > > > > > > > > > > >>> > >
> > > > > > > > > > > >>> > > Thanks for the summary. Just a few minor
> comments
> > > > blow.
> > > > > > > > > > > >>> > >
> > > > > > > > > > > >>> > > 30.  {"name": "id", "type": "string", "doc":
> > > "version
> > > > > > > id"}}:
> > > > > > > > > The
> > > > > > > > > > > doc
> > > > > > > > > > > >>> > should
> > > > > > > > > > > >>> > > say UUID. The issue seems unfixed.
> > > > > > > > > > > >>> > >
> > > > > > > > > > > >>> > > 40. Since UUID is public facing, could you
> > include
> > > > its
> > > > > > > > > > definition?
> > > > > > > > > > > >>> > >
> > > > > > > > > > > >>> > > 41. StopReplicaResponse still includes the
> topic
> > > > field.
> > > > > > > > > > > >>> > >
> > > > > > > > > > > >>> > > 42. "It is unnecessary to include the name of
> the
> > > > topic
> > > > > > in
> > > > > > > the
> > > > > > > > > > > >>> following
> > > > > > > > > > > >>> > > Request/Response calls" It would be useful to
> > > include
> > > > > all
> > > > > > > > > > modified
> > > > > > > > > > > >>> > requests
> > > > > > > > > > > >>> > > (e.g. produce) in the list below.
> > > > > > > > > > > >>> > >
> > > > > > > > > > > >>> > > Thanks,
> > > > > > > > > > > >>> > >
> > > > > > > > > > > >>> > > Jun
> > > > > > > > > > > >>> > >
> > > > > > > > > > > >>> > >
> > > > > > > > > > > >>> > > On Wed, Sep 30, 2020 at 10:17 AM Justine
> Olshan <
> > > > > > > > > > > >>> jolshan@confluent.io>
> > > > > > > > > > > >>> > > wrote:
> > > > > > > > > > > >>> > >
> > > > > > > > > > > >>> > > > Hello again,
> > > > > > > > > > > >>> > > >
> > > > > > > > > > > >>> > > > I've taken some time to discuss some of the
> > > > remaining
> > > > > > > points
> > > > > > > > > > > >>> brought up
> > > > > > > > > > > >>> > > by
> > > > > > > > > > > >>> > > > the previous emails offline. Here are some of
> > the
> > > > > > > > > conclusions.
> > > > > > > > > > > >>> > > >
> > > > > > > > > > > >>> > > > 1. Directory Structure:
> > > > > > > > > > > >>> > > > There was some talk about whether the
> directory
> > > > > > structure
> > > > > > > > > > should
> > > > > > > > > > > be
> > > > > > > > > > > >>> > > changed
> > > > > > > > > > > >>> > > > to replace all topic names with topic IDs.
> > > > > > > > > > > >>> > > > This will be a useful change, but will
> prevent
> > > > > > > downgrades. It
> > > > > > > > > > > will
> > > > > > > > > > > >>> be
> > > > > > > > > > > >>> > > best
> > > > > > > > > > > >>> > > > to wait until a major release, likely
> alongside
> > > > > KIP-500
> > > > > > > > > changes
> > > > > > > > > > > >>> that
> > > > > > > > > > > >>> > will
> > > > > > > > > > > >>> > > > prevent downgrades. I've updated the KIP to
> > > include
> > > > > > this
> > > > > > > > > change
> > > > > > > > > > > >>> with
> > > > > > > > > > > >>> > the
> > > > > > > > > > > >>> > > > note about migration and deprecation.
> > > > > > > > > > > >>> > > >
> > > > > > > > > > > >>> > > >
> > > > > > > > > > > >>> > >
> > > > > > > > > > > >>> >
> > > > > > > > > > > >>>
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-log.dirlayout
> > > > > > > > > > > >>> > > >
> > > > > > > > > > > >>> > > > 2. Partition Metadata file
> > > > > > > > > > > >>> > > > There was some disagreement about the
> inclusion
> > > of
> > > > > the
> > > > > > > > > > partition
> > > > > > > > > > > >>> > metadata
> > > > > > > > > > > >>> > > > file.
> > > > > > > > > > > >>> > > > This file will be needed to persist the topic
> > ID,
> > > > > > > especially
> > > > > > > > > > > while
> > > > > > > > > > > >>> we
> > > > > > > > > > > >>> > > still
> > > > > > > > > > > >>> > > > have the old directory structure. Even after
> > the
> > > > > > changes,
> > > > > > > > > this
> > > > > > > > > > > >>> file can
> > > > > > > > > > > >>> > > be
> > > > > > > > > > > >>> > > > useful for debugging and recovery.
> > > > > > > > > > > >>> > > > Creating a single mapping file from topic
> names
> > > to
> > > > > > topic
> > > > > > > IDs
> > > > > > > > > > was
> > > > > > > > > > > >>> > > > considered, but ultimately scrapped as it
> would
> > > not
> > > > > be
> > > > > > as
> > > > > > > > > easy
> > > > > > > > > > to
> > > > > > > > > > > >>> > > maintain.
> > > > > > > > > > > >>> > > >
> > > > > > > > > > > >>> > > >
> > > > > > > > > > > >>> > >
> > > > > > > > > > > >>> >
> > > > > > > > > > > >>>
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PartitionMetadatafile
> > > > > > > > > > > >>> > > >
> > > > > > > > > > > >>> > > >
> > > > > > > > > > > >>> > > >
> > > > > > > > > > > >>> > >
> > > > > > > > > > > >>> >
> > > > > > > > > > > >>>
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PersistingTopicIDs
> > > > > > > > > > > >>> > > >
> > > > > > > > > > > >>> > > > 3. Produce Protocols
> > > > > > > > > > > >>> > > > After some further discussion, this replacing
> > the
> > > > > topic
> > > > > > > name
> > > > > > > > > > with
> > > > > > > > > > > >>> topic
> > > > > > > > > > > >>> > > ID
> > > > > > > > > > > >>> > > > in these protocols has been added to the KIP.
> > > > > > > > > > > >>> > > > This will cut down on the size of the
> protocol.
> > > > Since
> > > > > > > changes
> > > > > > > > > > to
> > > > > > > > > > > >>> fetch
> > > > > > > > > > > >>> > > are
> > > > > > > > > > > >>> > > > included, it does make sense to update these
> > > > > protocols.
> > > > > > > > > > > >>> > > >
> > > > > > > > > > > >>> > > >
> > > > > > > > > > > >>> > >
> > > > > > > > > > > >>> >
> > > > > > > > > > > >>>
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-Produce
> > > > > > > > > > > >>> > > >
> > > > > > > > > > > >>> > > > 4. KIP-500 Compatibility
> > > > > > > > > > > >>> > > > After some discussion with those involved
> with
> > > > > KIP-500,
> > > > > > > it
> > > > > > > > > > seems
> > > > > > > > > > > >>> best
> > > > > > > > > > > >>> > to
> > > > > > > > > > > >>> > > > use a sentinel topic ID for the metadata
> topic
> > > that
> > > > > is
> > > > > > > used
> > > > > > > > > > > before
> > > > > > > > > > > >>> the
> > > > > > > > > > > >>> > > > first controller election.
> > > > > > > > > > > >>> > > > We had an issue where this metadata topic may
> > not
> > > > be
> > > > > > > assigned
> > > > > > > > > > an
> > > > > > > > > > > ID
> > > > > > > > > > > >>> > > before
> > > > > > > > > > > >>> > > > utilizing the Vote and Fetch protocols. It
> was
> > > > > decided
> > > > > > to
> > > > > > > > > > > reserve a
> > > > > > > > > > > >>> > > unique
> > > > > > > > > > > >>> > > > ID for this topic to be used until the
> > controller
> > > > > could
> > > > > > > give
> > > > > > > > > > the
> > > > > > > > > > > >>> topic
> > > > > > > > > > > >>> > a
> > > > > > > > > > > >>> > > > unique ID.
> > > > > > > > > > > >>> > > > Having the topic keep the sentinel ID (not be
> > > > > > reassigned
> > > > > > > to a
> > > > > > > > > > > >>> unique
> > > > > > > > > > > >>> > ID)
> > > > > > > > > > > >>> > > > was considered, but it seemed like a good
> idea
> > to
> > > > > leave
> > > > > > > the
> > > > > > > > > > > option
> > > > > > > > > > > >>> open
> > > > > > > > > > > >>> > > for
> > > > > > > > > > > >>> > > > the metadata topic to have a unique ID in
> cases
> > > > where
> > > > > > it
> > > > > > > > > would
> > > > > > > > > > > >>> need to
> > > > > > > > > > > >>> > be
> > > > > > > > > > > >>> > > > differentiated from other clusters' metadata
> > > > topics.
> > > > > > (ex.
> > > > > > > > > > tiered
> > > > > > > > > > > >>> > storage)
> > > > > > > > > > > >>> > > >
> > > > > > > > > > > >>> > > >
> > > > > > > > > > > >>> > >
> > > > > > > > > > > >>> >
> > > > > > > > > > > >>>
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-CompatibilitywithKIP-500
> > > > > > > > > > > >>> > > >
> > > > > > > > > > > >>> > > > I've also split up the KIP into sub-tasks on
> > the
> > > > > JIRA.
> > > > > > > > > > Hopefully
> > > > > > > > > > > >>> this
> > > > > > > > > > > >>> > > will
> > > > > > > > > > > >>> > > > provide a better idea about what tasks we
> have,
> > > and
> > > > > > > > > eventually
> > > > > > > > > > > >>> provide
> > > > > > > > > > > >>> > a
> > > > > > > > > > > >>> > > > place to see what's done and what is left.
> > > > > > > > > > > >>> > > > If there is a task I am missing, please let
> me
> > > > know!
> > > > > > > > > > > >>> > > >
> > https://issues.apache.org/jira/browse/KAFKA-8872
> > > > > > > > > > > >>> > > >
> > > > > > > > > > > >>> > > > Of course, these decisions are not set in
> > stone,
> > > > and
> > > > > I
> > > > > > > would
> > > > > > > > > > love
> > > > > > > > > > > >>> to
> > > > > > > > > > > >>> > hear
> > > > > > > > > > > >>> > > > any feedback.
> > > > > > > > > > > >>> > > >
> > > > > > > > > > > >>> > > > Thanks,
> > > > > > > > > > > >>> > > > Justine
> > > > > > > > > > > >>> > > >
> > > > > > > > > > > >>> > > > On Mon, Sep 28, 2020 at 11:38 AM Justine
> > Olshan <
> > > > > > > > > > > >>> jolshan@confluent.io>
> > > > > > > > > > > >>> > > > wrote:
> > > > > > > > > > > >>> > > >
> > > > > > > > > > > >>> > > > > Hello all,
> > > > > > > > > > > >>> > > > >
> > > > > > > > > > > >>> > > > > I just wanted to follow up on this
> > discussion.
> > > > Did
> > > > > we
> > > > > > > come
> > > > > > > > > to
> > > > > > > > > > > an
> > > > > > > > > > > >>> > > > > understanding about the directory
> structure?
> > > > > > > > > > > >>> > > > >
> > > > > > > > > > > >>> > > > > I think the biggest question here is what
> is
> > > > > > > acceptable to
> > > > > > > > > > > leave
> > > > > > > > > > > >>> out
> > > > > > > > > > > >>> > > due
> > > > > > > > > > > >>> > > > > to scope vs. what is considered to be too
> > much
> > > > tech
> > > > > > > debt.
> > > > > > > > > > > >>> > > > > This KIP is already pretty large with the
> > > number
> > > > of
> > > > > > > > > changes,
> > > > > > > > > > > but
> > > > > > > > > > > >>> it
> > > > > > > > > > > >>> > > also
> > > > > > > > > > > >>> > > > > makes sense to do things correctly, so I'd
> > love
> > > > to
> > > > > > hear
> > > > > > > > > > > >>> everyone's
> > > > > > > > > > > >>> > > > thoughts.
> > > > > > > > > > > >>> > > > >
> > > > > > > > > > > >>> > > > > Thanks,
> > > > > > > > > > > >>> > > > > Justine
> > > > > > > > > > > >>> > > > >
> > > > > > > > > > > >>> > > > > On Fri, Sep 25, 2020 at 8:19 AM Lucas
> > > Bradstreet
> > > > <
> > > > > > > > > > > >>> lucas@confluent.io
> > > > > > > > > > > >>> > >
> > > > > > > > > > > >>> > > > > wrote:
> > > > > > > > > > > >>> > > > >
> > > > > > > > > > > >>> > > > >> Hi Ismael,
> > > > > > > > > > > >>> > > > >>
> > > > > > > > > > > >>> > > > >> If you do not store it in a metadata file
> or
> > > in
> > > > > the
> > > > > > > > > > directory
> > > > > > > > > > > >>> > > structure
> > > > > > > > > > > >>> > > > >> would you then
> > > > > > > > > > > >>> > > > >> require the LeaderAndIsrRequest following
> > > > startup
> > > > > to
> > > > > > > give
> > > > > > > > > > you
> > > > > > > > > > > >>> some
> > > > > > > > > > > >>> > > > notion
> > > > > > > > > > > >>> > > > >> of
> > > > > > > > > > > >>> > > > >> topic name in memory? We would still need
> > this
> > > > > > > information
> > > > > > > > > > for
> > > > > > > > > > > >>> the
> > > > > > > > > > > >>> > > older
> > > > > > > > > > > >>> > > > >> protocols, but
> > > > > > > > > > > >>> > > > >> perhaps this is what's meant by tech debt.
> > > > > > > > > > > >>> > > > >>
> > > > > > > > > > > >>> > > > >> Once we're free of the old non-topicID
> > > requests
> > > > > > then I
> > > > > > > > > think
> > > > > > > > > > > you
> > > > > > > > > > > >>> > > > wouldn't
> > > > > > > > > > > >>> > > > >> need to retain the topic name.
> > > > > > > > > > > >>> > > > >> I think the ability to easily look up
> topic
> > > > names
> > > > > > > > > associated
> > > > > > > > > > > >>> with
> > > > > > > > > > > >>> > > > >> partition
> > > > > > > > > > > >>> > > > >> directories would still be missed
> > > > > > > > > > > >>> > > > >> when diagnosing issues, though maybe it
> > > wouldn't
> > > > > be
> > > > > > a
> > > > > > > deal
> > > > > > > > > > > >>> breaker
> > > > > > > > > > > >>> > > with
> > > > > > > > > > > >>> > > > >> the
> > > > > > > > > > > >>> > > > >> right tooling.
> > > > > > > > > > > >>> > > > >>
> > > > > > > > > > > >>> > > > >> Thanks,
> > > > > > > > > > > >>> > > > >>
> > > > > > > > > > > >>> > > > >> Lucas
> > > > > > > > > > > >>> > > > >>
> > > > > > > > > > > >>> > > > >> On Fri, Sep 25, 2020 at 7:55 AM Ismael
> Juma
> > <
> > > > > > > > > > > ismael@juma.me.uk>
> > > > > > > > > > > >>> > > wrote:
> > > > > > > > > > > >>> > > > >>
> > > > > > > > > > > >>> > > > >> > Hi Lucas,
> > > > > > > > > > > >>> > > > >> >
> > > > > > > > > > > >>> > > > >> > Why would you include the name and id? I
> > > think
> > > > > > you'd
> > > > > > > > > want
> > > > > > > > > > to
> > > > > > > > > > > >>> > remove
> > > > > > > > > > > >>> > > > the
> > > > > > > > > > > >>> > > > >> > name from the directory name right?
> > Jason's
> > > > > > > suggestion
> > > > > > > > > was
> > > > > > > > > > > >>> that if
> > > > > > > > > > > >>> > > you
> > > > > > > > > > > >>> > > > >> > remove the name from the directory, then
> > why
> > > > > would
> > > > > > > you
> > > > > > > > > > need
> > > > > > > > > > > >>> the id
> > > > > > > > > > > >>> > > > name
> > > > > > > > > > > >>> > > > >> > mapping file?
> > > > > > > > > > > >>> > > > >> >
> > > > > > > > > > > >>> > > > >> > Ismael
> > > > > > > > > > > >>> > > > >> >
> > > > > > > > > > > >>> > > > >> > On Thu, Sep 24, 2020 at 4:24 PM Lucas
> > > > > Bradstreet <
> > > > > > > > > > > >>> > > lucas@confluent.io>
> > > > > > > > > > > >>> > > > >> > wrote:
> > > > > > > > > > > >>> > > > >> >
> > > > > > > > > > > >>> > > > >> > > > 2. Part of the usage of the file is
> to
> > > > have
> > > > > > > > > persistent
> > > > > > > > > > > >>> storage
> > > > > > > > > > > >>> > > of
> > > > > > > > > > > >>> > > > >> the
> > > > > > > > > > > >>> > > > >> > > topic
> > > > > > > > > > > >>> > > > >> > > ID and use it to compare with the ID
> > > > supplied
> > > > > in
> > > > > > > the
> > > > > > > > > > > >>> > LeaderAndIsr
> > > > > > > > > > > >>> > > > >> > Request.
> > > > > > > > > > > >>> > > > >> > > There is some discussion in the KIP
> > about
> > > > > > changes
> > > > > > > to
> > > > > > > > > the
> > > > > > > > > > > >>> > directory
> > > > > > > > > > > >>> > > > >> > > structure, but I believe directory
> > changes
> > > > > were
> > > > > > > > > > considered
> > > > > > > > > > > >>> to be
> > > > > > > > > > > >>> > > out
> > > > > > > > > > > >>> > > > >> of
> > > > > > > > > > > >>> > > > >> > > scope when the KIP was written.
> > > > > > > > > > > >>> > > > >> > >
> > > > > > > > > > > >>> > > > >> > >
> > > > > > > > > > > >>> > > > >> > > Yeah, I was hoping to get a better
> > > > > understanding
> > > > > > > of
> > > > > > > > > why
> > > > > > > > > > it
> > > > > > > > > > > >>> was
> > > > > > > > > > > >>> > > taken
> > > > > > > > > > > >>> > > > >> out
> > > > > > > > > > > >>> > > > >> > of
> > > > > > > > > > > >>> > > > >> > > scope. Perhaps Lucas Bradstreet might
> > have
> > > > > more
> > > > > > > > > insight
> > > > > > > > > > > >>> about
> > > > > > > > > > > >>> > the
> > > > > > > > > > > >>> > > > >> > decision.
> > > > > > > > > > > >>> > > > >> > > Basically my point is that we have to
> > > create
> > > > > > > > > additional
> > > > > > > > > > > >>> > > > infrastructure
> > > > > > > > > > > >>> > > > >> > here
> > > > > > > > > > > >>> > > > >> > > to support the name/id mapping, so I
> > > wanted
> > > > to
> > > > > > > > > > understand
> > > > > > > > > > > >>> if we
> > > > > > > > > > > >>> > > just
> > > > > > > > > > > >>> > > > >> > > consider this a sort of tech debt. It
> > > would
> > > > be
> > > > > > > useful
> > > > > > > > > to
> > > > > > > > > > > >>> cover
> > > > > > > > > > > >>> > how
> > > > > > > > > > > >>> > > > we
> > > > > > > > > > > >>> > > > >> > > handle the case when this file gets
> > > > corrupted.
> > > > > > > Seems
> > > > > > > > > > like
> > > > > > > > > > > we
> > > > > > > > > > > >>> > just
> > > > > > > > > > > >>> > > > >> have to
> > > > > > > > > > > >>> > > > >> > > trust that it matches whatever the
> > > > controller
> > > > > > > tells us
> > > > > > > > > > and
> > > > > > > > > > > >>> > rewrite
> > > > > > > > > > > >>> > > > it?
> > > > > > > > > > > >>> > > > >> > >
> > > > > > > > > > > >>> > > > >> > >
> > > > > > > > > > > >>> > > > >> > > Hi Jason, Justine,
> > > > > > > > > > > >>> > > > >> > >
> > > > > > > > > > > >>> > > > >> > > My thought process is that we will
> > likely
> > > > need
> > > > > > the
> > > > > > > > > > > metadata
> > > > > > > > > > > >>> file
> > > > > > > > > > > >>> > > > >> whether
> > > > > > > > > > > >>> > > > >> > we
> > > > > > > > > > > >>> > > > >> > > rename the directories or not.
> > > > > > > > > > > >>> > > > >> > > Linux supports filenames of up to 255
> > > bytes
> > > > > and
> > > > > > > that
> > > > > > > > > > would
> > > > > > > > > > > >>> not
> > > > > > > > > > > >>> > be
> > > > > > > > > > > >>> > > > >> enough
> > > > > > > > > > > >>> > > > >> > to
> > > > > > > > > > > >>> > > > >> > > support a directory name
> > > > > > > > > > > >>> > > > >> > >  including both the name and topic ID.
> > > Given
> > > > > > that
> > > > > > > > > fact,
> > > > > > > > > > it
> > > > > > > > > > > >>> > seemed
> > > > > > > > > > > >>> > > > >> > > reasonable to add the metadata file
> > > > > > > > > > > >>> > > > >> > > and leave the directory rename until
> > some
> > > > time
> > > > > > in
> > > > > > > the
> > > > > > > > > > > future
> > > > > > > > > > > >>> > > > (possibly
> > > > > > > > > > > >>> > > > >> > > never).
> > > > > > > > > > > >>> > > > >> > >
> > > > > > > > > > > >>> > > > >> > > If the file does get corrupted I think
> > > > you're
> > > > > > > right.
> > > > > > > > > We
> > > > > > > > > > > >>> would
> > > > > > > > > > > >>> > > either
> > > > > > > > > > > >>> > > > >> have
> > > > > > > > > > > >>> > > > >> > > to trust it matches what the
> controller
> > > > tells
> > > > > us
> > > > > > > > > > > >>> > > > >> > >  or error out and let an administrator
> > > > resolve
> > > > > > it
> > > > > > > by
> > > > > > > > > > > >>> checking
> > > > > > > > > > > >>> > > across
> > > > > > > > > > > >>> > > > >> > > replicas for consistency.
> > > > > > > > > > > >>> > > > >> > >
> > > > > > > > > > > >>> > > > >> > > Lucas
> > > > > > > > > > > >>> > > > >> > >
> > > > > > > > > > > >>> > > > >> > >
> > > > > > > > > > > >>> > > > >> > > On Thu, Sep 24, 2020 at 3:41 PM Jason
> > > > > Gustafson
> > > > > > <
> > > > > > > > > > > >>> > > jason@confluent.io
> > > > > > > > > > > >>> > > > >
> > > > > > > > > > > >>> > > > >> > > wrote:
> > > > > > > > > > > >>> > > > >> > >
> > > > > > > > > > > >>> > > > >> > > > Thanks Justine. Responses below:
> > > > > > > > > > > >>> > > > >> > > >
> > > > > > > > > > > >>> > > > >> > > > > 1. Yes, the directory will still
> be
> > > > based
> > > > > on
> > > > > > > the
> > > > > > > > > > topic
> > > > > > > > > > > >>> > names.
> > > > > > > > > > > >>> > > > >> > > > LeaderAndIsrRequest is one of the
> few
> > > > > requests
> > > > > > > that
> > > > > > > > > > will
> > > > > > > > > > > >>> still
> > > > > > > > > > > >>> > > > >> contain
> > > > > > > > > > > >>> > > > >> > > the
> > > > > > > > > > > >>> > > > >> > > > topic name. So I think we have this
> > > > covered.
> > > > > > > Sorry
> > > > > > > > > for
> > > > > > > > > > > >>> > > confusion.
> > > > > > > > > > > >>> > > > >> > > >
> > > > > > > > > > > >>> > > > >> > > > Ah, you're right. My eyes passed
> right
> > > > over
> > > > > > the
> > > > > > > > > field.
> > > > > > > > > > > >>> > > > >> > > >
> > > > > > > > > > > >>> > > > >> > > > > 2. Part of the usage of the file
> is
> > to
> > > > > have
> > > > > > > > > > persistent
> > > > > > > > > > > >>> > storage
> > > > > > > > > > > >>> > > > of
> > > > > > > > > > > >>> > > > >> the
> > > > > > > > > > > >>> > > > >> > > > topic
> > > > > > > > > > > >>> > > > >> > > > ID and use it to compare with the ID
> > > > > supplied
> > > > > > > in the
> > > > > > > > > > > >>> > > LeaderAndIsr
> > > > > > > > > > > >>> > > > >> > > Request.
> > > > > > > > > > > >>> > > > >> > > > There is some discussion in the KIP
> > > about
> > > > > > > changes to
> > > > > > > > > > the
> > > > > > > > > > > >>> > > directory
> > > > > > > > > > > >>> > > > >> > > > structure, but I believe directory
> > > changes
> > > > > > were
> > > > > > > > > > > >>> considered to
> > > > > > > > > > > >>> > be
> > > > > > > > > > > >>> > > > >> out of
> > > > > > > > > > > >>> > > > >> > > > scope when the KIP was written.
> > > > > > > > > > > >>> > > > >> > > >
> > > > > > > > > > > >>> > > > >> > > > Yeah, I was hoping to get a better
> > > > > > > understanding of
> > > > > > > > > > why
> > > > > > > > > > > >>> it was
> > > > > > > > > > > >>> > > > taken
> > > > > > > > > > > >>> > > > >> > out
> > > > > > > > > > > >>> > > > >> > > of
> > > > > > > > > > > >>> > > > >> > > > scope. Perhaps Lucas Bradstreet
> might
> > > have
> > > > > > more
> > > > > > > > > > insight
> > > > > > > > > > > >>> about
> > > > > > > > > > > >>> > > the
> > > > > > > > > > > >>> > > > >> > > decision.
> > > > > > > > > > > >>> > > > >> > > > Basically my point is that we have
> to
> > > > create
> > > > > > > > > > additional
> > > > > > > > > > > >>> > > > >> infrastructure
> > > > > > > > > > > >>> > > > >> > > here
> > > > > > > > > > > >>> > > > >> > > > to support the name/id mapping, so I
> > > > wanted
> > > > > to
> > > > > > > > > > > understand
> > > > > > > > > > > >>> if
> > > > > > > > > > > >>> > we
> > > > > > > > > > > >>> > > > just
> > > > > > > > > > > >>> > > > >> > > > consider this a sort of tech debt.
> It
> > > > would
> > > > > be
> > > > > > > > > useful
> > > > > > > > > > to
> > > > > > > > > > > >>> cover
> > > > > > > > > > > >>> > > how
> > > > > > > > > > > >>> > > > >> we
> > > > > > > > > > > >>> > > > >> > > > handle the case when this file gets
> > > > > corrupted.
> > > > > > > Seems
> > > > > > > > > > > like
> > > > > > > > > > > >>> we
> > > > > > > > > > > >>> > > just
> > > > > > > > > > > >>> > > > >> have
> > > > > > > > > > > >>> > > > >> > to
> > > > > > > > > > > >>> > > > >> > > > trust that it matches whatever the
> > > > > controller
> > > > > > > tells
> > > > > > > > > us
> > > > > > > > > > > and
> > > > > > > > > > > >>> > > rewrite
> > > > > > > > > > > >>> > > > >> it?
> > > > > > > > > > > >>> > > > >> > > >
> > > > > > > > > > > >>> > > > >> > > > > 3. I think this is a good point,
> > but I
> > > > > > again I
> > > > > > > > > > wonder
> > > > > > > > > > > >>> about
> > > > > > > > > > > >>> > > the
> > > > > > > > > > > >>> > > > >> scope
> > > > > > > > > > > >>> > > > >> > > of
> > > > > > > > > > > >>> > > > >> > > > the KIP. Most of the changes
> mentioned
> > > in
> > > > > the
> > > > > > > KIP
> > > > > > > > > are
> > > > > > > > > > > for
> > > > > > > > > > > >>> > > > supporting
> > > > > > > > > > > >>> > > > >> > > topic
> > > > > > > > > > > >>> > > > >> > > > deletion and I believe that is why
> the
> > > > > produce
> > > > > > > > > request
> > > > > > > > > > > was
> > > > > > > > > > > >>> > > listed
> > > > > > > > > > > >>> > > > >> under
> > > > > > > > > > > >>> > > > >> > > > future work.
> > > > > > > > > > > >>> > > > >> > > >
> > > > > > > > > > > >>> > > > >> > > > That's fair. I brought it up since
> > > `Fetch`
> > > > > is
> > > > > > > > > already
> > > > > > > > > > > >>> > included.
> > > > > > > > > > > >>> > > If
> > > > > > > > > > > >>> > > > >> > we've
> > > > > > > > > > > >>> > > > >> > > > got `Metadata` and `Fetch`, seems we
> > may
> > > > as
> > > > > > > well do
> > > > > > > > > > > >>> `Produce`
> > > > > > > > > > > >>> > > and
> > > > > > > > > > > >>> > > > >> save
> > > > > > > > > > > >>> > > > >> > an
> > > > > > > > > > > >>> > > > >> > > > extra kip. No strong objection
> though
> > if
> > > > you
> > > > > > > want to
> > > > > > > > > > > >>> leave it
> > > > > > > > > > > >>> > > out.
> > > > > > > > > > > >>> > > > >> > > >
> > > > > > > > > > > >>> > > > >> > > >
> > > > > > > > > > > >>> > > > >> > > > -Jason
> > > > > > > > > > > >>> > > > >> > > >
> > > > > > > > > > > >>> > > > >> > > >
> > > > > > > > > > > >>> > > > >> > > > On Thu, Sep 24, 2020 at 3:26 PM
> > Justine
> > > > > > Olshan <
> > > > > > > > > > > >>> > > > >> jolshan@confluent.io>
> > > > > > > > > > > >>> > > > >> > > > wrote:
> > > > > > > > > > > >>> > > > >> > > >
> > > > > > > > > > > >>> > > > >> > > > > Hi Jason,
> > > > > > > > > > > >>> > > > >> > > > >
> > > > > > > > > > > >>> > > > >> > > > > Thanks for your comments.
> > > > > > > > > > > >>> > > > >> > > > >
> > > > > > > > > > > >>> > > > >> > > > > 1. Yes, the directory will still
> be
> > > > based
> > > > > on
> > > > > > > the
> > > > > > > > > > topic
> > > > > > > > > > > >>> > names.
> > > > > > > > > > > >>> > > > >> > > > > LeaderAndIsrRequest is one of the
> > few
> > > > > > requests
> > > > > > > > > that
> > > > > > > > > > > will
> > > > > > > > > > > >>> > still
> > > > > > > > > > > >>> > > > >> > contain
> > > > > > > > > > > >>> > > > >> > > > the
> > > > > > > > > > > >>> > > > >> > > > > topic name. So I think we have
> this
> > > > > covered.
> > > > > > > Sorry
> > > > > > > > > > for
> > > > > > > > > > > >>> > > > confusion.
> > > > > > > > > > > >>> > > > >> > > > >
> > > > > > > > > > > >>> > > > >> > > > > 2. Part of the usage of the file
> is
> > to
> > > > > have
> > > > > > > > > > persistent
> > > > > > > > > > > >>> > storage
> > > > > > > > > > > >>> > > > of
> > > > > > > > > > > >>> > > > >> the
> > > > > > > > > > > >>> > > > >> > > > topic
> > > > > > > > > > > >>> > > > >> > > > > ID and use it to compare with the
> ID
> > > > > > supplied
> > > > > > > in
> > > > > > > > > the
> > > > > > > > > > > >>> > > > LeaderAndIsr
> > > > > > > > > > > >>> > > > >> > > > Request.
> > > > > > > > > > > >>> > > > >> > > > > There is some discussion in the
> KIP
> > > > about
> > > > > > > changes
> > > > > > > > > to
> > > > > > > > > > > the
> > > > > > > > > > > >>> > > > directory
> > > > > > > > > > > >>> > > > >> > > > > structure, but I believe directory
> > > > changes
> > > > > > > were
> > > > > > > > > > > >>> considered
> > > > > > > > > > > >>> > to
> > > > > > > > > > > >>> > > be
> > > > > > > > > > > >>> > > > >> out
> > > > > > > > > > > >>> > > > >> > of
> > > > > > > > > > > >>> > > > >> > > > > scope when the KIP was written.
> > > > > > > > > > > >>> > > > >> > > > >
> > > > > > > > > > > >>> > > > >> > > > > 3. I think this is a good point,
> > but I
> > > > > > again I
> > > > > > > > > > wonder
> > > > > > > > > > > >>> about
> > > > > > > > > > > >>> > > the
> > > > > > > > > > > >>> > > > >> scope
> > > > > > > > > > > >>> > > > >> > > of
> > > > > > > > > > > >>> > > > >> > > > > the KIP. Most of the changes
> > mentioned
> > > > in
> > > > > > the
> > > > > > > KIP
> > > > > > > > > > are
> > > > > > > > > > > >>> for
> > > > > > > > > > > >>> > > > >> supporting
> > > > > > > > > > > >>> > > > >> > > > topic
> > > > > > > > > > > >>> > > > >> > > > > deletion and I believe that is why
> > the
> > > > > > produce
> > > > > > > > > > request
> > > > > > > > > > > >>> was
> > > > > > > > > > > >>> > > > listed
> > > > > > > > > > > >>> > > > >> > under
> > > > > > > > > > > >>> > > > >> > > > > future work.
> > > > > > > > > > > >>> > > > >> > > > >
> > > > > > > > > > > >>> > > > >> > > > > 4. This sounds like it might be a
> > good
> > > > > > > solution,
> > > > > > > > > > but I
> > > > > > > > > > > >>> will
> > > > > > > > > > > >>> > > need
> > > > > > > > > > > >>> > > > >> to
> > > > > > > > > > > >>> > > > >> > > > discuss
> > > > > > > > > > > >>> > > > >> > > > > more with KIP-500 folks to get the
> > > > details
> > > > > > > right.
> > > > > > > > > > > >>> > > > >> > > > >
> > > > > > > > > > > >>> > > > >> > > > > Thanks,
> > > > > > > > > > > >>> > > > >> > > > > Justine
> > > > > > > > > > > >>> > > > >> > > > >
> > > > > > > > > > > >>> > > > >> > > > > On Thu, Sep 24, 2020 at 12:30 PM
> > Jason
> > > > > > > Gustafson <
> > > > > > > > > > > >>> > > > >> jason@confluent.io
> > > > > > > > > > > >>> > > > >> > >
> > > > > > > > > > > >>> > > > >> > > > > wrote:
> > > > > > > > > > > >>> > > > >> > > > >
> > > > > > > > > > > >>> > > > >> > > > > > Hi Justine,
> > > > > > > > > > > >>> > > > >> > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > Thanks for picking up this
> work. I
> > > > have
> > > > > a
> > > > > > > few
> > > > > > > > > > > >>> > > > >> questions/comments:
> > > > > > > > > > > >>> > > > >> > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > 1. It sounds like the directory
> > > > > structure
> > > > > > is
> > > > > > > > > still
> > > > > > > > > > > >>> going
> > > > > > > > > > > >>> > to
> > > > > > > > > > > >>> > > be
> > > > > > > > > > > >>> > > > >> > based
> > > > > > > > > > > >>> > > > >> > > on
> > > > > > > > > > > >>> > > > >> > > > > > topic names. Do I have that
> right?
> > > One
> > > > > > > > > > complication
> > > > > > > > > > > is
> > > > > > > > > > > >>> > that
> > > > > > > > > > > >>> > > > the
> > > > > > > > > > > >>> > > > >> > > > > > LeaderAndIsr request does not
> > > include
> > > > > the
> > > > > > > topic
> > > > > > > > > > name
> > > > > > > > > > > >>> any
> > > > > > > > > > > >>> > > > longer.
> > > > > > > > > > > >>> > > > >> > This
> > > > > > > > > > > >>> > > > >> > > > > means
> > > > > > > > > > > >>> > > > >> > > > > > that a replica must wait for the
> > > > > > > UpdateMetadata
> > > > > > > > > > > >>> request to
> > > > > > > > > > > >>> > > > >> arrive
> > > > > > > > > > > >>> > > > >> > > with
> > > > > > > > > > > >>> > > > >> > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > topic name mapping before it can
> > > > create
> > > > > > the
> > > > > > > > > > > >>> directory. I
> > > > > > > > > > > >>> > > > wonder
> > > > > > > > > > > >>> > > > >> if
> > > > > > > > > > > >>> > > > >> > it
> > > > > > > > > > > >>> > > > >> > > > > would
> > > > > > > > > > > >>> > > > >> > > > > > be simpler to avoid assumptions
> on
> > > the
> > > > > > > ordering
> > > > > > > > > of
> > > > > > > > > > > >>> > > > >> UpdateMetadata
> > > > > > > > > > > >>> > > > >> > and
> > > > > > > > > > > >>> > > > >> > > > let
> > > > > > > > > > > >>> > > > >> > > > > > LeaderAndIsr include the topic
> > name
> > > as
> > > > > > well.
> > > > > > > > > Feels
> > > > > > > > > > > >>> like we
> > > > > > > > > > > >>> > > are
> > > > > > > > > > > >>> > > > >> not
> > > > > > > > > > > >>> > > > >> > > > saving
> > > > > > > > > > > >>> > > > >> > > > > > that much by excluding it.
> > > > > > > > > > > >>> > > > >> > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > 2. On a related note, it seems
> > that
> > > > the
> > > > > > > reason
> > > > > > > > > we
> > > > > > > > > > > >>> have the
> > > > > > > > > > > >>> > > > >> > partition
> > > > > > > > > > > >>> > > > >> > > > > > metadata file is because we are
> > not
> > > > > > > changing the
> > > > > > > > > > > >>> directory
> > > > > > > > > > > >>> > > > >> > structure.
> > > > > > > > > > > >>> > > > >> > > > We
> > > > > > > > > > > >>> > > > >> > > > > > need it so that we remember
> which
> > > > > > > directories
> > > > > > > > > map
> > > > > > > > > > to
> > > > > > > > > > > >>> which
> > > > > > > > > > > >>> > > > topic
> > > > > > > > > > > >>> > > > >> > id.
> > > > > > > > > > > >>> > > > >> > > I
> > > > > > > > > > > >>> > > > >> > > > > > think it would be useful to add
> > some
> > > > > > detail
> > > > > > > > > about
> > > > > > > > > > > why
> > > > > > > > > > > >>> > > changing
> > > > > > > > > > > >>> > > > >> the
> > > > > > > > > > > >>> > > > >> > > > > > directory layout was rejected.
> > > > Basically
> > > > > > > trying
> > > > > > > > > to
> > > > > > > > > > > >>> > > understand
> > > > > > > > > > > >>> > > > >> how
> > > > > > > > > > > >>> > > > >> > > much
> > > > > > > > > > > >>> > > > >> > > > > > effort we are saving by skipping
> > > this
> > > > > step
> > > > > > > if we
> > > > > > > > > > > have
> > > > > > > > > > > >>> to
> > > > > > > > > > > >>> > > > >> implement
> > > > > > > > > > > >>> > > > >> > > this
> > > > > > > > > > > >>> > > > >> > > > > > additional bookkeeping. One
> > concern
> > > I
> > > > > have
> > > > > > > for
> > > > > > > > > > > >>> example is
> > > > > > > > > > > >>> > > that
> > > > > > > > > > > >>> > > > >> the
> > > > > > > > > > > >>> > > > >> > > > > > partition metadata file gets
> > corrupt
> > > > and
> > > > > > > then
> > > > > > > > > all
> > > > > > > > > > > >>> bets are
> > > > > > > > > > > >>> > > off
> > > > > > > > > > > >>> > > > >> as
> > > > > > > > > > > >>> > > > >> > far
> > > > > > > > > > > >>> > > > >> > > > as
> > > > > > > > > > > >>> > > > >> > > > > > topic consistency.
> > > > > > > > > > > >>> > > > >> > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > 3. Is it worth adding support
> now
> > > for
> > > > > > topic
> > > > > > > ids
> > > > > > > > > in
> > > > > > > > > > > the
> > > > > > > > > > > >>> > > > `Produce`
> > > > > > > > > > > >>> > > > >> > > > request
> > > > > > > > > > > >>> > > > >> > > > > > now? Overhead is mentioned as
> one
> > of
> > > > the
> > > > > > > > > > motivations
> > > > > > > > > > > >>> and
> > > > > > > > > > > >>> > the
> > > > > > > > > > > >>> > > > >> best
> > > > > > > > > > > >>> > > > >> > > APIs
> > > > > > > > > > > >>> > > > >> > > > to
> > > > > > > > > > > >>> > > > >> > > > > > get savings from are `Produce`
> and
> > > > > > `Fetch`.
> > > > > > > > > > > >>> > > > >> > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > 4. When it comes to
> bootstrapping
> > > the
> > > > > > > metadata
> > > > > > > > > > topic
> > > > > > > > > > > >>> with
> > > > > > > > > > > >>> > > > >> respect
> > > > > > > > > > > >>> > > > >> > to
> > > > > > > > > > > >>> > > > >> > > > > > KIP-500, one suggestion would be
> > to
> > > > > > reserve
> > > > > > > a
> > > > > > > > > > > sentinel
> > > > > > > > > > > >>> > topic
> > > > > > > > > > > >>> > > > ID
> > > > > > > > > > > >>> > > > >> > which
> > > > > > > > > > > >>> > > > >> > > > can
> > > > > > > > > > > >>> > > > >> > > > > > be used initially by new
> brokers.
> > > When
> > > > > the
> > > > > > > first
> > > > > > > > > > > >>> > controller
> > > > > > > > > > > >>> > > is
> > > > > > > > > > > >>> > > > >> > > elected,
> > > > > > > > > > > >>> > > > >> > > > > it
> > > > > > > > > > > >>> > > > >> > > > > > can assign a topicId to the
> > metadata
> > > > > > topic.
> > > > > > > This
> > > > > > > > > > is
> > > > > > > > > > > a
> > > > > > > > > > > >>> bit
> > > > > > > > > > > >>> > > > >> similar
> > > > > > > > > > > >>> > > > >> > to
> > > > > > > > > > > >>> > > > >> > > > how
> > > > > > > > > > > >>> > > > >> > > > > we
> > > > > > > > > > > >>> > > > >> > > > > > were planning to generate the
> > > > clusterId.
> > > > > > > > > > > >>> > > > >> > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > Thanks,
> > > > > > > > > > > >>> > > > >> > > > > > Jason
> > > > > > > > > > > >>> > > > >> > > > > >
> > > > > > > > > > > >>> > > > >> > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > On Thu, Sep 24, 2020 at 11:10 AM
> > Jun
> > > > > Rao <
> > > > > > > > > > > >>> > jun@confluent.io>
> > > > > > > > > > > >>> > > > >> wrote:
> > > > > > > > > > > >>> > > > >> > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > Hi, Justine,
> > > > > > > > > > > >>> > > > >> > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > Thanks for the updated KIP. A
> > few
> > > > more
> > > > > > > > > comments
> > > > > > > > > > > >>> below.
> > > > > > > > > > > >>> > > > >> > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > 30.  {"name": "id", "type":
> > > > "string",
> > > > > > > "doc":
> > > > > > > > > > > >>> "version
> > > > > > > > > > > >>> > > id"}}:
> > > > > > > > > > > >>> > > > >> The
> > > > > > > > > > > >>> > > > >> > > doc
> > > > > > > > > > > >>> > > > >> > > > > > should
> > > > > > > > > > > >>> > > > >> > > > > > > say UUID.
> > > > > > > > > > > >>> > > > >> > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > 31. LeaderAndIsrResponse v5
> and
> > > > > > > > > > > StopReplicaResponse
> > > > > > > > > > > >>> v4 :
> > > > > > > > > > > >>> > > It
> > > > > > > > > > > >>> > > > >> seems
> > > > > > > > > > > >>> > > > >> > > > there
> > > > > > > > > > > >>> > > > >> > > > > > is
> > > > > > > > > > > >>> > > > >> > > > > > > no need to add topic_id at
> > > > partitions
> > > > > > > level.
> > > > > > > > > > > >>> > > > >> > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > 32. Regarding partition
> metadata
> > > > file.
> > > > > > > Perhaps
> > > > > > > > > > the
> > > > > > > > > > > >>> key
> > > > > > > > > > > >>> > can
> > > > > > > > > > > >>> > > > be
> > > > > > > > > > > >>> > > > >> a
> > > > > > > > > > > >>> > > > >> > > > single
> > > > > > > > > > > >>> > > > >> > > > > > > word, sth like the following.
> > > > > > > > > > > >>> > > > >> > > > > > > version: 0
> > > > > > > > > > > >>> > > > >> > > > > > > topic_id:
> > > > > > > 46bdb63f-9e8d-4a38-bf7b-ee4eb2a794e4
> > > > > > > > > > > >>> > > > >> > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > 33. Another tricky thing that
> I
> > > > > realized
> > > > > > > is
> > > > > > > > > how
> > > > > > > > > > to
> > > > > > > > > > > >>> > support
> > > > > > > > > > > >>> > > > the
> > > > > > > > > > > >>> > > > >> > > > metadata
> > > > > > > > > > > >>> > > > >> > > > > > > topic introduced in KIP-595.
> > It's
> > > a
> > > > > bit
> > > > > > > tricky
> > > > > > > > > > to
> > > > > > > > > > > >>> > assign a
> > > > > > > > > > > >>> > > > >> UUID
> > > > > > > > > > > >>> > > > >> > to
> > > > > > > > > > > >>> > > > >> > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > metadata topic since we have a
> > > > > > > chicken-and-egg
> > > > > > > > > > > >>> problem.
> > > > > > > > > > > >>> > > The
> > > > > > > > > > > >>> > > > >> > > > controller
> > > > > > > > > > > >>> > > > >> > > > > > > needs to persist the UUID in
> the
> > > > > > metadata
> > > > > > > > > topic
> > > > > > > > > > in
> > > > > > > > > > > >>> order
> > > > > > > > > > > >>> > > to
> > > > > > > > > > > >>> > > > >> > assign
> > > > > > > > > > > >>> > > > >> > > > one
> > > > > > > > > > > >>> > > > >> > > > > > > successfully, but the metadata
> > > topic
> > > > > is
> > > > > > > needed
> > > > > > > > > > to
> > > > > > > > > > > >>> elect
> > > > > > > > > > > >>> > a
> > > > > > > > > > > >>> > > > >> > > controller
> > > > > > > > > > > >>> > > > >> > > > > > > (KIP-631). So, this probably
> > > needs a
> > > > > bit
> > > > > > > more
> > > > > > > > > > > >>> thought.
> > > > > > > > > > > >>> > > > >> > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > Jun
> > > > > > > > > > > >>> > > > >> > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > On Thu, Sep 24, 2020 at 3:04
> AM
> > > > Ismael
> > > > > > > Juma <
> > > > > > > > > > > >>> > > > >> ismael@juma.me.uk>
> > > > > > > > > > > >>> > > > >> > > > wrote:
> > > > > > > > > > > >>> > > > >> > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > Also, can we provide more
> > > details
> > > > on
> > > > > > > how the
> > > > > > > > > > > >>> Partition
> > > > > > > > > > > >>> > > > >> Metadata
> > > > > > > > > > > >>> > > > >> > > > file
> > > > > > > > > > > >>> > > > >> > > > > > will
> > > > > > > > > > > >>> > > > >> > > > > > > > be used?
> > > > > > > > > > > >>> > > > >> > > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > Ismael
> > > > > > > > > > > >>> > > > >> > > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > On Thu, Sep 24, 2020 at 3:01
> > AM
> > > > > Ismael
> > > > > > > Juma
> > > > > > > > > <
> > > > > > > > > > > >>> > > > >> ismael@juma.me.uk
> > > > > > > > > > > >>> > > > >> > >
> > > > > > > > > > > >>> > > > >> > > > > wrote:
> > > > > > > > > > > >>> > > > >> > > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > > Hi Justine,
> > > > > > > > > > > >>> > > > >> > > > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > > I think we need to update
> > the
> > > > > > > "Rejected
> > > > > > > > > > > >>> > Alternatives"
> > > > > > > > > > > >>> > > > >> section
> > > > > > > > > > > >>> > > > >> > > to
> > > > > > > > > > > >>> > > > >> > > > > take
> > > > > > > > > > > >>> > > > >> > > > > > > > into
> > > > > > > > > > > >>> > > > >> > > > > > > > > account that the proposal
> > now
> > > > > > removes
> > > > > > > the
> > > > > > > > > > > topic
> > > > > > > > > > > >>> name
> > > > > > > > > > > >>> > > > from
> > > > > > > > > > > >>> > > > >> the
> > > > > > > > > > > >>> > > > >> > > > fetch
> > > > > > > > > > > >>> > > > >> > > > > > > > > request. Also, if we are
> > > > removing
> > > > > it
> > > > > > > from
> > > > > > > > > > the
> > > > > > > > > > > >>> Fetch
> > > > > > > > > > > >>> > > > >> request,
> > > > > > > > > > > >>> > > > >> > > does
> > > > > > > > > > > >>> > > > >> > > > > it
> > > > > > > > > > > >>> > > > >> > > > > > > make
> > > > > > > > > > > >>> > > > >> > > > > > > > > sense not to remove it
> from
> > > > > similar
> > > > > > > > > requests
> > > > > > > > > > > >>> like
> > > > > > > > > > > >>> > > > >> > > > > ListOffsetRequest?
> > > > > > > > > > > >>> > > > >> > > > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > > Ismael
> > > > > > > > > > > >>> > > > >> > > > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > > On Thu, Sep 24, 2020 at
> 2:46
> > > AM
> > > > > > David
> > > > > > > > > Jacot
> > > > > > > > > > <
> > > > > > > > > > > >>> > > > >> > > djacot@confluent.io
> > > > > > > > > > > >>> > > > >> > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > wrote:
> > > > > > > > > > > >>> > > > >> > > > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> Hi Justine,
> > > > > > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > > > > > >>> > > > >> > > > > > > > >> Thanks for the KIP. I
> > finally
> > > > had
> > > > > > > time to
> > > > > > > > > > > read
> > > > > > > > > > > >>> it
> > > > > > > > > > > >>> > :).
> > > > > > > > > > > >>> > > > It
> > > > > > > > > > > >>> > > > >> is
> > > > > > > > > > > >>> > > > >> > a
> > > > > > > > > > > >>> > > > >> > > > > great
> > > > > > > > > > > >>> > > > >> > > > > > > > >> improvement.
> > > > > > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > > > > > >>> > > > >> > > > > > > > >> I have a few
> > > > comments/questions:
> > > > > > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > > > > > >>> > > > >> > > > > > > > >> 1. It seems that the
> schema
> > > of
> > > > > the
> > > > > > > > > > > >>> > StopReplicaRequest
> > > > > > > > > > > >>> > > > is
> > > > > > > > > > > >>> > > > >> > > > slightly
> > > > > > > > > > > >>> > > > >> > > > > > > > >> outdated.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> We
> > > > > > > > > > > >>> > > > >> > > > > > > > >> did some changes as part
> of
> > > > > > KIP-570.
> > > > > > > V3
> > > > > > > > > is
> > > > > > > > > > > >>> already
> > > > > > > > > > > >>> > > > >> organized
> > > > > > > > > > > >>> > > > >> > > by
> > > > > > > > > > > >>> > > > >> > > > > > > topics.
> > > > > > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > > > > > >>> > > > >> > > > > > > > >> 2. I just want to make
> sure
> > > > that
> > > > > I
> > > > > > > > > > understand
> > > > > > > > > > > >>> the
> > > > > > > > > > > >>> > > > >> > > reconciliation
> > > > > > > > > > > >>> > > > >> > > > > > > > >> logic correctly. When
> > > > > > > > > > > >>> > > > >> > > > > > > > >> an "INCREMENTAL"
> > LeaderAndIsr
> > > > > > > Request is
> > > > > > > > > > > >>> received,
> > > > > > > > > > > >>> > > the
> > > > > > > > > > > >>> > > > >> > broker
> > > > > > > > > > > >>> > > > >> > > > will
> > > > > > > > > > > >>> > > > >> > > > > > > also
> > > > > > > > > > > >>> > > > >> > > > > > > > >> reconcile
> > > > > > > > > > > >>> > > > >> > > > > > > > >> when the local uuid does
> > not
> > > > > match
> > > > > > > the
> > > > > > > > > uuid
> > > > > > > > > > > in
> > > > > > > > > > > >>> the
> > > > > > > > > > > >>> > > > >> request,
> > > > > > > > > > > >>> > > > >> > > > right?
> > > > > > > > > > > >>> > > > >> > > > > > In
> > > > > > > > > > > >>> > > > >> > > > > > > > this
> > > > > > > > > > > >>> > > > >> > > > > > > > >> case, the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> local log is staged for
> > > > deletion.
> > > > > > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > > > > > >>> > > > >> > > > > > > > >> 3. In the documentation
> of
> > > the
> > > > `
> > > > > > > > > > > >>> > > > >> delete.stale.topic.delay.ms
> > > > > > > > > > > >>> > > > >> > `
> > > > > > > > > > > >>> > > > >> > > > > > config,
> > > > > > > > > > > >>> > > > >> > > > > > > it
> > > > > > > > > > > >>> > > > >> > > > > > > > >> says "When a
> > > > > > > > > > > >>> > > > >> > > > > > > > >> FULL LeaderAndIsrRequest
> is
> > > > > > > received..."
> > > > > > > > > > but
> > > > > > > > > > > I
> > > > > > > > > > > >>> > > suppose
> > > > > > > > > > > >>> > > > it
> > > > > > > > > > > >>> > > > >> > > > applies
> > > > > > > > > > > >>> > > > >> > > > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > both
> > > > > > > > > > > >>> > > > >> > > > > > > > >> types.
> > > > > > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > > > > > >>> > > > >> > > > > > > > >> Best,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> David
> > > > > > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > > > > > >>> > > > >> > > > > > > > >> On Thu, Sep 24, 2020 at
> > 1:40
> > > AM
> > > > > > > Justine
> > > > > > > > > > > Olshan
> > > > > > > > > > > >>> <
> > > > > > > > > > > >>> > > > >> > > > > > jolshan@confluent.io>
> > > > > > > > > > > >>> > > > >> > > > > > > > >> wrote:
> > > > > > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > Hi Jun,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > Thanks for the
> comments.
> > I
> > > > > > > apologize
> > > > > > > > > for
> > > > > > > > > > > >>> some of
> > > > > > > > > > > >>> > > the
> > > > > > > > > > > >>> > > > >> typos
> > > > > > > > > > > >>> > > > >> > > and
> > > > > > > > > > > >>> > > > >> > > > > > > > >> confusion.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > I’ve updated the KIP to
> > fix
> > > > > some
> > > > > > > of the
> > > > > > > > > > > >>> issues
> > > > > > > > > > > >>> > you
> > > > > > > > > > > >>> > > > >> > > mentioned.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > 20.2 I’ve changed the
> > type
> > > to
> > > > > > > String.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > 20.1/3 I’ve updated the
> > > > > > TopicZNode
> > > > > > > to
> > > > > > > > > fix
> > > > > > > > > > > >>> > > formatting
> > > > > > > > > > > >>> > > > >> and
> > > > > > > > > > > >>> > > > >> > > > reflect
> > > > > > > > > > > >>> > > > >> > > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> latest
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > version before this
> > change.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > 21. You are correct and
> > > I’ve
> > > > > > > removed
> > > > > > > > > this
> > > > > > > > > > > >>> line.
> > > > > > > > > > > >>> > > I’ve
> > > > > > > > > > > >>> > > > >> also
> > > > > > > > > > > >>> > > > >> > > > added
> > > > > > > > > > > >>> > > > >> > > > > a
> > > > > > > > > > > >>> > > > >> > > > > > > line
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > mentioning an IBP bump
> is
> > > > > > > necessary for
> > > > > > > > > > > >>> migration
> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > 22. I think the wording
> > was
> > > > > > > unclear but
> > > > > > > > > > > your
> > > > > > > > > > > >>> > > summary
> > > > > > > > > > > >>> > > > is
> > > > > > > > > > > >>> > > > >> > what
> > > > > > > > > > > >>> > > > >> > > > was
> > > > > > > > > > > >>> > > > >> > > > > > > > >> intended
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > by this line. I’ve
> > updated
> > > to
> > > > > > make
> > > > > > > this
> > > > > > > > > > > point
> > > > > > > > > > > >>> > more
> > > > > > > > > > > >>> > > > >> clear.
> > > > > > > > > > > >>> > > > >> > > > > “Remove
> > > > > > > > > > > >>> > > > >> > > > > > > > >> deleted
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > topics from replicas by
> > > > sending
> > > > > > > > > > > >>> > StopReplicaRequest
> > > > > > > > > > > >>> > > V3
> > > > > > > > > > > >>> > > > >> > before
> > > > > > > > > > > >>> > > > >> > > > the
> > > > > > > > > > > >>> > > > >> > > > > > IBP
> > > > > > > > > > > >>> > > > >> > > > > > > > >> bump
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > using the old logic,
> and
> > > > using
> > > > > V4
> > > > > > > and
> > > > > > > > > the
> > > > > > > > > > > new
> > > > > > > > > > > >>> > logic
> > > > > > > > > > > >>> > > > >> with
> > > > > > > > > > > >>> > > > >> > > topic
> > > > > > > > > > > >>> > > > >> > > > > IDs
> > > > > > > > > > > >>> > > > >> > > > > > > > after
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > the IBP bump.”
> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > 23. I’ve removed the
> > > > > unspecified
> > > > > > > type
> > > > > > > > > > from
> > > > > > > > > > > >>> the
> > > > > > > > > > > >>> > KIP
> > > > > > > > > > > >>> > > > and
> > > > > > > > > > > >>> > > > >> > > mention
> > > > > > > > > > > >>> > > > >> > > > > > that
> > > > > > > > > > > >>> > > > >> > > > > > > > IBP
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > will be used to handle
> > this
> > > > > > > request.
> > > > > > > > > “IBP
> > > > > > > > > > > >>> will be
> > > > > > > > > > > >>> > > > used
> > > > > > > > > > > >>> > > > >> to
> > > > > > > > > > > >>> > > > >> > > > > > determine
> > > > > > > > > > > >>> > > > >> > > > > > > > >> whether
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > this new form of the
> > > request
> > > > > will
> > > > > > > be
> > > > > > > > > > used.
> > > > > > > > > > > >>> For
> > > > > > > > > > > >>> > > older
> > > > > > > > > > > >>> > > > >> > > requests,
> > > > > > > > > > > >>> > > > >> > > > > we
> > > > > > > > > > > >>> > > > >> > > > > > > will
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > ignore this field and
> > > default
> > > > > to
> > > > > > > > > previous
> > > > > > > > > > > >>> > > behavior.”
> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > 24. I’ve fixed this
> typo.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > 25. I've added a topics
> > at
> > > a
> > > > > > higher
> > > > > > > > > level
> > > > > > > > > > > for
> > > > > > > > > > > >>> > > > >> > > > > LeaderAndIsrResponse
> > > > > > > > > > > >>> > > > >> > > > > > > v5,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > StopReplicaResponse v4.
> > > I've
> > > > > also
> > > > > > > > > changed
> > > > > > > > > > > >>> > > > >> > StopReplicaRequest
> > > > > > > > > > > >>> > > > >> > > > v4
> > > > > > > > > > > >>> > > > >> > > > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > have
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > topics at a higher
> level.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > 26. I’ve updated
> > > > > > > > > > > >>> forgotten_topics_data--added the
> > > > > > > > > > > >>> > > > >> topic ID
> > > > > > > > > > > >>> > > > >> > > and
> > > > > > > > > > > >>> > > > >> > > > > > > removed
> > > > > > > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > topic name
> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > 27. I’ve decided on
> plain
> > > > text,
> > > > > > and
> > > > > > > > > I’ve
> > > > > > > > > > > >>> added an
> > > > > > > > > > > >>> > > > >> example.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > 28. I’ve added this
> idea
> > to
> > > > > > future
> > > > > > > > > work.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > Thanks again for
> taking a
> > > > look,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > Justine
> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > On Wed, Sep 23, 2020 at
> > > 10:28
> > > > > AM
> > > > > > > Jun
> > > > > > > > > Rao
> > > > > > > > > > <
> > > > > > > > > > > >>> > > > >> > jun@confluent.io>
> > > > > > > > > > > >>> > > > >> > > > > > wrote:
> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > Hi, Justine,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > Thanks for the
> > response.
> > > > Made
> > > > > > > another
> > > > > > > > > > > >>> pass. A
> > > > > > > > > > > >>> > few
> > > > > > > > > > > >>> > > > >> more
> > > > > > > > > > > >>> > > > >> > > > > comments
> > > > > > > > > > > >>> > > > >> > > > > > > > below.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 20. znode schema:
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 20.1 It seems that
> > > {"name":
> > > > > > > > > "version",
> > > > > > > > > > > >>> "type":
> > > > > > > > > > > >>> > > > "int",
> > > > > > > > > > > >>> > > > >> > > "id":
> > > > > > > > > > > >>> > > > >> > > > > > > "UUID",
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > "doc":
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > "version id"} should
> be
> > > > > > {"name":
> > > > > > > > > > > "version",
> > > > > > > > > > > >>> > > "type":
> > > > > > > > > > > >>> > > > >> > > "int"},
> > > > > > > > > > > >>> > > > >> > > > > > > {"name":
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > "id",
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > "type": "UUID",
> "doc":
> > > > > "version
> > > > > > > id"}.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 20.2 The znode format
> > is
> > > > JSON
> > > > > > > which
> > > > > > > > > > > doesn't
> > > > > > > > > > > >>> > have
> > > > > > > > > > > >>> > > > UUID
> > > > > > > > > > > >>> > > > >> > > type.
> > > > > > > > > > > >>> > > > >> > > > So
> > > > > > > > > > > >>> > > > >> > > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> type
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > probably should be
> > > string?
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 20.3 Also, the
> existing
> > > > > format
> > > > > > > used
> > > > > > > > > > seems
> > > > > > > > > > > >>> > > outdated.
> > > > > > > > > > > >>> > > > >> It
> > > > > > > > > > > >>> > > > >> > > > should
> > > > > > > > > > > >>> > > > >> > > > > > have
> > > > > > > > > > > >>> > > > >> > > > > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > following format.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > >  Json.encodeAsBytes(Map(
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >       "version" -> 2,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >       "partitions" ->
> > > > > > > > > > > >>> > > replicaAssignmentJson.asJava,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>  "adding_replicas"
> > > ->
> > > > > > > > > > > >>> > > > >> > > > >
> addingReplicasAssignmentJson.asJava,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
> >  "removing_replicas"
> > > > ->
> > > > > > > > > > > >>> > > > >> > > > > >
> > > removingReplicasAssignmentJson.asJava
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >     ).asJava)
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >   }
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 21. Migration: The
> KIP
> > > says
> > > > > > "The
> > > > > > > > > > > migration
> > > > > > > > > > > >>> > > process
> > > > > > > > > > > >>> > > > >> can
> > > > > > > > > > > >>> > > > >> > > take
> > > > > > > > > > > >>> > > > >> > > > > > place
> > > > > > > > > > > >>> > > > >> > > > > > > > >> without
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > an inter-broker
> > protocol
> > > > > bump,
> > > > > > > as the
> > > > > > > > > > > >>> format
> > > > > > > > > > > >>> > > stored
> > > > > > > > > > > >>> > > > >> in
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
> /brokers/topics/[topic]
> > > > will
> > > > > be
> > > > > > > > > > > compatible
> > > > > > > > > > > >>> with
> > > > > > > > > > > >>> > > > older
> > > > > > > > > > > >>> > > > >> > > broker
> > > > > > > > > > > >>> > > > >> > > > > > > > >> versions."
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > However, since we are
> > > > bumping
> > > > > > up
> > > > > > > the
> > > > > > > > > > > >>> version of
> > > > > > > > > > > >>> > > > >> > > inter-broker
> > > > > > > > > > > >>> > > > >> > > > > > > > >> requests, it
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > seems that we need to
> > use
> > > > IBP
> > > > > > for
> > > > > > > > > > > >>> migration.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 22. The KIP says
> > "Remove
> > > > > > deleted
> > > > > > > > > topics
> > > > > > > > > > > >>> from
> > > > > > > > > > > >>> > > > >> replicas by
> > > > > > > > > > > >>> > > > >> > > > > sending
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > StopReplicaRequest V3
> > for
> > > > any
> > > > > > > topics
> > > > > > > > > > > which
> > > > > > > > > > > >>> do
> > > > > > > > > > > >>> > not
> > > > > > > > > > > >>> > > > >> > contain
> > > > > > > > > > > >>> > > > >> > > a
> > > > > > > > > > > >>> > > > >> > > > > > topic
> > > > > > > > > > > >>> > > > >> > > > > > > > ID,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> and
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > V4 for any topics
> which
> > > do
> > > > > > > contain a
> > > > > > > > > > > topic
> > > > > > > > > > > >>> > ID.".
> > > > > > > > > > > >>> > > > >> > However,
> > > > > > > > > > > >>> > > > >> > > if
> > > > > > > > > > > >>> > > > >> > > > > we
> > > > > > > > > > > >>> > > > >> > > > > > > use
> > > > > > > > > > > >>> > > > >> > > > > > > > >> IBP,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > it
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > seems that the
> > controller
> > > > > will
> > > > > > > either
> > > > > > > > > > > send
> > > > > > > > > > > >>> > > > >> > > > StopReplicaRequest
> > > > > > > > > > > >>> > > > >> > > > > V3
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > or StopReplicaRequest
> > V4,
> > > > but
> > > > > > > never
> > > > > > > > > > mixed
> > > > > > > > > > > >>> V3
> > > > > > > > > > > >>> > and
> > > > > > > > > > > >>> > > V4
> > > > > > > > > > > >>> > > > >> for
> > > > > > > > > > > >>> > > > >> > > > > > different
> > > > > > > > > > > >>> > > > >> > > > > > > > >> topics.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > Basically, before the
> > IBP
> > > > > bump,
> > > > > > > V3
> > > > > > > > > will
> > > > > > > > > > > be
> > > > > > > > > > > >>> > used.
> > > > > > > > > > > >>> > > > >> After
> > > > > > > > > > > >>> > > > >> > the
> > > > > > > > > > > >>> > > > >> > > > IBP
> > > > > > > > > > > >>> > > > >> > > > > > > bump,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > topicId will be
> created
> > > and
> > > > > V4
> > > > > > > will
> > > > > > > > > be
> > > > > > > > > > > >>> used.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 23. Given that we
> > depend
> > > on
> > > > > > IBP,
> > > > > > > do
> > > > > > > > > we
> > > > > > > > > > > >>> still
> > > > > > > > > > > >>> > need
> > > > > > > > > > > >>> > > > "0
> > > > > > > > > > > >>> > > > >> > > > > > UNSPECIFIED"
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > in LeaderAndIsr?
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 24.
> > LeaderAndIsrResponse
> > > > v5 :
> > > > > > It
> > > > > > > > > still
> > > > > > > > > > > has
> > > > > > > > > > > >>> the
> > > > > > > > > > > >>> > > > topic
> > > > > > > > > > > >>> > > > >> > > field.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 25.
> > LeaderAndIsrResponse
> > > > v5,
> > > > > > > > > > > >>> > StopReplicaResponse
> > > > > > > > > > > >>> > > > v4:
> > > > > > > > > > > >>> > > > >> > Could
> > > > > > > > > > > >>> > > > >> > > > we
> > > > > > > > > > > >>> > > > >> > > > > > use
> > > > > > > > > > > >>> > > > >> > > > > > > > this
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > opportunity to
> organize
> > > the
> > > > > > > response
> > > > > > > > > > in 2
> > > > > > > > > > > >>> > levels,
> > > > > > > > > > > >>> > > > >> first
> > > > > > > > > > > >>> > > > >> > by
> > > > > > > > > > > >>> > > > >> > > > > > topic,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> then by
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > partition, as most
> > other
> > > > > > > > > > > >>> requests/responses?
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 26. FetchRequest v13
> :
> > > > Should
> > > > > > > > > > > >>> > > forgotten_topics_data
> > > > > > > > > > > >>> > > > >> use
> > > > > > > > > > > >>> > > > >> > > > > topicId
> > > > > > > > > > > >>> > > > >> > > > > > > too?
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 27. "This file can
> > either
> > > > be
> > > > > > > plain
> > > > > > > > > text
> > > > > > > > > > > >>> > > (key/value
> > > > > > > > > > > >>> > > > >> > pairs)
> > > > > > > > > > > >>> > > > >> > > or
> > > > > > > > > > > >>> > > > >> > > > > > > JSON."
> > > > > > > > > > > >>> > > > >> > > > > > > > >> Have
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > we
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > decided which one to
> > use?
> > > > > Also,
> > > > > > > it
> > > > > > > > > > would
> > > > > > > > > > > be
> > > > > > > > > > > >>> > > helpful
> > > > > > > > > > > >>> > > > >> to
> > > > > > > > > > > >>> > > > >> > > > provide
> > > > > > > > > > > >>> > > > >> > > > > > an
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > example.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > 28. Future
> improvement:
> > > > > Another
> > > > > > > > > future
> > > > > > > > > > > >>> > > improvement
> > > > > > > > > > > >>> > > > >> > > > opportunity
> > > > > > > > > > > >>> > > > >> > > > > > is
> > > > > > > > > > > >>> > > > >> > > > > > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> use
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > topicId in
> > > > > > > > > > > >>> GroupMetadataManager.offsetCommitKey
> > > > > > > > > > > >>> > > in
> > > > > > > > > > > >>> > > > >> the
> > > > > > > > > > > >>> > > > >> > > > > > > offset_commit
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > topic.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > This may save some
> > space.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > Thanks,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > Jun
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > On Wed, Sep 23, 2020
> at
> > > > 8:50
> > > > > AM
> > > > > > > > > Justine
> > > > > > > > > > > >>> Olshan
> > > > > > > > > > > >>> > <
> > > > > > > > > > > >>> > > > >> > > > > > > > jolshan@confluent.io>
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > wrote:
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > Hi Tom,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > Thanks for the
> > > comment. I
> > > > > > think
> > > > > > > > > this
> > > > > > > > > > > is a
> > > > > > > > > > > >>> > > really
> > > > > > > > > > > >>> > > > >> good
> > > > > > > > > > > >>> > > > >> > > idea
> > > > > > > > > > > >>> > > > >> > > > > and
> > > > > > > > > > > >>> > > > >> > > > > > > it
> > > > > > > > > > > >>> > > > >> > > > > > > > >> has
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > been
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > added to the KIP
> > under
> > > > the
> > > > > > > newly
> > > > > > > > > > added
> > > > > > > > > > > >>> > tooling
> > > > > > > > > > > >>> > > > >> > section.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > Thanks again,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > Justine
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > On Wed, Sep 23,
> 2020
> > at
> > > > > 3:17
> > > > > > > AM Tom
> > > > > > > > > > > >>> Bentley <
> > > > > > > > > > > >>> > > > >> > > > > > > tbentley@redhat.com>
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > wrote:
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > Hi Justine,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > I know you
> started
> > > the
> > > > > vote
> > > > > > > > > thread,
> > > > > > > > > > > >>> but on
> > > > > > > > > > > >>> > > > >> > re-reading
> > > > > > > > > > > >>> > > > >> > > > the
> > > > > > > > > > > >>> > > > >> > > > > > KIP
> > > > > > > > > > > >>> > > > >> > > > > > > I
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > noticed
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that although the
> > > topic
> > > > > id
> > > > > > is
> > > > > > > > > > > included
> > > > > > > > > > > >>> in
> > > > > > > > > > > >>> > the
> > > > > > > > > > > >>> > > > >> > > > > > MetadataResponse
> > > > > > > > > > > >>> > > > >> > > > > > > > >> it's
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > not
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > surfaced in the
> > > output
> > > > > from
> > > > > > > > > > > >>> > `kafka-topics.sh
> > > > > > > > > > > >>> > > > >> > > > --describe`.
> > > > > > > > > > > >>> > > > >> > > > > > > Maybe
> > > > > > > > > > > >>> > > > >> > > > > > > > >> that
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > was
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > intentional
> because
> > > ids
> > > > > are
> > > > > > > > > > > >>> intentionally
> > > > > > > > > > > >>> > not
> > > > > > > > > > > >>> > > > >> really
> > > > > > > > > > > >>> > > > >> > > > > > something
> > > > > > > > > > > >>> > > > >> > > > > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > user
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > should care
> deeply
> > > > about,
> > > > > > > but it
> > > > > > > > > > > would
> > > > > > > > > > > >>> also
> > > > > > > > > > > >>> > > > make
> > > > > > > > > > > >>> > > > >> > life
> > > > > > > > > > > >>> > > > >> > > > > harder
> > > > > > > > > > > >>> > > > >> > > > > > > for
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > anyone
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > debugging Kafka
> and
> > > > this
> > > > > > > would
> > > > > > > > > > likely
> > > > > > > > > > > >>> get
> > > > > > > > > > > >>> > > worse
> > > > > > > > > > > >>> > > > >> the
> > > > > > > > > > > >>> > > > >> > > more
> > > > > > > > > > > >>> > > > >> > > > > > topic
> > > > > > > > > > > >>> > > > >> > > > > > > > ids
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > got
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > rolled out across
> > the
> > > > > > > protocols,
> > > > > > > > > > > >>> clients
> > > > > > > > > > > >>> > etc.
> > > > > > > > > > > >>> > > > It
> > > > > > > > > > > >>> > > > >> > seems
> > > > > > > > > > > >>> > > > >> > > > > > likely
> > > > > > > > > > > >>> > > > >> > > > > > > > that
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > `kafka-topics.sh`
> > > will
> > > > > > > eventually
> > > > > > > > > > > need
> > > > > > > > > > > >>> the
> > > > > > > > > > > >>> > > > >> ability
> > > > > > > > > > > >>> > > > >> > to
> > > > > > > > > > > >>> > > > >> > > > show
> > > > > > > > > > > >>> > > > >> > > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > id
> > > > > > > > > > > >>> > > > >> > > > > > > > >> of
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > a
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > topic and perhaps
> > > find
> > > > a
> > > > > > > topic
> > > > > > > > > name
> > > > > > > > > > > >>> given
> > > > > > > > > > > >>> > an
> > > > > > > > > > > >>> > > > id.
> > > > > > > > > > > >>> > > > >> Is
> > > > > > > > > > > >>> > > > >> > > > there
> > > > > > > > > > > >>> > > > >> > > > > > any
> > > > > > > > > > > >>> > > > >> > > > > > > > >> reason
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > not
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > implement that in
> > > this
> > > > > KIP?
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > Many thanks,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > Tom
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > On Mon, Sep 21,
> > 2020
> > > at
> > > > > > 9:54
> > > > > > > PM
> > > > > > > > > > > Justine
> > > > > > > > > > > >>> > > Olshan
> > > > > > > > > > > >>> > > > <
> > > > > > > > > > > >>> > > > >> > > > > > > > >> jolshan@confluent.io
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > wrote:
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Hi all,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > After thinking
> > > about
> > > > > it,
> > > > > > > I've
> > > > > > > > > > > >>> decided to
> > > > > > > > > > > >>> > > > remove
> > > > > > > > > > > >>> > > > >> > the
> > > > > > > > > > > >>> > > > >> > > > > topic
> > > > > > > > > > > >>> > > > >> > > > > > > name
> > > > > > > > > > > >>> > > > >> > > > > > > > >> from
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Fetch Request
> and
> > > > > > Response
> > > > > > > > > after
> > > > > > > > > > > all.
> > > > > > > > > > > >>> > Since
> > > > > > > > > > > >>> > > > >> there
> > > > > > > > > > > >>> > > > >> > > are
> > > > > > > > > > > >>> > > > >> > > > so
> > > > > > > > > > > >>> > > > >> > > > > > > many
> > > > > > > > > > > >>> > > > >> > > > > > > > of
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > these
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > requests per
> > > second,
> > > > it
> > > > > > is
> > > > > > > > > worth
> > > > > > > > > > > >>> removing
> > > > > > > > > > > >>> > > the
> > > > > > > > > > > >>> > > > >> > extra
> > > > > > > > > > > >>> > > > >> > > > > > > > information.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > I've
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > updated the KIP
> > to
> > > > > > reflect
> > > > > > > this
> > > > > > > > > > > >>> change.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Please let me
> > know
> > > if
> > > > > > > there is
> > > > > > > > > > > >>> anything
> > > > > > > > > > > >>> > > else
> > > > > > > > > > > >>> > > > we
> > > > > > > > > > > >>> > > > >> > > should
> > > > > > > > > > > >>> > > > >> > > > > > > discuss
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > before
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > voting.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Thank you,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Justine
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > On Fri, Sep 18,
> > > 2020
> > > > at
> > > > > > > 9:46 AM
> > > > > > > > > > > >>> Justine
> > > > > > > > > > > >>> > > > Olshan
> > > > > > > > > > > >>> > > > >> <
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > jolshan@confluent.io
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > wrote:
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > Hi Jun,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > I see what
> you
> > > are
> > > > > > > saying.
> > > > > > > > > For
> > > > > > > > > > > now
> > > > > > > > > > > >>> we
> > > > > > > > > > > >>> > can
> > > > > > > > > > > >>> > > > >> remove
> > > > > > > > > > > >>> > > > >> > > the
> > > > > > > > > > > >>> > > > >> > > > > > extra
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > information.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > I'll leave
> the
> > > > option
> > > > > > to
> > > > > > > add
> > > > > > > > > > more
> > > > > > > > > > > >>> > fields
> > > > > > > > > > > >>> > > to
> > > > > > > > > > > >>> > > > >> the
> > > > > > > > > > > >>> > > > >> > > file
> > > > > > > > > > > >>> > > > >> > > > > in
> > > > > > > > > > > >>> > > > >> > > > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > future.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > The
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > KIP
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > has been
> > updated
> > > to
> > > > > > > reflect
> > > > > > > > > > this
> > > > > > > > > > > >>> > change.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > Thanks,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > Justine
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > On Fri, Sep
> 18,
> > > > 2020
> > > > > at
> > > > > > > 8:46
> > > > > > > > > AM
> > > > > > > > > > > Jun
> > > > > > > > > > > >>> > Rao <
> > > > > > > > > > > >>> > > > >> > > > > > jun@confluent.io
> > > > > > > > > > > >>> > > > >> > > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > wrote:
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Hi, Justine,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Thanks for
> the
> > > > > reply.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> 13. If the
> log
> > > > > > > directory is
> > > > > > > > > > the
> > > > > > > > > > > >>> source
> > > > > > > > > > > >>> > > of
> > > > > > > > > > > >>> > > > >> > truth,
> > > > > > > > > > > >>> > > > >> > > it
> > > > > > > > > > > >>> > > > >> > > > > > means
> > > > > > > > > > > >>> > > > >> > > > > > > > >> that
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> redundant
> info
> > > in
> > > > > the
> > > > > > > > > metadata
> > > > > > > > > > > >>> file
> > > > > > > > > > > >>> > will
> > > > > > > > > > > >>> > > > be
> > > > > > > > > > > >>> > > > >> > > > ignored.
> > > > > > > > > > > >>> > > > >> > > > > > Then
> > > > > > > > > > > >>> > > > >> > > > > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > question
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > is
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> why do we
> need
> > > to
> > > > > put
> > > > > > > the
> > > > > > > > > > > >>> redundant
> > > > > > > > > > > >>> > info
> > > > > > > > > > > >>> > > > in
> > > > > > > > > > > >>> > > > >> the
> > > > > > > > > > > >>> > > > >> > > > > > metadata
> > > > > > > > > > > >>> > > > >> > > > > > > > file
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > now?
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Thanks,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Jun
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> On Thu, Sep
> > 17,
> > > > 2020
> > > > > > at
> > > > > > > 5:07
> > > > > > > > > > PM
> > > > > > > > > > > >>> > Justine
> > > > > > > > > > > >>> > > > >> Olshan
> > > > > > > > > > > >>> > > > >> > <
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
> jolshan@confluent.io
> > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> wrote:
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Hi Jun,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Thanks for
> > the
> > > > > quick
> > > > > > > > > > response!
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > 12. I've
> > > decided
> > > > > to
> > > > > > > bump
> > > > > > > > > up
> > > > > > > > > > > the
> > > > > > > > > > > >>> > > versions
> > > > > > > > > > > >>> > > > >> on
> > > > > > > > > > > >>> > > > >> > the
> > > > > > > > > > > >>> > > > >> > > > > > > requests
> > > > > > > > > > > >>> > > > >> > > > > > > > >> and
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > updated
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > KIP. I
> think
> > > > it's
> > > > > > > good we
> > > > > > > > > > > >>> thoroughly
> > > > > > > > > > > >>> > > > >> > discussed
> > > > > > > > > > > >>> > > > >> > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > options
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > here,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > so
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > we
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> know
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > we made a
> > good
> > > > > > > choice. :)
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > 13. This
> is
> > an
> > > > > > > interesting
> > > > > > > > > > > >>> > situation.
> > > > > > > > > > > >>> > > I
> > > > > > > > > > > >>> > > > >> think
> > > > > > > > > > > >>> > > > >> > > if
> > > > > > > > > > > >>> > > > >> > > > > this
> > > > > > > > > > > >>> > > > >> > > > > > > > does
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > occur
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > we
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> should
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > give a
> > > warning.
> > > > I
> > > > > > > agree
> > > > > > > > > that
> > > > > > > > > > > >>> it's
> > > > > > > > > > > >>> > hard
> > > > > > > > > > > >>> > > > to
> > > > > > > > > > > >>> > > > >> > know
> > > > > > > > > > > >>> > > > >> > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > source
> > > > > > > > > > > >>> > > > >> > > > > > > > >> of
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > truth
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > for
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> sure
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > since the
> > > > > directory
> > > > > > > or the
> > > > > > > > > > > file
> > > > > > > > > > > >>> > could
> > > > > > > > > > > >>> > > be
> > > > > > > > > > > >>> > > > >> > > manually
> > > > > > > > > > > >>> > > > >> > > > > > > > >> modified. I
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > guess
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > directory
> > > could
> > > > be
> > > > > > > used as
> > > > > > > > > > the
> > > > > > > > > > > >>> > source
> > > > > > > > > > > >>> > > of
> > > > > > > > > > > >>> > > > >> > truth.
> > > > > > > > > > > >>> > > > >> > > > To
> > > > > > > > > > > >>> > > > >> > > > > be
> > > > > > > > > > > >>> > > > >> > > > > > > > >> honest,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > I'm
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > not
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > really
> sure
> > > what
> > > > > > > happens
> > > > > > > > > in
> > > > > > > > > > > >>> kafka
> > > > > > > > > > > >>> > when
> > > > > > > > > > > >>> > > > the
> > > > > > > > > > > >>> > > > >> > log
> > > > > > > > > > > >>> > > > >> > > > > > > directory
> > > > > > > > > > > >>> > > > >> > > > > > > > is
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > renamed
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > manually
> in
> > > > such a
> > > > > > > way.
> > > > > > > > > I'm
> > > > > > > > > > > also
> > > > > > > > > > > >>> > > > >> wondering if
> > > > > > > > > > > >>> > > > >> > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> situation is
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> recoverable
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > in this
> > > > scenario.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Thanks,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Justine
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > On Thu,
> Sep
> > > 17,
> > > > > 2020
> > > > > > > at
> > > > > > > > > 4:28
> > > > > > > > > > > PM
> > > > > > > > > > > >>> Jun
> > > > > > > > > > > >>> > > Rao
> > > > > > > > > > > >>> > > > <
> > > > > > > > > > > >>> > > > >> > > > > > > > jun@confluent.io>
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > wrote:
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Hi,
> > Justine,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Thanks
> for
> > > the
> > > > > > > reply.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > 12. I
> > don't
> > > > > have a
> > > > > > > > > strong
> > > > > > > > > > > >>> > preference
> > > > > > > > > > > >>> > > > >> > either.
> > > > > > > > > > > >>> > > > >> > > > > > However,
> > > > > > > > > > > >>> > > > >> > > > > > > > if
> > > > > > > > > > > >>> > > > >> > > > > > > > >> we
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > need
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > IBP
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > anyway,
> > > maybe
> > > > > it's
> > > > > > > > > easier
> > > > > > > > > > to
> > > > > > > > > > > >>> just
> > > > > > > > > > > >>> > > bump
> > > > > > > > > > > >>> > > > >> up
> > > > > > > > > > > >>> > > > >> > the
> > > > > > > > > > > >>> > > > >> > > > > > version
> > > > > > > > > > > >>> > > > >> > > > > > > > for
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > all
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > inter
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > broker
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > requests
> > and
> > > > add
> > > > > > the
> > > > > > > > > topic
> > > > > > > > > > > id
> > > > > > > > > > > >>> > field
> > > > > > > > > > > >>> > > > as a
> > > > > > > > > > > >>> > > > >> > > > regular
> > > > > > > > > > > >>> > > > >> > > > > > > > field. A
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > regular
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> field
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > is
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > a bit
> more
> > > > > concise
> > > > > > > in
> > > > > > > > > wire
> > > > > > > > > > > >>> > transfer
> > > > > > > > > > > >>> > > > >> than a
> > > > > > > > > > > >>> > > > >> > > > > flexible
> > > > > > > > > > > >>> > > > >> > > > > > > > >> field.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > 13. The
> > > > > confusion
> > > > > > > that I
> > > > > > > > > > was
> > > > > > > > > > > >>> > > referring
> > > > > > > > > > > >>> > > > >> to
> > > > > > > > > > > >>> > > > >> > is
> > > > > > > > > > > >>> > > > >> > > > > > between
> > > > > > > > > > > >>> > > > >> > > > > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > topic
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > name
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> partition
> > > > number
> > > > > > > between
> > > > > > > > > > the
> > > > > > > > > > > >>> log
> > > > > > > > > > > >>> > dir
> > > > > > > > > > > >>> > > > and
> > > > > > > > > > > >>> > > > >> > the
> > > > > > > > > > > >>> > > > >> > > > > > metadata
> > > > > > > > > > > >>> > > > >> > > > > > > > >> file.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > For
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> example,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > if
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > the log
> > dir
> > > is
> > > > > > > topicA-1
> > > > > > > > > > and
> > > > > > > > > > > >>> the
> > > > > > > > > > > >>> > > > metadata
> > > > > > > > > > > >>> > > > >> > file
> > > > > > > > > > > >>> > > > >> > > > in
> > > > > > > > > > > >>> > > > >> > > > > it
> > > > > > > > > > > >>> > > > >> > > > > > > has
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > topicB
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > and
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> partition
> > 0
> > > > (say
> > > > > > > due to
> > > > > > > > > a
> > > > > > > > > > > bug
> > > > > > > > > > > >>> or
> > > > > > > > > > > >>> > > > manual
> > > > > > > > > > > >>> > > > >> > > > > > > modification),
> > > > > > > > > > > >>> > > > >> > > > > > > > >> which
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > one
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > do
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > we
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > use
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > as the
> > > source
> > > > of
> > > > > > > truth?
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Jun
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > On Thu,
> > Sep
> > > > 17,
> > > > > > > 2020 at
> > > > > > > > > > 3:43
> > > > > > > > > > > >>> PM
> > > > > > > > > > > >>> > > > Justine
> > > > > > > > > > > >>> > > > >> > > Olshan
> > > > > > > > > > > >>> > > > >> > > > <
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > jolshan@confluent.io
> > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > wrote:
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > Hi
> Jun,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > Thanks
> > for
> > > > the
> > > > > > > > > comments.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > 12. I
> > > bumped
> > > > > the
> > > > > > > > > > > >>> > > LeaderAndIsrRequest
> > > > > > > > > > > >>> > > > >> > > because
> > > > > > > > > > > >>> > > > >> > > > I
> > > > > > > > > > > >>> > > > >> > > > > > > > removed
> > > > > > > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > topic
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> name
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > field
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > in the
> > > > > response.
> > > > > > > It
> > > > > > > > > may
> > > > > > > > > > be
> > > > > > > > > > > >>> > > possible
> > > > > > > > > > > >>> > > > to
> > > > > > > > > > > >>> > > > >> > > avoid
> > > > > > > > > > > >>> > > > >> > > > > > > bumping
> > > > > > > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > version
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > without
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > that
> > > change,
> > > > > but
> > > > > > > I may
> > > > > > > > > > be
> > > > > > > > > > > >>> > missing
> > > > > > > > > > > >>> > > > >> > > something.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > I
> > believe
> > > > > > > StopReplica
> > > > > > > > > is
> > > > > > > > > > > >>> > actually
> > > > > > > > > > > >>> > > on
> > > > > > > > > > > >>> > > > >> > > version
> > > > > > > > > > > >>> > > > >> > > > 3
> > > > > > > > > > > >>> > > > >> > > > > > now,
> > > > > > > > > > > >>> > > > >> > > > > > > > but
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > because
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > version 2
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > is
> > > > flexible, I
> > > > > > > kept
> > > > > > > > > that
> > > > > > > > > > > >>> listed
> > > > > > > > > > > >>> > as
> > > > > > > > > > > >>> > > > >> > version
> > > > > > > > > > > >>> > > > >> > > 2
> > > > > > > > > > > >>> > > > >> > > > on
> > > > > > > > > > > >>> > > > >> > > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > KIP
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > page.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> However,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > you
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > may be
> > > right
> > > > > in
> > > > > > > that
> > > > > > > > > we
> > > > > > > > > > > may
> > > > > > > > > > > >>> need
> > > > > > > > > > > >>> > > to
> > > > > > > > > > > >>> > > > >> bump
> > > > > > > > > > > >>> > > > >> > > the
> > > > > > > > > > > >>> > > > >> > > > > > > version
> > > > > > > > > > > >>> > > > >> > > > > > > > on
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > StopReplica
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > deal
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > with
> > > > deletion
> > > > > > > > > > differently
> > > > > > > > > > > as
> > > > > > > > > > > >>> > > > mentioned
> > > > > > > > > > > >>> > > > >> > > > above. I
> > > > > > > > > > > >>> > > > >> > > > > > > don't
> > > > > > > > > > > >>> > > > >> > > > > > > > >> know
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > if
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > I
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> have a
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > big
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > preference
> > > > > over
> > > > > > > used
> > > > > > > > > > > tagged
> > > > > > > > > > > >>> > fields
> > > > > > > > > > > >>> > > > or
> > > > > > > > > > > >>> > > > >> > not.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > 13. I
> > was
> > > > > > thinking
> > > > > > > > > that
> > > > > > > > > > in
> > > > > > > > > > > >>> the
> > > > > > > > > > > >>> > > case
> > > > > > > > > > > >>> > > > >> where
> > > > > > > > > > > >>> > > > >> > > the
> > > > > > > > > > > >>> > > > >> > > > > > file
> > > > > > > > > > > >>> > > > >> > > > > > > > and
> > > > > > > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > request
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > topic
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > ids
> > don't
> > > > > match,
> > > > > > > it
> > > > > > > > > > means
> > > > > > > > > > > >>> that
> > > > > > > > > > > >>> > the
> > > > > > > > > > > >>> > > > >> > broker's
> > > > > > > > > > > >>> > > > >> > > > > > > topic/the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> one
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > in
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> file
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > has
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > been
> > > > deleted.
> > > > > In
> > > > > > > that
> > > > > > > > > > > case,
> > > > > > > > > > > >>> we
> > > > > > > > > > > >>> > > would
> > > > > > > > > > > >>> > > > >> need
> > > > > > > > > > > >>> > > > >> > > to
> > > > > > > > > > > >>> > > > >> > > > > > delete
> > > > > > > > > > > >>> > > > >> > > > > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > old
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > topic
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > start
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > receiving
> > > > the
> > > > > > new
> > > > > > > > > > version.
> > > > > > > > > > > >>> If
> > > > > > > > > > > >>> > the
> > > > > > > > > > > >>> > > > >> topic
> > > > > > > > > > > >>> > > > >> > > name
> > > > > > > > > > > >>> > > > >> > > > > were
> > > > > > > > > > > >>> > > > >> > > > > > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > change,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > but
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > ids
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > still
> > > > matched,
> > > > > > the
> > > > > > > > > file
> > > > > > > > > > > >>> would
> > > > > > > > > > > >>> > also
> > > > > > > > > > > >>> > > > >> need
> > > > > > > > > > > >>> > > > >> > to
> > > > > > > > > > > >>> > > > >> > > > > > update.
> > > > > > > > > > > >>> > > > >> > > > > > > > Am I
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > missing
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > a
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> case
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > where
> > the
> > > > file
> > > > > > > would
> > > > > > > > > be
> > > > > > > > > > > >>> correct
> > > > > > > > > > > >>> > > and
> > > > > > > > > > > >>> > > > >> not
> > > > > > > > > > > >>> > > > >> > the
> > > > > > > > > > > >>> > > > >> > > > > > > request?
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> Thanks,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> Justine
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > On
> Thu,
> > > Sep
> > > > > 17,
> > > > > > > 2020
> > > > > > > > > at
> > > > > > > > > > > >>> 3:18 PM
> > > > > > > > > > > >>> > > Jun
> > > > > > > > > > > >>> > > > >> Rao <
> > > > > > > > > > > >>> > > > >> > > > > > > > >> jun@confluent.io
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > wrote:
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Hi,
> > > > Justine,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> Thanks
> > > for
> > > > > the
> > > > > > > > > reply.
> > > > > > > > > > A
> > > > > > > > > > > >>> couple
> > > > > > > > > > > >>> > > of
> > > > > > > > > > > >>> > > > >> more
> > > > > > > > > > > >>> > > > >> > > > > comments
> > > > > > > > > > > >>> > > > >> > > > > > > > >> below.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > 12.
> > > > > ListOffset
> > > > > > > and
> > > > > > > > > > > >>> > > OffsetForLeader
> > > > > > > > > > > >>> > > > >> > > > currently
> > > > > > > > > > > >>> > > > >> > > > > > > don't
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > support
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> flexible
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> fields.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > So,
> we
> > > > have
> > > > > to
> > > > > > > bump
> > > > > > > > > up
> > > > > > > > > > > the
> > > > > > > > > > > >>> > > version
> > > > > > > > > > > >>> > > > >> > number
> > > > > > > > > > > >>> > > > >> > > > and
> > > > > > > > > > > >>> > > > >> > > > > > use
> > > > > > > > > > > >>> > > > >> > > > > > > > >> IBP at
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > least
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > for
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > these
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > two
> > > > > requests.
> > > > > > > Note
> > > > > > > > > > that
> > > > > > > > > > > it
> > > > > > > > > > > >>> > seems
> > > > > > > > > > > >>> > > > >> 2.7.0
> > > > > > > > > > > >>> > > > >> > > will
> > > > > > > > > > > >>> > > > >> > > > > > > require
> > > > > > > > > > > >>> > > > >> > > > > > > > >> IBP
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > anyway
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > because
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > of
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > changes
> > > in
> > > > > > > > > > KAFKA-10435.
> > > > > > > > > > > >>> Also,
> > > > > > > > > > > >>> > it
> > > > > > > > > > > >>> > > > >> seems
> > > > > > > > > > > >>> > > > >> > > that
> > > > > > > > > > > >>> > > > >> > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> version
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > for
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > LeaderAndIsrRequest
> > > > > > > > > > and
> > > > > > > > > > > >>> > > > StopReplica
> > > > > > > > > > > >>> > > > >> are
> > > > > > > > > > > >>> > > > >> > > > > bumped
> > > > > > > > > > > >>> > > > >> > > > > > > even
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > though
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > we
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > only
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > added
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > a
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> tagged
> > > > > field.
> > > > > > > But
> > > > > > > > > > since
> > > > > > > > > > > >>> IBP is
> > > > > > > > > > > >>> > > > >> needed
> > > > > > > > > > > >>> > > > >> > > > anyway,
> > > > > > > > > > > >>> > > > >> > > > > > we
> > > > > > > > > > > >>> > > > >> > > > > > > > may
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > want
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> revisit
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > overall
> > > > > tagged
> > > > > > > field
> > > > > > > > > > > >>> choice.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > 13.
> > The
> > > > only
> > > > > > > > > downside
> > > > > > > > > > is
> > > > > > > > > > > >>> the
> > > > > > > > > > > >>> > > > >> potential
> > > > > > > > > > > >>> > > > >> > > > > > confusion
> > > > > > > > > > > >>> > > > >> > > > > > > on
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > which
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > one
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > is
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > source
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > of
> > truth
> > > > if
> > > > > > they
> > > > > > > > > don't
> > > > > > > > > > > >>> match.
> > > > > > > > > > > >>> > > > >> Another
> > > > > > > > > > > >>> > > > >> > > > option
> > > > > > > > > > > >>> > > > >> > > > > is
> > > > > > > > > > > >>> > > > >> > > > > > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > include
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > those
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > fields
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > in
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > metadata
> > > > > > > file
> > > > > > > > > when
> > > > > > > > > > > we
> > > > > > > > > > > >>> > > actually
> > > > > > > > > > > >>> > > > >> > change
> > > > > > > > > > > >>> > > > >> > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> directory
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > structure.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > Thanks,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Jun
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > On
> > Thu,
> > > > Sep
> > > > > > 17,
> > > > > > > 2020
> > > > > > > > > > at
> > > > > > > > > > > >>> 2:01
> > > > > > > > > > > >>> > PM
> > > > > > > > > > > >>> > > > >> Justine
> > > > > > > > > > > >>> > > > >> > > > > Olshan
> > > > > > > > > > > >>> > > > >> > > > > > <
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > jolshan@confluent.io
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> wrote:
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > Hello
> > > > all,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> I've
> > > > > thought
> > > > > > > some
> > > > > > > > > > more
> > > > > > > > > > > >>> about
> > > > > > > > > > > >>> > > > >> removing
> > > > > > > > > > > >>> > > > >> > > the
> > > > > > > > > > > >>> > > > >> > > > > > topic
> > > > > > > > > > > >>> > > > >> > > > > > > > >> name
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > field
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > from
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > some
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > of
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > requests.
> > > > > On
> > > > > > > > > closer
> > > > > > > > > > > >>> > inspection
> > > > > > > > > > > >>> > > > of
> > > > > > > > > > > >>> > > > >> the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > requests/responses,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > it
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> seems
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > internal
> > > > > > > changes
> > > > > > > > > > would
> > > > > > > > > > > >>> be
> > > > > > > > > > > >>> > much
> > > > > > > > > > > >>> > > > >> larger
> > > > > > > > > > > >>> > > > >> > > > than
> > > > > > > > > > > >>> > > > >> > > > > I
> > > > > > > > > > > >>> > > > >> > > > > > > > >> expected.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > Some
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > protocols
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > involve
> > > > > > > clients,
> > > > > > > > > so
> > > > > > > > > > > they
> > > > > > > > > > > >>> > would
> > > > > > > > > > > >>> > > > >> > require
> > > > > > > > > > > >>> > > > >> > > > > > changes
> > > > > > > > > > > >>> > > > >> > > > > > > > too.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > I'm
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > thinking
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > that
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > for
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> now,
> > > > > > removing
> > > > > > > the
> > > > > > > > > > > topic
> > > > > > > > > > > >>> name
> > > > > > > > > > > >>> > > > from
> > > > > > > > > > > >>> > > > >> > these
> > > > > > > > > > > >>> > > > >> > > > > > > requests
> > > > > > > > > > > >>> > > > >> > > > > > > > >> and
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > responses
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> are
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > out
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > scope.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I
> > have
> > > > > > > decided to
> > > > > > > > > > just
> > > > > > > > > > > >>> keep
> > > > > > > > > > > >>> > > the
> > > > > > > > > > > >>> > > > >> > change
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
> LeaderAndIsrResponse
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > remove
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > topic
> > > > > name,
> > > > > > > and
> > > > > > > > > have
> > > > > > > > > > > >>> updated
> > > > > > > > > > > >>> > > the
> > > > > > > > > > > >>> > > > >> KIP
> > > > > > > > > > > >>> > > > >> > to
> > > > > > > > > > > >>> > > > >> > > > > > reflect
> > > > > > > > > > > >>> > > > >> > > > > > > > >> this
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > change. I
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> have
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > also
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > mentioned
> > > > > > the
> > > > > > > > > other
> > > > > > > > > > > >>> requests
> > > > > > > > > > > >>> > > and
> > > > > > > > > > > >>> > > > >> > > > responses
> > > > > > > > > > > >>> > > > >> > > > > in
> > > > > > > > > > > >>> > > > >> > > > > > > > >> future
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > work.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> I'm
> > > > hoping
> > > > > > to
> > > > > > > > > start
> > > > > > > > > > > the
> > > > > > > > > > > >>> > voting
> > > > > > > > > > > >>> > > > >> > process
> > > > > > > > > > > >>> > > > >> > > > > soon,
> > > > > > > > > > > >>> > > > >> > > > > > so
> > > > > > > > > > > >>> > > > >> > > > > > > > >> let me
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > know
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > if
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > there
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > is
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > anything
> > > > > > else
> > > > > > > we
> > > > > > > > > > > should
> > > > > > > > > > > >>> > > discuss.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > Thank
> > > > you,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > Justine
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > On
> > > Tue,
> > > > > Sep
> > > > > > > 15,
> > > > > > > > > 2020
> > > > > > > > > > > at
> > > > > > > > > > > >>> 3:57
> > > > > > > > > > > >>> > > PM
> > > > > > > > > > > >>> > > > >> > Justine
> > > > > > > > > > > >>> > > > >> > > > > > Olshan
> > > > > > > > > > > >>> > > > >> > > > > > > <
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > jolshan@confluent.io
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > wrote:
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > Hello
> > > > > > again,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> To
> > > > > follow
> > > > > > > up on
> > > > > > > > > > some
> > > > > > > > > > > >>> of
> > > > > > > > > > > >>> > the
> > > > > > > > > > > >>> > > > >> other
> > > > > > > > > > > >>> > > > >> > > > > comments:
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > 10/11)
> > > > > We
> > > > > > > can
> > > > > > > > > > remove
> > > > > > > > > > > >>> the
> > > > > > > > > > > >>> > > topic
> > > > > > > > > > > >>> > > > >> name
> > > > > > > > > > > >>> > > > >> > > > from
> > > > > > > > > > > >>> > > > >> > > > > > > these
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > requests/responses,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > and
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > that
> > > > > means
> > > > > > > we
> > > > > > > > > will
> > > > > > > > > > > >>> just
> > > > > > > > > > > >>> > have
> > > > > > > > > > > >>> > > > to
> > > > > > > > > > > >>> > > > >> > make
> > > > > > > > > > > >>> > > > >> > > a
> > > > > > > > > > > >>> > > > >> > > > > few
> > > > > > > > > > > >>> > > > >> > > > > > > > >> internal
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > changes
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > make
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > partitions
> > > > > > > > > > > accessible
> > > > > > > > > > > >>> by
> > > > > > > > > > > >>> > > topic
> > > > > > > > > > > >>> > > > >> id
> > > > > > > > > > > >>> > > > >> > and
> > > > > > > > > > > >>> > > > >> > > > > > > > partition.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> I
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > can
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > update
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > KIP
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > remove
> > > > > > them
> > > > > > > > > unless
> > > > > > > > > > > >>> anyone
> > > > > > > > > > > >>> > > > thinks
> > > > > > > > > > > >>> > > > >> > they
> > > > > > > > > > > >>> > > > >> > > > > > should
> > > > > > > > > > > >>> > > > >> > > > > > > > >> stay.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > 12)
> > > > > > > Addressed in
> > > > > > > > > > the
> > > > > > > > > > > >>> > > previous
> > > > > > > > > > > >>> > > > >> > email.
> > > > > > > > > > > >>> > > > >> > > > I've
> > > > > > > > > > > >>> > > > >> > > > > > > > updated
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > KIP
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > include
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > tagged
> > > > > > > fields
> > > > > > > > > for
> > > > > > > > > > > the
> > > > > > > > > > > >>> > > requests
> > > > > > > > > > > >>> > > > >> and
> > > > > > > > > > > >>> > > > >> > > > > > responses.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> (More
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > on
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > below)
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > 13)
> > > I
> > > > > > think
> > > > > > > part
> > > > > > > > > > of
> > > > > > > > > > > >>> the
> > > > > > > > > > > >>> > idea
> > > > > > > > > > > >>> > > > for
> > > > > > > > > > > >>> > > > >> > > > > including
> > > > > > > > > > > >>> > > > >> > > > > > > this
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > information
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> is to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > prepare
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > for
> > > > > future
> > > > > > > > > > changes.
> > > > > > > > > > > >>> > Perhaps
> > > > > > > > > > > >>> > > > the
> > > > > > > > > > > >>> > > > >> > > > directory
> > > > > > > > > > > >>> > > > >> > > > > > > > >> structure
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > might
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> change
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > from
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > > > > >>> topicName_partitionNumber
> > > > > > > > > > > >>> > to
> > > > > > > > > > > >>> > > > >> > > something
> > > > > > > > > > > >>> > > > >> > > > > like
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > topicID_partitionNumber.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Then
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> it
> > > > would
> > > > > > be
> > > > > > > > > useful
> > > > > > > > > > > to
> > > > > > > > > > > >>> have
> > > > > > > > > > > >>> > > the
> > > > > > > > > > > >>> > > > >> > topic
> > > > > > > > > > > >>> > > > >> > > > name
> > > > > > > > > > > >>> > > > >> > > > > > in
> > > > > > > > > > > >>> > > > >> > > > > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > file
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > since
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > it
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > would
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > not
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > be
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> in
> > > the
> > > > > > > directory
> > > > > > > > > > > >>> > structure.
> > > > > > > > > > > >>> > > > >> > > Supporting
> > > > > > > > > > > >>> > > > >> > > > > > topic
> > > > > > > > > > > >>> > > > >> > > > > > > > >> renames
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > might
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > be
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > easier
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > if
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > other
> > > > > > > fields are
> > > > > > > > > > > >>> included.
> > > > > > > > > > > >>> > > > Would
> > > > > > > > > > > >>> > > > >> > > there
> > > > > > > > > > > >>> > > > >> > > > be
> > > > > > > > > > > >>> > > > >> > > > > > any
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > downsides
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> including
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > this
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > information?
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > 14)
> > > > > Yes,
> > > > > > we
> > > > > > > > > would
> > > > > > > > > > > >>> need to
> > > > > > > > > > > >>> > > > copy
> > > > > > > > > > > >>> > > > >> the
> > > > > > > > > > > >>> > > > >> > > > > > partition
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > metadata
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > file
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > in
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > this
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > process.
> > > > > > > I've
> > > > > > > > > > > updated
> > > > > > > > > > > >>> the
> > > > > > > > > > > >>> > > KIP
> > > > > > > > > > > >>> > > > to
> > > > > > > > > > > >>> > > > >> > > > include
> > > > > > > > > > > >>> > > > >> > > > > > > this.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > 15)
> > > I
> > > > > > > believe
> > > > > > > > > > Lucas
> > > > > > > > > > > >>> meant
> > > > > > > > > > > >>> > v1
> > > > > > > > > > > >>> > > > >> and v2
> > > > > > > > > > > >>> > > > >> > > > here.
> > > > > > > > > > > >>> > > > >> > > > > > He
> > > > > > > > > > > >>> > > > >> > > > > > > > was
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > referring
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> how
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > requests
> > > > > > > would
> > > > > > > > > > fall
> > > > > > > > > > > >>> under
> > > > > > > > > > > >>> > > > >> different
> > > > > > > > > > > >>> > > > >> > > IBP
> > > > > > > > > > > >>> > > > >> > > > > and
> > > > > > > > > > > >>> > > > >> > > > > > > > meant
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > that
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > older
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > brokers
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > would
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > have
> > > > to
> > > > > > use
> > > > > > > the
> > > > > > > > > > > older
> > > > > > > > > > > >>> > > version
> > > > > > > > > > > >>> > > > of
> > > > > > > > > > > >>> > > > >> > the
> > > > > > > > > > > >>> > > > >> > > > > > request
> > > > > > > > > > > >>> > > > >> > > > > > > > and
> > > > > > > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > existing
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > topic
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > deletion
> > > > > > > > > process.
> > > > > > > > > > At
> > > > > > > > > > > >>> > first,
> > > > > > > > > > > >>> > > it
> > > > > > > > > > > >>> > > > >> > seemed
> > > > > > > > > > > >>> > > > >> > > > > like
> > > > > > > > > > > >>> > > > >> > > > > > > > tagged
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > fields
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > would
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> resolve
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > the
> > > > IBP
> > > > > > > issue.
> > > > > > > > > > > >>> However, we
> > > > > > > > > > > >>> > > may
> > > > > > > > > > > >>> > > > >> need
> > > > > > > > > > > >>> > > > >> > > IBP
> > > > > > > > > > > >>> > > > >> > > > > for
> > > > > > > > > > > >>> > > > >> > > > > > > > this
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > request
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > after
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > all
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> since
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > the
> > > > > > > controller
> > > > > > > > > > > >>> handles the
> > > > > > > > > > > >>> > > > topic
> > > > > > > > > > > >>> > > > >> > > > deletion
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > differently
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> depending
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > on
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> IBP
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > version.
> > > > > > In
> > > > > > > an
> > > > > > > > > > older
> > > > > > > > > > > >>> > > version,
> > > > > > > > > > > >>> > > > we
> > > > > > > > > > > >>> > > > >> > > can't
> > > > > > > > > > > >>> > > > >> > > > > just
> > > > > > > > > > > >>> > > > >> > > > > > > > send
> > > > > > > > > > > >>> > > > >> > > > > > > > >> a
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > StopReplica
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > delete
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > topic
> > > > > > > > > immediately
> > > > > > > > > > > like
> > > > > > > > > > > >>> > we'd
> > > > > > > > > > > >>> > > > >> want to
> > > > > > > > > > > >>> > > > >> > > for
> > > > > > > > > > > >>> > > > >> > > > > > this
> > > > > > > > > > > >>> > > > >> > > > > > > > KIP.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > This
> > > > > makes
> > > > > > > me
> > > > > > > > > > wonder
> > > > > > > > > > > >>> if we
> > > > > > > > > > > >>> > > > want
> > > > > > > > > > > >>> > > > >> > > tagged
> > > > > > > > > > > >>> > > > >> > > > > > fields
> > > > > > > > > > > >>> > > > >> > > > > > > > on
> > > > > > > > > > > >>> > > > >> > > > > > > > >> all
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> requests
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > after
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > all.
> > > > Let
> > > > > > me
> > > > > > > know
> > > > > > > > > > > your
> > > > > > > > > > > >>> > > > thoughts!
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > Justine
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> On
> > > > Tue,
> > > > > > Sep
> > > > > > > 15,
> > > > > > > > > > 2020
> > > > > > > > > > > >>> at
> > > > > > > > > > > >>> > 1:03
> > > > > > > > > > > >>> > > > PM
> > > > > > > > > > > >>> > > > >> > > Justine
> > > > > > > > > > > >>> > > > >> > > > > > > Olshan
> > > > > > > > > > > >>> > > > >> > > > > > > > <
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > jolshan@confluent.io
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > wrote:
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > Hi
> > > > all,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > Jun
> > > > > > > brought up
> > > > > > > > > a
> > > > > > > > > > > good
> > > > > > > > > > > >>> > point
> > > > > > > > > > > >>> > > > in
> > > > > > > > > > > >>> > > > >> his
> > > > > > > > > > > >>> > > > >> > > > last
> > > > > > > > > > > >>> > > > >> > > > > > > email
> > > > > > > > > > > >>> > > > >> > > > > > > > >> about
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > tagged
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > fields,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > and
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > I've
> > > > > > > updated
> > > > > > > > > the
> > > > > > > > > > > KIP
> > > > > > > > > > > >>> to
> > > > > > > > > > > >>> > > > reflect
> > > > > > > > > > > >>> > > > >> > that
> > > > > > > > > > > >>> > > > >> > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> changes to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > requests
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > responses
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > will
> > > > be
> > > > > > in
> > > > > > > the
> > > > > > > > > > form
> > > > > > > > > > > >>> of
> > > > > > > > > > > >>> > > tagged
> > > > > > > > > > > >>> > > > >> > fields
> > > > > > > > > > > >>> > > > >> > > > to
> > > > > > > > > > > >>> > > > >> > > > > > > avoid
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > changing
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > IBP.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > Jun:
> > > > I
> > > > > > > plan on
> > > > > > > > > > > >>> sending a
> > > > > > > > > > > >>> > > > >> followup
> > > > > > > > > > > >>> > > > >> > > > email
> > > > > > > > > > > >>> > > > >> > > > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> address
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > some
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > of
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > other
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > points.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > Thanks,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > Justine
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > On
> > > > Mon,
> > > > > > > Sep 14,
> > > > > > > > > > > 2020
> > > > > > > > > > > >>> at
> > > > > > > > > > > >>> > > 4:25
> > > > > > > > > > > >>> > > > PM
> > > > > > > > > > > >>> > > > >> > Jun
> > > > > > > > > > > >>> > > > >> > > > Rao
> > > > > > > > > > > >>> > > > >> > > > > <
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > jun@confluent.io
> >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > wrote:
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > Hi,
> > > > > > > Justine,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > Thanks
> > > > > > > for the
> > > > > > > > > > > >>> updated
> > > > > > > > > > > >>> > > KIP.
> > > > > > > > > > > >>> > > > A
> > > > > > > > > > > >>> > > > >> few
> > > > > > > > > > > >>> > > > >> > > > > > comments
> > > > > > > > > > > >>> > > > >> > > > > > > > >> below.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > 10.
> > > > > > > > > LeaderAndIsr
> > > > > > > > > > > >>> > Response:
> > > > > > > > > > > >>> > > > Do
> > > > > > > > > > > >>> > > > >> we
> > > > > > > > > > > >>> > > > >> > > need
> > > > > > > > > > > >>> > > > >> > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > topic
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > name?
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > 11.
> > > > > For
> > > > > > > the
> > > > > > > > > > > changed
> > > > > > > > > > > >>> > > > >> > > request/response,
> > > > > > > > > > > >>> > > > >> > > > > > other
> > > > > > > > > > > >>> > > > >> > > > > > > > >> than
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> LeaderAndIsr,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > > > > > UpdateMetadata,
> > > > > > > > > > > >>> > Metadata,
> > > > > > > > > > > >>> > > do
> > > > > > > > > > > >>> > > > >> we
> > > > > > > > > > > >>> > > > >> > > need
> > > > > > > > > > > >>> > > > >> > > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > include
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > topic
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> name?
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > 12.
> > > > It
> > > > > > > seems
> > > > > > > > > > that
> > > > > > > > > > > >>> > upgrades
> > > > > > > > > > > >>> > > > >> don't
> > > > > > > > > > > >>> > > > >> > > > > require
> > > > > > > > > > > >>> > > > >> > > > > > > IBP.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> Does
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > that
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > mean
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > new
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > fields
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > in
> > > > all
> > > > > > the
> > > > > > > > > > > >>> > > request/response
> > > > > > > > > > > >>> > > > >> are
> > > > > > > > > > > >>> > > > >> > > added
> > > > > > > > > > > >>> > > > >> > > > > as
> > > > > > > > > > > >>> > > > >> > > > > > > > tagged
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > fields
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> without
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > bumping
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > up
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > the
> > > > > > > request
> > > > > > > > > > > >>> version? It
> > > > > > > > > > > >>> > > > would
> > > > > > > > > > > >>> > > > >> be
> > > > > > > > > > > >>> > > > >> > > > useful
> > > > > > > > > > > >>> > > > >> > > > > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > make
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > that
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > clear.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > 13.
> > > > > > > Partition
> > > > > > > > > > > >>> Metadata
> > > > > > > > > > > >>> > > file:
> > > > > > > > > > > >>> > > > >> Do
> > > > > > > > > > > >>> > > > >> > we
> > > > > > > > > > > >>> > > > >> > > > need
> > > > > > > > > > > >>> > > > >> > > > > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> include
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > topic
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > name
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > and
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > > partition
> > > > > > > id
> > > > > > > > > > since
> > > > > > > > > > > >>> they
> > > > > > > > > > > >>> > > are
> > > > > > > > > > > >>> > > > >> > implied
> > > > > > > > > > > >>> > > > >> > > > in
> > > > > > > > > > > >>> > > > >> > > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > directory
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > name?
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > 14.
> > > > In
> > > > > > the
> > > > > > > > > JBOD
> > > > > > > > > > > >>> mode, we
> > > > > > > > > > > >>> > > > >> support
> > > > > > > > > > > >>> > > > >> > > > > moving a
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > partition's
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > data
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> from
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > one
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> disk
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > another.
> > > > > > > Will
> > > > > > > > > > the
> > > > > > > > > > > >>> new
> > > > > > > > > > > >>> > > > >> partition
> > > > > > > > > > > >>> > > > >> > > > > metadata
> > > > > > > > > > > >>> > > > >> > > > > > > file
> > > > > > > > > > > >>> > > > >> > > > > > > > >> be
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > copied
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> during
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > process?
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > 15.
> > > > > The
> > > > > > > KIP
> > > > > > > > > says
> > > > > > > > > > > >>> "Remove
> > > > > > > > > > > >>> > > > >> deleted
> > > > > > > > > > > >>> > > > >> > > > topics
> > > > > > > > > > > >>> > > > >> > > > > > > from
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > replicas
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > by
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > sending
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > > > > > > StopReplicaRequest
> > > > > > > > > > > >>> V2
> > > > > > > > > > > >>> > for
> > > > > > > > > > > >>> > > > any
> > > > > > > > > > > >>> > > > >> > > topics
> > > > > > > > > > > >>> > > > >> > > > > > which
> > > > > > > > > > > >>> > > > >> > > > > > > do
> > > > > > > > > > > >>> > > > >> > > > > > > > >> not
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > contain
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > a
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > topic
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > ID,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> and
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > V3
> > > > for
> > > > > > any
> > > > > > > > > > topics
> > > > > > > > > > > >>> which
> > > > > > > > > > > >>> > do
> > > > > > > > > > > >>> > > > >> > contain
> > > > > > > > > > > >>> > > > >> > > a
> > > > > > > > > > > >>> > > > >> > > > > > topic
> > > > > > > > > > > >>> > > > >> > > > > > > > >> ID.".
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > However,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > it
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > seems
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > updated
> > > > > > > > > > controller
> > > > > > > > > > > >>> will
> > > > > > > > > > > >>> > > > create
> > > > > > > > > > > >>> > > > >> > all
> > > > > > > > > > > >>> > > > >> > > > > > missing
> > > > > > > > > > > >>> > > > >> > > > > > > > >> topic
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > IDs
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > first
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > before
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> doing
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > other
> > > > > > > actions.
> > > > > > > > > > So,
> > > > > > > > > > > >>> is
> > > > > > > > > > > >>> > > > >> > > > > StopReplicaRequest
> > > > > > > > > > > >>> > > > >> > > > > > V2
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > needed?
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > Jun
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > On
> > > > > Fri,
> > > > > > > Sep
> > > > > > > > > 11,
> > > > > > > > > > > >>> 2020 at
> > > > > > > > > > > >>> > > > 10:31
> > > > > > > > > > > >>> > > > >> AM
> > > > > > > > > > > >>> > > > >> > > John
> > > > > > > > > > > >>> > > > >> > > > > > > > Roesler <
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > vvcephei@apache.org
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > wrote:
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > > > Thanks,
> > > > > > > > > > Justine!
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > Your
> > > > > > > > > response
> > > > > > > > > > > >>> seems
> > > > > > > > > > > >>> > > > >> compelling
> > > > > > > > > > > >>> > > > >> > to
> > > > > > > > > > > >>> > > > >> > > > me.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > > -John
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > On
> > > > > > Fri,
> > > > > > > > > > > >>> 2020-09-11 at
> > > > > > > > > > > >>> > > > 10:17
> > > > > > > > > > > >>> > > > >> > > -0700,
> > > > > > > > > > > >>> > > > >> > > > > > > Justine
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > Olshan
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > wrote:
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > > > Hello
> > > > > > > all,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > > > > Thanks for
> > > > > > > > > > > >>> > continuing
> > > > > > > > > > > >>> > > > the
> > > > > > > > > > > >>> > > > >> > > > > > discussion! I
> > > > > > > > > > > >>> > > > >> > > > > > > > >> have a
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > few
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > responses
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> your
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > > > points.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > > Tom:
> > > > > > > You
> > > > > > > > > are
> > > > > > > > > > > >>> correct
> > > > > > > > > > > >>> > > in
> > > > > > > > > > > >>> > > > >> that
> > > > > > > > > > > >>> > > > >> > > this
> > > > > > > > > > > >>> > > > >> > > > > KIP
> > > > > > > > > > > >>> > > > >> > > > > > > has
> > > > > > > > > > > >>> > > > >> > > > > > > > >> not
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > mentioned
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > > > > > > > > >>> > DeleteTopicsRequest. I
> > > > > > > > > > > >>> > > > >> think
> > > > > > > > > > > >>> > > > >> > > that
> > > > > > > > > > > >>> > > > >> > > > > > this
> > > > > > > > > > > >>> > > > >> > > > > > > > >> would
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > be
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > out
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > of
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > scope
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > for
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > now,
> > > > > but
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > > may
> > > > > > be
> > > > > > > > > > > something
> > > > > > > > > > > >>> > worth
> > > > > > > > > > > >>> > > > >> adding
> > > > > > > > > > > >>> > > > >> > > in
> > > > > > > > > > > >>> > > > >> > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> future.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > > > John:
> > > > > > > We
> > > > > > > > > did
> > > > > > > > > > > >>> > consider
> > > > > > > > > > > >>> > > > >> > sequence
> > > > > > > > > > > >>> > > > >> > > > ids,
> > > > > > > > > > > >>> > > > >> > > > > > but
> > > > > > > > > > > >>> > > > >> > > > > > > > >> there
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > are
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > a
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > few
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > reasons
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > favor
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > > > UUIDs.
> > > > > > > > > There
> > > > > > > > > > > are
> > > > > > > > > > > >>> > > several
> > > > > > > > > > > >>> > > > >> > cases
> > > > > > > > > > > >>> > > > >> > > > > where
> > > > > > > > > > > >>> > > > >> > > > > > > > topics
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > from
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> different
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > clusters
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > may
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > > > > interact
> > > > > > > > > now
> > > > > > > > > > > >>> and in
> > > > > > > > > > > >>> > > the
> > > > > > > > > > > >>> > > > >> > future.
> > > > > > > > > > > >>> > > > >> > > > For
> > > > > > > > > > > >>> > > > >> > > > > > > > >> example,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > Mirror
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Maker 2
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > may
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > benefit
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > > from
> > > > > > > being
> > > > > > > > > > > able
> > > > > > > > > > > >>> to
> > > > > > > > > > > >>> > > > detect
> > > > > > > > > > > >>> > > > >> > when
> > > > > > > > > > > >>> > > > >> > > a
> > > > > > > > > > > >>> > > > >> > > > > > > cluster
> > > > > > > > > > > >>> > > > >> > > > > > > > >> being
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > mirrored
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> is
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > deleted
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > and
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > > > > recreated
> > > > > > > > > > and
> > > > > > > > > > > >>> > globally
> > > > > > > > > > > >>> > > > >> unique
> > > > > > > > > > > >>> > > > >> > > > > > > identifiers
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > would
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > make
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> resolving
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > issues
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > > > easier
> > > > > > > > > than
> > > > > > > > > > > >>> sequence
> > > > > > > > > > > >>> > > IDs
> > > > > > > > > > > >>> > > > >> > which
> > > > > > > > > > > >>> > > > >> > > > may
> > > > > > > > > > > >>> > > > >> > > > > > > > collide
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > between
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > clusters.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > KIP-405
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > > > > (tiered
> > > > > > > > > > > storage)
> > > > > > > > > > > >>> > will
> > > > > > > > > > > >>> > > > also
> > > > > > > > > > > >>> > > > >> > > > benefit
> > > > > > > > > > > >>> > > > >> > > > > > from
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > globally
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > unique
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> IDs
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > as
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > shared
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > > > > buckets
> > > > > > > > > may
> > > > > > > > > > be
> > > > > > > > > > > >>> used
> > > > > > > > > > > >>> > > > >> between
> > > > > > > > > > > >>> > > > >> > > > > clusters.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > > > > Globally
> > > > > > > > > > > unique
> > > > > > > > > > > >>> IDs
> > > > > > > > > > > >>> > > > would
> > > > > > > > > > > >>> > > > >> > also
> > > > > > > > > > > >>> > > > >> > > > make
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > functionality
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > like
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > moving
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > topics
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > > > > between
> > > > > > > > > > > >>> disparate
> > > > > > > > > > > >>> > > > clusters
> > > > > > > > > > > >>> > > > >> > > easier
> > > > > > > > > > > >>> > > > >> > > > > in
> > > > > > > > > > > >>> > > > >> > > > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > future,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> simplify
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > any
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > future
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > > > > > > > > implementations
> > > > > > > > > > > >>> of
> > > > > > > > > > > >>> > > > backups
> > > > > > > > > > > >>> > > > >> > and
> > > > > > > > > > > >>> > > > >> > > > > > > restores,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> and
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > more.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > In
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > general,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > unique
> > > > > > IDs
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > > > would
> > > > > > > > > ensure
> > > > > > > > > > > >>> that
> > > > > > > > > > > >>> > the
> > > > > > > > > > > >>> > > > >> source
> > > > > > > > > > > >>> > > > >> > > > > cluster
> > > > > > > > > > > >>> > > > >> > > > > > > > >> topics do
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > not
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> conflict
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > with
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > > > > > > destination
> > > > > > > > > > > >>> cluster
> > > > > > > > > > > >>> > > > >> topics.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > If
> > > > > > we
> > > > > > > were
> > > > > > > > > > to
> > > > > > > > > > > >>> use
> > > > > > > > > > > >>> > > > sequence
> > > > > > > > > > > >>> > > > >> > ids,
> > > > > > > > > > > >>> > > > >> > > > we
> > > > > > > > > > > >>> > > > >> > > > > > > would
> > > > > > > > > > > >>> > > > >> > > > > > > > >> need
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> sufficiently
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > large
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > cluster
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > > ids
> > > > > > > to be
> > > > > > > > > > > stored
> > > > > > > > > > > >>> > with
> > > > > > > > > > > >>> > > > the
> > > > > > > > > > > >>> > > > >> > topic
> > > > > > > > > > > >>> > > > >> > > > > > > > >> identifiers or
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > we
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > run
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > risk
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > > > > > > collisions.
> > > > > > > > > > > This
> > > > > > > > > > > >>> > will
> > > > > > > > > > > >>> > > > >> give up
> > > > > > > > > > > >>> > > > >> > > any
> > > > > > > > > > > >>> > > > >> > > > > > > > >> advantage in
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> compactness
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > sequence
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > > > > numbers
> > > > > > > > > may
> > > > > > > > > > > >>> bring.
> > > > > > > > > > > >>> > > Given
> > > > > > > > > > > >>> > > > >> > these
> > > > > > > > > > > >>> > > > >> > > > > > > > advantages I
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > think
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > it
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> makes
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > sense
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > use
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > > > UUIDs.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > > > Gokul:
> > > > > > > > > This
> > > > > > > > > > is
> > > > > > > > > > > >>> an
> > > > > > > > > > > >>> > > > >> interesting
> > > > > > > > > > > >>> > > > >> > > > idea,
> > > > > > > > > > > >>> > > > >> > > > > > but
> > > > > > > > > > > >>> > > > >> > > > > > > > >> this
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > is
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > a
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> breaking
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > change.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > Out
> > > > of
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > > > scope
> > > > > > > for
> > > > > > > > > > now,
> > > > > > > > > > > >>> but
> > > > > > > > > > > >>> > > maybe
> > > > > > > > > > > >>> > > > >> > worth
> > > > > > > > > > > >>> > > > >> > > > > > > discussing
> > > > > > > > > > > >>> > > > >> > > > > > > > >> in
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > future.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > > Hope
> > > > > > > this
> > > > > > > > > > > >>> explains
> > > > > > > > > > > >>> > > some
> > > > > > > > > > > >>> > > > of
> > > > > > > > > > > >>> > > > >> > the
> > > > > > > > > > > >>> > > > >> > > > > > > decisions,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > > > > Justine
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > On
> > > > > > > Fri,
> > > > > > > > > Sep
> > > > > > > > > > > 11,
> > > > > > > > > > > >>> 2020
> > > > > > > > > > > >>> > > at
> > > > > > > > > > > >>> > > > >> 8:27
> > > > > > > > > > > >>> > > > >> > AM
> > > > > > > > > > > >>> > > > >> > > > > Gokul
> > > > > > > > > > > >>> > > > >> > > > > > > > >> Ramanan
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Subramanian
> <
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > > > > > > > > >>> > gokul2411s@gmail.com>
> > > > > > > > > > > >>> > > > >> wrote:
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > > Hi.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > > > Thanks
> > > > > > > > > for
> > > > > > > > > > > the
> > > > > > > > > > > >>> > KIP.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > > > Have you
> > > > > > > > > > > >>> thought
> > > > > > > > > > > >>> > > about
> > > > > > > > > > > >>> > > > >> > > whether
> > > > > > > > > > > >>> > > > >> > > > it
> > > > > > > > > > > >>> > > > >> > > > > > > makes
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > sense
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> support
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > > > authorizing a
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > > > > > principal
> > > > > > > > > > > for
> > > > > > > > > > > >>> a
> > > > > > > > > > > >>> > > topic
> > > > > > > > > > > >>> > > > ID
> > > > > > > > > > > >>> > > > >> > > rather
> > > > > > > > > > > >>> > > > >> > > > > > than
> > > > > > > > > > > >>> > > > >> > > > > > > a
> > > > > > > > > > > >>> > > > >> > > > > > > > >> topic
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > name
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > achieve
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > tighter
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > > > > > security?
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > Or
> > > > > > > is
> > > > > > > > > the
> > > > > > > > > > > >>> topic ID
> > > > > > > > > > > >>> > > > >> > > > fundamentally
> > > > > > > > > > > >>> > > > >> > > > > an
> > > > > > > > > > > >>> > > > >> > > > > > > > >> internal
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > detail
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > similar
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > epochs
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > used
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > > in a
> > > > > > > > > bunch
> > > > > > > > > > > of
> > > > > > > > > > > >>> > other
> > > > > > > > > > > >>> > > > >> places
> > > > > > > > > > > >>> > > > >> > in
> > > > > > > > > > > >>> > > > >> > > > > > Kafka?
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > > > Thanks.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > On
> > > > > > > Fri,
> > > > > > > > > > Sep
> > > > > > > > > > > >>> 11,
> > > > > > > > > > > >>> > 2020
> > > > > > > > > > > >>> > > > at
> > > > > > > > > > > >>> > > > >> > 4:06
> > > > > > > > > > > >>> > > > >> > > PM
> > > > > > > > > > > >>> > > > >> > > > > > John
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > Roesler <
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > > vvcephei@apache.org
> > > > > > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > > wrote:
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > Hello
> > > > > > > > > > > >>> Justine,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > Thanks
> > > > > > > > > > for
> > > > > > > > > > > >>> the
> > > > > > > > > > > >>> > > KIP!
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > I
> > > > > > > > > happen
> > > > > > > > > > > to
> > > > > > > > > > > >>> have
> > > > > > > > > > > >>> > > > been
> > > > > > > > > > > >>> > > > >> > > > > confronted
> > > > > > > > > > > >>> > > > >> > > > > > > > >> recently
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > with
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> need
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > keep
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > > track
> > > > > > of
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > a
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > large
> > > > > > > > > > > >>> number of
> > > > > > > > > > > >>> > > > >> topics as
> > > > > > > > > > > >>> > > > >> > > > > > compactly
> > > > > > > > > > > >>> > > > >> > > > > > > > as
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > possible. I
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> was
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > going
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > come
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > up
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > with
> > > > > > > > > > some
> > > > > > > > > > > >>> way to
> > > > > > > > > > > >>> > > > >> > dictionary
> > > > > > > > > > > >>> > > > >> > > > > > encode
> > > > > > > > > > > >>> > > > >> > > > > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > topic
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > names
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> as
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > integers,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > but
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > this
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > seems
> > > > > > > > > > much
> > > > > > > > > > > >>> > better!
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > > Apologies
> > > > > > > > > > > if
> > > > > > > > > > > >>> > this
> > > > > > > > > > > >>> > > > has
> > > > > > > > > > > >>> > > > >> > been
> > > > > > > > > > > >>> > > > >> > > > > raised
> > > > > > > > > > > >>> > > > >> > > > > > > > >> before,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > but
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > I’m
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> wondering
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > about
> > > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > choice
> > > > > > > > > > of
> > > > > > > > > > > >>> UUID
> > > > > > > > > > > >>> > vs
> > > > > > > > > > > >>> > > > >> > sequence
> > > > > > > > > > > >>> > > > >> > > > > number
> > > > > > > > > > > >>> > > > >> > > > > > > for
> > > > > > > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > ids.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Typically,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > I’ve
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > seen
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > > UUIDs
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > in two
> > > > > > > > > > > >>> > situations:
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > 1.
> > > > > > > > > When
> > > > > > > > > > > >>> > processes
> > > > > > > > > > > >>> > > > >> need to
> > > > > > > > > > > >>> > > > >> > > > > > generate
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > non-colliding
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > identifiers
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > without
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > > > >>> coordination.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > 2.
> > > > > > > > > When
> > > > > > > > > > > the
> > > > > > > > > > > >>> > > > identifier
> > > > > > > > > > > >>> > > > >> > > needs
> > > > > > > > > > > >>> > > > >> > > > to
> > > > > > > > > > > >>> > > > >> > > > > > be
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > “universally
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > unique”;
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> I.e.,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > > identifier
> > > > > > > > > > > >>> needs
> > > > > > > > > > > >>> > > to
> > > > > > > > > > > >>> > > > >> > > > distinguish
> > > > > > > > > > > >>> > > > >> > > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> entity
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > from
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > all
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > other
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > entities
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > that
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > could
> > > > > > > > > > ever
> > > > > > > > > > > >>> > exist.
> > > > > > > > > > > >>> > > > >> This is
> > > > > > > > > > > >>> > > > >> > > > > useful
> > > > > > > > > > > >>> > > > >> > > > > > in
> > > > > > > > > > > >>> > > > >> > > > > > > > >> cases
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > where
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > entities
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > from
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> all
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > > kinds
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > of
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > systems
> > > > > > > > > > > get
> > > > > > > > > > > >>> > mixed
> > > > > > > > > > > >>> > > > >> > together,
> > > > > > > > > > > >>> > > > >> > > > > such
> > > > > > > > > > > >>> > > > >> > > > > > as
> > > > > > > > > > > >>> > > > >> > > > > > > > >> when
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > dumping
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> logs
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > from
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > all
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > > > > processes
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > in
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > a
> > > > > > > > > > company
> > > > > > > > > > > >>> into a
> > > > > > > > > > > >>> > > > >> common
> > > > > > > > > > > >>> > > > >> > > > system.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > Maybe
> > > > > > > > > > I’m
> > > > > > > > > > > >>> being
> > > > > > > > > > > >>> > > > >> > > > short-sighted,
> > > > > > > > > > > >>> > > > >> > > > > > but
> > > > > > > > > > > >>> > > > >> > > > > > > it
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > doesn’t
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > seem
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> like
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> either
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > really
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > applies
> > > > > > > > > > > >>> here. It
> > > > > > > > > > > >>> > > > seems
> > > > > > > > > > > >>> > > > >> > like
> > > > > > > > > > > >>> > > > >> > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > brokers
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > could
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > and
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> would
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > achieve
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > > > > consensus
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > when
> > > > > > > > > > > >>> creating a
> > > > > > > > > > > >>> > > > topic
> > > > > > > > > > > >>> > > > >> > > anyway,
> > > > > > > > > > > >>> > > > >> > > > > > which
> > > > > > > > > > > >>> > > > >> > > > > > > > is
> > > > > > > > > > > >>> > > > >> > > > > > > > >> all
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that’s
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > required
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > > > generate
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > > > >>> non-colliding
> > > > > > > > > > > >>> > > > sequence
> > > > > > > > > > > >>> > > > >> > ids.
> > > > > > > > > > > >>> > > > >> > > > For
> > > > > > > > > > > >>> > > > >> > > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > second,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > as
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > you
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > mention,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > we
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > could
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > > > assign
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > a
> > > > > > > UUID
> > > > > > > > > > to
> > > > > > > > > > > >>> the
> > > > > > > > > > > >>> > > > cluster
> > > > > > > > > > > >>> > > > >> as
> > > > > > > > > > > >>> > > > >> > a
> > > > > > > > > > > >>> > > > >> > > > > whole,
> > > > > > > > > > > >>> > > > >> > > > > > > > which
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > would
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > render
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > any
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > resource
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > > scoped
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > the
> > > > > > > > > > broker
> > > > > > > > > > > >>> > > > universally
> > > > > > > > > > > >>> > > > >> > > unique
> > > > > > > > > > > >>> > > > >> > > > > as
> > > > > > > > > > > >>> > > > >> > > > > > > > well.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > The
> > > > > > > > > > > reason I
> > > > > > > > > > > >>> > > mention
> > > > > > > > > > > >>> > > > >> this
> > > > > > > > > > > >>> > > > >> > > is
> > > > > > > > > > > >>> > > > >> > > > > > that,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > although
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > a
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > UUID
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> is
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > way
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > more
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > > > compact
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > than
> > > > > > > > > > topic
> > > > > > > > > > > >>> > names,
> > > > > > > > > > > >>> > > > it’s
> > > > > > > > > > > >>> > > > >> > > still
> > > > > > > > > > > >>> > > > >> > > > 16
> > > > > > > > > > > >>> > > > >> > > > > > > > bytes.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> In
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > contrast,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> a
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > 4-byte
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > integer
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > sequence
> > > > > > > > > > > id
> > > > > > > > > > > >>> > would
> > > > > > > > > > > >>> > > > give
> > > > > > > > > > > >>> > > > >> > us 4
> > > > > > > > > > > >>> > > > >> > > > > > billion
> > > > > > > > > > > >>> > > > >> > > > > > > > >> unique
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > topics
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> per
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > cluster,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > which
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > > > seems
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > like
> > > > > > > > > > > enough
> > > > > > > > > > > >>> ;)
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > > > Considering
> > > > > > > > > > > >>> the
> > > > > > > > > > > >>> > > > >> number of
> > > > > > > > > > > >>> > > > >> > > > > > different
> > > > > > > > > > > >>> > > > >> > > > > > > > >> times
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > these
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> topic
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > > > identifiers
> > > > > > > > > are
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > > sent
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > over
> > > > > > > > > the
> > > > > > > > > > > >>> wire or
> > > > > > > > > > > >>> > > > >> stored
> > > > > > > > > > > >>> > > > >> > in
> > > > > > > > > > > >>> > > > >> > > > > > memory,
> > > > > > > > > > > >>> > > > >> > > > > > > it
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > seems
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > like
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > it
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > might
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > be
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > worth
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > > additional
> > > > > > > > > > > >>> 4x
> > > > > > > > > > > >>> > > space
> > > > > > > > > > > >>> > > > >> > > savings.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > What
> > > > > > > > > do
> > > > > > > > > > > you
> > > > > > > > > > > >>> > think
> > > > > > > > > > > >>> > > > >> about
> > > > > > > > > > > >>> > > > >> > > this?
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > John
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > On
> > > > > > > > > Fri,
> > > > > > > > > > > Sep
> > > > > > > > > > > >>> 11,
> > > > > > > > > > > >>> > > > 2020,
> > > > > > > > > > > >>> > > > >> at
> > > > > > > > > > > >>> > > > >> > > > 03:20,
> > > > > > > > > > > >>> > > > >> > > > > > Tom
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > Bentley
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > wrote:
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > Hi
> > > > > > > > > > > >>> Justine,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > This
> > > > > > > > > > > looks
> > > > > > > > > > > >>> > like
> > > > > > > > > > > >>> > > a
> > > > > > > > > > > >>> > > > >> very
> > > > > > > > > > > >>> > > > >> > > > > welcome
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > improvement.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Thanks!
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > Maybe
> > > > > > > > > > I
> > > > > > > > > > > >>> missed
> > > > > > > > > > > >>> > > it,
> > > > > > > > > > > >>> > > > >> but
> > > > > > > > > > > >>> > > > >> > > the
> > > > > > > > > > > >>> > > > >> > > > > KIP
> > > > > > > > > > > >>> > > > >> > > > > > > > >> doesn't
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > seem
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > mention
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > changing
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > > >>> > > > DeleteTopicsRequest
> > > > > > > > > > > >>> > > > >> to
> > > > > > > > > > > >>> > > > >> > > > > identify
> > > > > > > > > > > >>> > > > >> > > > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > topic
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > using
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > an
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > id.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> Maybe
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > > that's
> > > > > > > out
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > of
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > scope,
> > > > > > > > > > > but
> > > > > > > > > > > >>> > > > >> > > > > DeleteTopicsRequest
> > > > > > > > > > > >>> > > > >> > > > > > is
> > > > > > > > > > > >>> > > > >> > > > > > > > not
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > listed
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > among
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> Future
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > Work
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > APIs
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > either.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > Kind
> > > > > > > > > > > >>> regards,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > Tom
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > On
> > > > > > > > > > Thu,
> > > > > > > > > > > >>> Sep
> > > > > > > > > > > >>> > 10,
> > > > > > > > > > > >>> > > > >> 2020 at
> > > > > > > > > > > >>> > > > >> > > > 3:59
> > > > > > > > > > > >>> > > > >> > > > > PM
> > > > > > > > > > > >>> > > > >> > > > > > > > >> Satish
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > Duggana <
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > > > > > > > >>> > > > >> satish.duggana@gmail.com
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > wrote:
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > Thanks
> > > > > > > > > > > >>> > > > >> Lucas/Justine
> > > > > > > > > > > >>> > > > >> > > for
> > > > > > > > > > > >>> > > > >> > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > nice
> > > > > > > > > > > >>> > > > >> > > > > > > > >> KIP.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > It
> > > > > > > > > > has
> > > > > > > > > > > >>> > several
> > > > > > > > > > > >>> > > > >> > benefits
> > > > > > > > > > > >>> > > > >> > > > > which
> > > > > > > > > > > >>> > > > >> > > > > > > > also
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > include
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > simplifying
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > topic
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > > deletion
> > > > > > > > > > > >>> > > process
> > > > > > > > > > > >>> > > > >> by
> > > > > > > > > > > >>> > > > >> > > > > > controller
> > > > > > > > > > > >>> > > > >> > > > > > > > and
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > logs
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > cleanup
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> by
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > brokers
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > in
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > > corner
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > cases.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > Best,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > > Satish.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > On
> > > > > > > > > > > Wed,
> > > > > > > > > > > >>> Sep
> > > > > > > > > > > >>> > 9,
> > > > > > > > > > > >>> > > > >> 2020
> > > > > > > > > > > >>> > > > >> > at
> > > > > > > > > > > >>> > > > >> > > > > 10:07
> > > > > > > > > > > >>> > > > >> > > > > > PM
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > Justine
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Olshan <
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > > > > > > > > >>> jolshan@confluent.io
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > wrote:
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > > > Hello
> > > > > > > > > > > >>> all,
> > > > > > > > > > > >>> > > > it's
> > > > > > > > > > > >>> > > > >> > been
> > > > > > > > > > > >>> > > > >> > > > > > almost a
> > > > > > > > > > > >>> > > > >> > > > > > > > >> year!
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > I've
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > made
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > some
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > changes
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > this
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > KIP
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > and
> > > > > > > > > > > >>> hope to
> > > > > > > > > > > >>> > > > >> continue
> > > > > > > > > > > >>> > > > >> > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> discussion.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > > One
> > > > > > > > > > > >>> of the
> > > > > > > > > > > >>> > > > main
> > > > > > > > > > > >>> > > > >> > > changes
> > > > > > > > > > > >>> > > > >> > > > > > I've
> > > > > > > > > > > >>> > > > >> > > > > > > > >> added
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > is
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > now
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> metadata
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > response
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > > will
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > > include
> > > > > > > > > > > >>> the
> > > > > > > > > > > >>> > > > topic
> > > > > > > > > > > >>> > > > >> ID
> > > > > > > > > > > >>> > > > >> > > (as
> > > > > > > > > > > >>> > > > >> > > > > > Colin
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > suggested).
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Clients
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > can
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > obtain
> > > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > topicID
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > of a
> > > > > > > > > > > >>> given
> > > > > > > > > > > >>> > > topic
> > > > > > > > > > > >>> > > > >> > > through
> > > > > > > > > > > >>> > > > >> > > > a
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > TopicDescription.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > The
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> topicId
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> will
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > also
> > > > > > be
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > > included
> > > > > > > > > > > >>> > with
> > > > > > > > > > > >>> > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > UpdateMetadata
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > request.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > > Let
> > > > > > > > > > > me
> > > > > > > > > > > >>> > know
> > > > > > > > > > > >>> > > > what
> > > > > > > > > > > >>> > > > >> > you
> > > > > > > > > > > >>> > > > >> > > > all
> > > > > > > > > > > >>> > > > >> > > > > > > think.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > > > Thank
> > > > > > > > > > > >>> you,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > > > >>> Justine
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > On
> > > > > > > > > > > >>> > > 2019/09/13
> > > > > > > > > > > >>> > > > >> > > 16:38:26,
> > > > > > > > > > > >>> > > > >> > > > > > > "Colin
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > McCabe" <
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > > cmccabe@apache.org
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > > wrote:
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Hi
> > > > > > > > > > > >>> > Lucas,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > >>> Thanks
> > > > > > > > > > > >>> > for
> > > > > > > > > > > >>> > > > >> > tackling
> > > > > > > > > > > >>> > > > >> > > > > this.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> Topic
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > IDs
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > are a
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > great
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> idea,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > and
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > this
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > is
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > a
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > really
> > > > > > > > > > > >>> good
> > > > > > > > > > > >>> > > > >> writeup.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > For
> > > > > > > > > > > >>> > > > >> > > > > > > /brokers/topics/[topic],
> > > > > > > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > schema
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> version
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> should
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > be
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > > bumped
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > > version
> > > > > > > > > > > >>> 3,
> > > > > > > > > > > >>> > > > rather
> > > > > > > > > > > >>> > > > >> > than
> > > > > > > > > > > >>> > > > >> > > 2.
> > > > > > > > > > > >>> > > > >> > > > > > > > KIP-455
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > bumped
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > version
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> this
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > > znode
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > 2
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > > already
> > > > > > > > > > > >>> :)
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > >>> Given
> > > > > > > > > > > >>> > that
> > > > > > > > > > > >>> > > > >> we're
> > > > > > > > > > > >>> > > > >> > > > going
> > > > > > > > > > > >>> > > > >> > > > > to
> > > > > > > > > > > >>> > > > >> > > > > > > be
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > seeing
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > these
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > things
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > as
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > strings
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > as
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > > lot
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > (in
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > logs,
> > > > > > > > > > > in
> > > > > > > > > > > >>> > > > >> ZooKeeper,
> > > > > > > > > > > >>> > > > >> > on
> > > > > > > > > > > >>> > > > >> > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > command-line,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > etc.),
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > does
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > it
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> make
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > > sense
> > > > > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > use
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > base64
> > > > > > > > > > > >>> when
> > > > > > > > > > > >>> > > > >> > converting
> > > > > > > > > > > >>> > > > >> > > > them
> > > > > > > > > > > >>> > > > >> > > > > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > strings?
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > >>> Here is
> > > > > > > > > > > >>> > an
> > > > > > > > > > > >>> > > > >> > example
> > > > > > > > > > > >>> > > > >> > > of
> > > > > > > > > > > >>> > > > >> > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > hex
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > representation:
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > And
> > > > > > > > > > > >>> here
> > > > > > > > > > > >>> > > is
> > > > > > > > > > > >>> > > > an
> > > > > > > > > > > >>> > > > >> > > > example
> > > > > > > > > > > >>> > > > >> > > > > in
> > > > > > > > > > > >>> > > > >> > > > > > > > >> base64.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > >>> > > > >> > > > b8tRS7h4TJ2Vt43Dp85v2A
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > The
> > > > > > > > > > > >>> > base64
> > > > > > > > > > > >>> > > > >> > version
> > > > > > > > > > > >>> > > > >> > > > > saves
> > > > > > > > > > > >>> > > > >> > > > > > 15
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > letters
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > (to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > be
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > fair,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > 4
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > those
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > were
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > dashes
> > > > > > > > > > > >>> that
> > > > > > > > > > > >>> > we
> > > > > > > > > > > >>> > > > >> could
> > > > > > > > > > > >>> > > > >> > > have
> > > > > > > > > > > >>> > > > >> > > > > > > elided
> > > > > > > > > > > >>> > > > >> > > > > > > > in
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > hex
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > representation.)
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > >>> Another
> > > > > > > > > > > >>> > > > thing
> > > > > > > > > > > >>> > > > >> to
> > > > > > > > > > > >>> > > > >> > > > > consider
> > > > > > > > > > > >>> > > > >> > > > > > > is
> > > > > > > > > > > >>> > > > >> > > > > > > > >> that
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > we
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > should
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > specify
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> that
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > > >>> all-zeroes
> > > > > > > > > > > >>> > > UUID
> > > > > > > > > > > >>> > > > is
> > > > > > > > > > > >>> > > > >> > not
> > > > > > > > > > > >>> > > > >> > > a
> > > > > > > > > > > >>> > > > >> > > > > > valid
> > > > > > > > > > > >>> > > > >> > > > > > > > >> topic
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > UUID.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >  We
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > can't
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > use
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > null
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > for
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > > this
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > > because
> > > > > > > > > > > >>> we
> > > > > > > > > > > >>> > > can't
> > > > > > > > > > > >>> > > > >> > pass a
> > > > > > > > > > > >>> > > > >> > > > > null
> > > > > > > > > > > >>> > > > >> > > > > > > UUID
> > > > > > > > > > > >>> > > > >> > > > > > > > >> over
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > RPC
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > protocol
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > (there
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > is
> > > > > no
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > > special
> > > > > > > > > > > >>> > > pattern
> > > > > > > > > > > >>> > > > >> for
> > > > > > > > > > > >>> > > > >> > > null,
> > > > > > > > > > > >>> > > > >> > > > > nor
> > > > > > > > > > > >>> > > > >> > > > > > > do
> > > > > > > > > > > >>> > > > >> > > > > > > > we
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > want
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> waste
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > space
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > > reserving
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > > such
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > a
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > > >>> pattern.)
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > >>> Maybe I
> > > > > > > > > > > >>> > > > missed
> > > > > > > > > > > >>> > > > >> > it,
> > > > > > > > > > > >>> > > > >> > > > but
> > > > > > > > > > > >>> > > > >> > > > > > did
> > > > > > > > > > > >>> > > > >> > > > > > > > you
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > describe
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> "migration
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > of...
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > > > existing
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > > topic[s]
> > > > > > > > > > > >>> > > without
> > > > > > > > > > > >>> > > > >> > topic
> > > > > > > > > > > >>> > > > >> > > > IDs"
> > > > > > > > > > > >>> > > > >> > > > > > in
> > > > > > > > > > > >>> > > > >> > > > > > > > >> detail
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > in
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > any
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > section?
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > It
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > seems
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > like
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > when
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > the
> > > > > > > > > > > new
> > > > > > > > > > > >>> > > > controller
> > > > > > > > > > > >>> > > > >> > > > becomes
> > > > > > > > > > > >>> > > > >> > > > > > > > active,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> it
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > should
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> just
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > generate
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > random
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > UUIDs
> > > > > > > > > > for
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > these,
> > > > > > > > > > > >>> and
> > > > > > > > > > > >>> > > write
> > > > > > > > > > > >>> > > > >> the
> > > > > > > > > > > >>> > > > >> > > > random
> > > > > > > > > > > >>> > > > >> > > > > > > UUIDs
> > > > > > > > > > > >>> > > > >> > > > > > > > >> back
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > ZooKeeper.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > It
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > would
> > > > > be
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > > good
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > spell
> > > > > > > > > > > >>> that
> > > > > > > > > > > >>> > > out.
> > > > > > > > > > > >>> > > > >> We
> > > > > > > > > > > >>> > > > >> > > > should
> > > > > > > > > > > >>> > > > >> > > > > > make
> > > > > > > > > > > >>> > > > >> > > > > > > > it
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > clear
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> this
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > happens
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > > > > regardless
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > of
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > the
> > > > > > > > > > > >>> > > inter-broker
> > > > > > > > > > > >>> > > > >> > > protocol
> > > > > > > > > > > >>> > > > >> > > > > > > version
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > (it's
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > a
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > compatible
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > change).
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > >>> > > > >> > > "LeaderAndIsrRequests
> > > > > > > > > > > >>> > > > >> > > > > > > > >> including an
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > is_every_partition
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > flag"
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > > > seems a
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > bit
> > > > > > > > > > > >>> wordy.
> > > > > > > > > > > >>> > > Can
> > > > > > > > > > > >>> > > > we
> > > > > > > > > > > >>> > > > >> > just
> > > > > > > > > > > >>> > > > >> > > > > call
> > > > > > > > > > > >>> > > > >> > > > > > > > these
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > "full
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > > > > > > > >>> LeaderAndIsrRequests"?
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > Then
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > RPC
> > > > > > > > > > > >>> field
> > > > > > > > > > > >>> > > could
> > > > > > > > > > > >>> > > > be
> > > > > > > > > > > >>> > > > >> > > named
> > > > > > > > > > > >>> > > > >> > > > > > > "full".
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > Also,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > it
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > would
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> probably
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > be
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > > better
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > for
> > > > > > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > RPC
> > > > > > > > > > > >>> field to
> > > > > > > > > > > >>> > > be
> > > > > > > > > > > >>> > > > an
> > > > > > > > > > > >>> > > > >> > enum
> > > > > > > > > > > >>> > > > >> > > > of
> > > > > > > > > > > >>> > > > >> > > > > {
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > UNSPECIFIED,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > INCREMENTAL,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > FULL
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > },
> > > > so
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > > that
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > we
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > can
> > > > > > > > > > > >>> cleanly
> > > > > > > > > > > >>> > > > handle
> > > > > > > > > > > >>> > > > >> > old
> > > > > > > > > > > >>> > > > >> > > > > > versions
> > > > > > > > > > > >>> > > > >> > > > > > > > (by
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > treating
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> them
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > as
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > > > UNSPECIFIED)
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > In
> > > > > > > > > > > >>> the
> > > > > > > > > > > >>> > > > >> > > > > > LeaderAndIsrRequest
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > section,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > you
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> write
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > "A
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> final
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > > > deletion
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > event
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > will
> > > > > > > > > > > be
> > > > > > > > > > > >>> > > > secheduled
> > > > > > > > > > > >>> > > > >> > for
> > > > > > > > > > > >>> > > > >> > > X
> > > > > > > > > > > >>> > > > >> > > > ms
> > > > > > > > > > > >>> > > > >> > > > > > > after
> > > > > > > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > LeaderAndIsrRequest
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> was
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > > first
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > > >>> received..."
> > > > > > > > > > > >>> > > I
> > > > > > > > > > > >>> > > > >> guess
> > > > > > > > > > > >>> > > > >> > > > the X
> > > > > > > > > > > >>> > > > >> > > > > > > was a
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > placeholder
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> that
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > you
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > intended
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > replace
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > before
> > > > > > > > > > > >>> > > posting?
> > > > > > > > > > > >>> > > > :)
> > > > > > > > > > > >>> > > > >> > In
> > > > > > > > > > > >>> > > > >> > > > any
> > > > > > > > > > > >>> > > > >> > > > > > > case,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> this
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > seems
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > like
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > kind
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > of
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > > thing
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > > we'd
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > want a
> > > > > > > > > > > >>> > > > >> configuration
> > > > > > > > > > > >>> > > > >> > > for.
> > > > > > > > > > > >>> > > > >> > > > > > > Let's
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > describe
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > configuration
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > key
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > > somewhere
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > in
> > > > > > > > > > > this
> > > > > > > > > > > >>> KIP,
> > > > > > > > > > > >>> > > > >> > including
> > > > > > > > > > > >>> > > > >> > > > what
> > > > > > > > > > > >>> > > > >> > > > > > its
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > default
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > value
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> is.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > We
> > > > > > > > > > > >>> > should
> > > > > > > > > > > >>> > > > >> > probably
> > > > > > > > > > > >>> > > > >> > > > also
> > > > > > > > > > > >>> > > > >> > > > > > > log a
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > bunch
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > of
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> messages
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > at
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > WARN
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > level
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > > when
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > > >>> something is
> > > > > > > > > > > >>> > > > >> > scheduled
> > > > > > > > > > > >>> > > > >> > > > for
> > > > > > > > > > > >>> > > > >> > > > > > > > >> deletion,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > as
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > well.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > (Maybe
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > this
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > was
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > assumed,
> > > > > > > > > > > but
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > it
> > > > > > > > > > > >>> would be
> > > > > > > > > > > >>> > > good
> > > > > > > > > > > >>> > > > >> to
> > > > > > > > > > > >>> > > > >> > > > mention
> > > > > > > > > > > >>> > > > >> > > > > > > it).
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > I
> > > > > > > > > > > >>> feel
> > > > > > > > > > > >>> > > like
> > > > > > > > > > > >>> > > > >> there
> > > > > > > > > > > >>> > > > >> > > > are a
> > > > > > > > > > > >>> > > > >> > > > > > few
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > sections
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > should
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > be
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > moved
> > > > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > > "rejected
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > > >>> > alternatives."
> > > > > > > > > > > >>> > > > >> For
> > > > > > > > > > > >>> > > > >> > > > > example,
> > > > > > > > > > > >>> > > > >> > > > > > in
> > > > > > > > > > > >>> > > > >> > > > > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > DeleteTopics
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> section,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > since
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > > we're
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > not
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > going
> > > > > > > > > > > >>> to do
> > > > > > > > > > > >>> > > > >> option 1
> > > > > > > > > > > >>> > > > >> > or
> > > > > > > > > > > >>> > > > >> > > > 2,
> > > > > > > > > > > >>> > > > >> > > > > > > these
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > should
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > be
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > moved
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > into
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > > "rejected
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > > >>> > alternatives,"
> > > > > > > > > > > >>> > > > >> > rather
> > > > > > > > > > > >>> > > > >> > > > than
> > > > > > > > > > > >>> > > > >> > > > > > > > >> appearing
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > inline.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Another
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > case
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > is
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > "Should
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > we
> > > > > > > > > > > >>> remove
> > > > > > > > > > > >>> > > topic
> > > > > > > > > > > >>> > > > >> name
> > > > > > > > > > > >>> > > > >> > > from
> > > > > > > > > > > >>> > > > >> > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> protocol
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > where
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> possible"
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > section.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > This
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > is
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > > clearly
> > > > > > > > > > > >>> > > > >> discussing a
> > > > > > > > > > > >>> > > > >> > > > design
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > alternative
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > that
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> we're
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > not
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > > proposing
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > > >>> implement:
> > > > > > > > > > > >>> > > > >> removing
> > > > > > > > > > > >>> > > > >> > the
> > > > > > > > > > > >>> > > > >> > > > > topic
> > > > > > > > > > > >>> > > > >> > > > > > > > name
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > from
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > those
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > protocols.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Is
> > > > > > > > > > > >>> it
> > > > > > > > > > > >>> > > really
> > > > > > > > > > > >>> > > > >> > > > necessary
> > > > > > > > > > > >>> > > > >> > > > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > have
> > > > > > > > > > > >>> > > > >> > > > > > > > >> a
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > new
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > > > > > > > > >>> > > /admin/delete_topics_by_id
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > path
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > in
> > > > > > > > > > > >>> > ZooKeeper?
> > > > > > > > > > > >>> > > > It
> > > > > > > > > > > >>> > > > >> > seems
> > > > > > > > > > > >>> > > > >> > > > > like
> > > > > > > > > > > >>> > > > >> > > > > > we
> > > > > > > > > > > >>> > > > >> > > > > > > > >> don't
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > really
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> need
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > this.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > Whenever
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > there
> > > > > > > > > is
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > a
> > > > > > > > > > new
> > > > > > > > > > > >>> > > > controller,
> > > > > > > > > > > >>> > > > >> > we'll
> > > > > > > > > > > >>> > > > >> > > > > send
> > > > > > > > > > > >>> > > > >> > > > > > > out
> > > > > > > > > > > >>> > > > >> > > > > > > > >> full
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > LeaderAndIsrRequests
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > which
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > will
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > > trigger
> > > > > > > > > > > >>> the
> > > > > > > > > > > >>> > > > stale
> > > > > > > > > > > >>> > > > >> > > topics
> > > > > > > > > > > >>> > > > >> > > > to
> > > > > > > > > > > >>> > > > >> > > > > > be
> > > > > > > > > > > >>> > > > >> > > > > > > > >> cleaned
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > up.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >  The
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > active
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > > > > controller
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > > will
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > also
> > > > > > > > > > > >>> send
> > > > > > > > > > > >>> > the
> > > > > > > > > > > >>> > > > full
> > > > > > > > > > > >>> > > > >> > > > > > > > >> LeaderAndIsrRequest
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> brokers
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > are
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > just
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > starting
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > up.
> > > > > > > > > > > >>> So we
> > > > > > > > > > > >>> > > > don't
> > > > > > > > > > > >>> > > > >> > > really
> > > > > > > > > > > >>> > > > >> > > > > > need
> > > > > > > > > > > >>> > > > >> > > > > > > > this
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > kind
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > of
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > two-phase
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > commit
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > > (send
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > > out
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > > >>> > > > >> StopReplicasRequest,
> > > > > > > > > > > >>> > > > >> > > get
> > > > > > > > > > > >>> > > > >> > > > > ACKs
> > > > > > > > > > > >>> > > > >> > > > > > > > from
> > > > > > > > > > > >>> > > > >> > > > > > > > >> all
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > nodes,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > commit
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > by
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > removing
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > > >>> > > > >> /admin/delete_topics
> > > > > > > > > > > >>> > > > >> > > > node)
> > > > > > > > > > > >>> > > > >> > > > > > any
> > > > > > > > > > > >>> > > > >> > > > > > > > >> more.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > You
> > > > > > > > > > > >>> > > mention
> > > > > > > > > > > >>> > > > >> that
> > > > > > > > > > > >>> > > > >> > > > > > > FetchRequest
> > > > > > > > > > > >>> > > > >> > > > > > > > >> will
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > now
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> include
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > UUID
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > avoid
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > > > issues
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > where
> > > > > > > > > > > >>> > requests
> > > > > > > > > > > >>> > > > are
> > > > > > > > > > > >>> > > > >> > made
> > > > > > > > > > > >>> > > > >> > > > to
> > > > > > > > > > > >>> > > > >> > > > > > > stale
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > partitions.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > However,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > adding a
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > UUID
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > > >>> > > MetadataRequest
> > > > > > > > > > > >>> > > > is
> > > > > > > > > > > >>> > > > >> > > listed
> > > > > > > > > > > >>> > > > >> > > > > as
> > > > > > > > > > > >>> > > > >> > > > > > > > future
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > work,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > out
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > of
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > scope
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > for
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > this
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > KIP.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > How
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > will
> > > > > > > > > > > the
> > > > > > > > > > > >>> > > client
> > > > > > > > > > > >>> > > > >> learn
> > > > > > > > > > > >>> > > > >> > > > what
> > > > > > > > > > > >>> > > > >> > > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> topic
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > UUID
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > is,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > if
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > metadata
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > > > response
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > > doesn't
> > > > > > > > > > > >>> > > include
> > > > > > > > > > > >>> > > > >> that
> > > > > > > > > > > >>> > > > >> > > > > > > information?
> > > > > > > > > > > >>> > > > >> > > > > > > > >> It
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > seems
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > like
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > adding
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > UUID
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > > >>> > > MetadataResponse
> > > > > > > > > > > >>> > > > >> > would
> > > > > > > > > > > >>> > > > >> > > be
> > > > > > > > > > > >>> > > > >> > > > > an
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > improvement
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > here
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> that
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > might
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> not
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > be
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > too
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > hard
> > > > > > > > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > make.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > >>> best,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > >>> Colin
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On
> > > > > > > > > > > >>> Mon,
> > > > > > > > > > > >>> > > Sep
> > > > > > > > > > > >>> > > > 9,
> > > > > > > > > > > >>> > > > >> > > 2019,
> > > > > > > > > > > >>> > > > >> > > > at
> > > > > > > > > > > >>> > > > >> > > > > > > > 17:48,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > Ryanne
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Dolan
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > wrote:
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >>> Lucas,
> > > > > > > > > > > >>> > > > this
> > > > > > > > > > > >>> > > > >> > would
> > > > > > > > > > > >>> > > > >> > > > be
> > > > > > > > > > > >>> > > > >> > > > > > > great.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> I've
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > run
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > into
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > issues
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > with
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > > topics
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > being
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >>> > > > resurrected
> > > > > > > > > > > >>> > > > >> > > > > > accidentally,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> since
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > a
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > client
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > cannot
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > easily
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > > > distinguish
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > > between
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > a
> > > > > > > > > > > >>> > > deleted
> > > > > > > > > > > >>> > > > >> topic
> > > > > > > > > > > >>> > > > >> > > > and a
> > > > > > > > > > > >>> > > > >> > > > > > new
> > > > > > > > > > > >>> > > > >> > > > > > > > >> topic
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > with
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> same
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > name.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> I'd
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > need
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > ID
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >>> > > accessible
> > > > > > > > > > > >>> > > > >> from
> > > > > > > > > > > >>> > > > >> > > the
> > > > > > > > > > > >>> > > > >> > > > > > > client
> > > > > > > > > > > >>> > > > >> > > > > > > > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > solve
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > issue,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > but
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > this
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > is a
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > good
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > first
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >>> step.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >>> Ryanne
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > On
> > > > > > > > > > > >>> > Wed,
> > > > > > > > > > > >>> > > > Sep
> > > > > > > > > > > >>> > > > >> 4,
> > > > > > > > > > > >>> > > > >> > > 2019
> > > > > > > > > > > >>> > > > >> > > > > at
> > > > > > > > > > > >>> > > > >> > > > > > > 1:41
> > > > > > > > > > > >>> > > > >> > > > > > > > >> PM
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > Lucas
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > Bradstreet <
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > > > >>> > > lucas@confluent.io>
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > wrote:
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >>> Hi
> > > > > > > > > > > >>> > > all,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >>> I
> > > > > > > > > > > >>> > > would
> > > > > > > > > > > >>> > > > >> like
> > > > > > > > > > > >>> > > > >> > to
> > > > > > > > > > > >>> > > > >> > > > > kick
> > > > > > > > > > > >>> > > > >> > > > > > > off
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > discussion
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > of
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > KIP-516,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > an
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > > > >>> implementation
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > of
> > > > > > > > > > > topic
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >>> IDs
> > > > > > > > > > > >>> > > for
> > > > > > > > > > > >>> > > > >> > Kafka.
> > > > > > > > > > > >>> > > > >> > > > > Topic
> > > > > > > > > > > >>> > > > >> > > > > > > IDs
> > > > > > > > > > > >>> > > > >> > > > > > > > >> aim
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > to
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > solve
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> topic
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > > uniqueness
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > problems
> > > > > > > > > > > in
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > Kafka,
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >>> > where
> > > > > > > > > > > >>> > > > >> > referring
> > > > > > > > > > > >>> > > > >> > > > to
> > > > > > > > > > > >>> > > > >> > > > > a
> > > > > > > > > > > >>> > > > >> > > > > > > > topic
> > > > > > > > > > > >>> > > > >> > > > > > > > >> by
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > name
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> alone is
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > > > insufficient.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > > Such
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > cases
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >>> > > include
> > > > > > > > > > > >>> > > > >> when
> > > > > > > > > > > >>> > > > >> > a
> > > > > > > > > > > >>> > > > >> > > > > topic
> > > > > > > > > > > >>> > > > >> > > > > > > has
> > > > > > > > > > > >>> > > > >> > > > > > > > >> been
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > deleted
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > recreated
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > with
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > the
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > same
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > name.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >>> > Unique
> > > > > > > > > > > >>> > > > >> > > > identifiers
> > > > > > > > > > > >>> > > > >> > > > > > will
> > > > > > > > > > > >>> > > > >> > > > > > > > >> help
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > simplify
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> improve
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > Kafka's
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > > > topic
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > > deletion
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >>> > > process,
> > > > > > > > > > > >>> > > > >> as
> > > > > > > > > > > >>> > > > >> > > well
> > > > > > > > > > > >>> > > > >> > > > as
> > > > > > > > > > > >>> > > > >> > > > > > > > prevent
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > cases
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > where
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > brokers
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > may
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > > > > > > incorrectly
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > > interact
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >>> with
> > > > > > > > > > > >>> > > > stale
> > > > > > > > > > > >>> > > > >> > > > versions
> > > > > > > > > > > >>> > > > >> > > > > > of
> > > > > > > > > > > >>> > > > >> > > > > > > > >> topics.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > > > > > >>> > > > >> > > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > >
> > > > > > > > > > > >>> > > > >> > > > >
> > > > > > > > > > > >>> > > > >> > > >
> > > > > > > > > > > >>> > > > >> > >
> > > > > > > > > > > >>> > > > >> >
> > > > > > > > > > > >>> > > > >>
> > > > > > > > > > > >>> > > >
> > > > > > > > > > > >>> > >
> > > > > > > > > > > >>> >
> > > > > > > > > > > >>>
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >>> > > Looking
> > > > > > > > > > > >>> > > > >> > forward
> > > > > > > > > > > >>> > > > >> > > > to
> > > > > > > > > > > >>> > > > >> > > > > > your
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > thoughts.
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >>> > Lucas
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>>
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > > > > > >>> > > > >> > > > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > > >
> > > > > > > > > > > >>> > > > >> > > > > >
> > > > > > > > > > > >>> > > > >> > > > >
> > > > > > > > > > > >>> > > > >> > > >
> > > > > > > > > > > >>> > > > >> > >
> > > > > > > > > > > >>> > > > >> >
> > > > > > > > > > > >>> > > > >>
> > > > > > > > > > > >>> > > > >
> > > > > > > > > > > >>> > > >
> > > > > > > > > > > >>> > >
> > > > > > > > > > > >>> >
> > > > > > > > > > > >>>
> > > > > > > > > > > >>
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Jun Rao <ju...@confluent.io>]

Hi, Justine,

50. We can't change the format of existing versions since it will break
compatibility. Since we are bumping the version for MetadataRequest
and DeleteTopicsRequest, we can make the topic name field nullable from the
new version.

Thanks,

Jun

On Thu, Oct 8, 2020 at 2:19 PM Justine Olshan <jo...@confluent.io> wrote:

> Hi Jun,
>
> 50. Works for me. Is the idea that it will be nullable from this version
> onwards, but not past versions?
>
> 51. Added some information and example methods for AdminClient future work
>
> Thanks,
> Justine
>
>
> On Thu, Oct 8, 2020 at 10:59 AM Jun Rao <ju...@confluent.io> wrote:
>
> > Hi, Justine,
> >
> > The changes sound reasonable to me.
> >
> > 50. Should we make the topic name in the new MetadataRequest
> > and DeleteTopicsRequest nullable?
> >
> > 51. In the future work section, perhaps we could mention opportunities
> for
> > using topicId in other methods in AdminClient where topic name is used.
> >
> > Thanks,
> >
> > Jun
> >
> > On Wed, Oct 7, 2020 at 2:44 PM Justine Olshan <jo...@confluent.io>
> > wrote:
> >
> > > Hi all,
> > >
> > > Another update on the KIP. I've added some information about how topic
> > IDs
> > > will be supported in AdminClient
> > >
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-AdminClientSupport
> > >
> > >
> > > One specific decision I wanted to focus on is the inclusion of both
> topic
> > > names and topic Ids in metadata and deleteTopics requests.
> > >
> > > The idea is that when the request is received the topic id field is
> > checked
> > > first, but if it contains the default (zero) uuid, then the topic name
> > > field will be used.
> > >
> > > In the case where ids are used, if the ids do not have a corresponding
> > > topic, the request will fail. In the case where both ids and names are
> > > included, the request will fail if the name does not match the id.
> > >
> > >
> > > Initially the id field will be used to support the admin client use
> cases
> > > as described in the KIP, but over time, we can transfer to using topic
> > ids
> > > in all uses of the request.
> > >
> > >
> > > There will also be some support for returning the topic Id on the
> > > createtopics, deletetopics and metadata responses. I think it makes
> sense
> > > to include both the name and the id in the successful response for
> these
> > > cases.
> > >
> > >
> > > Let me know if you have any comments or concerns regarding the Admin
> > Client
> > > changes.
> > >
> > > On Wed, Oct 7, 2020 at 11:05 AM Jun Rao <ju...@confluent.io> wrote:
> > >
> > > > Hi, Justine,
> > > >
> > > > I was just wondering if we expose topicId publicly, how would people
> > use
> > > > it. One possibility is to use topicId in all those public interfaces
> > > where
> > > > we use topic name. I am not sure if the need is very strong though.
> So,
> > > > maybe we can list this as future considerations.
> > > >
> > > > Thanks,
> > > >
> > > > Jun
> > > >
> > > > On Wed, Oct 7, 2020 at 8:57 AM Justine Olshan <jo...@confluent.io>
> > > > wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > After some careful consideration, I've decided that the
> > > TopicIdPartition
> > > > > will be out of scope for this KIP. After investigating what it will
> > > look
> > > > > like using it in only the Fetch Request, I found that I would need
> to
> > > > > modify several public APIs in the consumer and I would need to take
> > > > special
> > > > > care in many situations to handle the request correctly based on
> what
> > > > > version of the request was sent. For these reasons, I've added
> > > > > TopicIdPartition to potential future work along with an alternative
> > to
> > > > > modify TopicPartition to include a topic ID field.
> > > > >
> > > > > Jun- I was also wondering about some of the details for supporting
> > > topic
> > > > > IDs for delete and describe topics in the admin client. Is the idea
> > > that
> > > > > instead of supplying topic names to describe or delete, we would
> > > supply a
> > > > > collection of ids? If that is the case, I'm wondering if it makes
> > sense
> > > > to
> > > > > add topic Ids to the metadata and delete topics requests.
> > > > >
> > > > > Thanks,
> > > > > Justine
> > > > >
> > > > > On Mon, Oct 5, 2020 at 9:25 PM Satish Duggana <
> > > satish.duggana@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hi Jun, Justine,
> > > > > > +1 to have TopicIdPartition in public packages. This is useful
> for
> > > > > > KIP-405 tiered storage too. We currently use TopicPartition for
> > > > > > pluggable log storage and metadata storage apis. We will be using
> > > > > > TopicIdPartition once the proposed changes in this KIP are
> merged.
> > > > > >
> > > > > > Thanks,
> > > > > > Satish.
> > > > > >
> > > > > > On Tue, Oct 6, 2020 at 5:21 AM Jun Rao <ju...@confluent.io> wrote:
> > > > > > >
> > > > > > > Hi, Justine,
> > > > > > >
> > > > > > > I guess we have to maintain the mapping between topic name and
> > > > topicId
> > > > > in
> > > > > > > both the client and the server. If TopicIdPartition is meant
> for
> > > > > internal
> > > > > > > usage only, we can create it under a package
> > > > > > > like org/apache/kafka/common/internal and we don't need to
> > document
> > > > it
> > > > > in
> > > > > > > the KIP. If it's meant for external public usage (and
> potentially
> > > > > > internal
> > > > > > > usage too), we can create it in org/apache/kafka/common and
> need
> > to
> > > > > > > document it in the KIP.
> > > > > > >
> > > > > > > Since topicId is now part of the topic metadata, perhaps it's
> > > useful
> > > > to
> > > > > > > expose it in TopicMetadataAndConfig too.
> > > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > > > Jun
> > > > > > >
> > > > > > > On Mon, Oct 5, 2020 at 3:35 PM Justine Olshan <
> > > jolshan@confluent.io>
> > > > > > wrote:
> > > > > > >
> > > > > > > > Hi Jun,
> > > > > > > >
> > > > > > > > This KIP removes the topic name from many requests. A lot of
> > > > requests
> > > > > > keep
> > > > > > > > track of the partitions internally with a TopicPartition. So
> > when
> > > > we
> > > > > > remove
> > > > > > > > the topic name, we can not differentiate unless we can also
> use
> > > > topic
> > > > > > ID.
> > > > > > > > The Fetch request relies heavily on this, but ListOffset,
> > > > > > > > OffsetsForLeaderEpoch, etc all use TopicPartition as a key
> > value
> > > > for
> > > > > > > > internal maps. Digging deeper into Fetch, I've noticed that
> the
> > > > code
> > > > > > path
> > > > > > > > runs into some public APIs that use the TopicPartition. Some
> > > > examples
> > > > > > are
> > > > > > > > Cluster.java or RecordTooLargeException.
> > > > > > > >
> > > > > > > > A related issue is that in some places like
> > > > > > TopicAuthorizationException,
> > > > > > > > the topic name is required and without the name in the
> request,
> > > we
> > > > > > won't
> > > > > > > > know it.
> > > > > > > >
> > > > > > > > I guess an alternative would be to have some mapping from
> topic
> > > > name
> > > > > to
> > > > > > > > topic ID, but I would have to think about that more.
> > > > > > > >
> > > > > > > > The KIP mentions exposing TopicId in TopicDescription. (The
> > font
> > > > was
> > > > > > > > weirdly colored, so I've made it easier to read.)
> > > > > > > >
> > > > > > > > Would the idea for the TopicMetadataAndConfig be that the
> topic
> > > ID
> > > > > > would be
> > > > > > > > returned as a result?
> > > > > > > > As for the other methods describe/deletetopics, it makes
> sense
> > to
> > > > > > include
> > > > > > > > them at some point, but I'm not sure how far we should go in
> > > terms
> > > > of
> > > > > > > > supporting topic IDs everywhere.
> > > > > > > >
> > > > > > > > Thank you for continuing to look at the KIP!
> > > > > > > > Justine
> > > > > > > >
> > > > > > > >
> > > > > > > > On Mon, Oct 5, 2020 at 2:53 PM Jun Rao <ju...@confluent.io>
> > wrote:
> > > > > > > >
> > > > > > > > > Hi, Justine,
> > > > > > > > >
> > > > > > > > > Adding a TopicIdPartition in the public interface may be
> > > useful.
> > > > > > Where
> > > > > > > > > would you expect it to be used?
> > > > > > > > >
> > > > > > > > > Related to this, since we are exposing topicId in the tool,
> > it
> > > > > seems
> > > > > > that
> > > > > > > > > we may want to expose topicId in at least TopicDescription
> > used
> > > > by
> > > > > > > > > AdminClient.describeTopics() and TopicMetadataAndConfig
> used
> > in
> > > > > > > > > AdminClient.createTopics(Collection<NewTopic> newTopics). I
> > am
> > > > also
> > > > > > not
> > > > > > > > > sure if we plan to expose topicId in other AdminClient
> > methods
> > > in
> > > > > > this
> > > > > > > > KIP
> > > > > > > > > or in some future KIP. For example, it might be useful to
> > > support
> > > > > > topicId
> > > > > > > > > in the following methods.
> > > > > > > > >
> > > > > > > > >     DescribeTopicsResult describeTopics(Collection<String>
> > > > > > topicNames,
> > > > > > > > > DescribeTopicsOptions options);
> > > > > > > > >     DeleteTopicsResult deleteTopics(Collection<String>
> > topics,
> > > > > > > > > DeleteTopicsOptions options);
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > >
> > > > > > > > > Jun
> > > > > > > > >
> > > > > > > > > On Mon, Oct 5, 2020 at 11:03 AM Justine Olshan <
> > > > > jolshan@confluent.io
> > > > > > >
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi all,
> > > > > > > > > >
> > > > > > > > > > I was thinking about this KIP some more. We are removing
> > the
> > > > > topic
> > > > > > name
> > > > > > > > > > from many of these request types, and some of them (eg.
> > > Fetch)
> > > > > use
> > > > > > > > > > TopicPartition as keys in a map when handling the
> request.
> > I
> > > > was
> > > > > > > > thinking
> > > > > > > > > > it would make sense to make some sort of TopicIdPartition
> > > with
> > > > > > topic
> > > > > > > > > name,
> > > > > > > > > > topic id, and partition fields (name and id would
> > potentially
> > > > be
> > > > > > > > optional
> > > > > > > > > > types since different request versions will contain only
> > one
> > > of
> > > > > the
> > > > > > > > two)
> > > > > > > > > I
> > > > > > > > > > was wondering if it made sense to include this as a
> public
> > > > facing
> > > > > > API
> > > > > > > > > > similar to TopicPartition. If so, I will add this class
> to
> > > the
> > > > > KIP
> > > > > > as
> > > > > > > > > well.
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > > Justine
> > > > > > > > > >
> > > > > > > > > > On Mon, Oct 5, 2020 at 8:52 AM Justine Olshan <
> > > > > > jolshan@confluent.io>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Hi all,
> > > > > > > > > > > I've made a few minor changes to the UUID class. I've
> > > added a
> > > > > few
> > > > > > > > more
> > > > > > > > > > > methods to support UUIDs more generally, by including a
> > > > > > constructor
> > > > > > > > > and a
> > > > > > > > > > > method to generate random UUIDs. Both of these will be
> > > useful
> > > > > for
> > > > > > > > > > testing.
> > > > > > > > > > >
> > > > > > > > > > > Let me know if there are any other areas for
> discussion.
> > > > > > > > > > >
> > > > > > > > > > > Thank you,
> > > > > > > > > > > Justine
> > > > > > > > > > >
> > > > > > > > > > > On Thu, Oct 1, 2020 at 5:29 PM Justine Olshan <
> > > > > > jolshan@confluent.io>
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > >> Hi Jun,
> > > > > > > > > > >> Thanks for looking at it again. Here's the new spec.
> (I
> > > > fixed
> > > > > > the
> > > > > > > > typo
> > > > > > > > > > in
> > > > > > > > > > >> it too.)
> > > > > > > > > > >> {"name": "id", "type": "string", "doc": option[UUID]}
> > > > > > > > > > >>
> > > > > > > > > > >> Justine
> > > > > > > > > > >>
> > > > > > > > > > >>
> > > > > > > > > > >> On Thu, Oct 1, 2020 at 5:03 PM Jun Rao <
> > jun@confluent.io>
> > > > > > wrote:
> > > > > > > > > > >>
> > > > > > > > > > >>> Hi, Justine,
> > > > > > > > > > >>>
> > > > > > > > > > >>> Thanks for the update. The KIP looks good to me now.
> > > Just a
> > > > > > minor
> > > > > > > > > > comment
> > > > > > > > > > >>> below.
> > > > > > > > > > >>>
> > > > > > > > > > >>> 30. Perhaps "option[UUID]" can be put in the doc.
> > > > > > > > > > >>>
> > > > > > > > > > >>> Jun
> > > > > > > > > > >>>
> > > > > > > > > > >>> On Thu, Oct 1, 2020 at 3:28 PM Justine Olshan <
> > > > > > > > jolshan@confluent.io>
> > > > > > > > > > >>> wrote:
> > > > > > > > > > >>>
> > > > > > > > > > >>> > Hi Jun,
> > > > > > > > > > >>> > Thanks for the response!
> > > > > > > > > > >>> >
> > > > > > > > > > >>> > 30. I think I might have changed this in between.
> The
> > > > > current
> > > > > > > > > version
> > > > > > > > > > >>> > says:  {"name":
> > > > > > > > > > >>> > "id", "type": "option[UUID]"}, "doc": topic id}
> > > > > > > > > > >>> > I have switched to the option type to cover the
> > > migration
> > > > > > case
> > > > > > > > > where
> > > > > > > > > > a
> > > > > > > > > > >>> > TopicZNode does not yet have a topic ID.
> > > > > > > > > > >>> > I understand that due to json, this field is
> written
> > > as a
> > > > > > string,
> > > > > > > > > so
> > > > > > > > > > >>> if I
> > > > > > > > > > >>> > should move the "option[uuid]" to the doc field and
> > the
> > > > > type
> > > > > > > > should
> > > > > > > > > > be
> > > > > > > > > > >>> > "string" please let me know.
> > > > > > > > > > >>> >
> > > > > > > > > > >>> > 40. I've added a definition for UUID.
> > > > > > > > > > >>> > 41,42. Fixed
> > > > > > > > > > >>> >
> > > > > > > > > > >>> > Thank you,
> > > > > > > > > > >>> > Justine
> > > > > > > > > > >>> >
> > > > > > > > > > >>> > On Wed, Sep 30, 2020 at 1:15 PM Jun Rao <
> > > > jun@confluent.io>
> > > > > > > > wrote:
> > > > > > > > > > >>> >
> > > > > > > > > > >>> > > Hi, Justine,
> > > > > > > > > > >>> > >
> > > > > > > > > > >>> > > Thanks for the summary. Just a few minor comments
> > > blow.
> > > > > > > > > > >>> > >
> > > > > > > > > > >>> > > 30.  {"name": "id", "type": "string", "doc":
> > "version
> > > > > > id"}}:
> > > > > > > > The
> > > > > > > > > > doc
> > > > > > > > > > >>> > should
> > > > > > > > > > >>> > > say UUID. The issue seems unfixed.
> > > > > > > > > > >>> > >
> > > > > > > > > > >>> > > 40. Since UUID is public facing, could you
> include
> > > its
> > > > > > > > > definition?
> > > > > > > > > > >>> > >
> > > > > > > > > > >>> > > 41. StopReplicaResponse still includes the topic
> > > field.
> > > > > > > > > > >>> > >
> > > > > > > > > > >>> > > 42. "It is unnecessary to include the name of the
> > > topic
> > > > > in
> > > > > > the
> > > > > > > > > > >>> following
> > > > > > > > > > >>> > > Request/Response calls" It would be useful to
> > include
> > > > all
> > > > > > > > > modified
> > > > > > > > > > >>> > requests
> > > > > > > > > > >>> > > (e.g. produce) in the list below.
> > > > > > > > > > >>> > >
> > > > > > > > > > >>> > > Thanks,
> > > > > > > > > > >>> > >
> > > > > > > > > > >>> > > Jun
> > > > > > > > > > >>> > >
> > > > > > > > > > >>> > >
> > > > > > > > > > >>> > > On Wed, Sep 30, 2020 at 10:17 AM Justine Olshan <
> > > > > > > > > > >>> jolshan@confluent.io>
> > > > > > > > > > >>> > > wrote:
> > > > > > > > > > >>> > >
> > > > > > > > > > >>> > > > Hello again,
> > > > > > > > > > >>> > > >
> > > > > > > > > > >>> > > > I've taken some time to discuss some of the
> > > remaining
> > > > > > points
> > > > > > > > > > >>> brought up
> > > > > > > > > > >>> > > by
> > > > > > > > > > >>> > > > the previous emails offline. Here are some of
> the
> > > > > > > > conclusions.
> > > > > > > > > > >>> > > >
> > > > > > > > > > >>> > > > 1. Directory Structure:
> > > > > > > > > > >>> > > > There was some talk about whether the directory
> > > > > structure
> > > > > > > > > should
> > > > > > > > > > be
> > > > > > > > > > >>> > > changed
> > > > > > > > > > >>> > > > to replace all topic names with topic IDs.
> > > > > > > > > > >>> > > > This will be a useful change, but will prevent
> > > > > > downgrades. It
> > > > > > > > > > will
> > > > > > > > > > >>> be
> > > > > > > > > > >>> > > best
> > > > > > > > > > >>> > > > to wait until a major release, likely alongside
> > > > KIP-500
> > > > > > > > changes
> > > > > > > > > > >>> that
> > > > > > > > > > >>> > will
> > > > > > > > > > >>> > > > prevent downgrades. I've updated the KIP to
> > include
> > > > > this
> > > > > > > > change
> > > > > > > > > > >>> with
> > > > > > > > > > >>> > the
> > > > > > > > > > >>> > > > note about migration and deprecation.
> > > > > > > > > > >>> > > >
> > > > > > > > > > >>> > > >
> > > > > > > > > > >>> > >
> > > > > > > > > > >>> >
> > > > > > > > > > >>>
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-log.dirlayout
> > > > > > > > > > >>> > > >
> > > > > > > > > > >>> > > > 2. Partition Metadata file
> > > > > > > > > > >>> > > > There was some disagreement about the inclusion
> > of
> > > > the
> > > > > > > > > partition
> > > > > > > > > > >>> > metadata
> > > > > > > > > > >>> > > > file.
> > > > > > > > > > >>> > > > This file will be needed to persist the topic
> ID,
> > > > > > especially
> > > > > > > > > > while
> > > > > > > > > > >>> we
> > > > > > > > > > >>> > > still
> > > > > > > > > > >>> > > > have the old directory structure. Even after
> the
> > > > > changes,
> > > > > > > > this
> > > > > > > > > > >>> file can
> > > > > > > > > > >>> > > be
> > > > > > > > > > >>> > > > useful for debugging and recovery.
> > > > > > > > > > >>> > > > Creating a single mapping file from topic names
> > to
> > > > > topic
> > > > > > IDs
> > > > > > > > > was
> > > > > > > > > > >>> > > > considered, but ultimately scrapped as it would
> > not
> > > > be
> > > > > as
> > > > > > > > easy
> > > > > > > > > to
> > > > > > > > > > >>> > > maintain.
> > > > > > > > > > >>> > > >
> > > > > > > > > > >>> > > >
> > > > > > > > > > >>> > >
> > > > > > > > > > >>> >
> > > > > > > > > > >>>
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PartitionMetadatafile
> > > > > > > > > > >>> > > >
> > > > > > > > > > >>> > > >
> > > > > > > > > > >>> > > >
> > > > > > > > > > >>> > >
> > > > > > > > > > >>> >
> > > > > > > > > > >>>
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PersistingTopicIDs
> > > > > > > > > > >>> > > >
> > > > > > > > > > >>> > > > 3. Produce Protocols
> > > > > > > > > > >>> > > > After some further discussion, this replacing
> the
> > > > topic
> > > > > > name
> > > > > > > > > with
> > > > > > > > > > >>> topic
> > > > > > > > > > >>> > > ID
> > > > > > > > > > >>> > > > in these protocols has been added to the KIP.
> > > > > > > > > > >>> > > > This will cut down on the size of the protocol.
> > > Since
> > > > > > changes
> > > > > > > > > to
> > > > > > > > > > >>> fetch
> > > > > > > > > > >>> > > are
> > > > > > > > > > >>> > > > included, it does make sense to update these
> > > > protocols.
> > > > > > > > > > >>> > > >
> > > > > > > > > > >>> > > >
> > > > > > > > > > >>> > >
> > > > > > > > > > >>> >
> > > > > > > > > > >>>
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-Produce
> > > > > > > > > > >>> > > >
> > > > > > > > > > >>> > > > 4. KIP-500 Compatibility
> > > > > > > > > > >>> > > > After some discussion with those involved with
> > > > KIP-500,
> > > > > > it
> > > > > > > > > seems
> > > > > > > > > > >>> best
> > > > > > > > > > >>> > to
> > > > > > > > > > >>> > > > use a sentinel topic ID for the metadata topic
> > that
> > > > is
> > > > > > used
> > > > > > > > > > before
> > > > > > > > > > >>> the
> > > > > > > > > > >>> > > > first controller election.
> > > > > > > > > > >>> > > > We had an issue where this metadata topic may
> not
> > > be
> > > > > > assigned
> > > > > > > > > an
> > > > > > > > > > ID
> > > > > > > > > > >>> > > before
> > > > > > > > > > >>> > > > utilizing the Vote and Fetch protocols. It was
> > > > decided
> > > > > to
> > > > > > > > > > reserve a
> > > > > > > > > > >>> > > unique
> > > > > > > > > > >>> > > > ID for this topic to be used until the
> controller
> > > > could
> > > > > > give
> > > > > > > > > the
> > > > > > > > > > >>> topic
> > > > > > > > > > >>> > a
> > > > > > > > > > >>> > > > unique ID.
> > > > > > > > > > >>> > > > Having the topic keep the sentinel ID (not be
> > > > > reassigned
> > > > > > to a
> > > > > > > > > > >>> unique
> > > > > > > > > > >>> > ID)
> > > > > > > > > > >>> > > > was considered, but it seemed like a good idea
> to
> > > > leave
> > > > > > the
> > > > > > > > > > option
> > > > > > > > > > >>> open
> > > > > > > > > > >>> > > for
> > > > > > > > > > >>> > > > the metadata topic to have a unique ID in cases
> > > where
> > > > > it
> > > > > > > > would
> > > > > > > > > > >>> need to
> > > > > > > > > > >>> > be
> > > > > > > > > > >>> > > > differentiated from other clusters' metadata
> > > topics.
> > > > > (ex.
> > > > > > > > > tiered
> > > > > > > > > > >>> > storage)
> > > > > > > > > > >>> > > >
> > > > > > > > > > >>> > > >
> > > > > > > > > > >>> > >
> > > > > > > > > > >>> >
> > > > > > > > > > >>>
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-CompatibilitywithKIP-500
> > > > > > > > > > >>> > > >
> > > > > > > > > > >>> > > > I've also split up the KIP into sub-tasks on
> the
> > > > JIRA.
> > > > > > > > > Hopefully
> > > > > > > > > > >>> this
> > > > > > > > > > >>> > > will
> > > > > > > > > > >>> > > > provide a better idea about what tasks we have,
> > and
> > > > > > > > eventually
> > > > > > > > > > >>> provide
> > > > > > > > > > >>> > a
> > > > > > > > > > >>> > > > place to see what's done and what is left.
> > > > > > > > > > >>> > > > If there is a task I am missing, please let me
> > > know!
> > > > > > > > > > >>> > > >
> https://issues.apache.org/jira/browse/KAFKA-8872
> > > > > > > > > > >>> > > >
> > > > > > > > > > >>> > > > Of course, these decisions are not set in
> stone,
> > > and
> > > > I
> > > > > > would
> > > > > > > > > love
> > > > > > > > > > >>> to
> > > > > > > > > > >>> > hear
> > > > > > > > > > >>> > > > any feedback.
> > > > > > > > > > >>> > > >
> > > > > > > > > > >>> > > > Thanks,
> > > > > > > > > > >>> > > > Justine
> > > > > > > > > > >>> > > >
> > > > > > > > > > >>> > > > On Mon, Sep 28, 2020 at 11:38 AM Justine
> Olshan <
> > > > > > > > > > >>> jolshan@confluent.io>
> > > > > > > > > > >>> > > > wrote:
> > > > > > > > > > >>> > > >
> > > > > > > > > > >>> > > > > Hello all,
> > > > > > > > > > >>> > > > >
> > > > > > > > > > >>> > > > > I just wanted to follow up on this
> discussion.
> > > Did
> > > > we
> > > > > > come
> > > > > > > > to
> > > > > > > > > > an
> > > > > > > > > > >>> > > > > understanding about the directory structure?
> > > > > > > > > > >>> > > > >
> > > > > > > > > > >>> > > > > I think the biggest question here is what is
> > > > > > acceptable to
> > > > > > > > > > leave
> > > > > > > > > > >>> out
> > > > > > > > > > >>> > > due
> > > > > > > > > > >>> > > > > to scope vs. what is considered to be too
> much
> > > tech
> > > > > > debt.
> > > > > > > > > > >>> > > > > This KIP is already pretty large with the
> > number
> > > of
> > > > > > > > changes,
> > > > > > > > > > but
> > > > > > > > > > >>> it
> > > > > > > > > > >>> > > also
> > > > > > > > > > >>> > > > > makes sense to do things correctly, so I'd
> love
> > > to
> > > > > hear
> > > > > > > > > > >>> everyone's
> > > > > > > > > > >>> > > > thoughts.
> > > > > > > > > > >>> > > > >
> > > > > > > > > > >>> > > > > Thanks,
> > > > > > > > > > >>> > > > > Justine
> > > > > > > > > > >>> > > > >
> > > > > > > > > > >>> > > > > On Fri, Sep 25, 2020 at 8:19 AM Lucas
> > Bradstreet
> > > <
> > > > > > > > > > >>> lucas@confluent.io
> > > > > > > > > > >>> > >
> > > > > > > > > > >>> > > > > wrote:
> > > > > > > > > > >>> > > > >
> > > > > > > > > > >>> > > > >> Hi Ismael,
> > > > > > > > > > >>> > > > >>
> > > > > > > > > > >>> > > > >> If you do not store it in a metadata file or
> > in
> > > > the
> > > > > > > > > directory
> > > > > > > > > > >>> > > structure
> > > > > > > > > > >>> > > > >> would you then
> > > > > > > > > > >>> > > > >> require the LeaderAndIsrRequest following
> > > startup
> > > > to
> > > > > > give
> > > > > > > > > you
> > > > > > > > > > >>> some
> > > > > > > > > > >>> > > > notion
> > > > > > > > > > >>> > > > >> of
> > > > > > > > > > >>> > > > >> topic name in memory? We would still need
> this
> > > > > > information
> > > > > > > > > for
> > > > > > > > > > >>> the
> > > > > > > > > > >>> > > older
> > > > > > > > > > >>> > > > >> protocols, but
> > > > > > > > > > >>> > > > >> perhaps this is what's meant by tech debt.
> > > > > > > > > > >>> > > > >>
> > > > > > > > > > >>> > > > >> Once we're free of the old non-topicID
> > requests
> > > > > then I
> > > > > > > > think
> > > > > > > > > > you
> > > > > > > > > > >>> > > > wouldn't
> > > > > > > > > > >>> > > > >> need to retain the topic name.
> > > > > > > > > > >>> > > > >> I think the ability to easily look up topic
> > > names
> > > > > > > > associated
> > > > > > > > > > >>> with
> > > > > > > > > > >>> > > > >> partition
> > > > > > > > > > >>> > > > >> directories would still be missed
> > > > > > > > > > >>> > > > >> when diagnosing issues, though maybe it
> > wouldn't
> > > > be
> > > > > a
> > > > > > deal
> > > > > > > > > > >>> breaker
> > > > > > > > > > >>> > > with
> > > > > > > > > > >>> > > > >> the
> > > > > > > > > > >>> > > > >> right tooling.
> > > > > > > > > > >>> > > > >>
> > > > > > > > > > >>> > > > >> Thanks,
> > > > > > > > > > >>> > > > >>
> > > > > > > > > > >>> > > > >> Lucas
> > > > > > > > > > >>> > > > >>
> > > > > > > > > > >>> > > > >> On Fri, Sep 25, 2020 at 7:55 AM Ismael Juma
> <
> > > > > > > > > > ismael@juma.me.uk>
> > > > > > > > > > >>> > > wrote:
> > > > > > > > > > >>> > > > >>
> > > > > > > > > > >>> > > > >> > Hi Lucas,
> > > > > > > > > > >>> > > > >> >
> > > > > > > > > > >>> > > > >> > Why would you include the name and id? I
> > think
> > > > > you'd
> > > > > > > > want
> > > > > > > > > to
> > > > > > > > > > >>> > remove
> > > > > > > > > > >>> > > > the
> > > > > > > > > > >>> > > > >> > name from the directory name right?
> Jason's
> > > > > > suggestion
> > > > > > > > was
> > > > > > > > > > >>> that if
> > > > > > > > > > >>> > > you
> > > > > > > > > > >>> > > > >> > remove the name from the directory, then
> why
> > > > would
> > > > > > you
> > > > > > > > > need
> > > > > > > > > > >>> the id
> > > > > > > > > > >>> > > > name
> > > > > > > > > > >>> > > > >> > mapping file?
> > > > > > > > > > >>> > > > >> >
> > > > > > > > > > >>> > > > >> > Ismael
> > > > > > > > > > >>> > > > >> >
> > > > > > > > > > >>> > > > >> > On Thu, Sep 24, 2020 at 4:24 PM Lucas
> > > > Bradstreet <
> > > > > > > > > > >>> > > lucas@confluent.io>
> > > > > > > > > > >>> > > > >> > wrote:
> > > > > > > > > > >>> > > > >> >
> > > > > > > > > > >>> > > > >> > > > 2. Part of the usage of the file is to
> > > have
> > > > > > > > persistent
> > > > > > > > > > >>> storage
> > > > > > > > > > >>> > > of
> > > > > > > > > > >>> > > > >> the
> > > > > > > > > > >>> > > > >> > > topic
> > > > > > > > > > >>> > > > >> > > ID and use it to compare with the ID
> > > supplied
> > > > in
> > > > > > the
> > > > > > > > > > >>> > LeaderAndIsr
> > > > > > > > > > >>> > > > >> > Request.
> > > > > > > > > > >>> > > > >> > > There is some discussion in the KIP
> about
> > > > > changes
> > > > > > to
> > > > > > > > the
> > > > > > > > > > >>> > directory
> > > > > > > > > > >>> > > > >> > > structure, but I believe directory
> changes
> > > > were
> > > > > > > > > considered
> > > > > > > > > > >>> to be
> > > > > > > > > > >>> > > out
> > > > > > > > > > >>> > > > >> of
> > > > > > > > > > >>> > > > >> > > scope when the KIP was written.
> > > > > > > > > > >>> > > > >> > >
> > > > > > > > > > >>> > > > >> > >
> > > > > > > > > > >>> > > > >> > > Yeah, I was hoping to get a better
> > > > understanding
> > > > > > of
> > > > > > > > why
> > > > > > > > > it
> > > > > > > > > > >>> was
> > > > > > > > > > >>> > > taken
> > > > > > > > > > >>> > > > >> out
> > > > > > > > > > >>> > > > >> > of
> > > > > > > > > > >>> > > > >> > > scope. Perhaps Lucas Bradstreet might
> have
> > > > more
> > > > > > > > insight
> > > > > > > > > > >>> about
> > > > > > > > > > >>> > the
> > > > > > > > > > >>> > > > >> > decision.
> > > > > > > > > > >>> > > > >> > > Basically my point is that we have to
> > create
> > > > > > > > additional
> > > > > > > > > > >>> > > > infrastructure
> > > > > > > > > > >>> > > > >> > here
> > > > > > > > > > >>> > > > >> > > to support the name/id mapping, so I
> > wanted
> > > to
> > > > > > > > > understand
> > > > > > > > > > >>> if we
> > > > > > > > > > >>> > > just
> > > > > > > > > > >>> > > > >> > > consider this a sort of tech debt. It
> > would
> > > be
> > > > > > useful
> > > > > > > > to
> > > > > > > > > > >>> cover
> > > > > > > > > > >>> > how
> > > > > > > > > > >>> > > > we
> > > > > > > > > > >>> > > > >> > > handle the case when this file gets
> > > corrupted.
> > > > > > Seems
> > > > > > > > > like
> > > > > > > > > > we
> > > > > > > > > > >>> > just
> > > > > > > > > > >>> > > > >> have to
> > > > > > > > > > >>> > > > >> > > trust that it matches whatever the
> > > controller
> > > > > > tells us
> > > > > > > > > and
> > > > > > > > > > >>> > rewrite
> > > > > > > > > > >>> > > > it?
> > > > > > > > > > >>> > > > >> > >
> > > > > > > > > > >>> > > > >> > >
> > > > > > > > > > >>> > > > >> > > Hi Jason, Justine,
> > > > > > > > > > >>> > > > >> > >
> > > > > > > > > > >>> > > > >> > > My thought process is that we will
> likely
> > > need
> > > > > the
> > > > > > > > > > metadata
> > > > > > > > > > >>> file
> > > > > > > > > > >>> > > > >> whether
> > > > > > > > > > >>> > > > >> > we
> > > > > > > > > > >>> > > > >> > > rename the directories or not.
> > > > > > > > > > >>> > > > >> > > Linux supports filenames of up to 255
> > bytes
> > > > and
> > > > > > that
> > > > > > > > > would
> > > > > > > > > > >>> not
> > > > > > > > > > >>> > be
> > > > > > > > > > >>> > > > >> enough
> > > > > > > > > > >>> > > > >> > to
> > > > > > > > > > >>> > > > >> > > support a directory name
> > > > > > > > > > >>> > > > >> > >  including both the name and topic ID.
> > Given
> > > > > that
> > > > > > > > fact,
> > > > > > > > > it
> > > > > > > > > > >>> > seemed
> > > > > > > > > > >>> > > > >> > > reasonable to add the metadata file
> > > > > > > > > > >>> > > > >> > > and leave the directory rename until
> some
> > > time
> > > > > in
> > > > > > the
> > > > > > > > > > future
> > > > > > > > > > >>> > > > (possibly
> > > > > > > > > > >>> > > > >> > > never).
> > > > > > > > > > >>> > > > >> > >
> > > > > > > > > > >>> > > > >> > > If the file does get corrupted I think
> > > you're
> > > > > > right.
> > > > > > > > We
> > > > > > > > > > >>> would
> > > > > > > > > > >>> > > either
> > > > > > > > > > >>> > > > >> have
> > > > > > > > > > >>> > > > >> > > to trust it matches what the controller
> > > tells
> > > > us
> > > > > > > > > > >>> > > > >> > >  or error out and let an administrator
> > > resolve
> > > > > it
> > > > > > by
> > > > > > > > > > >>> checking
> > > > > > > > > > >>> > > across
> > > > > > > > > > >>> > > > >> > > replicas for consistency.
> > > > > > > > > > >>> > > > >> > >
> > > > > > > > > > >>> > > > >> > > Lucas
> > > > > > > > > > >>> > > > >> > >
> > > > > > > > > > >>> > > > >> > >
> > > > > > > > > > >>> > > > >> > > On Thu, Sep 24, 2020 at 3:41 PM Jason
> > > > Gustafson
> > > > > <
> > > > > > > > > > >>> > > jason@confluent.io
> > > > > > > > > > >>> > > > >
> > > > > > > > > > >>> > > > >> > > wrote:
> > > > > > > > > > >>> > > > >> > >
> > > > > > > > > > >>> > > > >> > > > Thanks Justine. Responses below:
> > > > > > > > > > >>> > > > >> > > >
> > > > > > > > > > >>> > > > >> > > > > 1. Yes, the directory will still be
> > > based
> > > > on
> > > > > > the
> > > > > > > > > topic
> > > > > > > > > > >>> > names.
> > > > > > > > > > >>> > > > >> > > > LeaderAndIsrRequest is one of the few
> > > > requests
> > > > > > that
> > > > > > > > > will
> > > > > > > > > > >>> still
> > > > > > > > > > >>> > > > >> contain
> > > > > > > > > > >>> > > > >> > > the
> > > > > > > > > > >>> > > > >> > > > topic name. So I think we have this
> > > covered.
> > > > > > Sorry
> > > > > > > > for
> > > > > > > > > > >>> > > confusion.
> > > > > > > > > > >>> > > > >> > > >
> > > > > > > > > > >>> > > > >> > > > Ah, you're right. My eyes passed right
> > > over
> > > > > the
> > > > > > > > field.
> > > > > > > > > > >>> > > > >> > > >
> > > > > > > > > > >>> > > > >> > > > > 2. Part of the usage of the file is
> to
> > > > have
> > > > > > > > > persistent
> > > > > > > > > > >>> > storage
> > > > > > > > > > >>> > > > of
> > > > > > > > > > >>> > > > >> the
> > > > > > > > > > >>> > > > >> > > > topic
> > > > > > > > > > >>> > > > >> > > > ID and use it to compare with the ID
> > > > supplied
> > > > > > in the
> > > > > > > > > > >>> > > LeaderAndIsr
> > > > > > > > > > >>> > > > >> > > Request.
> > > > > > > > > > >>> > > > >> > > > There is some discussion in the KIP
> > about
> > > > > > changes to
> > > > > > > > > the
> > > > > > > > > > >>> > > directory
> > > > > > > > > > >>> > > > >> > > > structure, but I believe directory
> > changes
> > > > > were
> > > > > > > > > > >>> considered to
> > > > > > > > > > >>> > be
> > > > > > > > > > >>> > > > >> out of
> > > > > > > > > > >>> > > > >> > > > scope when the KIP was written.
> > > > > > > > > > >>> > > > >> > > >
> > > > > > > > > > >>> > > > >> > > > Yeah, I was hoping to get a better
> > > > > > understanding of
> > > > > > > > > why
> > > > > > > > > > >>> it was
> > > > > > > > > > >>> > > > taken
> > > > > > > > > > >>> > > > >> > out
> > > > > > > > > > >>> > > > >> > > of
> > > > > > > > > > >>> > > > >> > > > scope. Perhaps Lucas Bradstreet might
> > have
> > > > > more
> > > > > > > > > insight
> > > > > > > > > > >>> about
> > > > > > > > > > >>> > > the
> > > > > > > > > > >>> > > > >> > > decision.
> > > > > > > > > > >>> > > > >> > > > Basically my point is that we have to
> > > create
> > > > > > > > > additional
> > > > > > > > > > >>> > > > >> infrastructure
> > > > > > > > > > >>> > > > >> > > here
> > > > > > > > > > >>> > > > >> > > > to support the name/id mapping, so I
> > > wanted
> > > > to
> > > > > > > > > > understand
> > > > > > > > > > >>> if
> > > > > > > > > > >>> > we
> > > > > > > > > > >>> > > > just
> > > > > > > > > > >>> > > > >> > > > consider this a sort of tech debt. It
> > > would
> > > > be
> > > > > > > > useful
> > > > > > > > > to
> > > > > > > > > > >>> cover
> > > > > > > > > > >>> > > how
> > > > > > > > > > >>> > > > >> we
> > > > > > > > > > >>> > > > >> > > > handle the case when this file gets
> > > > corrupted.
> > > > > > Seems
> > > > > > > > > > like
> > > > > > > > > > >>> we
> > > > > > > > > > >>> > > just
> > > > > > > > > > >>> > > > >> have
> > > > > > > > > > >>> > > > >> > to
> > > > > > > > > > >>> > > > >> > > > trust that it matches whatever the
> > > > controller
> > > > > > tells
> > > > > > > > us
> > > > > > > > > > and
> > > > > > > > > > >>> > > rewrite
> > > > > > > > > > >>> > > > >> it?
> > > > > > > > > > >>> > > > >> > > >
> > > > > > > > > > >>> > > > >> > > > > 3. I think this is a good point,
> but I
> > > > > again I
> > > > > > > > > wonder
> > > > > > > > > > >>> about
> > > > > > > > > > >>> > > the
> > > > > > > > > > >>> > > > >> scope
> > > > > > > > > > >>> > > > >> > > of
> > > > > > > > > > >>> > > > >> > > > the KIP. Most of the changes mentioned
> > in
> > > > the
> > > > > > KIP
> > > > > > > > are
> > > > > > > > > > for
> > > > > > > > > > >>> > > > supporting
> > > > > > > > > > >>> > > > >> > > topic
> > > > > > > > > > >>> > > > >> > > > deletion and I believe that is why the
> > > > produce
> > > > > > > > request
> > > > > > > > > > was
> > > > > > > > > > >>> > > listed
> > > > > > > > > > >>> > > > >> under
> > > > > > > > > > >>> > > > >> > > > future work.
> > > > > > > > > > >>> > > > >> > > >
> > > > > > > > > > >>> > > > >> > > > That's fair. I brought it up since
> > `Fetch`
> > > > is
> > > > > > > > already
> > > > > > > > > > >>> > included.
> > > > > > > > > > >>> > > If
> > > > > > > > > > >>> > > > >> > we've
> > > > > > > > > > >>> > > > >> > > > got `Metadata` and `Fetch`, seems we
> may
> > > as
> > > > > > well do
> > > > > > > > > > >>> `Produce`
> > > > > > > > > > >>> > > and
> > > > > > > > > > >>> > > > >> save
> > > > > > > > > > >>> > > > >> > an
> > > > > > > > > > >>> > > > >> > > > extra kip. No strong objection though
> if
> > > you
> > > > > > want to
> > > > > > > > > > >>> leave it
> > > > > > > > > > >>> > > out.
> > > > > > > > > > >>> > > > >> > > >
> > > > > > > > > > >>> > > > >> > > >
> > > > > > > > > > >>> > > > >> > > > -Jason
> > > > > > > > > > >>> > > > >> > > >
> > > > > > > > > > >>> > > > >> > > >
> > > > > > > > > > >>> > > > >> > > > On Thu, Sep 24, 2020 at 3:26 PM
> Justine
> > > > > Olshan <
> > > > > > > > > > >>> > > > >> jolshan@confluent.io>
> > > > > > > > > > >>> > > > >> > > > wrote:
> > > > > > > > > > >>> > > > >> > > >
> > > > > > > > > > >>> > > > >> > > > > Hi Jason,
> > > > > > > > > > >>> > > > >> > > > >
> > > > > > > > > > >>> > > > >> > > > > Thanks for your comments.
> > > > > > > > > > >>> > > > >> > > > >
> > > > > > > > > > >>> > > > >> > > > > 1. Yes, the directory will still be
> > > based
> > > > on
> > > > > > the
> > > > > > > > > topic
> > > > > > > > > > >>> > names.
> > > > > > > > > > >>> > > > >> > > > > LeaderAndIsrRequest is one of the
> few
> > > > > requests
> > > > > > > > that
> > > > > > > > > > will
> > > > > > > > > > >>> > still
> > > > > > > > > > >>> > > > >> > contain
> > > > > > > > > > >>> > > > >> > > > the
> > > > > > > > > > >>> > > > >> > > > > topic name. So I think we have this
> > > > covered.
> > > > > > Sorry
> > > > > > > > > for
> > > > > > > > > > >>> > > > confusion.
> > > > > > > > > > >>> > > > >> > > > >
> > > > > > > > > > >>> > > > >> > > > > 2. Part of the usage of the file is
> to
> > > > have
> > > > > > > > > persistent
> > > > > > > > > > >>> > storage
> > > > > > > > > > >>> > > > of
> > > > > > > > > > >>> > > > >> the
> > > > > > > > > > >>> > > > >> > > > topic
> > > > > > > > > > >>> > > > >> > > > > ID and use it to compare with the ID
> > > > > supplied
> > > > > > in
> > > > > > > > the
> > > > > > > > > > >>> > > > LeaderAndIsr
> > > > > > > > > > >>> > > > >> > > > Request.
> > > > > > > > > > >>> > > > >> > > > > There is some discussion in the KIP
> > > about
> > > > > > changes
> > > > > > > > to
> > > > > > > > > > the
> > > > > > > > > > >>> > > > directory
> > > > > > > > > > >>> > > > >> > > > > structure, but I believe directory
> > > changes
> > > > > > were
> > > > > > > > > > >>> considered
> > > > > > > > > > >>> > to
> > > > > > > > > > >>> > > be
> > > > > > > > > > >>> > > > >> out
> > > > > > > > > > >>> > > > >> > of
> > > > > > > > > > >>> > > > >> > > > > scope when the KIP was written.
> > > > > > > > > > >>> > > > >> > > > >
> > > > > > > > > > >>> > > > >> > > > > 3. I think this is a good point,
> but I
> > > > > again I
> > > > > > > > > wonder
> > > > > > > > > > >>> about
> > > > > > > > > > >>> > > the
> > > > > > > > > > >>> > > > >> scope
> > > > > > > > > > >>> > > > >> > > of
> > > > > > > > > > >>> > > > >> > > > > the KIP. Most of the changes
> mentioned
> > > in
> > > > > the
> > > > > > KIP
> > > > > > > > > are
> > > > > > > > > > >>> for
> > > > > > > > > > >>> > > > >> supporting
> > > > > > > > > > >>> > > > >> > > > topic
> > > > > > > > > > >>> > > > >> > > > > deletion and I believe that is why
> the
> > > > > produce
> > > > > > > > > request
> > > > > > > > > > >>> was
> > > > > > > > > > >>> > > > listed
> > > > > > > > > > >>> > > > >> > under
> > > > > > > > > > >>> > > > >> > > > > future work.
> > > > > > > > > > >>> > > > >> > > > >
> > > > > > > > > > >>> > > > >> > > > > 4. This sounds like it might be a
> good
> > > > > > solution,
> > > > > > > > > but I
> > > > > > > > > > >>> will
> > > > > > > > > > >>> > > need
> > > > > > > > > > >>> > > > >> to
> > > > > > > > > > >>> > > > >> > > > discuss
> > > > > > > > > > >>> > > > >> > > > > more with KIP-500 folks to get the
> > > details
> > > > > > right.
> > > > > > > > > > >>> > > > >> > > > >
> > > > > > > > > > >>> > > > >> > > > > Thanks,
> > > > > > > > > > >>> > > > >> > > > > Justine
> > > > > > > > > > >>> > > > >> > > > >
> > > > > > > > > > >>> > > > >> > > > > On Thu, Sep 24, 2020 at 12:30 PM
> Jason
> > > > > > Gustafson <
> > > > > > > > > > >>> > > > >> jason@confluent.io
> > > > > > > > > > >>> > > > >> > >
> > > > > > > > > > >>> > > > >> > > > > wrote:
> > > > > > > > > > >>> > > > >> > > > >
> > > > > > > > > > >>> > > > >> > > > > > Hi Justine,
> > > > > > > > > > >>> > > > >> > > > > >
> > > > > > > > > > >>> > > > >> > > > > > Thanks for picking up this work. I
> > > have
> > > > a
> > > > > > few
> > > > > > > > > > >>> > > > >> questions/comments:
> > > > > > > > > > >>> > > > >> > > > > >
> > > > > > > > > > >>> > > > >> > > > > > 1. It sounds like the directory
> > > > structure
> > > > > is
> > > > > > > > still
> > > > > > > > > > >>> going
> > > > > > > > > > >>> > to
> > > > > > > > > > >>> > > be
> > > > > > > > > > >>> > > > >> > based
> > > > > > > > > > >>> > > > >> > > on
> > > > > > > > > > >>> > > > >> > > > > > topic names. Do I have that right?
> > One
> > > > > > > > > complication
> > > > > > > > > > is
> > > > > > > > > > >>> > that
> > > > > > > > > > >>> > > > the
> > > > > > > > > > >>> > > > >> > > > > > LeaderAndIsr request does not
> > include
> > > > the
> > > > > > topic
> > > > > > > > > name
> > > > > > > > > > >>> any
> > > > > > > > > > >>> > > > longer.
> > > > > > > > > > >>> > > > >> > This
> > > > > > > > > > >>> > > > >> > > > > means
> > > > > > > > > > >>> > > > >> > > > > > that a replica must wait for the
> > > > > > UpdateMetadata
> > > > > > > > > > >>> request to
> > > > > > > > > > >>> > > > >> arrive
> > > > > > > > > > >>> > > > >> > > with
> > > > > > > > > > >>> > > > >> > > > > the
> > > > > > > > > > >>> > > > >> > > > > > topic name mapping before it can
> > > create
> > > > > the
> > > > > > > > > > >>> directory. I
> > > > > > > > > > >>> > > > wonder
> > > > > > > > > > >>> > > > >> if
> > > > > > > > > > >>> > > > >> > it
> > > > > > > > > > >>> > > > >> > > > > would
> > > > > > > > > > >>> > > > >> > > > > > be simpler to avoid assumptions on
> > the
> > > > > > ordering
> > > > > > > > of
> > > > > > > > > > >>> > > > >> UpdateMetadata
> > > > > > > > > > >>> > > > >> > and
> > > > > > > > > > >>> > > > >> > > > let
> > > > > > > > > > >>> > > > >> > > > > > LeaderAndIsr include the topic
> name
> > as
> > > > > well.
> > > > > > > > Feels
> > > > > > > > > > >>> like we
> > > > > > > > > > >>> > > are
> > > > > > > > > > >>> > > > >> not
> > > > > > > > > > >>> > > > >> > > > saving
> > > > > > > > > > >>> > > > >> > > > > > that much by excluding it.
> > > > > > > > > > >>> > > > >> > > > > >
> > > > > > > > > > >>> > > > >> > > > > > 2. On a related note, it seems
> that
> > > the
> > > > > > reason
> > > > > > > > we
> > > > > > > > > > >>> have the
> > > > > > > > > > >>> > > > >> > partition
> > > > > > > > > > >>> > > > >> > > > > > metadata file is because we are
> not
> > > > > > changing the
> > > > > > > > > > >>> directory
> > > > > > > > > > >>> > > > >> > structure.
> > > > > > > > > > >>> > > > >> > > > We
> > > > > > > > > > >>> > > > >> > > > > > need it so that we remember which
> > > > > > directories
> > > > > > > > map
> > > > > > > > > to
> > > > > > > > > > >>> which
> > > > > > > > > > >>> > > > topic
> > > > > > > > > > >>> > > > >> > id.
> > > > > > > > > > >>> > > > >> > > I
> > > > > > > > > > >>> > > > >> > > > > > think it would be useful to add
> some
> > > > > detail
> > > > > > > > about
> > > > > > > > > > why
> > > > > > > > > > >>> > > changing
> > > > > > > > > > >>> > > > >> the
> > > > > > > > > > >>> > > > >> > > > > > directory layout was rejected.
> > > Basically
> > > > > > trying
> > > > > > > > to
> > > > > > > > > > >>> > > understand
> > > > > > > > > > >>> > > > >> how
> > > > > > > > > > >>> > > > >> > > much
> > > > > > > > > > >>> > > > >> > > > > > effort we are saving by skipping
> > this
> > > > step
> > > > > > if we
> > > > > > > > > > have
> > > > > > > > > > >>> to
> > > > > > > > > > >>> > > > >> implement
> > > > > > > > > > >>> > > > >> > > this
> > > > > > > > > > >>> > > > >> > > > > > additional bookkeeping. One
> concern
> > I
> > > > have
> > > > > > for
> > > > > > > > > > >>> example is
> > > > > > > > > > >>> > > that
> > > > > > > > > > >>> > > > >> the
> > > > > > > > > > >>> > > > >> > > > > > partition metadata file gets
> corrupt
> > > and
> > > > > > then
> > > > > > > > all
> > > > > > > > > > >>> bets are
> > > > > > > > > > >>> > > off
> > > > > > > > > > >>> > > > >> as
> > > > > > > > > > >>> > > > >> > far
> > > > > > > > > > >>> > > > >> > > > as
> > > > > > > > > > >>> > > > >> > > > > > topic consistency.
> > > > > > > > > > >>> > > > >> > > > > >
> > > > > > > > > > >>> > > > >> > > > > > 3. Is it worth adding support now
> > for
> > > > > topic
> > > > > > ids
> > > > > > > > in
> > > > > > > > > > the
> > > > > > > > > > >>> > > > `Produce`
> > > > > > > > > > >>> > > > >> > > > request
> > > > > > > > > > >>> > > > >> > > > > > now? Overhead is mentioned as one
> of
> > > the
> > > > > > > > > motivations
> > > > > > > > > > >>> and
> > > > > > > > > > >>> > the
> > > > > > > > > > >>> > > > >> best
> > > > > > > > > > >>> > > > >> > > APIs
> > > > > > > > > > >>> > > > >> > > > to
> > > > > > > > > > >>> > > > >> > > > > > get savings from are `Produce` and
> > > > > `Fetch`.
> > > > > > > > > > >>> > > > >> > > > > >
> > > > > > > > > > >>> > > > >> > > > > > 4. When it comes to bootstrapping
> > the
> > > > > > metadata
> > > > > > > > > topic
> > > > > > > > > > >>> with
> > > > > > > > > > >>> > > > >> respect
> > > > > > > > > > >>> > > > >> > to
> > > > > > > > > > >>> > > > >> > > > > > KIP-500, one suggestion would be
> to
> > > > > reserve
> > > > > > a
> > > > > > > > > > sentinel
> > > > > > > > > > >>> > topic
> > > > > > > > > > >>> > > > ID
> > > > > > > > > > >>> > > > >> > which
> > > > > > > > > > >>> > > > >> > > > can
> > > > > > > > > > >>> > > > >> > > > > > be used initially by new brokers.
> > When
> > > > the
> > > > > > first
> > > > > > > > > > >>> > controller
> > > > > > > > > > >>> > > is
> > > > > > > > > > >>> > > > >> > > elected,
> > > > > > > > > > >>> > > > >> > > > > it
> > > > > > > > > > >>> > > > >> > > > > > can assign a topicId to the
> metadata
> > > > > topic.
> > > > > > This
> > > > > > > > > is
> > > > > > > > > > a
> > > > > > > > > > >>> bit
> > > > > > > > > > >>> > > > >> similar
> > > > > > > > > > >>> > > > >> > to
> > > > > > > > > > >>> > > > >> > > > how
> > > > > > > > > > >>> > > > >> > > > > we
> > > > > > > > > > >>> > > > >> > > > > > were planning to generate the
> > > clusterId.
> > > > > > > > > > >>> > > > >> > > > > >
> > > > > > > > > > >>> > > > >> > > > > > Thanks,
> > > > > > > > > > >>> > > > >> > > > > > Jason
> > > > > > > > > > >>> > > > >> > > > > >
> > > > > > > > > > >>> > > > >> > > > > >
> > > > > > > > > > >>> > > > >> > > > > > On Thu, Sep 24, 2020 at 11:10 AM
> Jun
> > > > Rao <
> > > > > > > > > > >>> > jun@confluent.io>
> > > > > > > > > > >>> > > > >> wrote:
> > > > > > > > > > >>> > > > >> > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > Hi, Justine,
> > > > > > > > > > >>> > > > >> > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > Thanks for the updated KIP. A
> few
> > > more
> > > > > > > > comments
> > > > > > > > > > >>> below.
> > > > > > > > > > >>> > > > >> > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > 30.  {"name": "id", "type":
> > > "string",
> > > > > > "doc":
> > > > > > > > > > >>> "version
> > > > > > > > > > >>> > > id"}}:
> > > > > > > > > > >>> > > > >> The
> > > > > > > > > > >>> > > > >> > > doc
> > > > > > > > > > >>> > > > >> > > > > > should
> > > > > > > > > > >>> > > > >> > > > > > > say UUID.
> > > > > > > > > > >>> > > > >> > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > 31. LeaderAndIsrResponse v5 and
> > > > > > > > > > StopReplicaResponse
> > > > > > > > > > >>> v4 :
> > > > > > > > > > >>> > > It
> > > > > > > > > > >>> > > > >> seems
> > > > > > > > > > >>> > > > >> > > > there
> > > > > > > > > > >>> > > > >> > > > > > is
> > > > > > > > > > >>> > > > >> > > > > > > no need to add topic_id at
> > > partitions
> > > > > > level.
> > > > > > > > > > >>> > > > >> > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > 32. Regarding partition metadata
> > > file.
> > > > > > Perhaps
> > > > > > > > > the
> > > > > > > > > > >>> key
> > > > > > > > > > >>> > can
> > > > > > > > > > >>> > > > be
> > > > > > > > > > >>> > > > >> a
> > > > > > > > > > >>> > > > >> > > > single
> > > > > > > > > > >>> > > > >> > > > > > > word, sth like the following.
> > > > > > > > > > >>> > > > >> > > > > > > version: 0
> > > > > > > > > > >>> > > > >> > > > > > > topic_id:
> > > > > > 46bdb63f-9e8d-4a38-bf7b-ee4eb2a794e4
> > > > > > > > > > >>> > > > >> > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > 33. Another tricky thing that I
> > > > realized
> > > > > > is
> > > > > > > > how
> > > > > > > > > to
> > > > > > > > > > >>> > support
> > > > > > > > > > >>> > > > the
> > > > > > > > > > >>> > > > >> > > > metadata
> > > > > > > > > > >>> > > > >> > > > > > > topic introduced in KIP-595.
> It's
> > a
> > > > bit
> > > > > > tricky
> > > > > > > > > to
> > > > > > > > > > >>> > assign a
> > > > > > > > > > >>> > > > >> UUID
> > > > > > > > > > >>> > > > >> > to
> > > > > > > > > > >>> > > > >> > > > the
> > > > > > > > > > >>> > > > >> > > > > > > metadata topic since we have a
> > > > > > chicken-and-egg
> > > > > > > > > > >>> problem.
> > > > > > > > > > >>> > > The
> > > > > > > > > > >>> > > > >> > > > controller
> > > > > > > > > > >>> > > > >> > > > > > > needs to persist the UUID in the
> > > > > metadata
> > > > > > > > topic
> > > > > > > > > in
> > > > > > > > > > >>> order
> > > > > > > > > > >>> > > to
> > > > > > > > > > >>> > > > >> > assign
> > > > > > > > > > >>> > > > >> > > > one
> > > > > > > > > > >>> > > > >> > > > > > > successfully, but the metadata
> > topic
> > > > is
> > > > > > needed
> > > > > > > > > to
> > > > > > > > > > >>> elect
> > > > > > > > > > >>> > a
> > > > > > > > > > >>> > > > >> > > controller
> > > > > > > > > > >>> > > > >> > > > > > > (KIP-631). So, this probably
> > needs a
> > > > bit
> > > > > > more
> > > > > > > > > > >>> thought.
> > > > > > > > > > >>> > > > >> > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > Jun
> > > > > > > > > > >>> > > > >> > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > On Thu, Sep 24, 2020 at 3:04 AM
> > > Ismael
> > > > > > Juma <
> > > > > > > > > > >>> > > > >> ismael@juma.me.uk>
> > > > > > > > > > >>> > > > >> > > > wrote:
> > > > > > > > > > >>> > > > >> > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > Also, can we provide more
> > details
> > > on
> > > > > > how the
> > > > > > > > > > >>> Partition
> > > > > > > > > > >>> > > > >> Metadata
> > > > > > > > > > >>> > > > >> > > > file
> > > > > > > > > > >>> > > > >> > > > > > will
> > > > > > > > > > >>> > > > >> > > > > > > > be used?
> > > > > > > > > > >>> > > > >> > > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > Ismael
> > > > > > > > > > >>> > > > >> > > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > On Thu, Sep 24, 2020 at 3:01
> AM
> > > > Ismael
> > > > > > Juma
> > > > > > > > <
> > > > > > > > > > >>> > > > >> ismael@juma.me.uk
> > > > > > > > > > >>> > > > >> > >
> > > > > > > > > > >>> > > > >> > > > > wrote:
> > > > > > > > > > >>> > > > >> > > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > > Hi Justine,
> > > > > > > > > > >>> > > > >> > > > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > > I think we need to update
> the
> > > > > > "Rejected
> > > > > > > > > > >>> > Alternatives"
> > > > > > > > > > >>> > > > >> section
> > > > > > > > > > >>> > > > >> > > to
> > > > > > > > > > >>> > > > >> > > > > take
> > > > > > > > > > >>> > > > >> > > > > > > > into
> > > > > > > > > > >>> > > > >> > > > > > > > > account that the proposal
> now
> > > > > removes
> > > > > > the
> > > > > > > > > > topic
> > > > > > > > > > >>> name
> > > > > > > > > > >>> > > > from
> > > > > > > > > > >>> > > > >> the
> > > > > > > > > > >>> > > > >> > > > fetch
> > > > > > > > > > >>> > > > >> > > > > > > > > request. Also, if we are
> > > removing
> > > > it
> > > > > > from
> > > > > > > > > the
> > > > > > > > > > >>> Fetch
> > > > > > > > > > >>> > > > >> request,
> > > > > > > > > > >>> > > > >> > > does
> > > > > > > > > > >>> > > > >> > > > > it
> > > > > > > > > > >>> > > > >> > > > > > > make
> > > > > > > > > > >>> > > > >> > > > > > > > > sense not to remove it from
> > > > similar
> > > > > > > > requests
> > > > > > > > > > >>> like
> > > > > > > > > > >>> > > > >> > > > > ListOffsetRequest?
> > > > > > > > > > >>> > > > >> > > > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > > Ismael
> > > > > > > > > > >>> > > > >> > > > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > > On Thu, Sep 24, 2020 at 2:46
> > AM
> > > > > David
> > > > > > > > Jacot
> > > > > > > > > <
> > > > > > > > > > >>> > > > >> > > djacot@confluent.io
> > > > > > > > > > >>> > > > >> > > > >
> > > > > > > > > > >>> > > > >> > > > > > > wrote:
> > > > > > > > > > >>> > > > >> > > > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> Hi Justine,
> > > > > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > > > > >>> > > > >> > > > > > > > >> Thanks for the KIP. I
> finally
> > > had
> > > > > > time to
> > > > > > > > > > read
> > > > > > > > > > >>> it
> > > > > > > > > > >>> > :).
> > > > > > > > > > >>> > > > It
> > > > > > > > > > >>> > > > >> is
> > > > > > > > > > >>> > > > >> > a
> > > > > > > > > > >>> > > > >> > > > > great
> > > > > > > > > > >>> > > > >> > > > > > > > >> improvement.
> > > > > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > > > > >>> > > > >> > > > > > > > >> I have a few
> > > comments/questions:
> > > > > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > > > > >>> > > > >> > > > > > > > >> 1. It seems that the schema
> > of
> > > > the
> > > > > > > > > > >>> > StopReplicaRequest
> > > > > > > > > > >>> > > > is
> > > > > > > > > > >>> > > > >> > > > slightly
> > > > > > > > > > >>> > > > >> > > > > > > > >> outdated.
> > > > > > > > > > >>> > > > >> > > > > > > > >> We
> > > > > > > > > > >>> > > > >> > > > > > > > >> did some changes as part of
> > > > > KIP-570.
> > > > > > V3
> > > > > > > > is
> > > > > > > > > > >>> already
> > > > > > > > > > >>> > > > >> organized
> > > > > > > > > > >>> > > > >> > > by
> > > > > > > > > > >>> > > > >> > > > > > > topics.
> > > > > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > > > > >>> > > > >> > > > > > > > >> 2. I just want to make sure
> > > that
> > > > I
> > > > > > > > > understand
> > > > > > > > > > >>> the
> > > > > > > > > > >>> > > > >> > > reconciliation
> > > > > > > > > > >>> > > > >> > > > > > > > >> logic correctly. When
> > > > > > > > > > >>> > > > >> > > > > > > > >> an "INCREMENTAL"
> LeaderAndIsr
> > > > > > Request is
> > > > > > > > > > >>> received,
> > > > > > > > > > >>> > > the
> > > > > > > > > > >>> > > > >> > broker
> > > > > > > > > > >>> > > > >> > > > will
> > > > > > > > > > >>> > > > >> > > > > > > also
> > > > > > > > > > >>> > > > >> > > > > > > > >> reconcile
> > > > > > > > > > >>> > > > >> > > > > > > > >> when the local uuid does
> not
> > > > match
> > > > > > the
> > > > > > > > uuid
> > > > > > > > > > in
> > > > > > > > > > >>> the
> > > > > > > > > > >>> > > > >> request,
> > > > > > > > > > >>> > > > >> > > > right?
> > > > > > > > > > >>> > > > >> > > > > > In
> > > > > > > > > > >>> > > > >> > > > > > > > this
> > > > > > > > > > >>> > > > >> > > > > > > > >> case, the
> > > > > > > > > > >>> > > > >> > > > > > > > >> local log is staged for
> > > deletion.
> > > > > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > > > > >>> > > > >> > > > > > > > >> 3. In the documentation of
> > the
> > > `
> > > > > > > > > > >>> > > > >> delete.stale.topic.delay.ms
> > > > > > > > > > >>> > > > >> > `
> > > > > > > > > > >>> > > > >> > > > > > config,
> > > > > > > > > > >>> > > > >> > > > > > > it
> > > > > > > > > > >>> > > > >> > > > > > > > >> says "When a
> > > > > > > > > > >>> > > > >> > > > > > > > >> FULL LeaderAndIsrRequest is
> > > > > > received..."
> > > > > > > > > but
> > > > > > > > > > I
> > > > > > > > > > >>> > > suppose
> > > > > > > > > > >>> > > > it
> > > > > > > > > > >>> > > > >> > > > applies
> > > > > > > > > > >>> > > > >> > > > > to
> > > > > > > > > > >>> > > > >> > > > > > > > both
> > > > > > > > > > >>> > > > >> > > > > > > > >> types.
> > > > > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > > > > >>> > > > >> > > > > > > > >> Best,
> > > > > > > > > > >>> > > > >> > > > > > > > >> David
> > > > > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > > > > >>> > > > >> > > > > > > > >> On Thu, Sep 24, 2020 at
> 1:40
> > AM
> > > > > > Justine
> > > > > > > > > > Olshan
> > > > > > > > > > >>> <
> > > > > > > > > > >>> > > > >> > > > > > jolshan@confluent.io>
> > > > > > > > > > >>> > > > >> > > > > > > > >> wrote:
> > > > > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > > > > >>> > > > >> > > > > > > > >> > Hi Jun,
> > > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > Thanks for the comments.
> I
> > > > > > apologize
> > > > > > > > for
> > > > > > > > > > >>> some of
> > > > > > > > > > >>> > > the
> > > > > > > > > > >>> > > > >> typos
> > > > > > > > > > >>> > > > >> > > and
> > > > > > > > > > >>> > > > >> > > > > > > > >> confusion.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > I’ve updated the KIP to
> fix
> > > > some
> > > > > > of the
> > > > > > > > > > >>> issues
> > > > > > > > > > >>> > you
> > > > > > > > > > >>> > > > >> > > mentioned.
> > > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > 20.2 I’ve changed the
> type
> > to
> > > > > > String.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > 20.1/3 I’ve updated the
> > > > > TopicZNode
> > > > > > to
> > > > > > > > fix
> > > > > > > > > > >>> > > formatting
> > > > > > > > > > >>> > > > >> and
> > > > > > > > > > >>> > > > >> > > > reflect
> > > > > > > > > > >>> > > > >> > > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> latest
> > > > > > > > > > >>> > > > >> > > > > > > > >> > version before this
> change.
> > > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > 21. You are correct and
> > I’ve
> > > > > > removed
> > > > > > > > this
> > > > > > > > > > >>> line.
> > > > > > > > > > >>> > > I’ve
> > > > > > > > > > >>> > > > >> also
> > > > > > > > > > >>> > > > >> > > > added
> > > > > > > > > > >>> > > > >> > > > > a
> > > > > > > > > > >>> > > > >> > > > > > > line
> > > > > > > > > > >>> > > > >> > > > > > > > >> > mentioning an IBP bump is
> > > > > > necessary for
> > > > > > > > > > >>> migration
> > > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > 22. I think the wording
> was
> > > > > > unclear but
> > > > > > > > > > your
> > > > > > > > > > >>> > > summary
> > > > > > > > > > >>> > > > is
> > > > > > > > > > >>> > > > >> > what
> > > > > > > > > > >>> > > > >> > > > was
> > > > > > > > > > >>> > > > >> > > > > > > > >> intended
> > > > > > > > > > >>> > > > >> > > > > > > > >> > by this line. I’ve
> updated
> > to
> > > > > make
> > > > > > this
> > > > > > > > > > point
> > > > > > > > > > >>> > more
> > > > > > > > > > >>> > > > >> clear.
> > > > > > > > > > >>> > > > >> > > > > “Remove
> > > > > > > > > > >>> > > > >> > > > > > > > >> deleted
> > > > > > > > > > >>> > > > >> > > > > > > > >> > topics from replicas by
> > > sending
> > > > > > > > > > >>> > StopReplicaRequest
> > > > > > > > > > >>> > > V3
> > > > > > > > > > >>> > > > >> > before
> > > > > > > > > > >>> > > > >> > > > the
> > > > > > > > > > >>> > > > >> > > > > > IBP
> > > > > > > > > > >>> > > > >> > > > > > > > >> bump
> > > > > > > > > > >>> > > > >> > > > > > > > >> > using the old logic, and
> > > using
> > > > V4
> > > > > > and
> > > > > > > > the
> > > > > > > > > > new
> > > > > > > > > > >>> > logic
> > > > > > > > > > >>> > > > >> with
> > > > > > > > > > >>> > > > >> > > topic
> > > > > > > > > > >>> > > > >> > > > > IDs
> > > > > > > > > > >>> > > > >> > > > > > > > after
> > > > > > > > > > >>> > > > >> > > > > > > > >> > the IBP bump.”
> > > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > 23. I’ve removed the
> > > > unspecified
> > > > > > type
> > > > > > > > > from
> > > > > > > > > > >>> the
> > > > > > > > > > >>> > KIP
> > > > > > > > > > >>> > > > and
> > > > > > > > > > >>> > > > >> > > mention
> > > > > > > > > > >>> > > > >> > > > > > that
> > > > > > > > > > >>> > > > >> > > > > > > > IBP
> > > > > > > > > > >>> > > > >> > > > > > > > >> > will be used to handle
> this
> > > > > > request.
> > > > > > > > “IBP
> > > > > > > > > > >>> will be
> > > > > > > > > > >>> > > > used
> > > > > > > > > > >>> > > > >> to
> > > > > > > > > > >>> > > > >> > > > > > determine
> > > > > > > > > > >>> > > > >> > > > > > > > >> whether
> > > > > > > > > > >>> > > > >> > > > > > > > >> > this new form of the
> > request
> > > > will
> > > > > > be
> > > > > > > > > used.
> > > > > > > > > > >>> For
> > > > > > > > > > >>> > > older
> > > > > > > > > > >>> > > > >> > > requests,
> > > > > > > > > > >>> > > > >> > > > > we
> > > > > > > > > > >>> > > > >> > > > > > > will
> > > > > > > > > > >>> > > > >> > > > > > > > >> > ignore this field and
> > default
> > > > to
> > > > > > > > previous
> > > > > > > > > > >>> > > behavior.”
> > > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > 24. I’ve fixed this typo.
> > > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > 25. I've added a topics
> at
> > a
> > > > > higher
> > > > > > > > level
> > > > > > > > > > for
> > > > > > > > > > >>> > > > >> > > > > LeaderAndIsrResponse
> > > > > > > > > > >>> > > > >> > > > > > > v5,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > StopReplicaResponse v4.
> > I've
> > > > also
> > > > > > > > changed
> > > > > > > > > > >>> > > > >> > StopReplicaRequest
> > > > > > > > > > >>> > > > >> > > > v4
> > > > > > > > > > >>> > > > >> > > > > to
> > > > > > > > > > >>> > > > >> > > > > > > > have
> > > > > > > > > > >>> > > > >> > > > > > > > >> > topics at a higher level.
> > > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > 26. I’ve updated
> > > > > > > > > > >>> forgotten_topics_data--added the
> > > > > > > > > > >>> > > > >> topic ID
> > > > > > > > > > >>> > > > >> > > and
> > > > > > > > > > >>> > > > >> > > > > > > removed
> > > > > > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > topic name
> > > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > 27. I’ve decided on plain
> > > text,
> > > > > and
> > > > > > > > I’ve
> > > > > > > > > > >>> added an
> > > > > > > > > > >>> > > > >> example.
> > > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > 28. I’ve added this idea
> to
> > > > > future
> > > > > > > > work.
> > > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > Thanks again for taking a
> > > look,
> > > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > Justine
> > > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > On Wed, Sep 23, 2020 at
> > 10:28
> > > > AM
> > > > > > Jun
> > > > > > > > Rao
> > > > > > > > > <
> > > > > > > > > > >>> > > > >> > jun@confluent.io>
> > > > > > > > > > >>> > > > >> > > > > > wrote:
> > > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > Hi, Justine,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > Thanks for the
> response.
> > > Made
> > > > > > another
> > > > > > > > > > >>> pass. A
> > > > > > > > > > >>> > few
> > > > > > > > > > >>> > > > >> more
> > > > > > > > > > >>> > > > >> > > > > comments
> > > > > > > > > > >>> > > > >> > > > > > > > below.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > 20. znode schema:
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > 20.1 It seems that
> > {"name":
> > > > > > > > "version",
> > > > > > > > > > >>> "type":
> > > > > > > > > > >>> > > > "int",
> > > > > > > > > > >>> > > > >> > > "id":
> > > > > > > > > > >>> > > > >> > > > > > > "UUID",
> > > > > > > > > > >>> > > > >> > > > > > > > >> > "doc":
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > "version id"} should be
> > > > > {"name":
> > > > > > > > > > "version",
> > > > > > > > > > >>> > > "type":
> > > > > > > > > > >>> > > > >> > > "int"},
> > > > > > > > > > >>> > > > >> > > > > > > {"name":
> > > > > > > > > > >>> > > > >> > > > > > > > >> > "id",
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > "type": "UUID", "doc":
> > > > "version
> > > > > > id"}.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > 20.2 The znode format
> is
> > > JSON
> > > > > > which
> > > > > > > > > > doesn't
> > > > > > > > > > >>> > have
> > > > > > > > > > >>> > > > UUID
> > > > > > > > > > >>> > > > >> > > type.
> > > > > > > > > > >>> > > > >> > > > So
> > > > > > > > > > >>> > > > >> > > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> type
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > probably should be
> > string?
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > 20.3 Also, the existing
> > > > format
> > > > > > used
> > > > > > > > > seems
> > > > > > > > > > >>> > > outdated.
> > > > > > > > > > >>> > > > >> It
> > > > > > > > > > >>> > > > >> > > > should
> > > > > > > > > > >>> > > > >> > > > > > have
> > > > > > > > > > >>> > > > >> > > > > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > following format.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > >
> >  Json.encodeAsBytes(Map(
> > > > > > > > > > >>> > > > >> > > > > > > > >> > >       "version" -> 2,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > >       "partitions" ->
> > > > > > > > > > >>> > > replicaAssignmentJson.asJava,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > >       "adding_replicas"
> > ->
> > > > > > > > > > >>> > > > >> > > > > addingReplicasAssignmentJson.asJava,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > >
>  "removing_replicas"
> > > ->
> > > > > > > > > > >>> > > > >> > > > > >
> > removingReplicasAssignmentJson.asJava
> > > > > > > > > > >>> > > > >> > > > > > > > >> > >     ).asJava)
> > > > > > > > > > >>> > > > >> > > > > > > > >> > >   }
> > > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > 21. Migration: The KIP
> > says
> > > > > "The
> > > > > > > > > > migration
> > > > > > > > > > >>> > > process
> > > > > > > > > > >>> > > > >> can
> > > > > > > > > > >>> > > > >> > > take
> > > > > > > > > > >>> > > > >> > > > > > place
> > > > > > > > > > >>> > > > >> > > > > > > > >> without
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > an inter-broker
> protocol
> > > > bump,
> > > > > > as the
> > > > > > > > > > >>> format
> > > > > > > > > > >>> > > stored
> > > > > > > > > > >>> > > > >> in
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > /brokers/topics/[topic]
> > > will
> > > > be
> > > > > > > > > > compatible
> > > > > > > > > > >>> with
> > > > > > > > > > >>> > > > older
> > > > > > > > > > >>> > > > >> > > broker
> > > > > > > > > > >>> > > > >> > > > > > > > >> versions."
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > However, since we are
> > > bumping
> > > > > up
> > > > > > the
> > > > > > > > > > >>> version of
> > > > > > > > > > >>> > > > >> > > inter-broker
> > > > > > > > > > >>> > > > >> > > > > > > > >> requests, it
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > seems that we need to
> use
> > > IBP
> > > > > for
> > > > > > > > > > >>> migration.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > 22. The KIP says
> "Remove
> > > > > deleted
> > > > > > > > topics
> > > > > > > > > > >>> from
> > > > > > > > > > >>> > > > >> replicas by
> > > > > > > > > > >>> > > > >> > > > > sending
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > StopReplicaRequest V3
> for
> > > any
> > > > > > topics
> > > > > > > > > > which
> > > > > > > > > > >>> do
> > > > > > > > > > >>> > not
> > > > > > > > > > >>> > > > >> > contain
> > > > > > > > > > >>> > > > >> > > a
> > > > > > > > > > >>> > > > >> > > > > > topic
> > > > > > > > > > >>> > > > >> > > > > > > > ID,
> > > > > > > > > > >>> > > > >> > > > > > > > >> and
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > V4 for any topics which
> > do
> > > > > > contain a
> > > > > > > > > > topic
> > > > > > > > > > >>> > ID.".
> > > > > > > > > > >>> > > > >> > However,
> > > > > > > > > > >>> > > > >> > > if
> > > > > > > > > > >>> > > > >> > > > > we
> > > > > > > > > > >>> > > > >> > > > > > > use
> > > > > > > > > > >>> > > > >> > > > > > > > >> IBP,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > it
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > seems that the
> controller
> > > > will
> > > > > > either
> > > > > > > > > > send
> > > > > > > > > > >>> > > > >> > > > StopReplicaRequest
> > > > > > > > > > >>> > > > >> > > > > V3
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > or StopReplicaRequest
> V4,
> > > but
> > > > > > never
> > > > > > > > > mixed
> > > > > > > > > > >>> V3
> > > > > > > > > > >>> > and
> > > > > > > > > > >>> > > V4
> > > > > > > > > > >>> > > > >> for
> > > > > > > > > > >>> > > > >> > > > > > different
> > > > > > > > > > >>> > > > >> > > > > > > > >> topics.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > Basically, before the
> IBP
> > > > bump,
> > > > > > V3
> > > > > > > > will
> > > > > > > > > > be
> > > > > > > > > > >>> > used.
> > > > > > > > > > >>> > > > >> After
> > > > > > > > > > >>> > > > >> > the
> > > > > > > > > > >>> > > > >> > > > IBP
> > > > > > > > > > >>> > > > >> > > > > > > bump,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > topicId will be created
> > and
> > > > V4
> > > > > > will
> > > > > > > > be
> > > > > > > > > > >>> used.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > 23. Given that we
> depend
> > on
> > > > > IBP,
> > > > > > do
> > > > > > > > we
> > > > > > > > > > >>> still
> > > > > > > > > > >>> > need
> > > > > > > > > > >>> > > > "0
> > > > > > > > > > >>> > > > >> > > > > > UNSPECIFIED"
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > in LeaderAndIsr?
> > > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > 24.
> LeaderAndIsrResponse
> > > v5 :
> > > > > It
> > > > > > > > still
> > > > > > > > > > has
> > > > > > > > > > >>> the
> > > > > > > > > > >>> > > > topic
> > > > > > > > > > >>> > > > >> > > field.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > 25.
> LeaderAndIsrResponse
> > > v5,
> > > > > > > > > > >>> > StopReplicaResponse
> > > > > > > > > > >>> > > > v4:
> > > > > > > > > > >>> > > > >> > Could
> > > > > > > > > > >>> > > > >> > > > we
> > > > > > > > > > >>> > > > >> > > > > > use
> > > > > > > > > > >>> > > > >> > > > > > > > this
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > opportunity to organize
> > the
> > > > > > response
> > > > > > > > > in 2
> > > > > > > > > > >>> > levels,
> > > > > > > > > > >>> > > > >> first
> > > > > > > > > > >>> > > > >> > by
> > > > > > > > > > >>> > > > >> > > > > > topic,
> > > > > > > > > > >>> > > > >> > > > > > > > >> then by
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > partition, as most
> other
> > > > > > > > > > >>> requests/responses?
> > > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > 26. FetchRequest v13 :
> > > Should
> > > > > > > > > > >>> > > forgotten_topics_data
> > > > > > > > > > >>> > > > >> use
> > > > > > > > > > >>> > > > >> > > > > topicId
> > > > > > > > > > >>> > > > >> > > > > > > too?
> > > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > 27. "This file can
> either
> > > be
> > > > > > plain
> > > > > > > > text
> > > > > > > > > > >>> > > (key/value
> > > > > > > > > > >>> > > > >> > pairs)
> > > > > > > > > > >>> > > > >> > > or
> > > > > > > > > > >>> > > > >> > > > > > > JSON."
> > > > > > > > > > >>> > > > >> > > > > > > > >> Have
> > > > > > > > > > >>> > > > >> > > > > > > > >> > we
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > decided which one to
> use?
> > > > Also,
> > > > > > it
> > > > > > > > > would
> > > > > > > > > > be
> > > > > > > > > > >>> > > helpful
> > > > > > > > > > >>> > > > >> to
> > > > > > > > > > >>> > > > >> > > > provide
> > > > > > > > > > >>> > > > >> > > > > > an
> > > > > > > > > > >>> > > > >> > > > > > > > >> > example.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > 28. Future improvement:
> > > > Another
> > > > > > > > future
> > > > > > > > > > >>> > > improvement
> > > > > > > > > > >>> > > > >> > > > opportunity
> > > > > > > > > > >>> > > > >> > > > > > is
> > > > > > > > > > >>> > > > >> > > > > > > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> use
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > topicId in
> > > > > > > > > > >>> GroupMetadataManager.offsetCommitKey
> > > > > > > > > > >>> > > in
> > > > > > > > > > >>> > > > >> the
> > > > > > > > > > >>> > > > >> > > > > > > offset_commit
> > > > > > > > > > >>> > > > >> > > > > > > > >> > topic.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > This may save some
> space.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > Thanks,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > Jun
> > > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > On Wed, Sep 23, 2020 at
> > > 8:50
> > > > AM
> > > > > > > > Justine
> > > > > > > > > > >>> Olshan
> > > > > > > > > > >>> > <
> > > > > > > > > > >>> > > > >> > > > > > > > jolshan@confluent.io>
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > wrote:
> > > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > Hi Tom,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > Thanks for the
> > comment. I
> > > > > think
> > > > > > > > this
> > > > > > > > > > is a
> > > > > > > > > > >>> > > really
> > > > > > > > > > >>> > > > >> good
> > > > > > > > > > >>> > > > >> > > idea
> > > > > > > > > > >>> > > > >> > > > > and
> > > > > > > > > > >>> > > > >> > > > > > > it
> > > > > > > > > > >>> > > > >> > > > > > > > >> has
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > been
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > added to the KIP
> under
> > > the
> > > > > > newly
> > > > > > > > > added
> > > > > > > > > > >>> > tooling
> > > > > > > > > > >>> > > > >> > section.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > Thanks again,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > Justine
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > On Wed, Sep 23, 2020
> at
> > > > 3:17
> > > > > > AM Tom
> > > > > > > > > > >>> Bentley <
> > > > > > > > > > >>> > > > >> > > > > > > tbentley@redhat.com>
> > > > > > > > > > >>> > > > >> > > > > > > > >> > wrote:
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > Hi Justine,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > I know you started
> > the
> > > > vote
> > > > > > > > thread,
> > > > > > > > > > >>> but on
> > > > > > > > > > >>> > > > >> > re-reading
> > > > > > > > > > >>> > > > >> > > > the
> > > > > > > > > > >>> > > > >> > > > > > KIP
> > > > > > > > > > >>> > > > >> > > > > > > I
> > > > > > > > > > >>> > > > >> > > > > > > > >> > noticed
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that although the
> > topic
> > > > id
> > > > > is
> > > > > > > > > > included
> > > > > > > > > > >>> in
> > > > > > > > > > >>> > the
> > > > > > > > > > >>> > > > >> > > > > > MetadataResponse
> > > > > > > > > > >>> > > > >> > > > > > > > >> it's
> > > > > > > > > > >>> > > > >> > > > > > > > >> > not
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > surfaced in the
> > output
> > > > from
> > > > > > > > > > >>> > `kafka-topics.sh
> > > > > > > > > > >>> > > > >> > > > --describe`.
> > > > > > > > > > >>> > > > >> > > > > > > Maybe
> > > > > > > > > > >>> > > > >> > > > > > > > >> that
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > was
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > intentional because
> > ids
> > > > are
> > > > > > > > > > >>> intentionally
> > > > > > > > > > >>> > not
> > > > > > > > > > >>> > > > >> really
> > > > > > > > > > >>> > > > >> > > > > > something
> > > > > > > > > > >>> > > > >> > > > > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > user
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > should care deeply
> > > about,
> > > > > > but it
> > > > > > > > > > would
> > > > > > > > > > >>> also
> > > > > > > > > > >>> > > > make
> > > > > > > > > > >>> > > > >> > life
> > > > > > > > > > >>> > > > >> > > > > harder
> > > > > > > > > > >>> > > > >> > > > > > > for
> > > > > > > > > > >>> > > > >> > > > > > > > >> > anyone
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > debugging Kafka and
> > > this
> > > > > > would
> > > > > > > > > likely
> > > > > > > > > > >>> get
> > > > > > > > > > >>> > > worse
> > > > > > > > > > >>> > > > >> the
> > > > > > > > > > >>> > > > >> > > more
> > > > > > > > > > >>> > > > >> > > > > > topic
> > > > > > > > > > >>> > > > >> > > > > > > > ids
> > > > > > > > > > >>> > > > >> > > > > > > > >> > got
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > rolled out across
> the
> > > > > > protocols,
> > > > > > > > > > >>> clients
> > > > > > > > > > >>> > etc.
> > > > > > > > > > >>> > > > It
> > > > > > > > > > >>> > > > >> > seems
> > > > > > > > > > >>> > > > >> > > > > > likely
> > > > > > > > > > >>> > > > >> > > > > > > > that
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > `kafka-topics.sh`
> > will
> > > > > > eventually
> > > > > > > > > > need
> > > > > > > > > > >>> the
> > > > > > > > > > >>> > > > >> ability
> > > > > > > > > > >>> > > > >> > to
> > > > > > > > > > >>> > > > >> > > > show
> > > > > > > > > > >>> > > > >> > > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > id
> > > > > > > > > > >>> > > > >> > > > > > > > >> of
> > > > > > > > > > >>> > > > >> > > > > > > > >> > a
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > topic and perhaps
> > find
> > > a
> > > > > > topic
> > > > > > > > name
> > > > > > > > > > >>> given
> > > > > > > > > > >>> > an
> > > > > > > > > > >>> > > > id.
> > > > > > > > > > >>> > > > >> Is
> > > > > > > > > > >>> > > > >> > > > there
> > > > > > > > > > >>> > > > >> > > > > > any
> > > > > > > > > > >>> > > > >> > > > > > > > >> reason
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > not
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > implement that in
> > this
> > > > KIP?
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > Many thanks,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > Tom
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > On Mon, Sep 21,
> 2020
> > at
> > > > > 9:54
> > > > > > PM
> > > > > > > > > > Justine
> > > > > > > > > > >>> > > Olshan
> > > > > > > > > > >>> > > > <
> > > > > > > > > > >>> > > > >> > > > > > > > >> jolshan@confluent.io
> > > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > wrote:
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Hi all,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > After thinking
> > about
> > > > it,
> > > > > > I've
> > > > > > > > > > >>> decided to
> > > > > > > > > > >>> > > > remove
> > > > > > > > > > >>> > > > >> > the
> > > > > > > > > > >>> > > > >> > > > > topic
> > > > > > > > > > >>> > > > >> > > > > > > name
> > > > > > > > > > >>> > > > >> > > > > > > > >> from
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Fetch Request and
> > > > > Response
> > > > > > > > after
> > > > > > > > > > all.
> > > > > > > > > > >>> > Since
> > > > > > > > > > >>> > > > >> there
> > > > > > > > > > >>> > > > >> > > are
> > > > > > > > > > >>> > > > >> > > > so
> > > > > > > > > > >>> > > > >> > > > > > > many
> > > > > > > > > > >>> > > > >> > > > > > > > of
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > these
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > requests per
> > second,
> > > it
> > > > > is
> > > > > > > > worth
> > > > > > > > > > >>> removing
> > > > > > > > > > >>> > > the
> > > > > > > > > > >>> > > > >> > extra
> > > > > > > > > > >>> > > > >> > > > > > > > information.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > I've
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > updated the KIP
> to
> > > > > reflect
> > > > > > this
> > > > > > > > > > >>> change.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Please let me
> know
> > if
> > > > > > there is
> > > > > > > > > > >>> anything
> > > > > > > > > > >>> > > else
> > > > > > > > > > >>> > > > we
> > > > > > > > > > >>> > > > >> > > should
> > > > > > > > > > >>> > > > >> > > > > > > discuss
> > > > > > > > > > >>> > > > >> > > > > > > > >> > before
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > voting.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Thank you,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Justine
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > On Fri, Sep 18,
> > 2020
> > > at
> > > > > > 9:46 AM
> > > > > > > > > > >>> Justine
> > > > > > > > > > >>> > > > Olshan
> > > > > > > > > > >>> > > > >> <
> > > > > > > > > > >>> > > > >> > > > > > > > >> > jolshan@confluent.io
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > wrote:
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > Hi Jun,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > I see what you
> > are
> > > > > > saying.
> > > > > > > > For
> > > > > > > > > > now
> > > > > > > > > > >>> we
> > > > > > > > > > >>> > can
> > > > > > > > > > >>> > > > >> remove
> > > > > > > > > > >>> > > > >> > > the
> > > > > > > > > > >>> > > > >> > > > > > extra
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > information.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > I'll leave the
> > > option
> > > > > to
> > > > > > add
> > > > > > > > > more
> > > > > > > > > > >>> > fields
> > > > > > > > > > >>> > > to
> > > > > > > > > > >>> > > > >> the
> > > > > > > > > > >>> > > > >> > > file
> > > > > > > > > > >>> > > > >> > > > > in
> > > > > > > > > > >>> > > > >> > > > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > future.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > The
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > KIP
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > has been
> updated
> > to
> > > > > > reflect
> > > > > > > > > this
> > > > > > > > > > >>> > change.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > Thanks,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > Justine
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > On Fri, Sep 18,
> > > 2020
> > > > at
> > > > > > 8:46
> > > > > > > > AM
> > > > > > > > > > Jun
> > > > > > > > > > >>> > Rao <
> > > > > > > > > > >>> > > > >> > > > > > jun@confluent.io
> > > > > > > > > > >>> > > > >> > > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > wrote:
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Hi, Justine,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Thanks for the
> > > > reply.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> 13. If the log
> > > > > > directory is
> > > > > > > > > the
> > > > > > > > > > >>> source
> > > > > > > > > > >>> > > of
> > > > > > > > > > >>> > > > >> > truth,
> > > > > > > > > > >>> > > > >> > > it
> > > > > > > > > > >>> > > > >> > > > > > means
> > > > > > > > > > >>> > > > >> > > > > > > > >> that
> > > > > > > > > > >>> > > > >> > > > > > > > >> > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> redundant info
> > in
> > > > the
> > > > > > > > metadata
> > > > > > > > > > >>> file
> > > > > > > > > > >>> > will
> > > > > > > > > > >>> > > > be
> > > > > > > > > > >>> > > > >> > > > ignored.
> > > > > > > > > > >>> > > > >> > > > > > Then
> > > > > > > > > > >>> > > > >> > > > > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > question
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > is
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> why do we need
> > to
> > > > put
> > > > > > the
> > > > > > > > > > >>> redundant
> > > > > > > > > > >>> > info
> > > > > > > > > > >>> > > > in
> > > > > > > > > > >>> > > > >> the
> > > > > > > > > > >>> > > > >> > > > > > metadata
> > > > > > > > > > >>> > > > >> > > > > > > > file
> > > > > > > > > > >>> > > > >> > > > > > > > >> > now?
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Thanks,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Jun
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> On Thu, Sep
> 17,
> > > 2020
> > > > > at
> > > > > > 5:07
> > > > > > > > > PM
> > > > > > > > > > >>> > Justine
> > > > > > > > > > >>> > > > >> Olshan
> > > > > > > > > > >>> > > > >> > <
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > jolshan@confluent.io
> >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> wrote:
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Hi Jun,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Thanks for
> the
> > > > quick
> > > > > > > > > response!
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > 12. I've
> > decided
> > > > to
> > > > > > bump
> > > > > > > > up
> > > > > > > > > > the
> > > > > > > > > > >>> > > versions
> > > > > > > > > > >>> > > > >> on
> > > > > > > > > > >>> > > > >> > the
> > > > > > > > > > >>> > > > >> > > > > > > requests
> > > > > > > > > > >>> > > > >> > > > > > > > >> and
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > updated
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > KIP. I think
> > > it's
> > > > > > good we
> > > > > > > > > > >>> thoroughly
> > > > > > > > > > >>> > > > >> > discussed
> > > > > > > > > > >>> > > > >> > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > options
> > > > > > > > > > >>> > > > >> > > > > > > > >> > here,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > so
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > we
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> know
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > we made a
> good
> > > > > > choice. :)
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > 13. This is
> an
> > > > > > interesting
> > > > > > > > > > >>> > situation.
> > > > > > > > > > >>> > > I
> > > > > > > > > > >>> > > > >> think
> > > > > > > > > > >>> > > > >> > > if
> > > > > > > > > > >>> > > > >> > > > > this
> > > > > > > > > > >>> > > > >> > > > > > > > does
> > > > > > > > > > >>> > > > >> > > > > > > > >> > occur
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > we
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> should
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > give a
> > warning.
> > > I
> > > > > > agree
> > > > > > > > that
> > > > > > > > > > >>> it's
> > > > > > > > > > >>> > hard
> > > > > > > > > > >>> > > > to
> > > > > > > > > > >>> > > > >> > know
> > > > > > > > > > >>> > > > >> > > > the
> > > > > > > > > > >>> > > > >> > > > > > > source
> > > > > > > > > > >>> > > > >> > > > > > > > >> of
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > truth
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > for
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> sure
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > since the
> > > > directory
> > > > > > or the
> > > > > > > > > > file
> > > > > > > > > > >>> > could
> > > > > > > > > > >>> > > be
> > > > > > > > > > >>> > > > >> > > manually
> > > > > > > > > > >>> > > > >> > > > > > > > >> modified. I
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > guess
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > directory
> > could
> > > be
> > > > > > used as
> > > > > > > > > the
> > > > > > > > > > >>> > source
> > > > > > > > > > >>> > > of
> > > > > > > > > > >>> > > > >> > truth.
> > > > > > > > > > >>> > > > >> > > > To
> > > > > > > > > > >>> > > > >> > > > > be
> > > > > > > > > > >>> > > > >> > > > > > > > >> honest,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > I'm
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > not
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > really sure
> > what
> > > > > > happens
> > > > > > > > in
> > > > > > > > > > >>> kafka
> > > > > > > > > > >>> > when
> > > > > > > > > > >>> > > > the
> > > > > > > > > > >>> > > > >> > log
> > > > > > > > > > >>> > > > >> > > > > > > directory
> > > > > > > > > > >>> > > > >> > > > > > > > is
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > renamed
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > manually in
> > > such a
> > > > > > way.
> > > > > > > > I'm
> > > > > > > > > > also
> > > > > > > > > > >>> > > > >> wondering if
> > > > > > > > > > >>> > > > >> > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> situation is
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> recoverable
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > in this
> > > scenario.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Thanks,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Justine
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > On Thu, Sep
> > 17,
> > > > 2020
> > > > > > at
> > > > > > > > 4:28
> > > > > > > > > > PM
> > > > > > > > > > >>> Jun
> > > > > > > > > > >>> > > Rao
> > > > > > > > > > >>> > > > <
> > > > > > > > > > >>> > > > >> > > > > > > > jun@confluent.io>
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > wrote:
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Hi,
> Justine,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Thanks for
> > the
> > > > > > reply.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > 12. I
> don't
> > > > have a
> > > > > > > > strong
> > > > > > > > > > >>> > preference
> > > > > > > > > > >>> > > > >> > either.
> > > > > > > > > > >>> > > > >> > > > > > However,
> > > > > > > > > > >>> > > > >> > > > > > > > if
> > > > > > > > > > >>> > > > >> > > > > > > > >> we
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > need
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > IBP
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > anyway,
> > maybe
> > > > it's
> > > > > > > > easier
> > > > > > > > > to
> > > > > > > > > > >>> just
> > > > > > > > > > >>> > > bump
> > > > > > > > > > >>> > > > >> up
> > > > > > > > > > >>> > > > >> > the
> > > > > > > > > > >>> > > > >> > > > > > version
> > > > > > > > > > >>> > > > >> > > > > > > > for
> > > > > > > > > > >>> > > > >> > > > > > > > >> > all
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > inter
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > broker
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > requests
> and
> > > add
> > > > > the
> > > > > > > > topic
> > > > > > > > > > id
> > > > > > > > > > >>> > field
> > > > > > > > > > >>> > > > as a
> > > > > > > > > > >>> > > > >> > > > regular
> > > > > > > > > > >>> > > > >> > > > > > > > field. A
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > regular
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> field
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > is
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > a bit more
> > > > concise
> > > > > > in
> > > > > > > > wire
> > > > > > > > > > >>> > transfer
> > > > > > > > > > >>> > > > >> than a
> > > > > > > > > > >>> > > > >> > > > > flexible
> > > > > > > > > > >>> > > > >> > > > > > > > >> field.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > 13. The
> > > > confusion
> > > > > > that I
> > > > > > > > > was
> > > > > > > > > > >>> > > referring
> > > > > > > > > > >>> > > > >> to
> > > > > > > > > > >>> > > > >> > is
> > > > > > > > > > >>> > > > >> > > > > > between
> > > > > > > > > > >>> > > > >> > > > > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > topic
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > name
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > partition
> > > number
> > > > > > between
> > > > > > > > > the
> > > > > > > > > > >>> log
> > > > > > > > > > >>> > dir
> > > > > > > > > > >>> > > > and
> > > > > > > > > > >>> > > > >> > the
> > > > > > > > > > >>> > > > >> > > > > > metadata
> > > > > > > > > > >>> > > > >> > > > > > > > >> file.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > For
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> example,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > if
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > the log
> dir
> > is
> > > > > > topicA-1
> > > > > > > > > and
> > > > > > > > > > >>> the
> > > > > > > > > > >>> > > > metadata
> > > > > > > > > > >>> > > > >> > file
> > > > > > > > > > >>> > > > >> > > > in
> > > > > > > > > > >>> > > > >> > > > > it
> > > > > > > > > > >>> > > > >> > > > > > > has
> > > > > > > > > > >>> > > > >> > > > > > > > >> > topicB
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > and
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > partition
> 0
> > > (say
> > > > > > due to
> > > > > > > > a
> > > > > > > > > > bug
> > > > > > > > > > >>> or
> > > > > > > > > > >>> > > > manual
> > > > > > > > > > >>> > > > >> > > > > > > modification),
> > > > > > > > > > >>> > > > >> > > > > > > > >> which
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > one
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > do
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > we
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > use
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > as the
> > source
> > > of
> > > > > > truth?
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Jun
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > On Thu,
> Sep
> > > 17,
> > > > > > 2020 at
> > > > > > > > > 3:43
> > > > > > > > > > >>> PM
> > > > > > > > > > >>> > > > Justine
> > > > > > > > > > >>> > > > >> > > Olshan
> > > > > > > > > > >>> > > > >> > > > <
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > jolshan@confluent.io
> > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > wrote:
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > Hi Jun,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > Thanks
> for
> > > the
> > > > > > > > comments.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > 12. I
> > bumped
> > > > the
> > > > > > > > > > >>> > > LeaderAndIsrRequest
> > > > > > > > > > >>> > > > >> > > because
> > > > > > > > > > >>> > > > >> > > > I
> > > > > > > > > > >>> > > > >> > > > > > > > removed
> > > > > > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > topic
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> name
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > field
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > in the
> > > > response.
> > > > > > It
> > > > > > > > may
> > > > > > > > > be
> > > > > > > > > > >>> > > possible
> > > > > > > > > > >>> > > > to
> > > > > > > > > > >>> > > > >> > > avoid
> > > > > > > > > > >>> > > > >> > > > > > > bumping
> > > > > > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > version
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > without
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > that
> > change,
> > > > but
> > > > > > I may
> > > > > > > > > be
> > > > > > > > > > >>> > missing
> > > > > > > > > > >>> > > > >> > > something.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > I
> believe
> > > > > > StopReplica
> > > > > > > > is
> > > > > > > > > > >>> > actually
> > > > > > > > > > >>> > > on
> > > > > > > > > > >>> > > > >> > > version
> > > > > > > > > > >>> > > > >> > > > 3
> > > > > > > > > > >>> > > > >> > > > > > now,
> > > > > > > > > > >>> > > > >> > > > > > > > but
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > because
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > version 2
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > is
> > > flexible, I
> > > > > > kept
> > > > > > > > that
> > > > > > > > > > >>> listed
> > > > > > > > > > >>> > as
> > > > > > > > > > >>> > > > >> > version
> > > > > > > > > > >>> > > > >> > > 2
> > > > > > > > > > >>> > > > >> > > > on
> > > > > > > > > > >>> > > > >> > > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > KIP
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > page.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> However,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > you
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > may be
> > right
> > > > in
> > > > > > that
> > > > > > > > we
> > > > > > > > > > may
> > > > > > > > > > >>> need
> > > > > > > > > > >>> > > to
> > > > > > > > > > >>> > > > >> bump
> > > > > > > > > > >>> > > > >> > > the
> > > > > > > > > > >>> > > > >> > > > > > > version
> > > > > > > > > > >>> > > > >> > > > > > > > on
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > StopReplica
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > deal
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > with
> > > deletion
> > > > > > > > > differently
> > > > > > > > > > as
> > > > > > > > > > >>> > > > mentioned
> > > > > > > > > > >>> > > > >> > > > above. I
> > > > > > > > > > >>> > > > >> > > > > > > don't
> > > > > > > > > > >>> > > > >> > > > > > > > >> know
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > if
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > I
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> have a
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > big
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> preference
> > > > over
> > > > > > used
> > > > > > > > > > tagged
> > > > > > > > > > >>> > fields
> > > > > > > > > > >>> > > > or
> > > > > > > > > > >>> > > > >> > not.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > 13. I
> was
> > > > > thinking
> > > > > > > > that
> > > > > > > > > in
> > > > > > > > > > >>> the
> > > > > > > > > > >>> > > case
> > > > > > > > > > >>> > > > >> where
> > > > > > > > > > >>> > > > >> > > the
> > > > > > > > > > >>> > > > >> > > > > > file
> > > > > > > > > > >>> > > > >> > > > > > > > and
> > > > > > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > request
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > topic
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > ids
> don't
> > > > match,
> > > > > > it
> > > > > > > > > means
> > > > > > > > > > >>> that
> > > > > > > > > > >>> > the
> > > > > > > > > > >>> > > > >> > broker's
> > > > > > > > > > >>> > > > >> > > > > > > topic/the
> > > > > > > > > > >>> > > > >> > > > > > > > >> one
> > > > > > > > > > >>> > > > >> > > > > > > > >> > in
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> file
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > has
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > been
> > > deleted.
> > > > In
> > > > > > that
> > > > > > > > > > case,
> > > > > > > > > > >>> we
> > > > > > > > > > >>> > > would
> > > > > > > > > > >>> > > > >> need
> > > > > > > > > > >>> > > > >> > > to
> > > > > > > > > > >>> > > > >> > > > > > delete
> > > > > > > > > > >>> > > > >> > > > > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > old
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > topic
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > start
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> receiving
> > > the
> > > > > new
> > > > > > > > > version.
> > > > > > > > > > >>> If
> > > > > > > > > > >>> > the
> > > > > > > > > > >>> > > > >> topic
> > > > > > > > > > >>> > > > >> > > name
> > > > > > > > > > >>> > > > >> > > > > were
> > > > > > > > > > >>> > > > >> > > > > > > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > change,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > but
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > ids
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > still
> > > matched,
> > > > > the
> > > > > > > > file
> > > > > > > > > > >>> would
> > > > > > > > > > >>> > also
> > > > > > > > > > >>> > > > >> need
> > > > > > > > > > >>> > > > >> > to
> > > > > > > > > > >>> > > > >> > > > > > update.
> > > > > > > > > > >>> > > > >> > > > > > > > Am I
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > missing
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > a
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> case
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > where
> the
> > > file
> > > > > > would
> > > > > > > > be
> > > > > > > > > > >>> correct
> > > > > > > > > > >>> > > and
> > > > > > > > > > >>> > > > >> not
> > > > > > > > > > >>> > > > >> > the
> > > > > > > > > > >>> > > > >> > > > > > > request?
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > Thanks,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > Justine
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > On Thu,
> > Sep
> > > > 17,
> > > > > > 2020
> > > > > > > > at
> > > > > > > > > > >>> 3:18 PM
> > > > > > > > > > >>> > > Jun
> > > > > > > > > > >>> > > > >> Rao <
> > > > > > > > > > >>> > > > >> > > > > > > > >> jun@confluent.io
> > > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > wrote:
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Hi,
> > > Justine,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Thanks
> > for
> > > > the
> > > > > > > > reply.
> > > > > > > > > A
> > > > > > > > > > >>> couple
> > > > > > > > > > >>> > > of
> > > > > > > > > > >>> > > > >> more
> > > > > > > > > > >>> > > > >> > > > > comments
> > > > > > > > > > >>> > > > >> > > > > > > > >> below.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > 12.
> > > > ListOffset
> > > > > > and
> > > > > > > > > > >>> > > OffsetForLeader
> > > > > > > > > > >>> > > > >> > > > currently
> > > > > > > > > > >>> > > > >> > > > > > > don't
> > > > > > > > > > >>> > > > >> > > > > > > > >> > support
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> flexible
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > fields.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > So, we
> > > have
> > > > to
> > > > > > bump
> > > > > > > > up
> > > > > > > > > > the
> > > > > > > > > > >>> > > version
> > > > > > > > > > >>> > > > >> > number
> > > > > > > > > > >>> > > > >> > > > and
> > > > > > > > > > >>> > > > >> > > > > > use
> > > > > > > > > > >>> > > > >> > > > > > > > >> IBP at
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > least
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > for
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > these
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > two
> > > > requests.
> > > > > > Note
> > > > > > > > > that
> > > > > > > > > > it
> > > > > > > > > > >>> > seems
> > > > > > > > > > >>> > > > >> 2.7.0
> > > > > > > > > > >>> > > > >> > > will
> > > > > > > > > > >>> > > > >> > > > > > > require
> > > > > > > > > > >>> > > > >> > > > > > > > >> IBP
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > anyway
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > because
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > of
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> changes
> > in
> > > > > > > > > KAFKA-10435.
> > > > > > > > > > >>> Also,
> > > > > > > > > > >>> > it
> > > > > > > > > > >>> > > > >> seems
> > > > > > > > > > >>> > > > >> > > that
> > > > > > > > > > >>> > > > >> > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> version
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > for
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > LeaderAndIsrRequest
> > > > > > > > > and
> > > > > > > > > > >>> > > > StopReplica
> > > > > > > > > > >>> > > > >> are
> > > > > > > > > > >>> > > > >> > > > > bumped
> > > > > > > > > > >>> > > > >> > > > > > > even
> > > > > > > > > > >>> > > > >> > > > > > > > >> > though
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > we
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > only
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > added
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > a
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > tagged
> > > > field.
> > > > > > But
> > > > > > > > > since
> > > > > > > > > > >>> IBP is
> > > > > > > > > > >>> > > > >> needed
> > > > > > > > > > >>> > > > >> > > > anyway,
> > > > > > > > > > >>> > > > >> > > > > > we
> > > > > > > > > > >>> > > > >> > > > > > > > may
> > > > > > > > > > >>> > > > >> > > > > > > > >> > want
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> revisit
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> overall
> > > > tagged
> > > > > > field
> > > > > > > > > > >>> choice.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > 13.
> The
> > > only
> > > > > > > > downside
> > > > > > > > > is
> > > > > > > > > > >>> the
> > > > > > > > > > >>> > > > >> potential
> > > > > > > > > > >>> > > > >> > > > > > confusion
> > > > > > > > > > >>> > > > >> > > > > > > on
> > > > > > > > > > >>> > > > >> > > > > > > > >> > which
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > one
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > is
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > source
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > of
> truth
> > > if
> > > > > they
> > > > > > > > don't
> > > > > > > > > > >>> match.
> > > > > > > > > > >>> > > > >> Another
> > > > > > > > > > >>> > > > >> > > > option
> > > > > > > > > > >>> > > > >> > > > > is
> > > > > > > > > > >>> > > > >> > > > > > > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > include
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > those
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > fields
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > in
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > metadata
> > > > > > file
> > > > > > > > when
> > > > > > > > > > we
> > > > > > > > > > >>> > > actually
> > > > > > > > > > >>> > > > >> > change
> > > > > > > > > > >>> > > > >> > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> directory
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > structure.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> Thanks,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Jun
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > On
> Thu,
> > > Sep
> > > > > 17,
> > > > > > 2020
> > > > > > > > > at
> > > > > > > > > > >>> 2:01
> > > > > > > > > > >>> > PM
> > > > > > > > > > >>> > > > >> Justine
> > > > > > > > > > >>> > > > >> > > > > Olshan
> > > > > > > > > > >>> > > > >> > > > > > <
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > jolshan@confluent.io
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > wrote:
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> Hello
> > > all,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I've
> > > > thought
> > > > > > some
> > > > > > > > > more
> > > > > > > > > > >>> about
> > > > > > > > > > >>> > > > >> removing
> > > > > > > > > > >>> > > > >> > > the
> > > > > > > > > > >>> > > > >> > > > > > topic
> > > > > > > > > > >>> > > > >> > > > > > > > >> name
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > field
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > from
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > some
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > of
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > requests.
> > > > On
> > > > > > > > closer
> > > > > > > > > > >>> > inspection
> > > > > > > > > > >>> > > > of
> > > > > > > > > > >>> > > > >> the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > requests/responses,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > it
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> seems
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > internal
> > > > > > changes
> > > > > > > > > would
> > > > > > > > > > >>> be
> > > > > > > > > > >>> > much
> > > > > > > > > > >>> > > > >> larger
> > > > > > > > > > >>> > > > >> > > > than
> > > > > > > > > > >>> > > > >> > > > > I
> > > > > > > > > > >>> > > > >> > > > > > > > >> expected.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > Some
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > protocols
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > involve
> > > > > > clients,
> > > > > > > > so
> > > > > > > > > > they
> > > > > > > > > > >>> > would
> > > > > > > > > > >>> > > > >> > require
> > > > > > > > > > >>> > > > >> > > > > > changes
> > > > > > > > > > >>> > > > >> > > > > > > > too.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > I'm
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > thinking
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > that
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > for
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > now,
> > > > > removing
> > > > > > the
> > > > > > > > > > topic
> > > > > > > > > > >>> name
> > > > > > > > > > >>> > > > from
> > > > > > > > > > >>> > > > >> > these
> > > > > > > > > > >>> > > > >> > > > > > > requests
> > > > > > > > > > >>> > > > >> > > > > > > > >> and
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > responses
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> are
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > out
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> scope.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I
> have
> > > > > > decided to
> > > > > > > > > just
> > > > > > > > > > >>> keep
> > > > > > > > > > >>> > > the
> > > > > > > > > > >>> > > > >> > change
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > LeaderAndIsrResponse
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > remove
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> topic
> > > > name,
> > > > > > and
> > > > > > > > have
> > > > > > > > > > >>> updated
> > > > > > > > > > >>> > > the
> > > > > > > > > > >>> > > > >> KIP
> > > > > > > > > > >>> > > > >> > to
> > > > > > > > > > >>> > > > >> > > > > > reflect
> > > > > > > > > > >>> > > > >> > > > > > > > >> this
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > change. I
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> have
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > also
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > mentioned
> > > > > the
> > > > > > > > other
> > > > > > > > > > >>> requests
> > > > > > > > > > >>> > > and
> > > > > > > > > > >>> > > > >> > > > responses
> > > > > > > > > > >>> > > > >> > > > > in
> > > > > > > > > > >>> > > > >> > > > > > > > >> future
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > work.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I'm
> > > hoping
> > > > > to
> > > > > > > > start
> > > > > > > > > > the
> > > > > > > > > > >>> > voting
> > > > > > > > > > >>> > > > >> > process
> > > > > > > > > > >>> > > > >> > > > > soon,
> > > > > > > > > > >>> > > > >> > > > > > so
> > > > > > > > > > >>> > > > >> > > > > > > > >> let me
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > know
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > if
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > there
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > is
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > anything
> > > > > else
> > > > > > we
> > > > > > > > > > should
> > > > > > > > > > >>> > > discuss.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> Thank
> > > you,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > Justine
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > On
> > Tue,
> > > > Sep
> > > > > > 15,
> > > > > > > > 2020
> > > > > > > > > > at
> > > > > > > > > > >>> 3:57
> > > > > > > > > > >>> > > PM
> > > > > > > > > > >>> > > > >> > Justine
> > > > > > > > > > >>> > > > >> > > > > > Olshan
> > > > > > > > > > >>> > > > >> > > > > > > <
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > jolshan@confluent.io
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> wrote:
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > Hello
> > > > > again,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > To
> > > > follow
> > > > > > up on
> > > > > > > > > some
> > > > > > > > > > >>> of
> > > > > > > > > > >>> > the
> > > > > > > > > > >>> > > > >> other
> > > > > > > > > > >>> > > > >> > > > > comments:
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > 10/11)
> > > > We
> > > > > > can
> > > > > > > > > remove
> > > > > > > > > > >>> the
> > > > > > > > > > >>> > > topic
> > > > > > > > > > >>> > > > >> name
> > > > > > > > > > >>> > > > >> > > > from
> > > > > > > > > > >>> > > > >> > > > > > > these
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > requests/responses,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > and
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> that
> > > > means
> > > > > > we
> > > > > > > > will
> > > > > > > > > > >>> just
> > > > > > > > > > >>> > have
> > > > > > > > > > >>> > > > to
> > > > > > > > > > >>> > > > >> > make
> > > > > > > > > > >>> > > > >> > > a
> > > > > > > > > > >>> > > > >> > > > > few
> > > > > > > > > > >>> > > > >> > > > > > > > >> internal
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > changes
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > make
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > partitions
> > > > > > > > > > accessible
> > > > > > > > > > >>> by
> > > > > > > > > > >>> > > topic
> > > > > > > > > > >>> > > > >> id
> > > > > > > > > > >>> > > > >> > and
> > > > > > > > > > >>> > > > >> > > > > > > > partition.
> > > > > > > > > > >>> > > > >> > > > > > > > >> I
> > > > > > > > > > >>> > > > >> > > > > > > > >> > can
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > update
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > KIP
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > remove
> > > > > them
> > > > > > > > unless
> > > > > > > > > > >>> anyone
> > > > > > > > > > >>> > > > thinks
> > > > > > > > > > >>> > > > >> > they
> > > > > > > > > > >>> > > > >> > > > > > should
> > > > > > > > > > >>> > > > >> > > > > > > > >> stay.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> 12)
> > > > > > Addressed in
> > > > > > > > > the
> > > > > > > > > > >>> > > previous
> > > > > > > > > > >>> > > > >> > email.
> > > > > > > > > > >>> > > > >> > > > I've
> > > > > > > > > > >>> > > > >> > > > > > > > updated
> > > > > > > > > > >>> > > > >> > > > > > > > >> > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > KIP
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > include
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > tagged
> > > > > > fields
> > > > > > > > for
> > > > > > > > > > the
> > > > > > > > > > >>> > > requests
> > > > > > > > > > >>> > > > >> and
> > > > > > > > > > >>> > > > >> > > > > > responses.
> > > > > > > > > > >>> > > > >> > > > > > > > >> (More
> > > > > > > > > > >>> > > > >> > > > > > > > >> > on
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > below)
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> 13)
> > I
> > > > > think
> > > > > > part
> > > > > > > > > of
> > > > > > > > > > >>> the
> > > > > > > > > > >>> > idea
> > > > > > > > > > >>> > > > for
> > > > > > > > > > >>> > > > >> > > > > including
> > > > > > > > > > >>> > > > >> > > > > > > this
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > information
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> is to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> prepare
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> for
> > > > future
> > > > > > > > > changes.
> > > > > > > > > > >>> > Perhaps
> > > > > > > > > > >>> > > > the
> > > > > > > > > > >>> > > > >> > > > directory
> > > > > > > > > > >>> > > > >> > > > > > > > >> structure
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > might
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> change
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > from
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > > > >>> topicName_partitionNumber
> > > > > > > > > > >>> > to
> > > > > > > > > > >>> > > > >> > > something
> > > > > > > > > > >>> > > > >> > > > > like
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > topicID_partitionNumber.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Then
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > it
> > > would
> > > > > be
> > > > > > > > useful
> > > > > > > > > > to
> > > > > > > > > > >>> have
> > > > > > > > > > >>> > > the
> > > > > > > > > > >>> > > > >> > topic
> > > > > > > > > > >>> > > > >> > > > name
> > > > > > > > > > >>> > > > >> > > > > > in
> > > > > > > > > > >>> > > > >> > > > > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > file
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > since
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > it
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > would
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > not
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > be
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > in
> > the
> > > > > > directory
> > > > > > > > > > >>> > structure.
> > > > > > > > > > >>> > > > >> > > Supporting
> > > > > > > > > > >>> > > > >> > > > > > topic
> > > > > > > > > > >>> > > > >> > > > > > > > >> renames
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > might
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > be
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > easier
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > if
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > other
> > > > > > fields are
> > > > > > > > > > >>> included.
> > > > > > > > > > >>> > > > Would
> > > > > > > > > > >>> > > > >> > > there
> > > > > > > > > > >>> > > > >> > > > be
> > > > > > > > > > >>> > > > >> > > > > > any
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > downsides
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > including
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > this
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > information?
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> 14)
> > > > Yes,
> > > > > we
> > > > > > > > would
> > > > > > > > > > >>> need to
> > > > > > > > > > >>> > > > copy
> > > > > > > > > > >>> > > > >> the
> > > > > > > > > > >>> > > > >> > > > > > partition
> > > > > > > > > > >>> > > > >> > > > > > > > >> > metadata
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > file
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > in
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > this
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > process.
> > > > > > I've
> > > > > > > > > > updated
> > > > > > > > > > >>> the
> > > > > > > > > > >>> > > KIP
> > > > > > > > > > >>> > > > to
> > > > > > > > > > >>> > > > >> > > > include
> > > > > > > > > > >>> > > > >> > > > > > > this.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> 15)
> > I
> > > > > > believe
> > > > > > > > > Lucas
> > > > > > > > > > >>> meant
> > > > > > > > > > >>> > v1
> > > > > > > > > > >>> > > > >> and v2
> > > > > > > > > > >>> > > > >> > > > here.
> > > > > > > > > > >>> > > > >> > > > > > He
> > > > > > > > > > >>> > > > >> > > > > > > > was
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > referring
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> how
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > requests
> > > > > > would
> > > > > > > > > fall
> > > > > > > > > > >>> under
> > > > > > > > > > >>> > > > >> different
> > > > > > > > > > >>> > > > >> > > IBP
> > > > > > > > > > >>> > > > >> > > > > and
> > > > > > > > > > >>> > > > >> > > > > > > > meant
> > > > > > > > > > >>> > > > >> > > > > > > > >> > that
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > older
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > brokers
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> would
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> have
> > > to
> > > > > use
> > > > > > the
> > > > > > > > > > older
> > > > > > > > > > >>> > > version
> > > > > > > > > > >>> > > > of
> > > > > > > > > > >>> > > > >> > the
> > > > > > > > > > >>> > > > >> > > > > > request
> > > > > > > > > > >>> > > > >> > > > > > > > and
> > > > > > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > existing
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > topic
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > deletion
> > > > > > > > process.
> > > > > > > > > At
> > > > > > > > > > >>> > first,
> > > > > > > > > > >>> > > it
> > > > > > > > > > >>> > > > >> > seemed
> > > > > > > > > > >>> > > > >> > > > > like
> > > > > > > > > > >>> > > > >> > > > > > > > tagged
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > fields
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > would
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > resolve
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> the
> > > IBP
> > > > > > issue.
> > > > > > > > > > >>> However, we
> > > > > > > > > > >>> > > may
> > > > > > > > > > >>> > > > >> need
> > > > > > > > > > >>> > > > >> > > IBP
> > > > > > > > > > >>> > > > >> > > > > for
> > > > > > > > > > >>> > > > >> > > > > > > > this
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > request
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > after
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > all
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > since
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> the
> > > > > > controller
> > > > > > > > > > >>> handles the
> > > > > > > > > > >>> > > > topic
> > > > > > > > > > >>> > > > >> > > > deletion
> > > > > > > > > > >>> > > > >> > > > > > > > >> > differently
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> depending
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > on
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > IBP
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > version.
> > > > > In
> > > > > > an
> > > > > > > > > older
> > > > > > > > > > >>> > > version,
> > > > > > > > > > >>> > > > we
> > > > > > > > > > >>> > > > >> > > can't
> > > > > > > > > > >>> > > > >> > > > > just
> > > > > > > > > > >>> > > > >> > > > > > > > send
> > > > > > > > > > >>> > > > >> > > > > > > > >> a
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > StopReplica
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > delete
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > topic
> > > > > > > > immediately
> > > > > > > > > > like
> > > > > > > > > > >>> > we'd
> > > > > > > > > > >>> > > > >> want to
> > > > > > > > > > >>> > > > >> > > for
> > > > > > > > > > >>> > > > >> > > > > > this
> > > > > > > > > > >>> > > > >> > > > > > > > KIP.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> This
> > > > makes
> > > > > > me
> > > > > > > > > wonder
> > > > > > > > > > >>> if we
> > > > > > > > > > >>> > > > want
> > > > > > > > > > >>> > > > >> > > tagged
> > > > > > > > > > >>> > > > >> > > > > > fields
> > > > > > > > > > >>> > > > >> > > > > > > > on
> > > > > > > > > > >>> > > > >> > > > > > > > >> all
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> requests
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > after
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> all.
> > > Let
> > > > > me
> > > > > > know
> > > > > > > > > > your
> > > > > > > > > > >>> > > > thoughts!
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > Justine
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > On
> > > Tue,
> > > > > Sep
> > > > > > 15,
> > > > > > > > > 2020
> > > > > > > > > > >>> at
> > > > > > > > > > >>> > 1:03
> > > > > > > > > > >>> > > > PM
> > > > > > > > > > >>> > > > >> > > Justine
> > > > > > > > > > >>> > > > >> > > > > > > Olshan
> > > > > > > > > > >>> > > > >> > > > > > > > <
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > jolshan@confluent.io
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > wrote:
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> Hi
> > > all,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> Jun
> > > > > > brought up
> > > > > > > > a
> > > > > > > > > > good
> > > > > > > > > > >>> > point
> > > > > > > > > > >>> > > > in
> > > > > > > > > > >>> > > > >> his
> > > > > > > > > > >>> > > > >> > > > last
> > > > > > > > > > >>> > > > >> > > > > > > email
> > > > > > > > > > >>> > > > >> > > > > > > > >> about
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > tagged
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > fields,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > and
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > I've
> > > > > > updated
> > > > > > > > the
> > > > > > > > > > KIP
> > > > > > > > > > >>> to
> > > > > > > > > > >>> > > > reflect
> > > > > > > > > > >>> > > > >> > that
> > > > > > > > > > >>> > > > >> > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> changes to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > requests
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > responses
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > will
> > > be
> > > > > in
> > > > > > the
> > > > > > > > > form
> > > > > > > > > > >>> of
> > > > > > > > > > >>> > > tagged
> > > > > > > > > > >>> > > > >> > fields
> > > > > > > > > > >>> > > > >> > > > to
> > > > > > > > > > >>> > > > >> > > > > > > avoid
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > changing
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > IBP.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > Jun:
> > > I
> > > > > > plan on
> > > > > > > > > > >>> sending a
> > > > > > > > > > >>> > > > >> followup
> > > > > > > > > > >>> > > > >> > > > email
> > > > > > > > > > >>> > > > >> > > > > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> address
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > some
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > of
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > other
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > points.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > Thanks,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > Justine
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> On
> > > Mon,
> > > > > > Sep 14,
> > > > > > > > > > 2020
> > > > > > > > > > >>> at
> > > > > > > > > > >>> > > 4:25
> > > > > > > > > > >>> > > > PM
> > > > > > > > > > >>> > > > >> > Jun
> > > > > > > > > > >>> > > > >> > > > Rao
> > > > > > > > > > >>> > > > >> > > > > <
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > jun@confluent.io>
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > wrote:
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > Hi,
> > > > > > Justine,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > Thanks
> > > > > > for the
> > > > > > > > > > >>> updated
> > > > > > > > > > >>> > > KIP.
> > > > > > > > > > >>> > > > A
> > > > > > > > > > >>> > > > >> few
> > > > > > > > > > >>> > > > >> > > > > > comments
> > > > > > > > > > >>> > > > >> > > > > > > > >> below.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > 10.
> > > > > > > > LeaderAndIsr
> > > > > > > > > > >>> > Response:
> > > > > > > > > > >>> > > > Do
> > > > > > > > > > >>> > > > >> we
> > > > > > > > > > >>> > > > >> > > need
> > > > > > > > > > >>> > > > >> > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > topic
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > name?
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > 11.
> > > > For
> > > > > > the
> > > > > > > > > > changed
> > > > > > > > > > >>> > > > >> > > request/response,
> > > > > > > > > > >>> > > > >> > > > > > other
> > > > > > > > > > >>> > > > >> > > > > > > > >> than
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> LeaderAndIsr,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > UpdateMetadata,
> > > > > > > > > > >>> > Metadata,
> > > > > > > > > > >>> > > do
> > > > > > > > > > >>> > > > >> we
> > > > > > > > > > >>> > > > >> > > need
> > > > > > > > > > >>> > > > >> > > > to
> > > > > > > > > > >>> > > > >> > > > > > > > include
> > > > > > > > > > >>> > > > >> > > > > > > > >> > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > topic
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> name?
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > 12.
> > > It
> > > > > > seems
> > > > > > > > > that
> > > > > > > > > > >>> > upgrades
> > > > > > > > > > >>> > > > >> don't
> > > > > > > > > > >>> > > > >> > > > > require
> > > > > > > > > > >>> > > > >> > > > > > > IBP.
> > > > > > > > > > >>> > > > >> > > > > > > > >> Does
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > that
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > mean
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > new
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > fields
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> in
> > > all
> > > > > the
> > > > > > > > > > >>> > > request/response
> > > > > > > > > > >>> > > > >> are
> > > > > > > > > > >>> > > > >> > > added
> > > > > > > > > > >>> > > > >> > > > > as
> > > > > > > > > > >>> > > > >> > > > > > > > tagged
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > fields
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> without
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> bumping
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > up
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > the
> > > > > > request
> > > > > > > > > > >>> version? It
> > > > > > > > > > >>> > > > would
> > > > > > > > > > >>> > > > >> be
> > > > > > > > > > >>> > > > >> > > > useful
> > > > > > > > > > >>> > > > >> > > > > > to
> > > > > > > > > > >>> > > > >> > > > > > > > make
> > > > > > > > > > >>> > > > >> > > > > > > > >> > that
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > clear.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > 13.
> > > > > > Partition
> > > > > > > > > > >>> Metadata
> > > > > > > > > > >>> > > file:
> > > > > > > > > > >>> > > > >> Do
> > > > > > > > > > >>> > > > >> > we
> > > > > > > > > > >>> > > > >> > > > need
> > > > > > > > > > >>> > > > >> > > > > > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> include
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > topic
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > name
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > and
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > partition
> > > > > > id
> > > > > > > > > since
> > > > > > > > > > >>> they
> > > > > > > > > > >>> > > are
> > > > > > > > > > >>> > > > >> > implied
> > > > > > > > > > >>> > > > >> > > > in
> > > > > > > > > > >>> > > > >> > > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > directory
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > name?
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > 14.
> > > In
> > > > > the
> > > > > > > > JBOD
> > > > > > > > > > >>> mode, we
> > > > > > > > > > >>> > > > >> support
> > > > > > > > > > >>> > > > >> > > > > moving a
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > partition's
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > data
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> from
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > one
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > disk
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > another.
> > > > > > Will
> > > > > > > > > the
> > > > > > > > > > >>> new
> > > > > > > > > > >>> > > > >> partition
> > > > > > > > > > >>> > > > >> > > > > metadata
> > > > > > > > > > >>> > > > >> > > > > > > file
> > > > > > > > > > >>> > > > >> > > > > > > > >> be
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > copied
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> during
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > process?
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > 15.
> > > > The
> > > > > > KIP
> > > > > > > > says
> > > > > > > > > > >>> "Remove
> > > > > > > > > > >>> > > > >> deleted
> > > > > > > > > > >>> > > > >> > > > topics
> > > > > > > > > > >>> > > > >> > > > > > > from
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > replicas
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > by
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > sending
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > > StopReplicaRequest
> > > > > > > > > > >>> V2
> > > > > > > > > > >>> > for
> > > > > > > > > > >>> > > > any
> > > > > > > > > > >>> > > > >> > > topics
> > > > > > > > > > >>> > > > >> > > > > > which
> > > > > > > > > > >>> > > > >> > > > > > > do
> > > > > > > > > > >>> > > > >> > > > > > > > >> not
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > contain
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > a
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > topic
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > ID,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > and
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> V3
> > > for
> > > > > any
> > > > > > > > > topics
> > > > > > > > > > >>> which
> > > > > > > > > > >>> > do
> > > > > > > > > > >>> > > > >> > contain
> > > > > > > > > > >>> > > > >> > > a
> > > > > > > > > > >>> > > > >> > > > > > topic
> > > > > > > > > > >>> > > > >> > > > > > > > >> ID.".
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > However,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > it
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > seems
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > updated
> > > > > > > > > controller
> > > > > > > > > > >>> will
> > > > > > > > > > >>> > > > create
> > > > > > > > > > >>> > > > >> > all
> > > > > > > > > > >>> > > > >> > > > > > missing
> > > > > > > > > > >>> > > > >> > > > > > > > >> topic
> > > > > > > > > > >>> > > > >> > > > > > > > >> > IDs
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > first
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > before
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > doing
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > other
> > > > > > actions.
> > > > > > > > > So,
> > > > > > > > > > >>> is
> > > > > > > > > > >>> > > > >> > > > > StopReplicaRequest
> > > > > > > > > > >>> > > > >> > > > > > V2
> > > > > > > > > > >>> > > > >> > > > > > > > >> > needed?
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > Jun
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> On
> > > > Fri,
> > > > > > Sep
> > > > > > > > 11,
> > > > > > > > > > >>> 2020 at
> > > > > > > > > > >>> > > > 10:31
> > > > > > > > > > >>> > > > >> AM
> > > > > > > > > > >>> > > > >> > > John
> > > > > > > > > > >>> > > > >> > > > > > > > Roesler <
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > vvcephei@apache.org
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > wrote:
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > > > Thanks,
> > > > > > > > > Justine!
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > Your
> > > > > > > > response
> > > > > > > > > > >>> seems
> > > > > > > > > > >>> > > > >> compelling
> > > > > > > > > > >>> > > > >> > to
> > > > > > > > > > >>> > > > >> > > > me.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > > -John
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > On
> > > > > Fri,
> > > > > > > > > > >>> 2020-09-11 at
> > > > > > > > > > >>> > > > 10:17
> > > > > > > > > > >>> > > > >> > > -0700,
> > > > > > > > > > >>> > > > >> > > > > > > Justine
> > > > > > > > > > >>> > > > >> > > > > > > > >> > Olshan
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > wrote:
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > > > Hello
> > > > > > all,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > > > > Thanks for
> > > > > > > > > > >>> > continuing
> > > > > > > > > > >>> > > > the
> > > > > > > > > > >>> > > > >> > > > > > discussion! I
> > > > > > > > > > >>> > > > >> > > > > > > > >> have a
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > few
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > responses
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > your
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > > > points.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > > Tom:
> > > > > > You
> > > > > > > > are
> > > > > > > > > > >>> correct
> > > > > > > > > > >>> > > in
> > > > > > > > > > >>> > > > >> that
> > > > > > > > > > >>> > > > >> > > this
> > > > > > > > > > >>> > > > >> > > > > KIP
> > > > > > > > > > >>> > > > >> > > > > > > has
> > > > > > > > > > >>> > > > >> > > > > > > > >> not
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > mentioned
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > > > > > > > > >>> > DeleteTopicsRequest. I
> > > > > > > > > > >>> > > > >> think
> > > > > > > > > > >>> > > > >> > > that
> > > > > > > > > > >>> > > > >> > > > > > this
> > > > > > > > > > >>> > > > >> > > > > > > > >> would
> > > > > > > > > > >>> > > > >> > > > > > > > >> > be
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > out
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > of
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > scope
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > for
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > now,
> > > > but
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > > may
> > > > > be
> > > > > > > > > > something
> > > > > > > > > > >>> > worth
> > > > > > > > > > >>> > > > >> adding
> > > > > > > > > > >>> > > > >> > > in
> > > > > > > > > > >>> > > > >> > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> future.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > > > John:
> > > > > > We
> > > > > > > > did
> > > > > > > > > > >>> > consider
> > > > > > > > > > >>> > > > >> > sequence
> > > > > > > > > > >>> > > > >> > > > ids,
> > > > > > > > > > >>> > > > >> > > > > > but
> > > > > > > > > > >>> > > > >> > > > > > > > >> there
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > are
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > a
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > few
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > reasons
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > favor
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > > > UUIDs.
> > > > > > > > There
> > > > > > > > > > are
> > > > > > > > > > >>> > > several
> > > > > > > > > > >>> > > > >> > cases
> > > > > > > > > > >>> > > > >> > > > > where
> > > > > > > > > > >>> > > > >> > > > > > > > topics
> > > > > > > > > > >>> > > > >> > > > > > > > >> > from
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> different
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> clusters
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > may
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > > > > interact
> > > > > > > > now
> > > > > > > > > > >>> and in
> > > > > > > > > > >>> > > the
> > > > > > > > > > >>> > > > >> > future.
> > > > > > > > > > >>> > > > >> > > > For
> > > > > > > > > > >>> > > > >> > > > > > > > >> example,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > Mirror
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Maker 2
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > may
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > benefit
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > > from
> > > > > > being
> > > > > > > > > > able
> > > > > > > > > > >>> to
> > > > > > > > > > >>> > > > detect
> > > > > > > > > > >>> > > > >> > when
> > > > > > > > > > >>> > > > >> > > a
> > > > > > > > > > >>> > > > >> > > > > > > cluster
> > > > > > > > > > >>> > > > >> > > > > > > > >> being
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > mirrored
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> is
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> deleted
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > and
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > > > > recreated
> > > > > > > > > and
> > > > > > > > > > >>> > globally
> > > > > > > > > > >>> > > > >> unique
> > > > > > > > > > >>> > > > >> > > > > > > identifiers
> > > > > > > > > > >>> > > > >> > > > > > > > >> > would
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > make
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > resolving
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> issues
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > > > easier
> > > > > > > > than
> > > > > > > > > > >>> sequence
> > > > > > > > > > >>> > > IDs
> > > > > > > > > > >>> > > > >> > which
> > > > > > > > > > >>> > > > >> > > > may
> > > > > > > > > > >>> > > > >> > > > > > > > collide
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > between
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > clusters.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > KIP-405
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > > > > (tiered
> > > > > > > > > > storage)
> > > > > > > > > > >>> > will
> > > > > > > > > > >>> > > > also
> > > > > > > > > > >>> > > > >> > > > benefit
> > > > > > > > > > >>> > > > >> > > > > > from
> > > > > > > > > > >>> > > > >> > > > > > > > >> > globally
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > unique
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> IDs
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > as
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> shared
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > > > > buckets
> > > > > > > > may
> > > > > > > > > be
> > > > > > > > > > >>> used
> > > > > > > > > > >>> > > > >> between
> > > > > > > > > > >>> > > > >> > > > > clusters.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > > > > Globally
> > > > > > > > > > unique
> > > > > > > > > > >>> IDs
> > > > > > > > > > >>> > > > would
> > > > > > > > > > >>> > > > >> > also
> > > > > > > > > > >>> > > > >> > > > make
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > functionality
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > like
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > moving
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> topics
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > > > > between
> > > > > > > > > > >>> disparate
> > > > > > > > > > >>> > > > clusters
> > > > > > > > > > >>> > > > >> > > easier
> > > > > > > > > > >>> > > > >> > > > > in
> > > > > > > > > > >>> > > > >> > > > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > future,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> simplify
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > any
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> future
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > > > > > > > > implementations
> > > > > > > > > > >>> of
> > > > > > > > > > >>> > > > backups
> > > > > > > > > > >>> > > > >> > and
> > > > > > > > > > >>> > > > >> > > > > > > restores,
> > > > > > > > > > >>> > > > >> > > > > > > > >> and
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > more.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > In
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > general,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > unique
> > > > > IDs
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > > > would
> > > > > > > > ensure
> > > > > > > > > > >>> that
> > > > > > > > > > >>> > the
> > > > > > > > > > >>> > > > >> source
> > > > > > > > > > >>> > > > >> > > > > cluster
> > > > > > > > > > >>> > > > >> > > > > > > > >> topics do
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > not
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> conflict
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > with
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > > > > > > destination
> > > > > > > > > > >>> cluster
> > > > > > > > > > >>> > > > >> topics.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > If
> > > > > we
> > > > > > were
> > > > > > > > > to
> > > > > > > > > > >>> use
> > > > > > > > > > >>> > > > sequence
> > > > > > > > > > >>> > > > >> > ids,
> > > > > > > > > > >>> > > > >> > > > we
> > > > > > > > > > >>> > > > >> > > > > > > would
> > > > > > > > > > >>> > > > >> > > > > > > > >> need
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> sufficiently
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > large
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > cluster
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > > ids
> > > > > > to be
> > > > > > > > > > stored
> > > > > > > > > > >>> > with
> > > > > > > > > > >>> > > > the
> > > > > > > > > > >>> > > > >> > topic
> > > > > > > > > > >>> > > > >> > > > > > > > >> identifiers or
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > we
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > run
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > risk
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > > > > > > collisions.
> > > > > > > > > > This
> > > > > > > > > > >>> > will
> > > > > > > > > > >>> > > > >> give up
> > > > > > > > > > >>> > > > >> > > any
> > > > > > > > > > >>> > > > >> > > > > > > > >> advantage in
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> compactness
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > sequence
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > > > > numbers
> > > > > > > > may
> > > > > > > > > > >>> bring.
> > > > > > > > > > >>> > > Given
> > > > > > > > > > >>> > > > >> > these
> > > > > > > > > > >>> > > > >> > > > > > > > advantages I
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > think
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > it
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> makes
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > sense
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > use
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > > > UUIDs.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > > > Gokul:
> > > > > > > > This
> > > > > > > > > is
> > > > > > > > > > >>> an
> > > > > > > > > > >>> > > > >> interesting
> > > > > > > > > > >>> > > > >> > > > idea,
> > > > > > > > > > >>> > > > >> > > > > > but
> > > > > > > > > > >>> > > > >> > > > > > > > >> this
> > > > > > > > > > >>> > > > >> > > > > > > > >> > is
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > a
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> breaking
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> change.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > Out
> > > of
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > > > scope
> > > > > > for
> > > > > > > > > now,
> > > > > > > > > > >>> but
> > > > > > > > > > >>> > > maybe
> > > > > > > > > > >>> > > > >> > worth
> > > > > > > > > > >>> > > > >> > > > > > > discussing
> > > > > > > > > > >>> > > > >> > > > > > > > >> in
> > > > > > > > > > >>> > > > >> > > > > > > > >> > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > future.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > > Hope
> > > > > > this
> > > > > > > > > > >>> explains
> > > > > > > > > > >>> > > some
> > > > > > > > > > >>> > > > of
> > > > > > > > > > >>> > > > >> > the
> > > > > > > > > > >>> > > > >> > > > > > > decisions,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > > > > Justine
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > On
> > > > > > Fri,
> > > > > > > > Sep
> > > > > > > > > > 11,
> > > > > > > > > > >>> 2020
> > > > > > > > > > >>> > > at
> > > > > > > > > > >>> > > > >> 8:27
> > > > > > > > > > >>> > > > >> > AM
> > > > > > > > > > >>> > > > >> > > > > Gokul
> > > > > > > > > > >>> > > > >> > > > > > > > >> Ramanan
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Subramanian <
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > > > > > > > > >>> > gokul2411s@gmail.com>
> > > > > > > > > > >>> > > > >> wrote:
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > > Hi.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > > > Thanks
> > > > > > > > for
> > > > > > > > > > the
> > > > > > > > > > >>> > KIP.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > > > Have you
> > > > > > > > > > >>> thought
> > > > > > > > > > >>> > > about
> > > > > > > > > > >>> > > > >> > > whether
> > > > > > > > > > >>> > > > >> > > > it
> > > > > > > > > > >>> > > > >> > > > > > > makes
> > > > > > > > > > >>> > > > >> > > > > > > > >> > sense
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> support
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > authorizing a
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > > > > > principal
> > > > > > > > > > for
> > > > > > > > > > >>> a
> > > > > > > > > > >>> > > topic
> > > > > > > > > > >>> > > > ID
> > > > > > > > > > >>> > > > >> > > rather
> > > > > > > > > > >>> > > > >> > > > > > than
> > > > > > > > > > >>> > > > >> > > > > > > a
> > > > > > > > > > >>> > > > >> > > > > > > > >> topic
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > name
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > achieve
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > tighter
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > > > > > security?
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > Or
> > > > > > is
> > > > > > > > the
> > > > > > > > > > >>> topic ID
> > > > > > > > > > >>> > > > >> > > > fundamentally
> > > > > > > > > > >>> > > > >> > > > > an
> > > > > > > > > > >>> > > > >> > > > > > > > >> internal
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > detail
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > similar
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > epochs
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > used
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > > in a
> > > > > > > > bunch
> > > > > > > > > > of
> > > > > > > > > > >>> > other
> > > > > > > > > > >>> > > > >> places
> > > > > > > > > > >>> > > > >> > in
> > > > > > > > > > >>> > > > >> > > > > > Kafka?
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > > > Thanks.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > On
> > > > > > Fri,
> > > > > > > > > Sep
> > > > > > > > > > >>> 11,
> > > > > > > > > > >>> > 2020
> > > > > > > > > > >>> > > > at
> > > > > > > > > > >>> > > > >> > 4:06
> > > > > > > > > > >>> > > > >> > > PM
> > > > > > > > > > >>> > > > >> > > > > > John
> > > > > > > > > > >>> > > > >> > > > > > > > >> > Roesler <
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > vvcephei@apache.org
> > > > > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > > wrote:
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > Hello
> > > > > > > > > > >>> Justine,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > Thanks
> > > > > > > > > for
> > > > > > > > > > >>> the
> > > > > > > > > > >>> > > KIP!
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > I
> > > > > > > > happen
> > > > > > > > > > to
> > > > > > > > > > >>> have
> > > > > > > > > > >>> > > > been
> > > > > > > > > > >>> > > > >> > > > > confronted
> > > > > > > > > > >>> > > > >> > > > > > > > >> recently
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > with
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> need
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > keep
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > > track
> > > > > of
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > a
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > large
> > > > > > > > > > >>> number of
> > > > > > > > > > >>> > > > >> topics as
> > > > > > > > > > >>> > > > >> > > > > > compactly
> > > > > > > > > > >>> > > > >> > > > > > > > as
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > possible. I
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> was
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > going
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > come
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > up
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > with
> > > > > > > > > some
> > > > > > > > > > >>> way to
> > > > > > > > > > >>> > > > >> > dictionary
> > > > > > > > > > >>> > > > >> > > > > > encode
> > > > > > > > > > >>> > > > >> > > > > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > topic
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > names
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> as
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > integers,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > but
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > this
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > seems
> > > > > > > > > much
> > > > > > > > > > >>> > better!
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > > > > Apologies
> > > > > > > > > > if
> > > > > > > > > > >>> > this
> > > > > > > > > > >>> > > > has
> > > > > > > > > > >>> > > > >> > been
> > > > > > > > > > >>> > > > >> > > > > raised
> > > > > > > > > > >>> > > > >> > > > > > > > >> before,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > but
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > I’m
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > wondering
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > about
> > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > choice
> > > > > > > > > of
> > > > > > > > > > >>> UUID
> > > > > > > > > > >>> > vs
> > > > > > > > > > >>> > > > >> > sequence
> > > > > > > > > > >>> > > > >> > > > > number
> > > > > > > > > > >>> > > > >> > > > > > > for
> > > > > > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > ids.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Typically,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > I’ve
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > seen
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > > UUIDs
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > in two
> > > > > > > > > > >>> > situations:
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > 1.
> > > > > > > > When
> > > > > > > > > > >>> > processes
> > > > > > > > > > >>> > > > >> need to
> > > > > > > > > > >>> > > > >> > > > > > generate
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > non-colliding
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > identifiers
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > without
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > > > > > >>> coordination.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > 2.
> > > > > > > > When
> > > > > > > > > > the
> > > > > > > > > > >>> > > > identifier
> > > > > > > > > > >>> > > > >> > > needs
> > > > > > > > > > >>> > > > >> > > > to
> > > > > > > > > > >>> > > > >> > > > > > be
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > “universally
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > unique”;
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > I.e.,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > > > > identifier
> > > > > > > > > > >>> needs
> > > > > > > > > > >>> > > to
> > > > > > > > > > >>> > > > >> > > > distinguish
> > > > > > > > > > >>> > > > >> > > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> entity
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > from
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > all
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > other
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > entities
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > that
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > could
> > > > > > > > > ever
> > > > > > > > > > >>> > exist.
> > > > > > > > > > >>> > > > >> This is
> > > > > > > > > > >>> > > > >> > > > > useful
> > > > > > > > > > >>> > > > >> > > > > > in
> > > > > > > > > > >>> > > > >> > > > > > > > >> cases
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > where
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > entities
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > from
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > all
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > > kinds
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > of
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > > > systems
> > > > > > > > > > get
> > > > > > > > > > >>> > mixed
> > > > > > > > > > >>> > > > >> > together,
> > > > > > > > > > >>> > > > >> > > > > such
> > > > > > > > > > >>> > > > >> > > > > > as
> > > > > > > > > > >>> > > > >> > > > > > > > >> when
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > dumping
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> logs
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > from
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > all
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > > > > processes
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > in
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > a
> > > > > > > > > company
> > > > > > > > > > >>> into a
> > > > > > > > > > >>> > > > >> common
> > > > > > > > > > >>> > > > >> > > > system.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > Maybe
> > > > > > > > > I’m
> > > > > > > > > > >>> being
> > > > > > > > > > >>> > > > >> > > > short-sighted,
> > > > > > > > > > >>> > > > >> > > > > > but
> > > > > > > > > > >>> > > > >> > > > > > > it
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > doesn’t
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > seem
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> like
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > either
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > really
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > > > applies
> > > > > > > > > > >>> here. It
> > > > > > > > > > >>> > > > seems
> > > > > > > > > > >>> > > > >> > like
> > > > > > > > > > >>> > > > >> > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > brokers
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > could
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > and
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> would
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> achieve
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > > > > consensus
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > when
> > > > > > > > > > >>> creating a
> > > > > > > > > > >>> > > > topic
> > > > > > > > > > >>> > > > >> > > anyway,
> > > > > > > > > > >>> > > > >> > > > > > which
> > > > > > > > > > >>> > > > >> > > > > > > > is
> > > > > > > > > > >>> > > > >> > > > > > > > >> all
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that’s
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > required
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > > > generate
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > > > > > >>> non-colliding
> > > > > > > > > > >>> > > > sequence
> > > > > > > > > > >>> > > > >> > ids.
> > > > > > > > > > >>> > > > >> > > > For
> > > > > > > > > > >>> > > > >> > > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > second,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > as
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > you
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > mention,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > we
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > could
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > > > assign
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > a
> > > > > > UUID
> > > > > > > > > to
> > > > > > > > > > >>> the
> > > > > > > > > > >>> > > > cluster
> > > > > > > > > > >>> > > > >> as
> > > > > > > > > > >>> > > > >> > a
> > > > > > > > > > >>> > > > >> > > > > whole,
> > > > > > > > > > >>> > > > >> > > > > > > > which
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > would
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > render
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > any
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > resource
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > > scoped
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > the
> > > > > > > > > broker
> > > > > > > > > > >>> > > > universally
> > > > > > > > > > >>> > > > >> > > unique
> > > > > > > > > > >>> > > > >> > > > > as
> > > > > > > > > > >>> > > > >> > > > > > > > well.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > The
> > > > > > > > > > reason I
> > > > > > > > > > >>> > > mention
> > > > > > > > > > >>> > > > >> this
> > > > > > > > > > >>> > > > >> > > is
> > > > > > > > > > >>> > > > >> > > > > > that,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > although
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > a
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > UUID
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> is
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > way
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > more
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > > > compact
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > than
> > > > > > > > > topic
> > > > > > > > > > >>> > names,
> > > > > > > > > > >>> > > > it’s
> > > > > > > > > > >>> > > > >> > > still
> > > > > > > > > > >>> > > > >> > > > 16
> > > > > > > > > > >>> > > > >> > > > > > > > bytes.
> > > > > > > > > > >>> > > > >> > > > > > > > >> In
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > contrast,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> a
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > 4-byte
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > integer
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > > > sequence
> > > > > > > > > > id
> > > > > > > > > > >>> > would
> > > > > > > > > > >>> > > > give
> > > > > > > > > > >>> > > > >> > us 4
> > > > > > > > > > >>> > > > >> > > > > > billion
> > > > > > > > > > >>> > > > >> > > > > > > > >> unique
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > topics
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> per
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> cluster,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > which
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > > > seems
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > like
> > > > > > > > > > enough
> > > > > > > > > > >>> ;)
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > > > > > Considering
> > > > > > > > > > >>> the
> > > > > > > > > > >>> > > > >> number of
> > > > > > > > > > >>> > > > >> > > > > > different
> > > > > > > > > > >>> > > > >> > > > > > > > >> times
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > these
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> topic
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > identifiers
> > > > > > > > are
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > > sent
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > over
> > > > > > > > the
> > > > > > > > > > >>> wire or
> > > > > > > > > > >>> > > > >> stored
> > > > > > > > > > >>> > > > >> > in
> > > > > > > > > > >>> > > > >> > > > > > memory,
> > > > > > > > > > >>> > > > >> > > > > > > it
> > > > > > > > > > >>> > > > >> > > > > > > > >> > seems
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > like
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > it
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > might
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > be
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > worth
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > > > > additional
> > > > > > > > > > >>> 4x
> > > > > > > > > > >>> > > space
> > > > > > > > > > >>> > > > >> > > savings.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > What
> > > > > > > > do
> > > > > > > > > > you
> > > > > > > > > > >>> > think
> > > > > > > > > > >>> > > > >> about
> > > > > > > > > > >>> > > > >> > > this?
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > > > Thanks,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > John
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > On
> > > > > > > > Fri,
> > > > > > > > > > Sep
> > > > > > > > > > >>> 11,
> > > > > > > > > > >>> > > > 2020,
> > > > > > > > > > >>> > > > >> at
> > > > > > > > > > >>> > > > >> > > > 03:20,
> > > > > > > > > > >>> > > > >> > > > > > Tom
> > > > > > > > > > >>> > > > >> > > > > > > > >> > Bentley
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > wrote:
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > Hi
> > > > > > > > > > >>> Justine,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > This
> > > > > > > > > > looks
> > > > > > > > > > >>> > like
> > > > > > > > > > >>> > > a
> > > > > > > > > > >>> > > > >> very
> > > > > > > > > > >>> > > > >> > > > > welcome
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > improvement.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Thanks!
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > Maybe
> > > > > > > > > I
> > > > > > > > > > >>> missed
> > > > > > > > > > >>> > > it,
> > > > > > > > > > >>> > > > >> but
> > > > > > > > > > >>> > > > >> > > the
> > > > > > > > > > >>> > > > >> > > > > KIP
> > > > > > > > > > >>> > > > >> > > > > > > > >> doesn't
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > seem
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > mention
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > changing
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > > >>> > > > DeleteTopicsRequest
> > > > > > > > > > >>> > > > >> to
> > > > > > > > > > >>> > > > >> > > > > identify
> > > > > > > > > > >>> > > > >> > > > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > topic
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > using
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > an
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > id.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Maybe
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > > that's
> > > > > > out
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > of
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > scope,
> > > > > > > > > > but
> > > > > > > > > > >>> > > > >> > > > > DeleteTopicsRequest
> > > > > > > > > > >>> > > > >> > > > > > is
> > > > > > > > > > >>> > > > >> > > > > > > > not
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > listed
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > among
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Future
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > Work
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > APIs
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > either.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > Kind
> > > > > > > > > > >>> regards,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > Tom
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > On
> > > > > > > > > Thu,
> > > > > > > > > > >>> Sep
> > > > > > > > > > >>> > 10,
> > > > > > > > > > >>> > > > >> 2020 at
> > > > > > > > > > >>> > > > >> > > > 3:59
> > > > > > > > > > >>> > > > >> > > > > PM
> > > > > > > > > > >>> > > > >> > > > > > > > >> Satish
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > Duggana <
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > > > > > > > >>> > > > >> satish.duggana@gmail.com
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > wrote:
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > Thanks
> > > > > > > > > > >>> > > > >> Lucas/Justine
> > > > > > > > > > >>> > > > >> > > for
> > > > > > > > > > >>> > > > >> > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > nice
> > > > > > > > > > >>> > > > >> > > > > > > > >> KIP.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > It
> > > > > > > > > has
> > > > > > > > > > >>> > several
> > > > > > > > > > >>> > > > >> > benefits
> > > > > > > > > > >>> > > > >> > > > > which
> > > > > > > > > > >>> > > > >> > > > > > > > also
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > include
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> simplifying
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > topic
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > deletion
> > > > > > > > > > >>> > > process
> > > > > > > > > > >>> > > > >> by
> > > > > > > > > > >>> > > > >> > > > > > controller
> > > > > > > > > > >>> > > > >> > > > > > > > and
> > > > > > > > > > >>> > > > >> > > > > > > > >> > logs
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > cleanup
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> by
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> brokers
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > in
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > > corner
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > cases.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > Best,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > Satish.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > On
> > > > > > > > > > Wed,
> > > > > > > > > > >>> Sep
> > > > > > > > > > >>> > 9,
> > > > > > > > > > >>> > > > >> 2020
> > > > > > > > > > >>> > > > >> > at
> > > > > > > > > > >>> > > > >> > > > > 10:07
> > > > > > > > > > >>> > > > >> > > > > > PM
> > > > > > > > > > >>> > > > >> > > > > > > > >> > Justine
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Olshan <
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > > > > > > > > >>> jolshan@confluent.io
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > wrote:
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > > > Hello
> > > > > > > > > > >>> all,
> > > > > > > > > > >>> > > > it's
> > > > > > > > > > >>> > > > >> > been
> > > > > > > > > > >>> > > > >> > > > > > almost a
> > > > > > > > > > >>> > > > >> > > > > > > > >> year!
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > I've
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > made
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > some
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > changes
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > this
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > KIP
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > and
> > > > > > > > > > >>> hope to
> > > > > > > > > > >>> > > > >> continue
> > > > > > > > > > >>> > > > >> > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> discussion.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > > One
> > > > > > > > > > >>> of the
> > > > > > > > > > >>> > > > main
> > > > > > > > > > >>> > > > >> > > changes
> > > > > > > > > > >>> > > > >> > > > > > I've
> > > > > > > > > > >>> > > > >> > > > > > > > >> added
> > > > > > > > > > >>> > > > >> > > > > > > > >> > is
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > now
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > metadata
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > response
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > > will
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > include
> > > > > > > > > > >>> the
> > > > > > > > > > >>> > > > topic
> > > > > > > > > > >>> > > > >> ID
> > > > > > > > > > >>> > > > >> > > (as
> > > > > > > > > > >>> > > > >> > > > > > Colin
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > suggested).
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Clients
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > can
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > obtain
> > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > > > topicID
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > of a
> > > > > > > > > > >>> given
> > > > > > > > > > >>> > > topic
> > > > > > > > > > >>> > > > >> > > through
> > > > > > > > > > >>> > > > >> > > > a
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > TopicDescription.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > The
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > topicId
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > will
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > also
> > > > > be
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > included
> > > > > > > > > > >>> > with
> > > > > > > > > > >>> > > > the
> > > > > > > > > > >>> > > > >> > > > > > > UpdateMetadata
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > request.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > > Let
> > > > > > > > > > me
> > > > > > > > > > >>> > know
> > > > > > > > > > >>> > > > what
> > > > > > > > > > >>> > > > >> > you
> > > > > > > > > > >>> > > > >> > > > all
> > > > > > > > > > >>> > > > >> > > > > > > think.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > > > Thank
> > > > > > > > > > >>> you,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > > > >>> Justine
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > On
> > > > > > > > > > >>> > > 2019/09/13
> > > > > > > > > > >>> > > > >> > > 16:38:26,
> > > > > > > > > > >>> > > > >> > > > > > > "Colin
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > McCabe" <
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > cmccabe@apache.org
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > > wrote:
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > Hi
> > > > > > > > > > >>> > Lucas,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > > >>> Thanks
> > > > > > > > > > >>> > for
> > > > > > > > > > >>> > > > >> > tackling
> > > > > > > > > > >>> > > > >> > > > > this.
> > > > > > > > > > >>> > > > >> > > > > > > > >> Topic
> > > > > > > > > > >>> > > > >> > > > > > > > >> > IDs
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > are a
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > great
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > idea,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > and
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > this
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > is
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > a
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > really
> > > > > > > > > > >>> good
> > > > > > > > > > >>> > > > >> writeup.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > > For
> > > > > > > > > > >>> > > > >> > > > > > > /brokers/topics/[topic],
> > > > > > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > schema
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> version
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > should
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > be
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > > bumped
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > version
> > > > > > > > > > >>> 3,
> > > > > > > > > > >>> > > > rather
> > > > > > > > > > >>> > > > >> > than
> > > > > > > > > > >>> > > > >> > > 2.
> > > > > > > > > > >>> > > > >> > > > > > > > KIP-455
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > bumped
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > version
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > this
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > > znode
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > 2
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > already
> > > > > > > > > > >>> :)
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > > >>> Given
> > > > > > > > > > >>> > that
> > > > > > > > > > >>> > > > >> we're
> > > > > > > > > > >>> > > > >> > > > going
> > > > > > > > > > >>> > > > >> > > > > to
> > > > > > > > > > >>> > > > >> > > > > > > be
> > > > > > > > > > >>> > > > >> > > > > > > > >> > seeing
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > these
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > things
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > as
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > strings
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > as
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > > lot
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > (in
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > logs,
> > > > > > > > > > in
> > > > > > > > > > >>> > > > >> ZooKeeper,
> > > > > > > > > > >>> > > > >> > on
> > > > > > > > > > >>> > > > >> > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > command-line,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > etc.),
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > does
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > it
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > make
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > > sense
> > > > > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > use
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > base64
> > > > > > > > > > >>> when
> > > > > > > > > > >>> > > > >> > converting
> > > > > > > > > > >>> > > > >> > > > them
> > > > > > > > > > >>> > > > >> > > > > > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > strings?
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > > >>> Here is
> > > > > > > > > > >>> > an
> > > > > > > > > > >>> > > > >> > example
> > > > > > > > > > >>> > > > >> > > of
> > > > > > > > > > >>> > > > >> > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > hex
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> representation:
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > > And
> > > > > > > > > > >>> here
> > > > > > > > > > >>> > > is
> > > > > > > > > > >>> > > > an
> > > > > > > > > > >>> > > > >> > > > example
> > > > > > > > > > >>> > > > >> > > > > in
> > > > > > > > > > >>> > > > >> > > > > > > > >> base64.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > > >>> > > > >> > > > b8tRS7h4TJ2Vt43Dp85v2A
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > > The
> > > > > > > > > > >>> > base64
> > > > > > > > > > >>> > > > >> > version
> > > > > > > > > > >>> > > > >> > > > > saves
> > > > > > > > > > >>> > > > >> > > > > > 15
> > > > > > > > > > >>> > > > >> > > > > > > > >> > letters
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > (to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > be
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > fair,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > 4
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > those
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > were
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > dashes
> > > > > > > > > > >>> that
> > > > > > > > > > >>> > we
> > > > > > > > > > >>> > > > >> could
> > > > > > > > > > >>> > > > >> > > have
> > > > > > > > > > >>> > > > >> > > > > > > elided
> > > > > > > > > > >>> > > > >> > > > > > > > in
> > > > > > > > > > >>> > > > >> > > > > > > > >> > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > hex
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > representation.)
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > > >>> Another
> > > > > > > > > > >>> > > > thing
> > > > > > > > > > >>> > > > >> to
> > > > > > > > > > >>> > > > >> > > > > consider
> > > > > > > > > > >>> > > > >> > > > > > > is
> > > > > > > > > > >>> > > > >> > > > > > > > >> that
> > > > > > > > > > >>> > > > >> > > > > > > > >> > we
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > should
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > specify
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > that
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > >>> all-zeroes
> > > > > > > > > > >>> > > UUID
> > > > > > > > > > >>> > > > is
> > > > > > > > > > >>> > > > >> > not
> > > > > > > > > > >>> > > > >> > > a
> > > > > > > > > > >>> > > > >> > > > > > valid
> > > > > > > > > > >>> > > > >> > > > > > > > >> topic
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > UUID.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >  We
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > can't
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > use
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > null
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > for
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > > this
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > because
> > > > > > > > > > >>> we
> > > > > > > > > > >>> > > can't
> > > > > > > > > > >>> > > > >> > pass a
> > > > > > > > > > >>> > > > >> > > > > null
> > > > > > > > > > >>> > > > >> > > > > > > UUID
> > > > > > > > > > >>> > > > >> > > > > > > > >> over
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > RPC
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > protocol
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > (there
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > is
> > > > no
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > special
> > > > > > > > > > >>> > > pattern
> > > > > > > > > > >>> > > > >> for
> > > > > > > > > > >>> > > > >> > > null,
> > > > > > > > > > >>> > > > >> > > > > nor
> > > > > > > > > > >>> > > > >> > > > > > > do
> > > > > > > > > > >>> > > > >> > > > > > > > we
> > > > > > > > > > >>> > > > >> > > > > > > > >> > want
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> waste
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > space
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > reserving
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > > such
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > a
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > >>> pattern.)
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > > >>> Maybe I
> > > > > > > > > > >>> > > > missed
> > > > > > > > > > >>> > > > >> > it,
> > > > > > > > > > >>> > > > >> > > > but
> > > > > > > > > > >>> > > > >> > > > > > did
> > > > > > > > > > >>> > > > >> > > > > > > > you
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > describe
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > "migration
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> of...
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > > > existing
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > topic[s]
> > > > > > > > > > >>> > > without
> > > > > > > > > > >>> > > > >> > topic
> > > > > > > > > > >>> > > > >> > > > IDs"
> > > > > > > > > > >>> > > > >> > > > > > in
> > > > > > > > > > >>> > > > >> > > > > > > > >> detail
> > > > > > > > > > >>> > > > >> > > > > > > > >> > in
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > any
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > section?
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > It
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > seems
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > like
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > when
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > the
> > > > > > > > > > new
> > > > > > > > > > >>> > > > controller
> > > > > > > > > > >>> > > > >> > > > becomes
> > > > > > > > > > >>> > > > >> > > > > > > > active,
> > > > > > > > > > >>> > > > >> > > > > > > > >> it
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > should
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> just
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> generate
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > random
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > UUIDs
> > > > > > > > > for
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > these,
> > > > > > > > > > >>> and
> > > > > > > > > > >>> > > write
> > > > > > > > > > >>> > > > >> the
> > > > > > > > > > >>> > > > >> > > > random
> > > > > > > > > > >>> > > > >> > > > > > > UUIDs
> > > > > > > > > > >>> > > > >> > > > > > > > >> back
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > ZooKeeper.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > It
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > would
> > > > be
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > > good
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > spell
> > > > > > > > > > >>> that
> > > > > > > > > > >>> > > out.
> > > > > > > > > > >>> > > > >> We
> > > > > > > > > > >>> > > > >> > > > should
> > > > > > > > > > >>> > > > >> > > > > > make
> > > > > > > > > > >>> > > > >> > > > > > > > it
> > > > > > > > > > >>> > > > >> > > > > > > > >> > clear
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> this
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> happens
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > > > > regardless
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > of
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > the
> > > > > > > > > > >>> > > inter-broker
> > > > > > > > > > >>> > > > >> > > protocol
> > > > > > > > > > >>> > > > >> > > > > > > version
> > > > > > > > > > >>> > > > >> > > > > > > > >> > (it's
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > a
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > compatible
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > change).
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > > >>> > > > >> > > "LeaderAndIsrRequests
> > > > > > > > > > >>> > > > >> > > > > > > > >> including an
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > is_every_partition
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > flag"
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > > > seems a
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > bit
> > > > > > > > > > >>> wordy.
> > > > > > > > > > >>> > > Can
> > > > > > > > > > >>> > > > we
> > > > > > > > > > >>> > > > >> > just
> > > > > > > > > > >>> > > > >> > > > > call
> > > > > > > > > > >>> > > > >> > > > > > > > these
> > > > > > > > > > >>> > > > >> > > > > > > > >> > "full
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > > > >>> LeaderAndIsrRequests"?
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > Then
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > RPC
> > > > > > > > > > >>> field
> > > > > > > > > > >>> > > could
> > > > > > > > > > >>> > > > be
> > > > > > > > > > >>> > > > >> > > named
> > > > > > > > > > >>> > > > >> > > > > > > "full".
> > > > > > > > > > >>> > > > >> > > > > > > > >> > Also,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > it
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > would
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > probably
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > be
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > > better
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > for
> > > > > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > RPC
> > > > > > > > > > >>> field to
> > > > > > > > > > >>> > > be
> > > > > > > > > > >>> > > > an
> > > > > > > > > > >>> > > > >> > enum
> > > > > > > > > > >>> > > > >> > > > of
> > > > > > > > > > >>> > > > >> > > > > {
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > UNSPECIFIED,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> INCREMENTAL,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > FULL
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> },
> > > so
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > > that
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > we
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > can
> > > > > > > > > > >>> cleanly
> > > > > > > > > > >>> > > > handle
> > > > > > > > > > >>> > > > >> > old
> > > > > > > > > > >>> > > > >> > > > > > versions
> > > > > > > > > > >>> > > > >> > > > > > > > (by
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > treating
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> them
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > as
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > UNSPECIFIED)
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > In
> > > > > > > > > > >>> the
> > > > > > > > > > >>> > > > >> > > > > > LeaderAndIsrRequest
> > > > > > > > > > >>> > > > >> > > > > > > > >> > section,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > you
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> write
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > "A
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > final
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > > > deletion
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > event
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > will
> > > > > > > > > > be
> > > > > > > > > > >>> > > > secheduled
> > > > > > > > > > >>> > > > >> > for
> > > > > > > > > > >>> > > > >> > > X
> > > > > > > > > > >>> > > > >> > > > ms
> > > > > > > > > > >>> > > > >> > > > > > > after
> > > > > > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > LeaderAndIsrRequest
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > was
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > > first
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > >>> received..."
> > > > > > > > > > >>> > > I
> > > > > > > > > > >>> > > > >> guess
> > > > > > > > > > >>> > > > >> > > > the X
> > > > > > > > > > >>> > > > >> > > > > > > was a
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > placeholder
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> that
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > you
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > intended
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > > > replace
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > before
> > > > > > > > > > >>> > > posting?
> > > > > > > > > > >>> > > > :)
> > > > > > > > > > >>> > > > >> > In
> > > > > > > > > > >>> > > > >> > > > any
> > > > > > > > > > >>> > > > >> > > > > > > case,
> > > > > > > > > > >>> > > > >> > > > > > > > >> this
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > seems
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > like
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > kind
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > of
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > > thing
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > > we'd
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > want a
> > > > > > > > > > >>> > > > >> configuration
> > > > > > > > > > >>> > > > >> > > for.
> > > > > > > > > > >>> > > > >> > > > > > > Let's
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > describe
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > configuration
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > key
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > > > > somewhere
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > in
> > > > > > > > > > this
> > > > > > > > > > >>> KIP,
> > > > > > > > > > >>> > > > >> > including
> > > > > > > > > > >>> > > > >> > > > what
> > > > > > > > > > >>> > > > >> > > > > > its
> > > > > > > > > > >>> > > > >> > > > > > > > >> > default
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > value
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> is.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > We
> > > > > > > > > > >>> > should
> > > > > > > > > > >>> > > > >> > probably
> > > > > > > > > > >>> > > > >> > > > also
> > > > > > > > > > >>> > > > >> > > > > > > log a
> > > > > > > > > > >>> > > > >> > > > > > > > >> > bunch
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > of
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> messages
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > at
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > WARN
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > level
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > > when
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > >>> something is
> > > > > > > > > > >>> > > > >> > scheduled
> > > > > > > > > > >>> > > > >> > > > for
> > > > > > > > > > >>> > > > >> > > > > > > > >> deletion,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > as
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > well.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > (Maybe
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > this
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > was
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > > > assumed,
> > > > > > > > > > but
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > it
> > > > > > > > > > >>> would be
> > > > > > > > > > >>> > > good
> > > > > > > > > > >>> > > > >> to
> > > > > > > > > > >>> > > > >> > > > mention
> > > > > > > > > > >>> > > > >> > > > > > > it).
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > I
> > > > > > > > > > >>> feel
> > > > > > > > > > >>> > > like
> > > > > > > > > > >>> > > > >> there
> > > > > > > > > > >>> > > > >> > > > are a
> > > > > > > > > > >>> > > > >> > > > > > few
> > > > > > > > > > >>> > > > >> > > > > > > > >> > sections
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > should
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > be
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > moved
> > > > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > > > > "rejected
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > >>> > alternatives."
> > > > > > > > > > >>> > > > >> For
> > > > > > > > > > >>> > > > >> > > > > example,
> > > > > > > > > > >>> > > > >> > > > > > in
> > > > > > > > > > >>> > > > >> > > > > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > DeleteTopics
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > section,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > since
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > > we're
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > not
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > going
> > > > > > > > > > >>> to do
> > > > > > > > > > >>> > > > >> option 1
> > > > > > > > > > >>> > > > >> > or
> > > > > > > > > > >>> > > > >> > > > 2,
> > > > > > > > > > >>> > > > >> > > > > > > these
> > > > > > > > > > >>> > > > >> > > > > > > > >> > should
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > be
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > moved
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > into
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > "rejected
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > >>> > alternatives,"
> > > > > > > > > > >>> > > > >> > rather
> > > > > > > > > > >>> > > > >> > > > than
> > > > > > > > > > >>> > > > >> > > > > > > > >> appearing
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > inline.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Another
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > case
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> is
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > > > "Should
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > we
> > > > > > > > > > >>> remove
> > > > > > > > > > >>> > > topic
> > > > > > > > > > >>> > > > >> name
> > > > > > > > > > >>> > > > >> > > from
> > > > > > > > > > >>> > > > >> > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> protocol
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > where
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > possible"
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > section.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > This
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > is
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > clearly
> > > > > > > > > > >>> > > > >> discussing a
> > > > > > > > > > >>> > > > >> > > > design
> > > > > > > > > > >>> > > > >> > > > > > > > >> > alternative
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > that
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> we're
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > not
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > proposing
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > >>> implement:
> > > > > > > > > > >>> > > > >> removing
> > > > > > > > > > >>> > > > >> > the
> > > > > > > > > > >>> > > > >> > > > > topic
> > > > > > > > > > >>> > > > >> > > > > > > > name
> > > > > > > > > > >>> > > > >> > > > > > > > >> > from
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > those
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> protocols.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > Is
> > > > > > > > > > >>> it
> > > > > > > > > > >>> > > really
> > > > > > > > > > >>> > > > >> > > > necessary
> > > > > > > > > > >>> > > > >> > > > > to
> > > > > > > > > > >>> > > > >> > > > > > > > have
> > > > > > > > > > >>> > > > >> > > > > > > > >> a
> > > > > > > > > > >>> > > > >> > > > > > > > >> > new
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > > > > > > > > >>> > > /admin/delete_topics_by_id
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > path
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > in
> > > > > > > > > > >>> > ZooKeeper?
> > > > > > > > > > >>> > > > It
> > > > > > > > > > >>> > > > >> > seems
> > > > > > > > > > >>> > > > >> > > > > like
> > > > > > > > > > >>> > > > >> > > > > > we
> > > > > > > > > > >>> > > > >> > > > > > > > >> don't
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > really
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> need
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > this.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > Whenever
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > there
> > > > > > > > is
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > a
> > > > > > > > > new
> > > > > > > > > > >>> > > > controller,
> > > > > > > > > > >>> > > > >> > we'll
> > > > > > > > > > >>> > > > >> > > > > send
> > > > > > > > > > >>> > > > >> > > > > > > out
> > > > > > > > > > >>> > > > >> > > > > > > > >> full
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > LeaderAndIsrRequests
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > which
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > will
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > trigger
> > > > > > > > > > >>> the
> > > > > > > > > > >>> > > > stale
> > > > > > > > > > >>> > > > >> > > topics
> > > > > > > > > > >>> > > > >> > > > to
> > > > > > > > > > >>> > > > >> > > > > > be
> > > > > > > > > > >>> > > > >> > > > > > > > >> cleaned
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > up.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >  The
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > active
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > > > > controller
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > > will
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > also
> > > > > > > > > > >>> send
> > > > > > > > > > >>> > the
> > > > > > > > > > >>> > > > full
> > > > > > > > > > >>> > > > >> > > > > > > > >> LeaderAndIsrRequest
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> brokers
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > are
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > just
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > > > starting
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > up.
> > > > > > > > > > >>> So we
> > > > > > > > > > >>> > > > don't
> > > > > > > > > > >>> > > > >> > > really
> > > > > > > > > > >>> > > > >> > > > > > need
> > > > > > > > > > >>> > > > >> > > > > > > > this
> > > > > > > > > > >>> > > > >> > > > > > > > >> > kind
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > of
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > two-phase
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> commit
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > > (send
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > > out
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > >>> > > > >> StopReplicasRequest,
> > > > > > > > > > >>> > > > >> > > get
> > > > > > > > > > >>> > > > >> > > > > ACKs
> > > > > > > > > > >>> > > > >> > > > > > > > from
> > > > > > > > > > >>> > > > >> > > > > > > > >> all
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > nodes,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > commit
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > by
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > removing
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > >>> > > > >> /admin/delete_topics
> > > > > > > > > > >>> > > > >> > > > node)
> > > > > > > > > > >>> > > > >> > > > > > any
> > > > > > > > > > >>> > > > >> > > > > > > > >> more.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > > You
> > > > > > > > > > >>> > > mention
> > > > > > > > > > >>> > > > >> that
> > > > > > > > > > >>> > > > >> > > > > > > FetchRequest
> > > > > > > > > > >>> > > > >> > > > > > > > >> will
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > now
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> include
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > UUID
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > avoid
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > > > issues
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > where
> > > > > > > > > > >>> > requests
> > > > > > > > > > >>> > > > are
> > > > > > > > > > >>> > > > >> > made
> > > > > > > > > > >>> > > > >> > > > to
> > > > > > > > > > >>> > > > >> > > > > > > stale
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > partitions.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > However,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > adding a
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > UUID
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > >>> > > MetadataRequest
> > > > > > > > > > >>> > > > is
> > > > > > > > > > >>> > > > >> > > listed
> > > > > > > > > > >>> > > > >> > > > > as
> > > > > > > > > > >>> > > > >> > > > > > > > future
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > work,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > out
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > of
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > scope
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > for
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > this
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > KIP.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > How
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > will
> > > > > > > > > > the
> > > > > > > > > > >>> > > client
> > > > > > > > > > >>> > > > >> learn
> > > > > > > > > > >>> > > > >> > > > what
> > > > > > > > > > >>> > > > >> > > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> topic
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > UUID
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > is,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > if
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > metadata
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > > > response
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > doesn't
> > > > > > > > > > >>> > > include
> > > > > > > > > > >>> > > > >> that
> > > > > > > > > > >>> > > > >> > > > > > > information?
> > > > > > > > > > >>> > > > >> > > > > > > > >> It
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > seems
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > like
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > adding
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > UUID
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > >>> > > MetadataResponse
> > > > > > > > > > >>> > > > >> > would
> > > > > > > > > > >>> > > > >> > > be
> > > > > > > > > > >>> > > > >> > > > > an
> > > > > > > > > > >>> > > > >> > > > > > > > >> > improvement
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > here
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> that
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > might
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > not
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> be
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > too
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > hard
> > > > > > > > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > make.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > > >>> best,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > > >>> Colin
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > On
> > > > > > > > > > >>> Mon,
> > > > > > > > > > >>> > > Sep
> > > > > > > > > > >>> > > > 9,
> > > > > > > > > > >>> > > > >> > > 2019,
> > > > > > > > > > >>> > > > >> > > > at
> > > > > > > > > > >>> > > > >> > > > > > > > 17:48,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > Ryanne
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Dolan
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > wrote:
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >>> Lucas,
> > > > > > > > > > >>> > > > this
> > > > > > > > > > >>> > > > >> > would
> > > > > > > > > > >>> > > > >> > > > be
> > > > > > > > > > >>> > > > >> > > > > > > great.
> > > > > > > > > > >>> > > > >> > > > > > > > >> I've
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > run
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > into
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > issues
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > with
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > > topics
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > being
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >>> > > > resurrected
> > > > > > > > > > >>> > > > >> > > > > > accidentally,
> > > > > > > > > > >>> > > > >> > > > > > > > >> since
> > > > > > > > > > >>> > > > >> > > > > > > > >> > a
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > client
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > cannot
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> easily
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > > > > > distinguish
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > between
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > a
> > > > > > > > > > >>> > > deleted
> > > > > > > > > > >>> > > > >> topic
> > > > > > > > > > >>> > > > >> > > > and a
> > > > > > > > > > >>> > > > >> > > > > > new
> > > > > > > > > > >>> > > > >> > > > > > > > >> topic
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > with
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> same
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > name.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I'd
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > need
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > ID
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >>> > > accessible
> > > > > > > > > > >>> > > > >> from
> > > > > > > > > > >>> > > > >> > > the
> > > > > > > > > > >>> > > > >> > > > > > > client
> > > > > > > > > > >>> > > > >> > > > > > > > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > solve
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > issue,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > but
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > this
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > is a
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > good
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > first
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >>> step.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >>> Ryanne
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On
> > > > > > > > > > >>> > Wed,
> > > > > > > > > > >>> > > > Sep
> > > > > > > > > > >>> > > > >> 4,
> > > > > > > > > > >>> > > > >> > > 2019
> > > > > > > > > > >>> > > > >> > > > > at
> > > > > > > > > > >>> > > > >> > > > > > > 1:41
> > > > > > > > > > >>> > > > >> > > > > > > > >> PM
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > Lucas
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> Bradstreet <
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > > > > > >>> > > lucas@confluent.io>
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > wrote:
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >>> Hi
> > > > > > > > > > >>> > > all,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >>> I
> > > > > > > > > > >>> > > would
> > > > > > > > > > >>> > > > >> like
> > > > > > > > > > >>> > > > >> > to
> > > > > > > > > > >>> > > > >> > > > > kick
> > > > > > > > > > >>> > > > >> > > > > > > off
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > discussion
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > of
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > KIP-516,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > an
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > > > > > >>> implementation
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > of
> > > > > > > > > > topic
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >>> IDs
> > > > > > > > > > >>> > > for
> > > > > > > > > > >>> > > > >> > Kafka.
> > > > > > > > > > >>> > > > >> > > > > Topic
> > > > > > > > > > >>> > > > >> > > > > > > IDs
> > > > > > > > > > >>> > > > >> > > > > > > > >> aim
> > > > > > > > > > >>> > > > >> > > > > > > > >> > to
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > solve
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> topic
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > uniqueness
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > > > problems
> > > > > > > > > > in
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > Kafka,
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >>> > where
> > > > > > > > > > >>> > > > >> > referring
> > > > > > > > > > >>> > > > >> > > > to
> > > > > > > > > > >>> > > > >> > > > > a
> > > > > > > > > > >>> > > > >> > > > > > > > topic
> > > > > > > > > > >>> > > > >> > > > > > > > >> by
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > name
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> alone is
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > insufficient.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > > Such
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > cases
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >>> > > include
> > > > > > > > > > >>> > > > >> when
> > > > > > > > > > >>> > > > >> > a
> > > > > > > > > > >>> > > > >> > > > > topic
> > > > > > > > > > >>> > > > >> > > > > > > has
> > > > > > > > > > >>> > > > >> > > > > > > > >> been
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > deleted
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > recreated
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > with
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > the
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > > same
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > name.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >>> > Unique
> > > > > > > > > > >>> > > > >> > > > identifiers
> > > > > > > > > > >>> > > > >> > > > > > will
> > > > > > > > > > >>> > > > >> > > > > > > > >> help
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > simplify
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > improve
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > Kafka's
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > > > topic
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > deletion
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >>> > > process,
> > > > > > > > > > >>> > > > >> as
> > > > > > > > > > >>> > > > >> > > well
> > > > > > > > > > >>> > > > >> > > > as
> > > > > > > > > > >>> > > > >> > > > > > > > prevent
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > cases
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > where
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > brokers
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > may
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > > > > > > incorrectly
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > interact
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >>> with
> > > > > > > > > > >>> > > > stale
> > > > > > > > > > >>> > > > >> > > > versions
> > > > > > > > > > >>> > > > >> > > > > > of
> > > > > > > > > > >>> > > > >> > > > > > > > >> topics.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > > > > >>> > > > >> > > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > >
> > > > > > > > > > >>> > > > >> > > > > >
> > > > > > > > > > >>> > > > >> > > > >
> > > > > > > > > > >>> > > > >> > > >
> > > > > > > > > > >>> > > > >> > >
> > > > > > > > > > >>> > > > >> >
> > > > > > > > > > >>> > > > >>
> > > > > > > > > > >>> > > >
> > > > > > > > > > >>> > >
> > > > > > > > > > >>> >
> > > > > > > > > > >>>
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >>> > > Looking
> > > > > > > > > > >>> > > > >> > forward
> > > > > > > > > > >>> > > > >> > > > to
> > > > > > > > > > >>> > > > >> > > > > > your
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > thoughts.
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >>> > Lucas
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > > > > >>> > > > >> > > > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > > >
> > > > > > > > > > >>> > > > >> > > > > > >
> > > > > > > > > > >>> > > > >> > > > > >
> > > > > > > > > > >>> > > > >> > > > >
> > > > > > > > > > >>> > > > >> > > >
> > > > > > > > > > >>> > > > >> > >
> > > > > > > > > > >>> > > > >> >
> > > > > > > > > > >>> > > > >>
> > > > > > > > > > >>> > > > >
> > > > > > > > > > >>> > > >
> > > > > > > > > > >>> > >
> > > > > > > > > > >>> >
> > > > > > > > > > >>>
> > > > > > > > > > >>
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Justine Olshan <jo...@confluent.io>]

Hi Jun,

50. Works for me. Is the idea that it will be nullable from this version
onwards, but not past versions?

51. Added some information and example methods for AdminClient future work

Thanks,
Justine


On Thu, Oct 8, 2020 at 10:59 AM Jun Rao <ju...@confluent.io> wrote:

> Hi, Justine,
>
> The changes sound reasonable to me.
>
> 50. Should we make the topic name in the new MetadataRequest
> and DeleteTopicsRequest nullable?
>
> 51. In the future work section, perhaps we could mention opportunities for
> using topicId in other methods in AdminClient where topic name is used.
>
> Thanks,
>
> Jun
>
> On Wed, Oct 7, 2020 at 2:44 PM Justine Olshan <jo...@confluent.io>
> wrote:
>
> > Hi all,
> >
> > Another update on the KIP. I've added some information about how topic
> IDs
> > will be supported in AdminClient
> >
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-AdminClientSupport
> >
> >
> > One specific decision I wanted to focus on is the inclusion of both topic
> > names and topic Ids in metadata and deleteTopics requests.
> >
> > The idea is that when the request is received the topic id field is
> checked
> > first, but if it contains the default (zero) uuid, then the topic name
> > field will be used.
> >
> > In the case where ids are used, if the ids do not have a corresponding
> > topic, the request will fail. In the case where both ids and names are
> > included, the request will fail if the name does not match the id.
> >
> >
> > Initially the id field will be used to support the admin client use cases
> > as described in the KIP, but over time, we can transfer to using topic
> ids
> > in all uses of the request.
> >
> >
> > There will also be some support for returning the topic Id on the
> > createtopics, deletetopics and metadata responses. I think it makes sense
> > to include both the name and the id in the successful response for these
> > cases.
> >
> >
> > Let me know if you have any comments or concerns regarding the Admin
> Client
> > changes.
> >
> > On Wed, Oct 7, 2020 at 11:05 AM Jun Rao <ju...@confluent.io> wrote:
> >
> > > Hi, Justine,
> > >
> > > I was just wondering if we expose topicId publicly, how would people
> use
> > > it. One possibility is to use topicId in all those public interfaces
> > where
> > > we use topic name. I am not sure if the need is very strong though. So,
> > > maybe we can list this as future considerations.
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Wed, Oct 7, 2020 at 8:57 AM Justine Olshan <jo...@confluent.io>
> > > wrote:
> > >
> > > > Hi all,
> > > >
> > > > After some careful consideration, I've decided that the
> > TopicIdPartition
> > > > will be out of scope for this KIP. After investigating what it will
> > look
> > > > like using it in only the Fetch Request, I found that I would need to
> > > > modify several public APIs in the consumer and I would need to take
> > > special
> > > > care in many situations to handle the request correctly based on what
> > > > version of the request was sent. For these reasons, I've added
> > > > TopicIdPartition to potential future work along with an alternative
> to
> > > > modify TopicPartition to include a topic ID field.
> > > >
> > > > Jun- I was also wondering about some of the details for supporting
> > topic
> > > > IDs for delete and describe topics in the admin client. Is the idea
> > that
> > > > instead of supplying topic names to describe or delete, we would
> > supply a
> > > > collection of ids? If that is the case, I'm wondering if it makes
> sense
> > > to
> > > > add topic Ids to the metadata and delete topics requests.
> > > >
> > > > Thanks,
> > > > Justine
> > > >
> > > > On Mon, Oct 5, 2020 at 9:25 PM Satish Duggana <
> > satish.duggana@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi Jun, Justine,
> > > > > +1 to have TopicIdPartition in public packages. This is useful for
> > > > > KIP-405 tiered storage too. We currently use TopicPartition for
> > > > > pluggable log storage and metadata storage apis. We will be using
> > > > > TopicIdPartition once the proposed changes in this KIP are merged.
> > > > >
> > > > > Thanks,
> > > > > Satish.
> > > > >
> > > > > On Tue, Oct 6, 2020 at 5:21 AM Jun Rao <ju...@confluent.io> wrote:
> > > > > >
> > > > > > Hi, Justine,
> > > > > >
> > > > > > I guess we have to maintain the mapping between topic name and
> > > topicId
> > > > in
> > > > > > both the client and the server. If TopicIdPartition is meant for
> > > > internal
> > > > > > usage only, we can create it under a package
> > > > > > like org/apache/kafka/common/internal and we don't need to
> document
> > > it
> > > > in
> > > > > > the KIP. If it's meant for external public usage (and potentially
> > > > > internal
> > > > > > usage too), we can create it in org/apache/kafka/common and need
> to
> > > > > > document it in the KIP.
> > > > > >
> > > > > > Since topicId is now part of the topic metadata, perhaps it's
> > useful
> > > to
> > > > > > expose it in TopicMetadataAndConfig too.
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Jun
> > > > > >
> > > > > > On Mon, Oct 5, 2020 at 3:35 PM Justine Olshan <
> > jolshan@confluent.io>
> > > > > wrote:
> > > > > >
> > > > > > > Hi Jun,
> > > > > > >
> > > > > > > This KIP removes the topic name from many requests. A lot of
> > > requests
> > > > > keep
> > > > > > > track of the partitions internally with a TopicPartition. So
> when
> > > we
> > > > > remove
> > > > > > > the topic name, we can not differentiate unless we can also use
> > > topic
> > > > > ID.
> > > > > > > The Fetch request relies heavily on this, but ListOffset,
> > > > > > > OffsetsForLeaderEpoch, etc all use TopicPartition as a key
> value
> > > for
> > > > > > > internal maps. Digging deeper into Fetch, I've noticed that the
> > > code
> > > > > path
> > > > > > > runs into some public APIs that use the TopicPartition. Some
> > > examples
> > > > > are
> > > > > > > Cluster.java or RecordTooLargeException.
> > > > > > >
> > > > > > > A related issue is that in some places like
> > > > > TopicAuthorizationException,
> > > > > > > the topic name is required and without the name in the request,
> > we
> > > > > won't
> > > > > > > know it.
> > > > > > >
> > > > > > > I guess an alternative would be to have some mapping from topic
> > > name
> > > > to
> > > > > > > topic ID, but I would have to think about that more.
> > > > > > >
> > > > > > > The KIP mentions exposing TopicId in TopicDescription. (The
> font
> > > was
> > > > > > > weirdly colored, so I've made it easier to read.)
> > > > > > >
> > > > > > > Would the idea for the TopicMetadataAndConfig be that the topic
> > ID
> > > > > would be
> > > > > > > returned as a result?
> > > > > > > As for the other methods describe/deletetopics, it makes sense
> to
> > > > > include
> > > > > > > them at some point, but I'm not sure how far we should go in
> > terms
> > > of
> > > > > > > supporting topic IDs everywhere.
> > > > > > >
> > > > > > > Thank you for continuing to look at the KIP!
> > > > > > > Justine
> > > > > > >
> > > > > > >
> > > > > > > On Mon, Oct 5, 2020 at 2:53 PM Jun Rao <ju...@confluent.io>
> wrote:
> > > > > > >
> > > > > > > > Hi, Justine,
> > > > > > > >
> > > > > > > > Adding a TopicIdPartition in the public interface may be
> > useful.
> > > > > Where
> > > > > > > > would you expect it to be used?
> > > > > > > >
> > > > > > > > Related to this, since we are exposing topicId in the tool,
> it
> > > > seems
> > > > > that
> > > > > > > > we may want to expose topicId in at least TopicDescription
> used
> > > by
> > > > > > > > AdminClient.describeTopics() and TopicMetadataAndConfig used
> in
> > > > > > > > AdminClient.createTopics(Collection<NewTopic> newTopics). I
> am
> > > also
> > > > > not
> > > > > > > > sure if we plan to expose topicId in other AdminClient
> methods
> > in
> > > > > this
> > > > > > > KIP
> > > > > > > > or in some future KIP. For example, it might be useful to
> > support
> > > > > topicId
> > > > > > > > in the following methods.
> > > > > > > >
> > > > > > > >     DescribeTopicsResult describeTopics(Collection<String>
> > > > > topicNames,
> > > > > > > > DescribeTopicsOptions options);
> > > > > > > >     DeleteTopicsResult deleteTopics(Collection<String>
> topics,
> > > > > > > > DeleteTopicsOptions options);
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > >
> > > > > > > > Jun
> > > > > > > >
> > > > > > > > On Mon, Oct 5, 2020 at 11:03 AM Justine Olshan <
> > > > jolshan@confluent.io
> > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi all,
> > > > > > > > >
> > > > > > > > > I was thinking about this KIP some more. We are removing
> the
> > > > topic
> > > > > name
> > > > > > > > > from many of these request types, and some of them (eg.
> > Fetch)
> > > > use
> > > > > > > > > TopicPartition as keys in a map when handling the request.
> I
> > > was
> > > > > > > thinking
> > > > > > > > > it would make sense to make some sort of TopicIdPartition
> > with
> > > > > topic
> > > > > > > > name,
> > > > > > > > > topic id, and partition fields (name and id would
> potentially
> > > be
> > > > > > > optional
> > > > > > > > > types since different request versions will contain only
> one
> > of
> > > > the
> > > > > > > two)
> > > > > > > > I
> > > > > > > > > was wondering if it made sense to include this as a public
> > > facing
> > > > > API
> > > > > > > > > similar to TopicPartition. If so, I will add this class to
> > the
> > > > KIP
> > > > > as
> > > > > > > > well.
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > Justine
> > > > > > > > >
> > > > > > > > > On Mon, Oct 5, 2020 at 8:52 AM Justine Olshan <
> > > > > jolshan@confluent.io>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi all,
> > > > > > > > > > I've made a few minor changes to the UUID class. I've
> > added a
> > > > few
> > > > > > > more
> > > > > > > > > > methods to support UUIDs more generally, by including a
> > > > > constructor
> > > > > > > > and a
> > > > > > > > > > method to generate random UUIDs. Both of these will be
> > useful
> > > > for
> > > > > > > > > testing.
> > > > > > > > > >
> > > > > > > > > > Let me know if there are any other areas for discussion.
> > > > > > > > > >
> > > > > > > > > > Thank you,
> > > > > > > > > > Justine
> > > > > > > > > >
> > > > > > > > > > On Thu, Oct 1, 2020 at 5:29 PM Justine Olshan <
> > > > > jolshan@confluent.io>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > >> Hi Jun,
> > > > > > > > > >> Thanks for looking at it again. Here's the new spec. (I
> > > fixed
> > > > > the
> > > > > > > typo
> > > > > > > > > in
> > > > > > > > > >> it too.)
> > > > > > > > > >> {"name": "id", "type": "string", "doc": option[UUID]}
> > > > > > > > > >>
> > > > > > > > > >> Justine
> > > > > > > > > >>
> > > > > > > > > >>
> > > > > > > > > >> On Thu, Oct 1, 2020 at 5:03 PM Jun Rao <
> jun@confluent.io>
> > > > > wrote:
> > > > > > > > > >>
> > > > > > > > > >>> Hi, Justine,
> > > > > > > > > >>>
> > > > > > > > > >>> Thanks for the update. The KIP looks good to me now.
> > Just a
> > > > > minor
> > > > > > > > > comment
> > > > > > > > > >>> below.
> > > > > > > > > >>>
> > > > > > > > > >>> 30. Perhaps "option[UUID]" can be put in the doc.
> > > > > > > > > >>>
> > > > > > > > > >>> Jun
> > > > > > > > > >>>
> > > > > > > > > >>> On Thu, Oct 1, 2020 at 3:28 PM Justine Olshan <
> > > > > > > jolshan@confluent.io>
> > > > > > > > > >>> wrote:
> > > > > > > > > >>>
> > > > > > > > > >>> > Hi Jun,
> > > > > > > > > >>> > Thanks for the response!
> > > > > > > > > >>> >
> > > > > > > > > >>> > 30. I think I might have changed this in between. The
> > > > current
> > > > > > > > version
> > > > > > > > > >>> > says:  {"name":
> > > > > > > > > >>> > "id", "type": "option[UUID]"}, "doc": topic id}
> > > > > > > > > >>> > I have switched to the option type to cover the
> > migration
> > > > > case
> > > > > > > > where
> > > > > > > > > a
> > > > > > > > > >>> > TopicZNode does not yet have a topic ID.
> > > > > > > > > >>> > I understand that due to json, this field is written
> > as a
> > > > > string,
> > > > > > > > so
> > > > > > > > > >>> if I
> > > > > > > > > >>> > should move the "option[uuid]" to the doc field and
> the
> > > > type
> > > > > > > should
> > > > > > > > > be
> > > > > > > > > >>> > "string" please let me know.
> > > > > > > > > >>> >
> > > > > > > > > >>> > 40. I've added a definition for UUID.
> > > > > > > > > >>> > 41,42. Fixed
> > > > > > > > > >>> >
> > > > > > > > > >>> > Thank you,
> > > > > > > > > >>> > Justine
> > > > > > > > > >>> >
> > > > > > > > > >>> > On Wed, Sep 30, 2020 at 1:15 PM Jun Rao <
> > > jun@confluent.io>
> > > > > > > wrote:
> > > > > > > > > >>> >
> > > > > > > > > >>> > > Hi, Justine,
> > > > > > > > > >>> > >
> > > > > > > > > >>> > > Thanks for the summary. Just a few minor comments
> > blow.
> > > > > > > > > >>> > >
> > > > > > > > > >>> > > 30.  {"name": "id", "type": "string", "doc":
> "version
> > > > > id"}}:
> > > > > > > The
> > > > > > > > > doc
> > > > > > > > > >>> > should
> > > > > > > > > >>> > > say UUID. The issue seems unfixed.
> > > > > > > > > >>> > >
> > > > > > > > > >>> > > 40. Since UUID is public facing, could you include
> > its
> > > > > > > > definition?
> > > > > > > > > >>> > >
> > > > > > > > > >>> > > 41. StopReplicaResponse still includes the topic
> > field.
> > > > > > > > > >>> > >
> > > > > > > > > >>> > > 42. "It is unnecessary to include the name of the
> > topic
> > > > in
> > > > > the
> > > > > > > > > >>> following
> > > > > > > > > >>> > > Request/Response calls" It would be useful to
> include
> > > all
> > > > > > > > modified
> > > > > > > > > >>> > requests
> > > > > > > > > >>> > > (e.g. produce) in the list below.
> > > > > > > > > >>> > >
> > > > > > > > > >>> > > Thanks,
> > > > > > > > > >>> > >
> > > > > > > > > >>> > > Jun
> > > > > > > > > >>> > >
> > > > > > > > > >>> > >
> > > > > > > > > >>> > > On Wed, Sep 30, 2020 at 10:17 AM Justine Olshan <
> > > > > > > > > >>> jolshan@confluent.io>
> > > > > > > > > >>> > > wrote:
> > > > > > > > > >>> > >
> > > > > > > > > >>> > > > Hello again,
> > > > > > > > > >>> > > >
> > > > > > > > > >>> > > > I've taken some time to discuss some of the
> > remaining
> > > > > points
> > > > > > > > > >>> brought up
> > > > > > > > > >>> > > by
> > > > > > > > > >>> > > > the previous emails offline. Here are some of the
> > > > > > > conclusions.
> > > > > > > > > >>> > > >
> > > > > > > > > >>> > > > 1. Directory Structure:
> > > > > > > > > >>> > > > There was some talk about whether the directory
> > > > structure
> > > > > > > > should
> > > > > > > > > be
> > > > > > > > > >>> > > changed
> > > > > > > > > >>> > > > to replace all topic names with topic IDs.
> > > > > > > > > >>> > > > This will be a useful change, but will prevent
> > > > > downgrades. It
> > > > > > > > > will
> > > > > > > > > >>> be
> > > > > > > > > >>> > > best
> > > > > > > > > >>> > > > to wait until a major release, likely alongside
> > > KIP-500
> > > > > > > changes
> > > > > > > > > >>> that
> > > > > > > > > >>> > will
> > > > > > > > > >>> > > > prevent downgrades. I've updated the KIP to
> include
> > > > this
> > > > > > > change
> > > > > > > > > >>> with
> > > > > > > > > >>> > the
> > > > > > > > > >>> > > > note about migration and deprecation.
> > > > > > > > > >>> > > >
> > > > > > > > > >>> > > >
> > > > > > > > > >>> > >
> > > > > > > > > >>> >
> > > > > > > > > >>>
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-log.dirlayout
> > > > > > > > > >>> > > >
> > > > > > > > > >>> > > > 2. Partition Metadata file
> > > > > > > > > >>> > > > There was some disagreement about the inclusion
> of
> > > the
> > > > > > > > partition
> > > > > > > > > >>> > metadata
> > > > > > > > > >>> > > > file.
> > > > > > > > > >>> > > > This file will be needed to persist the topic ID,
> > > > > especially
> > > > > > > > > while
> > > > > > > > > >>> we
> > > > > > > > > >>> > > still
> > > > > > > > > >>> > > > have the old directory structure. Even after the
> > > > changes,
> > > > > > > this
> > > > > > > > > >>> file can
> > > > > > > > > >>> > > be
> > > > > > > > > >>> > > > useful for debugging and recovery.
> > > > > > > > > >>> > > > Creating a single mapping file from topic names
> to
> > > > topic
> > > > > IDs
> > > > > > > > was
> > > > > > > > > >>> > > > considered, but ultimately scrapped as it would
> not
> > > be
> > > > as
> > > > > > > easy
> > > > > > > > to
> > > > > > > > > >>> > > maintain.
> > > > > > > > > >>> > > >
> > > > > > > > > >>> > > >
> > > > > > > > > >>> > >
> > > > > > > > > >>> >
> > > > > > > > > >>>
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PartitionMetadatafile
> > > > > > > > > >>> > > >
> > > > > > > > > >>> > > >
> > > > > > > > > >>> > > >
> > > > > > > > > >>> > >
> > > > > > > > > >>> >
> > > > > > > > > >>>
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PersistingTopicIDs
> > > > > > > > > >>> > > >
> > > > > > > > > >>> > > > 3. Produce Protocols
> > > > > > > > > >>> > > > After some further discussion, this replacing the
> > > topic
> > > > > name
> > > > > > > > with
> > > > > > > > > >>> topic
> > > > > > > > > >>> > > ID
> > > > > > > > > >>> > > > in these protocols has been added to the KIP.
> > > > > > > > > >>> > > > This will cut down on the size of the protocol.
> > Since
> > > > > changes
> > > > > > > > to
> > > > > > > > > >>> fetch
> > > > > > > > > >>> > > are
> > > > > > > > > >>> > > > included, it does make sense to update these
> > > protocols.
> > > > > > > > > >>> > > >
> > > > > > > > > >>> > > >
> > > > > > > > > >>> > >
> > > > > > > > > >>> >
> > > > > > > > > >>>
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-Produce
> > > > > > > > > >>> > > >
> > > > > > > > > >>> > > > 4. KIP-500 Compatibility
> > > > > > > > > >>> > > > After some discussion with those involved with
> > > KIP-500,
> > > > > it
> > > > > > > > seems
> > > > > > > > > >>> best
> > > > > > > > > >>> > to
> > > > > > > > > >>> > > > use a sentinel topic ID for the metadata topic
> that
> > > is
> > > > > used
> > > > > > > > > before
> > > > > > > > > >>> the
> > > > > > > > > >>> > > > first controller election.
> > > > > > > > > >>> > > > We had an issue where this metadata topic may not
> > be
> > > > > assigned
> > > > > > > > an
> > > > > > > > > ID
> > > > > > > > > >>> > > before
> > > > > > > > > >>> > > > utilizing the Vote and Fetch protocols. It was
> > > decided
> > > > to
> > > > > > > > > reserve a
> > > > > > > > > >>> > > unique
> > > > > > > > > >>> > > > ID for this topic to be used until the controller
> > > could
> > > > > give
> > > > > > > > the
> > > > > > > > > >>> topic
> > > > > > > > > >>> > a
> > > > > > > > > >>> > > > unique ID.
> > > > > > > > > >>> > > > Having the topic keep the sentinel ID (not be
> > > > reassigned
> > > > > to a
> > > > > > > > > >>> unique
> > > > > > > > > >>> > ID)
> > > > > > > > > >>> > > > was considered, but it seemed like a good idea to
> > > leave
> > > > > the
> > > > > > > > > option
> > > > > > > > > >>> open
> > > > > > > > > >>> > > for
> > > > > > > > > >>> > > > the metadata topic to have a unique ID in cases
> > where
> > > > it
> > > > > > > would
> > > > > > > > > >>> need to
> > > > > > > > > >>> > be
> > > > > > > > > >>> > > > differentiated from other clusters' metadata
> > topics.
> > > > (ex.
> > > > > > > > tiered
> > > > > > > > > >>> > storage)
> > > > > > > > > >>> > > >
> > > > > > > > > >>> > > >
> > > > > > > > > >>> > >
> > > > > > > > > >>> >
> > > > > > > > > >>>
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-CompatibilitywithKIP-500
> > > > > > > > > >>> > > >
> > > > > > > > > >>> > > > I've also split up the KIP into sub-tasks on the
> > > JIRA.
> > > > > > > > Hopefully
> > > > > > > > > >>> this
> > > > > > > > > >>> > > will
> > > > > > > > > >>> > > > provide a better idea about what tasks we have,
> and
> > > > > > > eventually
> > > > > > > > > >>> provide
> > > > > > > > > >>> > a
> > > > > > > > > >>> > > > place to see what's done and what is left.
> > > > > > > > > >>> > > > If there is a task I am missing, please let me
> > know!
> > > > > > > > > >>> > > > https://issues.apache.org/jira/browse/KAFKA-8872
> > > > > > > > > >>> > > >
> > > > > > > > > >>> > > > Of course, these decisions are not set in stone,
> > and
> > > I
> > > > > would
> > > > > > > > love
> > > > > > > > > >>> to
> > > > > > > > > >>> > hear
> > > > > > > > > >>> > > > any feedback.
> > > > > > > > > >>> > > >
> > > > > > > > > >>> > > > Thanks,
> > > > > > > > > >>> > > > Justine
> > > > > > > > > >>> > > >
> > > > > > > > > >>> > > > On Mon, Sep 28, 2020 at 11:38 AM Justine Olshan <
> > > > > > > > > >>> jolshan@confluent.io>
> > > > > > > > > >>> > > > wrote:
> > > > > > > > > >>> > > >
> > > > > > > > > >>> > > > > Hello all,
> > > > > > > > > >>> > > > >
> > > > > > > > > >>> > > > > I just wanted to follow up on this discussion.
> > Did
> > > we
> > > > > come
> > > > > > > to
> > > > > > > > > an
> > > > > > > > > >>> > > > > understanding about the directory structure?
> > > > > > > > > >>> > > > >
> > > > > > > > > >>> > > > > I think the biggest question here is what is
> > > > > acceptable to
> > > > > > > > > leave
> > > > > > > > > >>> out
> > > > > > > > > >>> > > due
> > > > > > > > > >>> > > > > to scope vs. what is considered to be too much
> > tech
> > > > > debt.
> > > > > > > > > >>> > > > > This KIP is already pretty large with the
> number
> > of
> > > > > > > changes,
> > > > > > > > > but
> > > > > > > > > >>> it
> > > > > > > > > >>> > > also
> > > > > > > > > >>> > > > > makes sense to do things correctly, so I'd love
> > to
> > > > hear
> > > > > > > > > >>> everyone's
> > > > > > > > > >>> > > > thoughts.
> > > > > > > > > >>> > > > >
> > > > > > > > > >>> > > > > Thanks,
> > > > > > > > > >>> > > > > Justine
> > > > > > > > > >>> > > > >
> > > > > > > > > >>> > > > > On Fri, Sep 25, 2020 at 8:19 AM Lucas
> Bradstreet
> > <
> > > > > > > > > >>> lucas@confluent.io
> > > > > > > > > >>> > >
> > > > > > > > > >>> > > > > wrote:
> > > > > > > > > >>> > > > >
> > > > > > > > > >>> > > > >> Hi Ismael,
> > > > > > > > > >>> > > > >>
> > > > > > > > > >>> > > > >> If you do not store it in a metadata file or
> in
> > > the
> > > > > > > > directory
> > > > > > > > > >>> > > structure
> > > > > > > > > >>> > > > >> would you then
> > > > > > > > > >>> > > > >> require the LeaderAndIsrRequest following
> > startup
> > > to
> > > > > give
> > > > > > > > you
> > > > > > > > > >>> some
> > > > > > > > > >>> > > > notion
> > > > > > > > > >>> > > > >> of
> > > > > > > > > >>> > > > >> topic name in memory? We would still need this
> > > > > information
> > > > > > > > for
> > > > > > > > > >>> the
> > > > > > > > > >>> > > older
> > > > > > > > > >>> > > > >> protocols, but
> > > > > > > > > >>> > > > >> perhaps this is what's meant by tech debt.
> > > > > > > > > >>> > > > >>
> > > > > > > > > >>> > > > >> Once we're free of the old non-topicID
> requests
> > > > then I
> > > > > > > think
> > > > > > > > > you
> > > > > > > > > >>> > > > wouldn't
> > > > > > > > > >>> > > > >> need to retain the topic name.
> > > > > > > > > >>> > > > >> I think the ability to easily look up topic
> > names
> > > > > > > associated
> > > > > > > > > >>> with
> > > > > > > > > >>> > > > >> partition
> > > > > > > > > >>> > > > >> directories would still be missed
> > > > > > > > > >>> > > > >> when diagnosing issues, though maybe it
> wouldn't
> > > be
> > > > a
> > > > > deal
> > > > > > > > > >>> breaker
> > > > > > > > > >>> > > with
> > > > > > > > > >>> > > > >> the
> > > > > > > > > >>> > > > >> right tooling.
> > > > > > > > > >>> > > > >>
> > > > > > > > > >>> > > > >> Thanks,
> > > > > > > > > >>> > > > >>
> > > > > > > > > >>> > > > >> Lucas
> > > > > > > > > >>> > > > >>
> > > > > > > > > >>> > > > >> On Fri, Sep 25, 2020 at 7:55 AM Ismael Juma <
> > > > > > > > > ismael@juma.me.uk>
> > > > > > > > > >>> > > wrote:
> > > > > > > > > >>> > > > >>
> > > > > > > > > >>> > > > >> > Hi Lucas,
> > > > > > > > > >>> > > > >> >
> > > > > > > > > >>> > > > >> > Why would you include the name and id? I
> think
> > > > you'd
> > > > > > > want
> > > > > > > > to
> > > > > > > > > >>> > remove
> > > > > > > > > >>> > > > the
> > > > > > > > > >>> > > > >> > name from the directory name right? Jason's
> > > > > suggestion
> > > > > > > was
> > > > > > > > > >>> that if
> > > > > > > > > >>> > > you
> > > > > > > > > >>> > > > >> > remove the name from the directory, then why
> > > would
> > > > > you
> > > > > > > > need
> > > > > > > > > >>> the id
> > > > > > > > > >>> > > > name
> > > > > > > > > >>> > > > >> > mapping file?
> > > > > > > > > >>> > > > >> >
> > > > > > > > > >>> > > > >> > Ismael
> > > > > > > > > >>> > > > >> >
> > > > > > > > > >>> > > > >> > On Thu, Sep 24, 2020 at 4:24 PM Lucas
> > > Bradstreet <
> > > > > > > > > >>> > > lucas@confluent.io>
> > > > > > > > > >>> > > > >> > wrote:
> > > > > > > > > >>> > > > >> >
> > > > > > > > > >>> > > > >> > > > 2. Part of the usage of the file is to
> > have
> > > > > > > persistent
> > > > > > > > > >>> storage
> > > > > > > > > >>> > > of
> > > > > > > > > >>> > > > >> the
> > > > > > > > > >>> > > > >> > > topic
> > > > > > > > > >>> > > > >> > > ID and use it to compare with the ID
> > supplied
> > > in
> > > > > the
> > > > > > > > > >>> > LeaderAndIsr
> > > > > > > > > >>> > > > >> > Request.
> > > > > > > > > >>> > > > >> > > There is some discussion in the KIP about
> > > > changes
> > > > > to
> > > > > > > the
> > > > > > > > > >>> > directory
> > > > > > > > > >>> > > > >> > > structure, but I believe directory changes
> > > were
> > > > > > > > considered
> > > > > > > > > >>> to be
> > > > > > > > > >>> > > out
> > > > > > > > > >>> > > > >> of
> > > > > > > > > >>> > > > >> > > scope when the KIP was written.
> > > > > > > > > >>> > > > >> > >
> > > > > > > > > >>> > > > >> > >
> > > > > > > > > >>> > > > >> > > Yeah, I was hoping to get a better
> > > understanding
> > > > > of
> > > > > > > why
> > > > > > > > it
> > > > > > > > > >>> was
> > > > > > > > > >>> > > taken
> > > > > > > > > >>> > > > >> out
> > > > > > > > > >>> > > > >> > of
> > > > > > > > > >>> > > > >> > > scope. Perhaps Lucas Bradstreet might have
> > > more
> > > > > > > insight
> > > > > > > > > >>> about
> > > > > > > > > >>> > the
> > > > > > > > > >>> > > > >> > decision.
> > > > > > > > > >>> > > > >> > > Basically my point is that we have to
> create
> > > > > > > additional
> > > > > > > > > >>> > > > infrastructure
> > > > > > > > > >>> > > > >> > here
> > > > > > > > > >>> > > > >> > > to support the name/id mapping, so I
> wanted
> > to
> > > > > > > > understand
> > > > > > > > > >>> if we
> > > > > > > > > >>> > > just
> > > > > > > > > >>> > > > >> > > consider this a sort of tech debt. It
> would
> > be
> > > > > useful
> > > > > > > to
> > > > > > > > > >>> cover
> > > > > > > > > >>> > how
> > > > > > > > > >>> > > > we
> > > > > > > > > >>> > > > >> > > handle the case when this file gets
> > corrupted.
> > > > > Seems
> > > > > > > > like
> > > > > > > > > we
> > > > > > > > > >>> > just
> > > > > > > > > >>> > > > >> have to
> > > > > > > > > >>> > > > >> > > trust that it matches whatever the
> > controller
> > > > > tells us
> > > > > > > > and
> > > > > > > > > >>> > rewrite
> > > > > > > > > >>> > > > it?
> > > > > > > > > >>> > > > >> > >
> > > > > > > > > >>> > > > >> > >
> > > > > > > > > >>> > > > >> > > Hi Jason, Justine,
> > > > > > > > > >>> > > > >> > >
> > > > > > > > > >>> > > > >> > > My thought process is that we will likely
> > need
> > > > the
> > > > > > > > > metadata
> > > > > > > > > >>> file
> > > > > > > > > >>> > > > >> whether
> > > > > > > > > >>> > > > >> > we
> > > > > > > > > >>> > > > >> > > rename the directories or not.
> > > > > > > > > >>> > > > >> > > Linux supports filenames of up to 255
> bytes
> > > and
> > > > > that
> > > > > > > > would
> > > > > > > > > >>> not
> > > > > > > > > >>> > be
> > > > > > > > > >>> > > > >> enough
> > > > > > > > > >>> > > > >> > to
> > > > > > > > > >>> > > > >> > > support a directory name
> > > > > > > > > >>> > > > >> > >  including both the name and topic ID.
> Given
> > > > that
> > > > > > > fact,
> > > > > > > > it
> > > > > > > > > >>> > seemed
> > > > > > > > > >>> > > > >> > > reasonable to add the metadata file
> > > > > > > > > >>> > > > >> > > and leave the directory rename until some
> > time
> > > > in
> > > > > the
> > > > > > > > > future
> > > > > > > > > >>> > > > (possibly
> > > > > > > > > >>> > > > >> > > never).
> > > > > > > > > >>> > > > >> > >
> > > > > > > > > >>> > > > >> > > If the file does get corrupted I think
> > you're
> > > > > right.
> > > > > > > We
> > > > > > > > > >>> would
> > > > > > > > > >>> > > either
> > > > > > > > > >>> > > > >> have
> > > > > > > > > >>> > > > >> > > to trust it matches what the controller
> > tells
> > > us
> > > > > > > > > >>> > > > >> > >  or error out and let an administrator
> > resolve
> > > > it
> > > > > by
> > > > > > > > > >>> checking
> > > > > > > > > >>> > > across
> > > > > > > > > >>> > > > >> > > replicas for consistency.
> > > > > > > > > >>> > > > >> > >
> > > > > > > > > >>> > > > >> > > Lucas
> > > > > > > > > >>> > > > >> > >
> > > > > > > > > >>> > > > >> > >
> > > > > > > > > >>> > > > >> > > On Thu, Sep 24, 2020 at 3:41 PM Jason
> > > Gustafson
> > > > <
> > > > > > > > > >>> > > jason@confluent.io
> > > > > > > > > >>> > > > >
> > > > > > > > > >>> > > > >> > > wrote:
> > > > > > > > > >>> > > > >> > >
> > > > > > > > > >>> > > > >> > > > Thanks Justine. Responses below:
> > > > > > > > > >>> > > > >> > > >
> > > > > > > > > >>> > > > >> > > > > 1. Yes, the directory will still be
> > based
> > > on
> > > > > the
> > > > > > > > topic
> > > > > > > > > >>> > names.
> > > > > > > > > >>> > > > >> > > > LeaderAndIsrRequest is one of the few
> > > requests
> > > > > that
> > > > > > > > will
> > > > > > > > > >>> still
> > > > > > > > > >>> > > > >> contain
> > > > > > > > > >>> > > > >> > > the
> > > > > > > > > >>> > > > >> > > > topic name. So I think we have this
> > covered.
> > > > > Sorry
> > > > > > > for
> > > > > > > > > >>> > > confusion.
> > > > > > > > > >>> > > > >> > > >
> > > > > > > > > >>> > > > >> > > > Ah, you're right. My eyes passed right
> > over
> > > > the
> > > > > > > field.
> > > > > > > > > >>> > > > >> > > >
> > > > > > > > > >>> > > > >> > > > > 2. Part of the usage of the file is to
> > > have
> > > > > > > > persistent
> > > > > > > > > >>> > storage
> > > > > > > > > >>> > > > of
> > > > > > > > > >>> > > > >> the
> > > > > > > > > >>> > > > >> > > > topic
> > > > > > > > > >>> > > > >> > > > ID and use it to compare with the ID
> > > supplied
> > > > > in the
> > > > > > > > > >>> > > LeaderAndIsr
> > > > > > > > > >>> > > > >> > > Request.
> > > > > > > > > >>> > > > >> > > > There is some discussion in the KIP
> about
> > > > > changes to
> > > > > > > > the
> > > > > > > > > >>> > > directory
> > > > > > > > > >>> > > > >> > > > structure, but I believe directory
> changes
> > > > were
> > > > > > > > > >>> considered to
> > > > > > > > > >>> > be
> > > > > > > > > >>> > > > >> out of
> > > > > > > > > >>> > > > >> > > > scope when the KIP was written.
> > > > > > > > > >>> > > > >> > > >
> > > > > > > > > >>> > > > >> > > > Yeah, I was hoping to get a better
> > > > > understanding of
> > > > > > > > why
> > > > > > > > > >>> it was
> > > > > > > > > >>> > > > taken
> > > > > > > > > >>> > > > >> > out
> > > > > > > > > >>> > > > >> > > of
> > > > > > > > > >>> > > > >> > > > scope. Perhaps Lucas Bradstreet might
> have
> > > > more
> > > > > > > > insight
> > > > > > > > > >>> about
> > > > > > > > > >>> > > the
> > > > > > > > > >>> > > > >> > > decision.
> > > > > > > > > >>> > > > >> > > > Basically my point is that we have to
> > create
> > > > > > > > additional
> > > > > > > > > >>> > > > >> infrastructure
> > > > > > > > > >>> > > > >> > > here
> > > > > > > > > >>> > > > >> > > > to support the name/id mapping, so I
> > wanted
> > > to
> > > > > > > > > understand
> > > > > > > > > >>> if
> > > > > > > > > >>> > we
> > > > > > > > > >>> > > > just
> > > > > > > > > >>> > > > >> > > > consider this a sort of tech debt. It
> > would
> > > be
> > > > > > > useful
> > > > > > > > to
> > > > > > > > > >>> cover
> > > > > > > > > >>> > > how
> > > > > > > > > >>> > > > >> we
> > > > > > > > > >>> > > > >> > > > handle the case when this file gets
> > > corrupted.
> > > > > Seems
> > > > > > > > > like
> > > > > > > > > >>> we
> > > > > > > > > >>> > > just
> > > > > > > > > >>> > > > >> have
> > > > > > > > > >>> > > > >> > to
> > > > > > > > > >>> > > > >> > > > trust that it matches whatever the
> > > controller
> > > > > tells
> > > > > > > us
> > > > > > > > > and
> > > > > > > > > >>> > > rewrite
> > > > > > > > > >>> > > > >> it?
> > > > > > > > > >>> > > > >> > > >
> > > > > > > > > >>> > > > >> > > > > 3. I think this is a good point, but I
> > > > again I
> > > > > > > > wonder
> > > > > > > > > >>> about
> > > > > > > > > >>> > > the
> > > > > > > > > >>> > > > >> scope
> > > > > > > > > >>> > > > >> > > of
> > > > > > > > > >>> > > > >> > > > the KIP. Most of the changes mentioned
> in
> > > the
> > > > > KIP
> > > > > > > are
> > > > > > > > > for
> > > > > > > > > >>> > > > supporting
> > > > > > > > > >>> > > > >> > > topic
> > > > > > > > > >>> > > > >> > > > deletion and I believe that is why the
> > > produce
> > > > > > > request
> > > > > > > > > was
> > > > > > > > > >>> > > listed
> > > > > > > > > >>> > > > >> under
> > > > > > > > > >>> > > > >> > > > future work.
> > > > > > > > > >>> > > > >> > > >
> > > > > > > > > >>> > > > >> > > > That's fair. I brought it up since
> `Fetch`
> > > is
> > > > > > > already
> > > > > > > > > >>> > included.
> > > > > > > > > >>> > > If
> > > > > > > > > >>> > > > >> > we've
> > > > > > > > > >>> > > > >> > > > got `Metadata` and `Fetch`, seems we may
> > as
> > > > > well do
> > > > > > > > > >>> `Produce`
> > > > > > > > > >>> > > and
> > > > > > > > > >>> > > > >> save
> > > > > > > > > >>> > > > >> > an
> > > > > > > > > >>> > > > >> > > > extra kip. No strong objection though if
> > you
> > > > > want to
> > > > > > > > > >>> leave it
> > > > > > > > > >>> > > out.
> > > > > > > > > >>> > > > >> > > >
> > > > > > > > > >>> > > > >> > > >
> > > > > > > > > >>> > > > >> > > > -Jason
> > > > > > > > > >>> > > > >> > > >
> > > > > > > > > >>> > > > >> > > >
> > > > > > > > > >>> > > > >> > > > On Thu, Sep 24, 2020 at 3:26 PM Justine
> > > > Olshan <
> > > > > > > > > >>> > > > >> jolshan@confluent.io>
> > > > > > > > > >>> > > > >> > > > wrote:
> > > > > > > > > >>> > > > >> > > >
> > > > > > > > > >>> > > > >> > > > > Hi Jason,
> > > > > > > > > >>> > > > >> > > > >
> > > > > > > > > >>> > > > >> > > > > Thanks for your comments.
> > > > > > > > > >>> > > > >> > > > >
> > > > > > > > > >>> > > > >> > > > > 1. Yes, the directory will still be
> > based
> > > on
> > > > > the
> > > > > > > > topic
> > > > > > > > > >>> > names.
> > > > > > > > > >>> > > > >> > > > > LeaderAndIsrRequest is one of the few
> > > > requests
> > > > > > > that
> > > > > > > > > will
> > > > > > > > > >>> > still
> > > > > > > > > >>> > > > >> > contain
> > > > > > > > > >>> > > > >> > > > the
> > > > > > > > > >>> > > > >> > > > > topic name. So I think we have this
> > > covered.
> > > > > Sorry
> > > > > > > > for
> > > > > > > > > >>> > > > confusion.
> > > > > > > > > >>> > > > >> > > > >
> > > > > > > > > >>> > > > >> > > > > 2. Part of the usage of the file is to
> > > have
> > > > > > > > persistent
> > > > > > > > > >>> > storage
> > > > > > > > > >>> > > > of
> > > > > > > > > >>> > > > >> the
> > > > > > > > > >>> > > > >> > > > topic
> > > > > > > > > >>> > > > >> > > > > ID and use it to compare with the ID
> > > > supplied
> > > > > in
> > > > > > > the
> > > > > > > > > >>> > > > LeaderAndIsr
> > > > > > > > > >>> > > > >> > > > Request.
> > > > > > > > > >>> > > > >> > > > > There is some discussion in the KIP
> > about
> > > > > changes
> > > > > > > to
> > > > > > > > > the
> > > > > > > > > >>> > > > directory
> > > > > > > > > >>> > > > >> > > > > structure, but I believe directory
> > changes
> > > > > were
> > > > > > > > > >>> considered
> > > > > > > > > >>> > to
> > > > > > > > > >>> > > be
> > > > > > > > > >>> > > > >> out
> > > > > > > > > >>> > > > >> > of
> > > > > > > > > >>> > > > >> > > > > scope when the KIP was written.
> > > > > > > > > >>> > > > >> > > > >
> > > > > > > > > >>> > > > >> > > > > 3. I think this is a good point, but I
> > > > again I
> > > > > > > > wonder
> > > > > > > > > >>> about
> > > > > > > > > >>> > > the
> > > > > > > > > >>> > > > >> scope
> > > > > > > > > >>> > > > >> > > of
> > > > > > > > > >>> > > > >> > > > > the KIP. Most of the changes mentioned
> > in
> > > > the
> > > > > KIP
> > > > > > > > are
> > > > > > > > > >>> for
> > > > > > > > > >>> > > > >> supporting
> > > > > > > > > >>> > > > >> > > > topic
> > > > > > > > > >>> > > > >> > > > > deletion and I believe that is why the
> > > > produce
> > > > > > > > request
> > > > > > > > > >>> was
> > > > > > > > > >>> > > > listed
> > > > > > > > > >>> > > > >> > under
> > > > > > > > > >>> > > > >> > > > > future work.
> > > > > > > > > >>> > > > >> > > > >
> > > > > > > > > >>> > > > >> > > > > 4. This sounds like it might be a good
> > > > > solution,
> > > > > > > > but I
> > > > > > > > > >>> will
> > > > > > > > > >>> > > need
> > > > > > > > > >>> > > > >> to
> > > > > > > > > >>> > > > >> > > > discuss
> > > > > > > > > >>> > > > >> > > > > more with KIP-500 folks to get the
> > details
> > > > > right.
> > > > > > > > > >>> > > > >> > > > >
> > > > > > > > > >>> > > > >> > > > > Thanks,
> > > > > > > > > >>> > > > >> > > > > Justine
> > > > > > > > > >>> > > > >> > > > >
> > > > > > > > > >>> > > > >> > > > > On Thu, Sep 24, 2020 at 12:30 PM Jason
> > > > > Gustafson <
> > > > > > > > > >>> > > > >> jason@confluent.io
> > > > > > > > > >>> > > > >> > >
> > > > > > > > > >>> > > > >> > > > > wrote:
> > > > > > > > > >>> > > > >> > > > >
> > > > > > > > > >>> > > > >> > > > > > Hi Justine,
> > > > > > > > > >>> > > > >> > > > > >
> > > > > > > > > >>> > > > >> > > > > > Thanks for picking up this work. I
> > have
> > > a
> > > > > few
> > > > > > > > > >>> > > > >> questions/comments:
> > > > > > > > > >>> > > > >> > > > > >
> > > > > > > > > >>> > > > >> > > > > > 1. It sounds like the directory
> > > structure
> > > > is
> > > > > > > still
> > > > > > > > > >>> going
> > > > > > > > > >>> > to
> > > > > > > > > >>> > > be
> > > > > > > > > >>> > > > >> > based
> > > > > > > > > >>> > > > >> > > on
> > > > > > > > > >>> > > > >> > > > > > topic names. Do I have that right?
> One
> > > > > > > > complication
> > > > > > > > > is
> > > > > > > > > >>> > that
> > > > > > > > > >>> > > > the
> > > > > > > > > >>> > > > >> > > > > > LeaderAndIsr request does not
> include
> > > the
> > > > > topic
> > > > > > > > name
> > > > > > > > > >>> any
> > > > > > > > > >>> > > > longer.
> > > > > > > > > >>> > > > >> > This
> > > > > > > > > >>> > > > >> > > > > means
> > > > > > > > > >>> > > > >> > > > > > that a replica must wait for the
> > > > > UpdateMetadata
> > > > > > > > > >>> request to
> > > > > > > > > >>> > > > >> arrive
> > > > > > > > > >>> > > > >> > > with
> > > > > > > > > >>> > > > >> > > > > the
> > > > > > > > > >>> > > > >> > > > > > topic name mapping before it can
> > create
> > > > the
> > > > > > > > > >>> directory. I
> > > > > > > > > >>> > > > wonder
> > > > > > > > > >>> > > > >> if
> > > > > > > > > >>> > > > >> > it
> > > > > > > > > >>> > > > >> > > > > would
> > > > > > > > > >>> > > > >> > > > > > be simpler to avoid assumptions on
> the
> > > > > ordering
> > > > > > > of
> > > > > > > > > >>> > > > >> UpdateMetadata
> > > > > > > > > >>> > > > >> > and
> > > > > > > > > >>> > > > >> > > > let
> > > > > > > > > >>> > > > >> > > > > > LeaderAndIsr include the topic name
> as
> > > > well.
> > > > > > > Feels
> > > > > > > > > >>> like we
> > > > > > > > > >>> > > are
> > > > > > > > > >>> > > > >> not
> > > > > > > > > >>> > > > >> > > > saving
> > > > > > > > > >>> > > > >> > > > > > that much by excluding it.
> > > > > > > > > >>> > > > >> > > > > >
> > > > > > > > > >>> > > > >> > > > > > 2. On a related note, it seems that
> > the
> > > > > reason
> > > > > > > we
> > > > > > > > > >>> have the
> > > > > > > > > >>> > > > >> > partition
> > > > > > > > > >>> > > > >> > > > > > metadata file is because we are not
> > > > > changing the
> > > > > > > > > >>> directory
> > > > > > > > > >>> > > > >> > structure.
> > > > > > > > > >>> > > > >> > > > We
> > > > > > > > > >>> > > > >> > > > > > need it so that we remember which
> > > > > directories
> > > > > > > map
> > > > > > > > to
> > > > > > > > > >>> which
> > > > > > > > > >>> > > > topic
> > > > > > > > > >>> > > > >> > id.
> > > > > > > > > >>> > > > >> > > I
> > > > > > > > > >>> > > > >> > > > > > think it would be useful to add some
> > > > detail
> > > > > > > about
> > > > > > > > > why
> > > > > > > > > >>> > > changing
> > > > > > > > > >>> > > > >> the
> > > > > > > > > >>> > > > >> > > > > > directory layout was rejected.
> > Basically
> > > > > trying
> > > > > > > to
> > > > > > > > > >>> > > understand
> > > > > > > > > >>> > > > >> how
> > > > > > > > > >>> > > > >> > > much
> > > > > > > > > >>> > > > >> > > > > > effort we are saving by skipping
> this
> > > step
> > > > > if we
> > > > > > > > > have
> > > > > > > > > >>> to
> > > > > > > > > >>> > > > >> implement
> > > > > > > > > >>> > > > >> > > this
> > > > > > > > > >>> > > > >> > > > > > additional bookkeeping. One concern
> I
> > > have
> > > > > for
> > > > > > > > > >>> example is
> > > > > > > > > >>> > > that
> > > > > > > > > >>> > > > >> the
> > > > > > > > > >>> > > > >> > > > > > partition metadata file gets corrupt
> > and
> > > > > then
> > > > > > > all
> > > > > > > > > >>> bets are
> > > > > > > > > >>> > > off
> > > > > > > > > >>> > > > >> as
> > > > > > > > > >>> > > > >> > far
> > > > > > > > > >>> > > > >> > > > as
> > > > > > > > > >>> > > > >> > > > > > topic consistency.
> > > > > > > > > >>> > > > >> > > > > >
> > > > > > > > > >>> > > > >> > > > > > 3. Is it worth adding support now
> for
> > > > topic
> > > > > ids
> > > > > > > in
> > > > > > > > > the
> > > > > > > > > >>> > > > `Produce`
> > > > > > > > > >>> > > > >> > > > request
> > > > > > > > > >>> > > > >> > > > > > now? Overhead is mentioned as one of
> > the
> > > > > > > > motivations
> > > > > > > > > >>> and
> > > > > > > > > >>> > the
> > > > > > > > > >>> > > > >> best
> > > > > > > > > >>> > > > >> > > APIs
> > > > > > > > > >>> > > > >> > > > to
> > > > > > > > > >>> > > > >> > > > > > get savings from are `Produce` and
> > > > `Fetch`.
> > > > > > > > > >>> > > > >> > > > > >
> > > > > > > > > >>> > > > >> > > > > > 4. When it comes to bootstrapping
> the
> > > > > metadata
> > > > > > > > topic
> > > > > > > > > >>> with
> > > > > > > > > >>> > > > >> respect
> > > > > > > > > >>> > > > >> > to
> > > > > > > > > >>> > > > >> > > > > > KIP-500, one suggestion would be to
> > > > reserve
> > > > > a
> > > > > > > > > sentinel
> > > > > > > > > >>> > topic
> > > > > > > > > >>> > > > ID
> > > > > > > > > >>> > > > >> > which
> > > > > > > > > >>> > > > >> > > > can
> > > > > > > > > >>> > > > >> > > > > > be used initially by new brokers.
> When
> > > the
> > > > > first
> > > > > > > > > >>> > controller
> > > > > > > > > >>> > > is
> > > > > > > > > >>> > > > >> > > elected,
> > > > > > > > > >>> > > > >> > > > > it
> > > > > > > > > >>> > > > >> > > > > > can assign a topicId to the metadata
> > > > topic.
> > > > > This
> > > > > > > > is
> > > > > > > > > a
> > > > > > > > > >>> bit
> > > > > > > > > >>> > > > >> similar
> > > > > > > > > >>> > > > >> > to
> > > > > > > > > >>> > > > >> > > > how
> > > > > > > > > >>> > > > >> > > > > we
> > > > > > > > > >>> > > > >> > > > > > were planning to generate the
> > clusterId.
> > > > > > > > > >>> > > > >> > > > > >
> > > > > > > > > >>> > > > >> > > > > > Thanks,
> > > > > > > > > >>> > > > >> > > > > > Jason
> > > > > > > > > >>> > > > >> > > > > >
> > > > > > > > > >>> > > > >> > > > > >
> > > > > > > > > >>> > > > >> > > > > > On Thu, Sep 24, 2020 at 11:10 AM Jun
> > > Rao <
> > > > > > > > > >>> > jun@confluent.io>
> > > > > > > > > >>> > > > >> wrote:
> > > > > > > > > >>> > > > >> > > > > >
> > > > > > > > > >>> > > > >> > > > > > > Hi, Justine,
> > > > > > > > > >>> > > > >> > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > Thanks for the updated KIP. A few
> > more
> > > > > > > comments
> > > > > > > > > >>> below.
> > > > > > > > > >>> > > > >> > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > 30.  {"name": "id", "type":
> > "string",
> > > > > "doc":
> > > > > > > > > >>> "version
> > > > > > > > > >>> > > id"}}:
> > > > > > > > > >>> > > > >> The
> > > > > > > > > >>> > > > >> > > doc
> > > > > > > > > >>> > > > >> > > > > > should
> > > > > > > > > >>> > > > >> > > > > > > say UUID.
> > > > > > > > > >>> > > > >> > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > 31. LeaderAndIsrResponse v5 and
> > > > > > > > > StopReplicaResponse
> > > > > > > > > >>> v4 :
> > > > > > > > > >>> > > It
> > > > > > > > > >>> > > > >> seems
> > > > > > > > > >>> > > > >> > > > there
> > > > > > > > > >>> > > > >> > > > > > is
> > > > > > > > > >>> > > > >> > > > > > > no need to add topic_id at
> > partitions
> > > > > level.
> > > > > > > > > >>> > > > >> > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > 32. Regarding partition metadata
> > file.
> > > > > Perhaps
> > > > > > > > the
> > > > > > > > > >>> key
> > > > > > > > > >>> > can
> > > > > > > > > >>> > > > be
> > > > > > > > > >>> > > > >> a
> > > > > > > > > >>> > > > >> > > > single
> > > > > > > > > >>> > > > >> > > > > > > word, sth like the following.
> > > > > > > > > >>> > > > >> > > > > > > version: 0
> > > > > > > > > >>> > > > >> > > > > > > topic_id:
> > > > > 46bdb63f-9e8d-4a38-bf7b-ee4eb2a794e4
> > > > > > > > > >>> > > > >> > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > 33. Another tricky thing that I
> > > realized
> > > > > is
> > > > > > > how
> > > > > > > > to
> > > > > > > > > >>> > support
> > > > > > > > > >>> > > > the
> > > > > > > > > >>> > > > >> > > > metadata
> > > > > > > > > >>> > > > >> > > > > > > topic introduced in KIP-595. It's
> a
> > > bit
> > > > > tricky
> > > > > > > > to
> > > > > > > > > >>> > assign a
> > > > > > > > > >>> > > > >> UUID
> > > > > > > > > >>> > > > >> > to
> > > > > > > > > >>> > > > >> > > > the
> > > > > > > > > >>> > > > >> > > > > > > metadata topic since we have a
> > > > > chicken-and-egg
> > > > > > > > > >>> problem.
> > > > > > > > > >>> > > The
> > > > > > > > > >>> > > > >> > > > controller
> > > > > > > > > >>> > > > >> > > > > > > needs to persist the UUID in the
> > > > metadata
> > > > > > > topic
> > > > > > > > in
> > > > > > > > > >>> order
> > > > > > > > > >>> > > to
> > > > > > > > > >>> > > > >> > assign
> > > > > > > > > >>> > > > >> > > > one
> > > > > > > > > >>> > > > >> > > > > > > successfully, but the metadata
> topic
> > > is
> > > > > needed
> > > > > > > > to
> > > > > > > > > >>> elect
> > > > > > > > > >>> > a
> > > > > > > > > >>> > > > >> > > controller
> > > > > > > > > >>> > > > >> > > > > > > (KIP-631). So, this probably
> needs a
> > > bit
> > > > > more
> > > > > > > > > >>> thought.
> > > > > > > > > >>> > > > >> > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > Jun
> > > > > > > > > >>> > > > >> > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > On Thu, Sep 24, 2020 at 3:04 AM
> > Ismael
> > > > > Juma <
> > > > > > > > > >>> > > > >> ismael@juma.me.uk>
> > > > > > > > > >>> > > > >> > > > wrote:
> > > > > > > > > >>> > > > >> > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > Also, can we provide more
> details
> > on
> > > > > how the
> > > > > > > > > >>> Partition
> > > > > > > > > >>> > > > >> Metadata
> > > > > > > > > >>> > > > >> > > > file
> > > > > > > > > >>> > > > >> > > > > > will
> > > > > > > > > >>> > > > >> > > > > > > > be used?
> > > > > > > > > >>> > > > >> > > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > Ismael
> > > > > > > > > >>> > > > >> > > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > On Thu, Sep 24, 2020 at 3:01 AM
> > > Ismael
> > > > > Juma
> > > > > > > <
> > > > > > > > > >>> > > > >> ismael@juma.me.uk
> > > > > > > > > >>> > > > >> > >
> > > > > > > > > >>> > > > >> > > > > wrote:
> > > > > > > > > >>> > > > >> > > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > > Hi Justine,
> > > > > > > > > >>> > > > >> > > > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > > I think we need to update the
> > > > > "Rejected
> > > > > > > > > >>> > Alternatives"
> > > > > > > > > >>> > > > >> section
> > > > > > > > > >>> > > > >> > > to
> > > > > > > > > >>> > > > >> > > > > take
> > > > > > > > > >>> > > > >> > > > > > > > into
> > > > > > > > > >>> > > > >> > > > > > > > > account that the proposal now
> > > > removes
> > > > > the
> > > > > > > > > topic
> > > > > > > > > >>> name
> > > > > > > > > >>> > > > from
> > > > > > > > > >>> > > > >> the
> > > > > > > > > >>> > > > >> > > > fetch
> > > > > > > > > >>> > > > >> > > > > > > > > request. Also, if we are
> > removing
> > > it
> > > > > from
> > > > > > > > the
> > > > > > > > > >>> Fetch
> > > > > > > > > >>> > > > >> request,
> > > > > > > > > >>> > > > >> > > does
> > > > > > > > > >>> > > > >> > > > > it
> > > > > > > > > >>> > > > >> > > > > > > make
> > > > > > > > > >>> > > > >> > > > > > > > > sense not to remove it from
> > > similar
> > > > > > > requests
> > > > > > > > > >>> like
> > > > > > > > > >>> > > > >> > > > > ListOffsetRequest?
> > > > > > > > > >>> > > > >> > > > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > > Ismael
> > > > > > > > > >>> > > > >> > > > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > > On Thu, Sep 24, 2020 at 2:46
> AM
> > > > David
> > > > > > > Jacot
> > > > > > > > <
> > > > > > > > > >>> > > > >> > > djacot@confluent.io
> > > > > > > > > >>> > > > >> > > > >
> > > > > > > > > >>> > > > >> > > > > > > wrote:
> > > > > > > > > >>> > > > >> > > > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> Hi Justine,
> > > > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > > > >>> > > > >> > > > > > > > >> Thanks for the KIP. I finally
> > had
> > > > > time to
> > > > > > > > > read
> > > > > > > > > >>> it
> > > > > > > > > >>> > :).
> > > > > > > > > >>> > > > It
> > > > > > > > > >>> > > > >> is
> > > > > > > > > >>> > > > >> > a
> > > > > > > > > >>> > > > >> > > > > great
> > > > > > > > > >>> > > > >> > > > > > > > >> improvement.
> > > > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > > > >>> > > > >> > > > > > > > >> I have a few
> > comments/questions:
> > > > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > > > >>> > > > >> > > > > > > > >> 1. It seems that the schema
> of
> > > the
> > > > > > > > > >>> > StopReplicaRequest
> > > > > > > > > >>> > > > is
> > > > > > > > > >>> > > > >> > > > slightly
> > > > > > > > > >>> > > > >> > > > > > > > >> outdated.
> > > > > > > > > >>> > > > >> > > > > > > > >> We
> > > > > > > > > >>> > > > >> > > > > > > > >> did some changes as part of
> > > > KIP-570.
> > > > > V3
> > > > > > > is
> > > > > > > > > >>> already
> > > > > > > > > >>> > > > >> organized
> > > > > > > > > >>> > > > >> > > by
> > > > > > > > > >>> > > > >> > > > > > > topics.
> > > > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > > > >>> > > > >> > > > > > > > >> 2. I just want to make sure
> > that
> > > I
> > > > > > > > understand
> > > > > > > > > >>> the
> > > > > > > > > >>> > > > >> > > reconciliation
> > > > > > > > > >>> > > > >> > > > > > > > >> logic correctly. When
> > > > > > > > > >>> > > > >> > > > > > > > >> an "INCREMENTAL" LeaderAndIsr
> > > > > Request is
> > > > > > > > > >>> received,
> > > > > > > > > >>> > > the
> > > > > > > > > >>> > > > >> > broker
> > > > > > > > > >>> > > > >> > > > will
> > > > > > > > > >>> > > > >> > > > > > > also
> > > > > > > > > >>> > > > >> > > > > > > > >> reconcile
> > > > > > > > > >>> > > > >> > > > > > > > >> when the local uuid does not
> > > match
> > > > > the
> > > > > > > uuid
> > > > > > > > > in
> > > > > > > > > >>> the
> > > > > > > > > >>> > > > >> request,
> > > > > > > > > >>> > > > >> > > > right?
> > > > > > > > > >>> > > > >> > > > > > In
> > > > > > > > > >>> > > > >> > > > > > > > this
> > > > > > > > > >>> > > > >> > > > > > > > >> case, the
> > > > > > > > > >>> > > > >> > > > > > > > >> local log is staged for
> > deletion.
> > > > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > > > >>> > > > >> > > > > > > > >> 3. In the documentation of
> the
> > `
> > > > > > > > > >>> > > > >> delete.stale.topic.delay.ms
> > > > > > > > > >>> > > > >> > `
> > > > > > > > > >>> > > > >> > > > > > config,
> > > > > > > > > >>> > > > >> > > > > > > it
> > > > > > > > > >>> > > > >> > > > > > > > >> says "When a
> > > > > > > > > >>> > > > >> > > > > > > > >> FULL LeaderAndIsrRequest is
> > > > > received..."
> > > > > > > > but
> > > > > > > > > I
> > > > > > > > > >>> > > suppose
> > > > > > > > > >>> > > > it
> > > > > > > > > >>> > > > >> > > > applies
> > > > > > > > > >>> > > > >> > > > > to
> > > > > > > > > >>> > > > >> > > > > > > > both
> > > > > > > > > >>> > > > >> > > > > > > > >> types.
> > > > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > > > >>> > > > >> > > > > > > > >> Best,
> > > > > > > > > >>> > > > >> > > > > > > > >> David
> > > > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > > > >>> > > > >> > > > > > > > >> On Thu, Sep 24, 2020 at 1:40
> AM
> > > > > Justine
> > > > > > > > > Olshan
> > > > > > > > > >>> <
> > > > > > > > > >>> > > > >> > > > > > jolshan@confluent.io>
> > > > > > > > > >>> > > > >> > > > > > > > >> wrote:
> > > > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > > > >>> > > > >> > > > > > > > >> > Hi Jun,
> > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > >>> > > > >> > > > > > > > >> > Thanks for the comments. I
> > > > > apologize
> > > > > > > for
> > > > > > > > > >>> some of
> > > > > > > > > >>> > > the
> > > > > > > > > >>> > > > >> typos
> > > > > > > > > >>> > > > >> > > and
> > > > > > > > > >>> > > > >> > > > > > > > >> confusion.
> > > > > > > > > >>> > > > >> > > > > > > > >> > I’ve updated the KIP to fix
> > > some
> > > > > of the
> > > > > > > > > >>> issues
> > > > > > > > > >>> > you
> > > > > > > > > >>> > > > >> > > mentioned.
> > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > >>> > > > >> > > > > > > > >> > 20.2 I’ve changed the type
> to
> > > > > String.
> > > > > > > > > >>> > > > >> > > > > > > > >> > 20.1/3 I’ve updated the
> > > > TopicZNode
> > > > > to
> > > > > > > fix
> > > > > > > > > >>> > > formatting
> > > > > > > > > >>> > > > >> and
> > > > > > > > > >>> > > > >> > > > reflect
> > > > > > > > > >>> > > > >> > > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> latest
> > > > > > > > > >>> > > > >> > > > > > > > >> > version before this change.
> > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > >>> > > > >> > > > > > > > >> > 21. You are correct and
> I’ve
> > > > > removed
> > > > > > > this
> > > > > > > > > >>> line.
> > > > > > > > > >>> > > I’ve
> > > > > > > > > >>> > > > >> also
> > > > > > > > > >>> > > > >> > > > added
> > > > > > > > > >>> > > > >> > > > > a
> > > > > > > > > >>> > > > >> > > > > > > line
> > > > > > > > > >>> > > > >> > > > > > > > >> > mentioning an IBP bump is
> > > > > necessary for
> > > > > > > > > >>> migration
> > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > >>> > > > >> > > > > > > > >> > 22. I think the wording was
> > > > > unclear but
> > > > > > > > > your
> > > > > > > > > >>> > > summary
> > > > > > > > > >>> > > > is
> > > > > > > > > >>> > > > >> > what
> > > > > > > > > >>> > > > >> > > > was
> > > > > > > > > >>> > > > >> > > > > > > > >> intended
> > > > > > > > > >>> > > > >> > > > > > > > >> > by this line. I’ve updated
> to
> > > > make
> > > > > this
> > > > > > > > > point
> > > > > > > > > >>> > more
> > > > > > > > > >>> > > > >> clear.
> > > > > > > > > >>> > > > >> > > > > “Remove
> > > > > > > > > >>> > > > >> > > > > > > > >> deleted
> > > > > > > > > >>> > > > >> > > > > > > > >> > topics from replicas by
> > sending
> > > > > > > > > >>> > StopReplicaRequest
> > > > > > > > > >>> > > V3
> > > > > > > > > >>> > > > >> > before
> > > > > > > > > >>> > > > >> > > > the
> > > > > > > > > >>> > > > >> > > > > > IBP
> > > > > > > > > >>> > > > >> > > > > > > > >> bump
> > > > > > > > > >>> > > > >> > > > > > > > >> > using the old logic, and
> > using
> > > V4
> > > > > and
> > > > > > > the
> > > > > > > > > new
> > > > > > > > > >>> > logic
> > > > > > > > > >>> > > > >> with
> > > > > > > > > >>> > > > >> > > topic
> > > > > > > > > >>> > > > >> > > > > IDs
> > > > > > > > > >>> > > > >> > > > > > > > after
> > > > > > > > > >>> > > > >> > > > > > > > >> > the IBP bump.”
> > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > >>> > > > >> > > > > > > > >> > 23. I’ve removed the
> > > unspecified
> > > > > type
> > > > > > > > from
> > > > > > > > > >>> the
> > > > > > > > > >>> > KIP
> > > > > > > > > >>> > > > and
> > > > > > > > > >>> > > > >> > > mention
> > > > > > > > > >>> > > > >> > > > > > that
> > > > > > > > > >>> > > > >> > > > > > > > IBP
> > > > > > > > > >>> > > > >> > > > > > > > >> > will be used to handle this
> > > > > request.
> > > > > > > “IBP
> > > > > > > > > >>> will be
> > > > > > > > > >>> > > > used
> > > > > > > > > >>> > > > >> to
> > > > > > > > > >>> > > > >> > > > > > determine
> > > > > > > > > >>> > > > >> > > > > > > > >> whether
> > > > > > > > > >>> > > > >> > > > > > > > >> > this new form of the
> request
> > > will
> > > > > be
> > > > > > > > used.
> > > > > > > > > >>> For
> > > > > > > > > >>> > > older
> > > > > > > > > >>> > > > >> > > requests,
> > > > > > > > > >>> > > > >> > > > > we
> > > > > > > > > >>> > > > >> > > > > > > will
> > > > > > > > > >>> > > > >> > > > > > > > >> > ignore this field and
> default
> > > to
> > > > > > > previous
> > > > > > > > > >>> > > behavior.”
> > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > >>> > > > >> > > > > > > > >> > 24. I’ve fixed this typo.
> > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > >>> > > > >> > > > > > > > >> > 25. I've added a topics at
> a
> > > > higher
> > > > > > > level
> > > > > > > > > for
> > > > > > > > > >>> > > > >> > > > > LeaderAndIsrResponse
> > > > > > > > > >>> > > > >> > > > > > > v5,
> > > > > > > > > >>> > > > >> > > > > > > > >> > StopReplicaResponse v4.
> I've
> > > also
> > > > > > > changed
> > > > > > > > > >>> > > > >> > StopReplicaRequest
> > > > > > > > > >>> > > > >> > > > v4
> > > > > > > > > >>> > > > >> > > > > to
> > > > > > > > > >>> > > > >> > > > > > > > have
> > > > > > > > > >>> > > > >> > > > > > > > >> > topics at a higher level.
> > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > >>> > > > >> > > > > > > > >> > 26. I’ve updated
> > > > > > > > > >>> forgotten_topics_data--added the
> > > > > > > > > >>> > > > >> topic ID
> > > > > > > > > >>> > > > >> > > and
> > > > > > > > > >>> > > > >> > > > > > > removed
> > > > > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > > > > >>> > > > >> > > > > > > > >> > topic name
> > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > >>> > > > >> > > > > > > > >> > 27. I’ve decided on plain
> > text,
> > > > and
> > > > > > > I’ve
> > > > > > > > > >>> added an
> > > > > > > > > >>> > > > >> example.
> > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > >>> > > > >> > > > > > > > >> > 28. I’ve added this idea to
> > > > future
> > > > > > > work.
> > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > >>> > > > >> > > > > > > > >> > Thanks again for taking a
> > look,
> > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > >>> > > > >> > > > > > > > >> > Justine
> > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > >>> > > > >> > > > > > > > >> > On Wed, Sep 23, 2020 at
> 10:28
> > > AM
> > > > > Jun
> > > > > > > Rao
> > > > > > > > <
> > > > > > > > > >>> > > > >> > jun@confluent.io>
> > > > > > > > > >>> > > > >> > > > > > wrote:
> > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > Hi, Justine,
> > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > Thanks for the response.
> > Made
> > > > > another
> > > > > > > > > >>> pass. A
> > > > > > > > > >>> > few
> > > > > > > > > >>> > > > >> more
> > > > > > > > > >>> > > > >> > > > > comments
> > > > > > > > > >>> > > > >> > > > > > > > below.
> > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > 20. znode schema:
> > > > > > > > > >>> > > > >> > > > > > > > >> > > 20.1 It seems that
> {"name":
> > > > > > > "version",
> > > > > > > > > >>> "type":
> > > > > > > > > >>> > > > "int",
> > > > > > > > > >>> > > > >> > > "id":
> > > > > > > > > >>> > > > >> > > > > > > "UUID",
> > > > > > > > > >>> > > > >> > > > > > > > >> > "doc":
> > > > > > > > > >>> > > > >> > > > > > > > >> > > "version id"} should be
> > > > {"name":
> > > > > > > > > "version",
> > > > > > > > > >>> > > "type":
> > > > > > > > > >>> > > > >> > > "int"},
> > > > > > > > > >>> > > > >> > > > > > > {"name":
> > > > > > > > > >>> > > > >> > > > > > > > >> > "id",
> > > > > > > > > >>> > > > >> > > > > > > > >> > > "type": "UUID", "doc":
> > > "version
> > > > > id"}.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > 20.2 The znode format is
> > JSON
> > > > > which
> > > > > > > > > doesn't
> > > > > > > > > >>> > have
> > > > > > > > > >>> > > > UUID
> > > > > > > > > >>> > > > >> > > type.
> > > > > > > > > >>> > > > >> > > > So
> > > > > > > > > >>> > > > >> > > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> type
> > > > > > > > > >>> > > > >> > > > > > > > >> > > probably should be
> string?
> > > > > > > > > >>> > > > >> > > > > > > > >> > > 20.3 Also, the existing
> > > format
> > > > > used
> > > > > > > > seems
> > > > > > > > > >>> > > outdated.
> > > > > > > > > >>> > > > >> It
> > > > > > > > > >>> > > > >> > > > should
> > > > > > > > > >>> > > > >> > > > > > have
> > > > > > > > > >>> > > > >> > > > > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > following format.
> > > > > > > > > >>> > > > >> > > > > > > > >> > >
>  Json.encodeAsBytes(Map(
> > > > > > > > > >>> > > > >> > > > > > > > >> > >       "version" -> 2,
> > > > > > > > > >>> > > > >> > > > > > > > >> > >       "partitions" ->
> > > > > > > > > >>> > > replicaAssignmentJson.asJava,
> > > > > > > > > >>> > > > >> > > > > > > > >> > >       "adding_replicas"
> ->
> > > > > > > > > >>> > > > >> > > > > addingReplicasAssignmentJson.asJava,
> > > > > > > > > >>> > > > >> > > > > > > > >> > >       "removing_replicas"
> > ->
> > > > > > > > > >>> > > > >> > > > > >
> removingReplicasAssignmentJson.asJava
> > > > > > > > > >>> > > > >> > > > > > > > >> > >     ).asJava)
> > > > > > > > > >>> > > > >> > > > > > > > >> > >   }
> > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > 21. Migration: The KIP
> says
> > > > "The
> > > > > > > > > migration
> > > > > > > > > >>> > > process
> > > > > > > > > >>> > > > >> can
> > > > > > > > > >>> > > > >> > > take
> > > > > > > > > >>> > > > >> > > > > > place
> > > > > > > > > >>> > > > >> > > > > > > > >> without
> > > > > > > > > >>> > > > >> > > > > > > > >> > > an inter-broker protocol
> > > bump,
> > > > > as the
> > > > > > > > > >>> format
> > > > > > > > > >>> > > stored
> > > > > > > > > >>> > > > >> in
> > > > > > > > > >>> > > > >> > > > > > > > >> > > /brokers/topics/[topic]
> > will
> > > be
> > > > > > > > > compatible
> > > > > > > > > >>> with
> > > > > > > > > >>> > > > older
> > > > > > > > > >>> > > > >> > > broker
> > > > > > > > > >>> > > > >> > > > > > > > >> versions."
> > > > > > > > > >>> > > > >> > > > > > > > >> > > However, since we are
> > bumping
> > > > up
> > > > > the
> > > > > > > > > >>> version of
> > > > > > > > > >>> > > > >> > > inter-broker
> > > > > > > > > >>> > > > >> > > > > > > > >> requests, it
> > > > > > > > > >>> > > > >> > > > > > > > >> > > seems that we need to use
> > IBP
> > > > for
> > > > > > > > > >>> migration.
> > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > 22. The KIP says "Remove
> > > > deleted
> > > > > > > topics
> > > > > > > > > >>> from
> > > > > > > > > >>> > > > >> replicas by
> > > > > > > > > >>> > > > >> > > > > sending
> > > > > > > > > >>> > > > >> > > > > > > > >> > > StopReplicaRequest V3 for
> > any
> > > > > topics
> > > > > > > > > which
> > > > > > > > > >>> do
> > > > > > > > > >>> > not
> > > > > > > > > >>> > > > >> > contain
> > > > > > > > > >>> > > > >> > > a
> > > > > > > > > >>> > > > >> > > > > > topic
> > > > > > > > > >>> > > > >> > > > > > > > ID,
> > > > > > > > > >>> > > > >> > > > > > > > >> and
> > > > > > > > > >>> > > > >> > > > > > > > >> > > V4 for any topics which
> do
> > > > > contain a
> > > > > > > > > topic
> > > > > > > > > >>> > ID.".
> > > > > > > > > >>> > > > >> > However,
> > > > > > > > > >>> > > > >> > > if
> > > > > > > > > >>> > > > >> > > > > we
> > > > > > > > > >>> > > > >> > > > > > > use
> > > > > > > > > >>> > > > >> > > > > > > > >> IBP,
> > > > > > > > > >>> > > > >> > > > > > > > >> > it
> > > > > > > > > >>> > > > >> > > > > > > > >> > > seems that the controller
> > > will
> > > > > either
> > > > > > > > > send
> > > > > > > > > >>> > > > >> > > > StopReplicaRequest
> > > > > > > > > >>> > > > >> > > > > V3
> > > > > > > > > >>> > > > >> > > > > > > > >> > > or StopReplicaRequest V4,
> > but
> > > > > never
> > > > > > > > mixed
> > > > > > > > > >>> V3
> > > > > > > > > >>> > and
> > > > > > > > > >>> > > V4
> > > > > > > > > >>> > > > >> for
> > > > > > > > > >>> > > > >> > > > > > different
> > > > > > > > > >>> > > > >> > > > > > > > >> topics.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > Basically, before the IBP
> > > bump,
> > > > > V3
> > > > > > > will
> > > > > > > > > be
> > > > > > > > > >>> > used.
> > > > > > > > > >>> > > > >> After
> > > > > > > > > >>> > > > >> > the
> > > > > > > > > >>> > > > >> > > > IBP
> > > > > > > > > >>> > > > >> > > > > > > bump,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > topicId will be created
> and
> > > V4
> > > > > will
> > > > > > > be
> > > > > > > > > >>> used.
> > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > 23. Given that we depend
> on
> > > > IBP,
> > > > > do
> > > > > > > we
> > > > > > > > > >>> still
> > > > > > > > > >>> > need
> > > > > > > > > >>> > > > "0
> > > > > > > > > >>> > > > >> > > > > > UNSPECIFIED"
> > > > > > > > > >>> > > > >> > > > > > > > >> > > in LeaderAndIsr?
> > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > 24. LeaderAndIsrResponse
> > v5 :
> > > > It
> > > > > > > still
> > > > > > > > > has
> > > > > > > > > >>> the
> > > > > > > > > >>> > > > topic
> > > > > > > > > >>> > > > >> > > field.
> > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > 25. LeaderAndIsrResponse
> > v5,
> > > > > > > > > >>> > StopReplicaResponse
> > > > > > > > > >>> > > > v4:
> > > > > > > > > >>> > > > >> > Could
> > > > > > > > > >>> > > > >> > > > we
> > > > > > > > > >>> > > > >> > > > > > use
> > > > > > > > > >>> > > > >> > > > > > > > this
> > > > > > > > > >>> > > > >> > > > > > > > >> > > opportunity to organize
> the
> > > > > response
> > > > > > > > in 2
> > > > > > > > > >>> > levels,
> > > > > > > > > >>> > > > >> first
> > > > > > > > > >>> > > > >> > by
> > > > > > > > > >>> > > > >> > > > > > topic,
> > > > > > > > > >>> > > > >> > > > > > > > >> then by
> > > > > > > > > >>> > > > >> > > > > > > > >> > > partition, as most other
> > > > > > > > > >>> requests/responses?
> > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > 26. FetchRequest v13 :
> > Should
> > > > > > > > > >>> > > forgotten_topics_data
> > > > > > > > > >>> > > > >> use
> > > > > > > > > >>> > > > >> > > > > topicId
> > > > > > > > > >>> > > > >> > > > > > > too?
> > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > 27. "This file can either
> > be
> > > > > plain
> > > > > > > text
> > > > > > > > > >>> > > (key/value
> > > > > > > > > >>> > > > >> > pairs)
> > > > > > > > > >>> > > > >> > > or
> > > > > > > > > >>> > > > >> > > > > > > JSON."
> > > > > > > > > >>> > > > >> > > > > > > > >> Have
> > > > > > > > > >>> > > > >> > > > > > > > >> > we
> > > > > > > > > >>> > > > >> > > > > > > > >> > > decided which one to use?
> > > Also,
> > > > > it
> > > > > > > > would
> > > > > > > > > be
> > > > > > > > > >>> > > helpful
> > > > > > > > > >>> > > > >> to
> > > > > > > > > >>> > > > >> > > > provide
> > > > > > > > > >>> > > > >> > > > > > an
> > > > > > > > > >>> > > > >> > > > > > > > >> > example.
> > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > 28. Future improvement:
> > > Another
> > > > > > > future
> > > > > > > > > >>> > > improvement
> > > > > > > > > >>> > > > >> > > > opportunity
> > > > > > > > > >>> > > > >> > > > > > is
> > > > > > > > > >>> > > > >> > > > > > > to
> > > > > > > > > >>> > > > >> > > > > > > > >> use
> > > > > > > > > >>> > > > >> > > > > > > > >> > > topicId in
> > > > > > > > > >>> GroupMetadataManager.offsetCommitKey
> > > > > > > > > >>> > > in
> > > > > > > > > >>> > > > >> the
> > > > > > > > > >>> > > > >> > > > > > > offset_commit
> > > > > > > > > >>> > > > >> > > > > > > > >> > topic.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > This may save some space.
> > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > Thanks,
> > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > Jun
> > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > On Wed, Sep 23, 2020 at
> > 8:50
> > > AM
> > > > > > > Justine
> > > > > > > > > >>> Olshan
> > > > > > > > > >>> > <
> > > > > > > > > >>> > > > >> > > > > > > > jolshan@confluent.io>
> > > > > > > > > >>> > > > >> > > > > > > > >> > > wrote:
> > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > Hi Tom,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > Thanks for the
> comment. I
> > > > think
> > > > > > > this
> > > > > > > > > is a
> > > > > > > > > >>> > > really
> > > > > > > > > >>> > > > >> good
> > > > > > > > > >>> > > > >> > > idea
> > > > > > > > > >>> > > > >> > > > > and
> > > > > > > > > >>> > > > >> > > > > > > it
> > > > > > > > > >>> > > > >> > > > > > > > >> has
> > > > > > > > > >>> > > > >> > > > > > > > >> > > been
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > added to the KIP under
> > the
> > > > > newly
> > > > > > > > added
> > > > > > > > > >>> > tooling
> > > > > > > > > >>> > > > >> > section.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > Thanks again,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > Justine
> > > > > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > On Wed, Sep 23, 2020 at
> > > 3:17
> > > > > AM Tom
> > > > > > > > > >>> Bentley <
> > > > > > > > > >>> > > > >> > > > > > > tbentley@redhat.com>
> > > > > > > > > >>> > > > >> > > > > > > > >> > wrote:
> > > > > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > Hi Justine,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > I know you started
> the
> > > vote
> > > > > > > thread,
> > > > > > > > > >>> but on
> > > > > > > > > >>> > > > >> > re-reading
> > > > > > > > > >>> > > > >> > > > the
> > > > > > > > > >>> > > > >> > > > > > KIP
> > > > > > > > > >>> > > > >> > > > > > > I
> > > > > > > > > >>> > > > >> > > > > > > > >> > noticed
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that although the
> topic
> > > id
> > > > is
> > > > > > > > > included
> > > > > > > > > >>> in
> > > > > > > > > >>> > the
> > > > > > > > > >>> > > > >> > > > > > MetadataResponse
> > > > > > > > > >>> > > > >> > > > > > > > >> it's
> > > > > > > > > >>> > > > >> > > > > > > > >> > not
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > surfaced in the
> output
> > > from
> > > > > > > > > >>> > `kafka-topics.sh
> > > > > > > > > >>> > > > >> > > > --describe`.
> > > > > > > > > >>> > > > >> > > > > > > Maybe
> > > > > > > > > >>> > > > >> > > > > > > > >> that
> > > > > > > > > >>> > > > >> > > > > > > > >> > > was
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > intentional because
> ids
> > > are
> > > > > > > > > >>> intentionally
> > > > > > > > > >>> > not
> > > > > > > > > >>> > > > >> really
> > > > > > > > > >>> > > > >> > > > > > something
> > > > > > > > > >>> > > > >> > > > > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > user
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > should care deeply
> > about,
> > > > > but it
> > > > > > > > > would
> > > > > > > > > >>> also
> > > > > > > > > >>> > > > make
> > > > > > > > > >>> > > > >> > life
> > > > > > > > > >>> > > > >> > > > > harder
> > > > > > > > > >>> > > > >> > > > > > > for
> > > > > > > > > >>> > > > >> > > > > > > > >> > anyone
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > debugging Kafka and
> > this
> > > > > would
> > > > > > > > likely
> > > > > > > > > >>> get
> > > > > > > > > >>> > > worse
> > > > > > > > > >>> > > > >> the
> > > > > > > > > >>> > > > >> > > more
> > > > > > > > > >>> > > > >> > > > > > topic
> > > > > > > > > >>> > > > >> > > > > > > > ids
> > > > > > > > > >>> > > > >> > > > > > > > >> > got
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > rolled out across the
> > > > > protocols,
> > > > > > > > > >>> clients
> > > > > > > > > >>> > etc.
> > > > > > > > > >>> > > > It
> > > > > > > > > >>> > > > >> > seems
> > > > > > > > > >>> > > > >> > > > > > likely
> > > > > > > > > >>> > > > >> > > > > > > > that
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > `kafka-topics.sh`
> will
> > > > > eventually
> > > > > > > > > need
> > > > > > > > > >>> the
> > > > > > > > > >>> > > > >> ability
> > > > > > > > > >>> > > > >> > to
> > > > > > > > > >>> > > > >> > > > show
> > > > > > > > > >>> > > > >> > > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > id
> > > > > > > > > >>> > > > >> > > > > > > > >> of
> > > > > > > > > >>> > > > >> > > > > > > > >> > a
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > topic and perhaps
> find
> > a
> > > > > topic
> > > > > > > name
> > > > > > > > > >>> given
> > > > > > > > > >>> > an
> > > > > > > > > >>> > > > id.
> > > > > > > > > >>> > > > >> Is
> > > > > > > > > >>> > > > >> > > > there
> > > > > > > > > >>> > > > >> > > > > > any
> > > > > > > > > >>> > > > >> > > > > > > > >> reason
> > > > > > > > > >>> > > > >> > > > > > > > >> > > not
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > implement that in
> this
> > > KIP?
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > Many thanks,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > Tom
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > On Mon, Sep 21, 2020
> at
> > > > 9:54
> > > > > PM
> > > > > > > > > Justine
> > > > > > > > > >>> > > Olshan
> > > > > > > > > >>> > > > <
> > > > > > > > > >>> > > > >> > > > > > > > >> jolshan@confluent.io
> > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > wrote:
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Hi all,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > After thinking
> about
> > > it,
> > > > > I've
> > > > > > > > > >>> decided to
> > > > > > > > > >>> > > > remove
> > > > > > > > > >>> > > > >> > the
> > > > > > > > > >>> > > > >> > > > > topic
> > > > > > > > > >>> > > > >> > > > > > > name
> > > > > > > > > >>> > > > >> > > > > > > > >> from
> > > > > > > > > >>> > > > >> > > > > > > > >> > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Fetch Request and
> > > > Response
> > > > > > > after
> > > > > > > > > all.
> > > > > > > > > >>> > Since
> > > > > > > > > >>> > > > >> there
> > > > > > > > > >>> > > > >> > > are
> > > > > > > > > >>> > > > >> > > > so
> > > > > > > > > >>> > > > >> > > > > > > many
> > > > > > > > > >>> > > > >> > > > > > > > of
> > > > > > > > > >>> > > > >> > > > > > > > >> > > these
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > requests per
> second,
> > it
> > > > is
> > > > > > > worth
> > > > > > > > > >>> removing
> > > > > > > > > >>> > > the
> > > > > > > > > >>> > > > >> > extra
> > > > > > > > > >>> > > > >> > > > > > > > information.
> > > > > > > > > >>> > > > >> > > > > > > > >> > I've
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > updated the KIP to
> > > > reflect
> > > > > this
> > > > > > > > > >>> change.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Please let me know
> if
> > > > > there is
> > > > > > > > > >>> anything
> > > > > > > > > >>> > > else
> > > > > > > > > >>> > > > we
> > > > > > > > > >>> > > > >> > > should
> > > > > > > > > >>> > > > >> > > > > > > discuss
> > > > > > > > > >>> > > > >> > > > > > > > >> > before
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > voting.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Thank you,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Justine
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > On Fri, Sep 18,
> 2020
> > at
> > > > > 9:46 AM
> > > > > > > > > >>> Justine
> > > > > > > > > >>> > > > Olshan
> > > > > > > > > >>> > > > >> <
> > > > > > > > > >>> > > > >> > > > > > > > >> > jolshan@confluent.io
> > > > > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > wrote:
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > Hi Jun,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > I see what you
> are
> > > > > saying.
> > > > > > > For
> > > > > > > > > now
> > > > > > > > > >>> we
> > > > > > > > > >>> > can
> > > > > > > > > >>> > > > >> remove
> > > > > > > > > >>> > > > >> > > the
> > > > > > > > > >>> > > > >> > > > > > extra
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > information.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > I'll leave the
> > option
> > > > to
> > > > > add
> > > > > > > > more
> > > > > > > > > >>> > fields
> > > > > > > > > >>> > > to
> > > > > > > > > >>> > > > >> the
> > > > > > > > > >>> > > > >> > > file
> > > > > > > > > >>> > > > >> > > > > in
> > > > > > > > > >>> > > > >> > > > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > future.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > The
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > KIP
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > has been updated
> to
> > > > > reflect
> > > > > > > > this
> > > > > > > > > >>> > change.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > Thanks,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > Justine
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > On Fri, Sep 18,
> > 2020
> > > at
> > > > > 8:46
> > > > > > > AM
> > > > > > > > > Jun
> > > > > > > > > >>> > Rao <
> > > > > > > > > >>> > > > >> > > > > > jun@confluent.io
> > > > > > > > > >>> > > > >> > > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > wrote:
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Hi, Justine,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Thanks for the
> > > reply.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> 13. If the log
> > > > > directory is
> > > > > > > > the
> > > > > > > > > >>> source
> > > > > > > > > >>> > > of
> > > > > > > > > >>> > > > >> > truth,
> > > > > > > > > >>> > > > >> > > it
> > > > > > > > > >>> > > > >> > > > > > means
> > > > > > > > > >>> > > > >> > > > > > > > >> that
> > > > > > > > > >>> > > > >> > > > > > > > >> > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> redundant info
> in
> > > the
> > > > > > > metadata
> > > > > > > > > >>> file
> > > > > > > > > >>> > will
> > > > > > > > > >>> > > > be
> > > > > > > > > >>> > > > >> > > > ignored.
> > > > > > > > > >>> > > > >> > > > > > Then
> > > > > > > > > >>> > > > >> > > > > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > question
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > is
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> why do we need
> to
> > > put
> > > > > the
> > > > > > > > > >>> redundant
> > > > > > > > > >>> > info
> > > > > > > > > >>> > > > in
> > > > > > > > > >>> > > > >> the
> > > > > > > > > >>> > > > >> > > > > > metadata
> > > > > > > > > >>> > > > >> > > > > > > > file
> > > > > > > > > >>> > > > >> > > > > > > > >> > now?
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Thanks,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Jun
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> On Thu, Sep 17,
> > 2020
> > > > at
> > > > > 5:07
> > > > > > > > PM
> > > > > > > > > >>> > Justine
> > > > > > > > > >>> > > > >> Olshan
> > > > > > > > > >>> > > > >> > <
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > jolshan@confluent.io>
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> wrote:
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Hi Jun,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Thanks for the
> > > quick
> > > > > > > > response!
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > 12. I've
> decided
> > > to
> > > > > bump
> > > > > > > up
> > > > > > > > > the
> > > > > > > > > >>> > > versions
> > > > > > > > > >>> > > > >> on
> > > > > > > > > >>> > > > >> > the
> > > > > > > > > >>> > > > >> > > > > > > requests
> > > > > > > > > >>> > > > >> > > > > > > > >> and
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > updated
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > KIP. I think
> > it's
> > > > > good we
> > > > > > > > > >>> thoroughly
> > > > > > > > > >>> > > > >> > discussed
> > > > > > > > > >>> > > > >> > > > the
> > > > > > > > > >>> > > > >> > > > > > > > options
> > > > > > > > > >>> > > > >> > > > > > > > >> > here,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > so
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > we
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> know
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > we made a good
> > > > > choice. :)
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > 13. This is an
> > > > > interesting
> > > > > > > > > >>> > situation.
> > > > > > > > > >>> > > I
> > > > > > > > > >>> > > > >> think
> > > > > > > > > >>> > > > >> > > if
> > > > > > > > > >>> > > > >> > > > > this
> > > > > > > > > >>> > > > >> > > > > > > > does
> > > > > > > > > >>> > > > >> > > > > > > > >> > occur
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > we
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> should
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > give a
> warning.
> > I
> > > > > agree
> > > > > > > that
> > > > > > > > > >>> it's
> > > > > > > > > >>> > hard
> > > > > > > > > >>> > > > to
> > > > > > > > > >>> > > > >> > know
> > > > > > > > > >>> > > > >> > > > the
> > > > > > > > > >>> > > > >> > > > > > > source
> > > > > > > > > >>> > > > >> > > > > > > > >> of
> > > > > > > > > >>> > > > >> > > > > > > > >> > > truth
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > for
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> sure
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > since the
> > > directory
> > > > > or the
> > > > > > > > > file
> > > > > > > > > >>> > could
> > > > > > > > > >>> > > be
> > > > > > > > > >>> > > > >> > > manually
> > > > > > > > > >>> > > > >> > > > > > > > >> modified. I
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > guess
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > directory
> could
> > be
> > > > > used as
> > > > > > > > the
> > > > > > > > > >>> > source
> > > > > > > > > >>> > > of
> > > > > > > > > >>> > > > >> > truth.
> > > > > > > > > >>> > > > >> > > > To
> > > > > > > > > >>> > > > >> > > > > be
> > > > > > > > > >>> > > > >> > > > > > > > >> honest,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > I'm
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > not
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > really sure
> what
> > > > > happens
> > > > > > > in
> > > > > > > > > >>> kafka
> > > > > > > > > >>> > when
> > > > > > > > > >>> > > > the
> > > > > > > > > >>> > > > >> > log
> > > > > > > > > >>> > > > >> > > > > > > directory
> > > > > > > > > >>> > > > >> > > > > > > > is
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > renamed
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > manually in
> > such a
> > > > > way.
> > > > > > > I'm
> > > > > > > > > also
> > > > > > > > > >>> > > > >> wondering if
> > > > > > > > > >>> > > > >> > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> situation is
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> recoverable
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > in this
> > scenario.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Thanks,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Justine
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > On Thu, Sep
> 17,
> > > 2020
> > > > > at
> > > > > > > 4:28
> > > > > > > > > PM
> > > > > > > > > >>> Jun
> > > > > > > > > >>> > > Rao
> > > > > > > > > >>> > > > <
> > > > > > > > > >>> > > > >> > > > > > > > jun@confluent.io>
> > > > > > > > > >>> > > > >> > > > > > > > >> > > wrote:
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Hi, Justine,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Thanks for
> the
> > > > > reply.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > 12. I don't
> > > have a
> > > > > > > strong
> > > > > > > > > >>> > preference
> > > > > > > > > >>> > > > >> > either.
> > > > > > > > > >>> > > > >> > > > > > However,
> > > > > > > > > >>> > > > >> > > > > > > > if
> > > > > > > > > >>> > > > >> > > > > > > > >> we
> > > > > > > > > >>> > > > >> > > > > > > > >> > > need
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > IBP
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > anyway,
> maybe
> > > it's
> > > > > > > easier
> > > > > > > > to
> > > > > > > > > >>> just
> > > > > > > > > >>> > > bump
> > > > > > > > > >>> > > > >> up
> > > > > > > > > >>> > > > >> > the
> > > > > > > > > >>> > > > >> > > > > > version
> > > > > > > > > >>> > > > >> > > > > > > > for
> > > > > > > > > >>> > > > >> > > > > > > > >> > all
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > inter
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > broker
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > requests and
> > add
> > > > the
> > > > > > > topic
> > > > > > > > > id
> > > > > > > > > >>> > field
> > > > > > > > > >>> > > > as a
> > > > > > > > > >>> > > > >> > > > regular
> > > > > > > > > >>> > > > >> > > > > > > > field. A
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > regular
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> field
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > is
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > a bit more
> > > concise
> > > > > in
> > > > > > > wire
> > > > > > > > > >>> > transfer
> > > > > > > > > >>> > > > >> than a
> > > > > > > > > >>> > > > >> > > > > flexible
> > > > > > > > > >>> > > > >> > > > > > > > >> field.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > 13. The
> > > confusion
> > > > > that I
> > > > > > > > was
> > > > > > > > > >>> > > referring
> > > > > > > > > >>> > > > >> to
> > > > > > > > > >>> > > > >> > is
> > > > > > > > > >>> > > > >> > > > > > between
> > > > > > > > > >>> > > > >> > > > > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > topic
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > name
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > partition
> > number
> > > > > between
> > > > > > > > the
> > > > > > > > > >>> log
> > > > > > > > > >>> > dir
> > > > > > > > > >>> > > > and
> > > > > > > > > >>> > > > >> > the
> > > > > > > > > >>> > > > >> > > > > > metadata
> > > > > > > > > >>> > > > >> > > > > > > > >> file.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > For
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> example,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > if
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > the log dir
> is
> > > > > topicA-1
> > > > > > > > and
> > > > > > > > > >>> the
> > > > > > > > > >>> > > > metadata
> > > > > > > > > >>> > > > >> > file
> > > > > > > > > >>> > > > >> > > > in
> > > > > > > > > >>> > > > >> > > > > it
> > > > > > > > > >>> > > > >> > > > > > > has
> > > > > > > > > >>> > > > >> > > > > > > > >> > topicB
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > and
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > partition 0
> > (say
> > > > > due to
> > > > > > > a
> > > > > > > > > bug
> > > > > > > > > >>> or
> > > > > > > > > >>> > > > manual
> > > > > > > > > >>> > > > >> > > > > > > modification),
> > > > > > > > > >>> > > > >> > > > > > > > >> which
> > > > > > > > > >>> > > > >> > > > > > > > >> > > one
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > do
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > we
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > use
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > as the
> source
> > of
> > > > > truth?
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Jun
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > On Thu, Sep
> > 17,
> > > > > 2020 at
> > > > > > > > 3:43
> > > > > > > > > >>> PM
> > > > > > > > > >>> > > > Justine
> > > > > > > > > >>> > > > >> > > Olshan
> > > > > > > > > >>> > > > >> > > > <
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
> jolshan@confluent.io
> > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > wrote:
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > Hi Jun,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > Thanks for
> > the
> > > > > > > comments.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > 12. I
> bumped
> > > the
> > > > > > > > > >>> > > LeaderAndIsrRequest
> > > > > > > > > >>> > > > >> > > because
> > > > > > > > > >>> > > > >> > > > I
> > > > > > > > > >>> > > > >> > > > > > > > removed
> > > > > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > topic
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> name
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > field
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > in the
> > > response.
> > > > > It
> > > > > > > may
> > > > > > > > be
> > > > > > > > > >>> > > possible
> > > > > > > > > >>> > > > to
> > > > > > > > > >>> > > > >> > > avoid
> > > > > > > > > >>> > > > >> > > > > > > bumping
> > > > > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > version
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > without
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > that
> change,
> > > but
> > > > > I may
> > > > > > > > be
> > > > > > > > > >>> > missing
> > > > > > > > > >>> > > > >> > > something.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > I believe
> > > > > StopReplica
> > > > > > > is
> > > > > > > > > >>> > actually
> > > > > > > > > >>> > > on
> > > > > > > > > >>> > > > >> > > version
> > > > > > > > > >>> > > > >> > > > 3
> > > > > > > > > >>> > > > >> > > > > > now,
> > > > > > > > > >>> > > > >> > > > > > > > but
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > because
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > version 2
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > is
> > flexible, I
> > > > > kept
> > > > > > > that
> > > > > > > > > >>> listed
> > > > > > > > > >>> > as
> > > > > > > > > >>> > > > >> > version
> > > > > > > > > >>> > > > >> > > 2
> > > > > > > > > >>> > > > >> > > > on
> > > > > > > > > >>> > > > >> > > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > KIP
> > > > > > > > > >>> > > > >> > > > > > > > >> > > page.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> However,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > you
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > may be
> right
> > > in
> > > > > that
> > > > > > > we
> > > > > > > > > may
> > > > > > > > > >>> need
> > > > > > > > > >>> > > to
> > > > > > > > > >>> > > > >> bump
> > > > > > > > > >>> > > > >> > > the
> > > > > > > > > >>> > > > >> > > > > > > version
> > > > > > > > > >>> > > > >> > > > > > > > on
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > StopReplica
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > deal
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > with
> > deletion
> > > > > > > > differently
> > > > > > > > > as
> > > > > > > > > >>> > > > mentioned
> > > > > > > > > >>> > > > >> > > > above. I
> > > > > > > > > >>> > > > >> > > > > > > don't
> > > > > > > > > >>> > > > >> > > > > > > > >> know
> > > > > > > > > >>> > > > >> > > > > > > > >> > > if
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > I
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> have a
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > big
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > preference
> > > over
> > > > > used
> > > > > > > > > tagged
> > > > > > > > > >>> > fields
> > > > > > > > > >>> > > > or
> > > > > > > > > >>> > > > >> > not.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > 13. I was
> > > > thinking
> > > > > > > that
> > > > > > > > in
> > > > > > > > > >>> the
> > > > > > > > > >>> > > case
> > > > > > > > > >>> > > > >> where
> > > > > > > > > >>> > > > >> > > the
> > > > > > > > > >>> > > > >> > > > > > file
> > > > > > > > > >>> > > > >> > > > > > > > and
> > > > > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > request
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > topic
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > ids don't
> > > match,
> > > > > it
> > > > > > > > means
> > > > > > > > > >>> that
> > > > > > > > > >>> > the
> > > > > > > > > >>> > > > >> > broker's
> > > > > > > > > >>> > > > >> > > > > > > topic/the
> > > > > > > > > >>> > > > >> > > > > > > > >> one
> > > > > > > > > >>> > > > >> > > > > > > > >> > in
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> file
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > has
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > been
> > deleted.
> > > In
> > > > > that
> > > > > > > > > case,
> > > > > > > > > >>> we
> > > > > > > > > >>> > > would
> > > > > > > > > >>> > > > >> need
> > > > > > > > > >>> > > > >> > > to
> > > > > > > > > >>> > > > >> > > > > > delete
> > > > > > > > > >>> > > > >> > > > > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > old
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > topic
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > start
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > receiving
> > the
> > > > new
> > > > > > > > version.
> > > > > > > > > >>> If
> > > > > > > > > >>> > the
> > > > > > > > > >>> > > > >> topic
> > > > > > > > > >>> > > > >> > > name
> > > > > > > > > >>> > > > >> > > > > were
> > > > > > > > > >>> > > > >> > > > > > > to
> > > > > > > > > >>> > > > >> > > > > > > > >> > change,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > but
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > ids
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > still
> > matched,
> > > > the
> > > > > > > file
> > > > > > > > > >>> would
> > > > > > > > > >>> > also
> > > > > > > > > >>> > > > >> need
> > > > > > > > > >>> > > > >> > to
> > > > > > > > > >>> > > > >> > > > > > update.
> > > > > > > > > >>> > > > >> > > > > > > > Am I
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > missing
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > a
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> case
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > where the
> > file
> > > > > would
> > > > > > > be
> > > > > > > > > >>> correct
> > > > > > > > > >>> > > and
> > > > > > > > > >>> > > > >> not
> > > > > > > > > >>> > > > >> > the
> > > > > > > > > >>> > > > >> > > > > > > request?
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > Thanks,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > Justine
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > On Thu,
> Sep
> > > 17,
> > > > > 2020
> > > > > > > at
> > > > > > > > > >>> 3:18 PM
> > > > > > > > > >>> > > Jun
> > > > > > > > > >>> > > > >> Rao <
> > > > > > > > > >>> > > > >> > > > > > > > >> jun@confluent.io
> > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > wrote:
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Hi,
> > Justine,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Thanks
> for
> > > the
> > > > > > > reply.
> > > > > > > > A
> > > > > > > > > >>> couple
> > > > > > > > > >>> > > of
> > > > > > > > > >>> > > > >> more
> > > > > > > > > >>> > > > >> > > > > comments
> > > > > > > > > >>> > > > >> > > > > > > > >> below.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > 12.
> > > ListOffset
> > > > > and
> > > > > > > > > >>> > > OffsetForLeader
> > > > > > > > > >>> > > > >> > > > currently
> > > > > > > > > >>> > > > >> > > > > > > don't
> > > > > > > > > >>> > > > >> > > > > > > > >> > support
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> flexible
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > fields.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > So, we
> > have
> > > to
> > > > > bump
> > > > > > > up
> > > > > > > > > the
> > > > > > > > > >>> > > version
> > > > > > > > > >>> > > > >> > number
> > > > > > > > > >>> > > > >> > > > and
> > > > > > > > > >>> > > > >> > > > > > use
> > > > > > > > > >>> > > > >> > > > > > > > >> IBP at
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > least
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > for
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > these
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > two
> > > requests.
> > > > > Note
> > > > > > > > that
> > > > > > > > > it
> > > > > > > > > >>> > seems
> > > > > > > > > >>> > > > >> 2.7.0
> > > > > > > > > >>> > > > >> > > will
> > > > > > > > > >>> > > > >> > > > > > > require
> > > > > > > > > >>> > > > >> > > > > > > > >> IBP
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > anyway
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > because
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > of
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > changes
> in
> > > > > > > > KAFKA-10435.
> > > > > > > > > >>> Also,
> > > > > > > > > >>> > it
> > > > > > > > > >>> > > > >> seems
> > > > > > > > > >>> > > > >> > > that
> > > > > > > > > >>> > > > >> > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> version
> > > > > > > > > >>> > > > >> > > > > > > > >> > > for
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > LeaderAndIsrRequest
> > > > > > > > and
> > > > > > > > > >>> > > > StopReplica
> > > > > > > > > >>> > > > >> are
> > > > > > > > > >>> > > > >> > > > > bumped
> > > > > > > > > >>> > > > >> > > > > > > even
> > > > > > > > > >>> > > > >> > > > > > > > >> > though
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > we
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > only
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > added
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > a
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > tagged
> > > field.
> > > > > But
> > > > > > > > since
> > > > > > > > > >>> IBP is
> > > > > > > > > >>> > > > >> needed
> > > > > > > > > >>> > > > >> > > > anyway,
> > > > > > > > > >>> > > > >> > > > > > we
> > > > > > > > > >>> > > > >> > > > > > > > may
> > > > > > > > > >>> > > > >> > > > > > > > >> > want
> > > > > > > > > >>> > > > >> > > > > > > > >> > > to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> revisit
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > overall
> > > tagged
> > > > > field
> > > > > > > > > >>> choice.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > 13. The
> > only
> > > > > > > downside
> > > > > > > > is
> > > > > > > > > >>> the
> > > > > > > > > >>> > > > >> potential
> > > > > > > > > >>> > > > >> > > > > > confusion
> > > > > > > > > >>> > > > >> > > > > > > on
> > > > > > > > > >>> > > > >> > > > > > > > >> > which
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > one
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > is
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > source
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > of truth
> > if
> > > > they
> > > > > > > don't
> > > > > > > > > >>> match.
> > > > > > > > > >>> > > > >> Another
> > > > > > > > > >>> > > > >> > > > option
> > > > > > > > > >>> > > > >> > > > > is
> > > > > > > > > >>> > > > >> > > > > > > to
> > > > > > > > > >>> > > > >> > > > > > > > >> > include
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > those
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > fields
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > in
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > metadata
> > > > > file
> > > > > > > when
> > > > > > > > > we
> > > > > > > > > >>> > > actually
> > > > > > > > > >>> > > > >> > change
> > > > > > > > > >>> > > > >> > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> directory
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > structure.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Thanks,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Jun
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > On Thu,
> > Sep
> > > > 17,
> > > > > 2020
> > > > > > > > at
> > > > > > > > > >>> 2:01
> > > > > > > > > >>> > PM
> > > > > > > > > >>> > > > >> Justine
> > > > > > > > > >>> > > > >> > > > > Olshan
> > > > > > > > > >>> > > > >> > > > > > <
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > jolshan@confluent.io
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > wrote:
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > Hello
> > all,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I've
> > > thought
> > > > > some
> > > > > > > > more
> > > > > > > > > >>> about
> > > > > > > > > >>> > > > >> removing
> > > > > > > > > >>> > > > >> > > the
> > > > > > > > > >>> > > > >> > > > > > topic
> > > > > > > > > >>> > > > >> > > > > > > > >> name
> > > > > > > > > >>> > > > >> > > > > > > > >> > > field
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > from
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > some
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > of
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > requests.
> > > On
> > > > > > > closer
> > > > > > > > > >>> > inspection
> > > > > > > > > >>> > > > of
> > > > > > > > > >>> > > > >> the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > requests/responses,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > it
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> seems
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> internal
> > > > > changes
> > > > > > > > would
> > > > > > > > > >>> be
> > > > > > > > > >>> > much
> > > > > > > > > >>> > > > >> larger
> > > > > > > > > >>> > > > >> > > > than
> > > > > > > > > >>> > > > >> > > > > I
> > > > > > > > > >>> > > > >> > > > > > > > >> expected.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > Some
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > protocols
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> involve
> > > > > clients,
> > > > > > > so
> > > > > > > > > they
> > > > > > > > > >>> > would
> > > > > > > > > >>> > > > >> > require
> > > > > > > > > >>> > > > >> > > > > > changes
> > > > > > > > > >>> > > > >> > > > > > > > too.
> > > > > > > > > >>> > > > >> > > > > > > > >> > I'm
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > thinking
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > that
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > for
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > now,
> > > > removing
> > > > > the
> > > > > > > > > topic
> > > > > > > > > >>> name
> > > > > > > > > >>> > > > from
> > > > > > > > > >>> > > > >> > these
> > > > > > > > > >>> > > > >> > > > > > > requests
> > > > > > > > > >>> > > > >> > > > > > > > >> and
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > responses
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> are
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > out
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > scope.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I have
> > > > > decided to
> > > > > > > > just
> > > > > > > > > >>> keep
> > > > > > > > > >>> > > the
> > > > > > > > > >>> > > > >> > change
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > LeaderAndIsrResponse
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > remove
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > topic
> > > name,
> > > > > and
> > > > > > > have
> > > > > > > > > >>> updated
> > > > > > > > > >>> > > the
> > > > > > > > > >>> > > > >> KIP
> > > > > > > > > >>> > > > >> > to
> > > > > > > > > >>> > > > >> > > > > > reflect
> > > > > > > > > >>> > > > >> > > > > > > > >> this
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > change. I
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> have
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > also
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > mentioned
> > > > the
> > > > > > > other
> > > > > > > > > >>> requests
> > > > > > > > > >>> > > and
> > > > > > > > > >>> > > > >> > > > responses
> > > > > > > > > >>> > > > >> > > > > in
> > > > > > > > > >>> > > > >> > > > > > > > >> future
> > > > > > > > > >>> > > > >> > > > > > > > >> > > work.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I'm
> > hoping
> > > > to
> > > > > > > start
> > > > > > > > > the
> > > > > > > > > >>> > voting
> > > > > > > > > >>> > > > >> > process
> > > > > > > > > >>> > > > >> > > > > soon,
> > > > > > > > > >>> > > > >> > > > > > so
> > > > > > > > > >>> > > > >> > > > > > > > >> let me
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > know
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > if
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > there
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > is
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> anything
> > > > else
> > > > > we
> > > > > > > > > should
> > > > > > > > > >>> > > discuss.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > Thank
> > you,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> Justine
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > On
> Tue,
> > > Sep
> > > > > 15,
> > > > > > > 2020
> > > > > > > > > at
> > > > > > > > > >>> 3:57
> > > > > > > > > >>> > > PM
> > > > > > > > > >>> > > > >> > Justine
> > > > > > > > > >>> > > > >> > > > > > Olshan
> > > > > > > > > >>> > > > >> > > > > > > <
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > jolshan@confluent.io
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > wrote:
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> Hello
> > > > again,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > To
> > > follow
> > > > > up on
> > > > > > > > some
> > > > > > > > > >>> of
> > > > > > > > > >>> > the
> > > > > > > > > >>> > > > >> other
> > > > > > > > > >>> > > > >> > > > > comments:
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> 10/11)
> > > We
> > > > > can
> > > > > > > > remove
> > > > > > > > > >>> the
> > > > > > > > > >>> > > topic
> > > > > > > > > >>> > > > >> name
> > > > > > > > > >>> > > > >> > > > from
> > > > > > > > > >>> > > > >> > > > > > > these
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > requests/responses,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > and
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > that
> > > means
> > > > > we
> > > > > > > will
> > > > > > > > > >>> just
> > > > > > > > > >>> > have
> > > > > > > > > >>> > > > to
> > > > > > > > > >>> > > > >> > make
> > > > > > > > > >>> > > > >> > > a
> > > > > > > > > >>> > > > >> > > > > few
> > > > > > > > > >>> > > > >> > > > > > > > >> internal
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > changes
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > make
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > partitions
> > > > > > > > > accessible
> > > > > > > > > >>> by
> > > > > > > > > >>> > > topic
> > > > > > > > > >>> > > > >> id
> > > > > > > > > >>> > > > >> > and
> > > > > > > > > >>> > > > >> > > > > > > > partition.
> > > > > > > > > >>> > > > >> > > > > > > > >> I
> > > > > > > > > >>> > > > >> > > > > > > > >> > can
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > update
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > KIP
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> remove
> > > > them
> > > > > > > unless
> > > > > > > > > >>> anyone
> > > > > > > > > >>> > > > thinks
> > > > > > > > > >>> > > > >> > they
> > > > > > > > > >>> > > > >> > > > > > should
> > > > > > > > > >>> > > > >> > > > > > > > >> stay.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 12)
> > > > > Addressed in
> > > > > > > > the
> > > > > > > > > >>> > > previous
> > > > > > > > > >>> > > > >> > email.
> > > > > > > > > >>> > > > >> > > > I've
> > > > > > > > > >>> > > > >> > > > > > > > updated
> > > > > > > > > >>> > > > >> > > > > > > > >> > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > KIP
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > include
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> tagged
> > > > > fields
> > > > > > > for
> > > > > > > > > the
> > > > > > > > > >>> > > requests
> > > > > > > > > >>> > > > >> and
> > > > > > > > > >>> > > > >> > > > > > responses.
> > > > > > > > > >>> > > > >> > > > > > > > >> (More
> > > > > > > > > >>> > > > >> > > > > > > > >> > on
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > below)
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 13)
> I
> > > > think
> > > > > part
> > > > > > > > of
> > > > > > > > > >>> the
> > > > > > > > > >>> > idea
> > > > > > > > > >>> > > > for
> > > > > > > > > >>> > > > >> > > > > including
> > > > > > > > > >>> > > > >> > > > > > > this
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > information
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> is to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > prepare
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > for
> > > future
> > > > > > > > changes.
> > > > > > > > > >>> > Perhaps
> > > > > > > > > >>> > > > the
> > > > > > > > > >>> > > > >> > > > directory
> > > > > > > > > >>> > > > >> > > > > > > > >> structure
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > might
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> change
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > from
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > > >>> topicName_partitionNumber
> > > > > > > > > >>> > to
> > > > > > > > > >>> > > > >> > > something
> > > > > > > > > >>> > > > >> > > > > like
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > topicID_partitionNumber.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Then
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > it
> > would
> > > > be
> > > > > > > useful
> > > > > > > > > to
> > > > > > > > > >>> have
> > > > > > > > > >>> > > the
> > > > > > > > > >>> > > > >> > topic
> > > > > > > > > >>> > > > >> > > > name
> > > > > > > > > >>> > > > >> > > > > > in
> > > > > > > > > >>> > > > >> > > > > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > file
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > since
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > it
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > would
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > not
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > be
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > in
> the
> > > > > directory
> > > > > > > > > >>> > structure.
> > > > > > > > > >>> > > > >> > > Supporting
> > > > > > > > > >>> > > > >> > > > > > topic
> > > > > > > > > >>> > > > >> > > > > > > > >> renames
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > might
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > be
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > easier
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > if
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> other
> > > > > fields are
> > > > > > > > > >>> included.
> > > > > > > > > >>> > > > Would
> > > > > > > > > >>> > > > >> > > there
> > > > > > > > > >>> > > > >> > > > be
> > > > > > > > > >>> > > > >> > > > > > any
> > > > > > > > > >>> > > > >> > > > > > > > >> > > downsides
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > including
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > this
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > information?
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 14)
> > > Yes,
> > > > we
> > > > > > > would
> > > > > > > > > >>> need to
> > > > > > > > > >>> > > > copy
> > > > > > > > > >>> > > > >> the
> > > > > > > > > >>> > > > >> > > > > > partition
> > > > > > > > > >>> > > > >> > > > > > > > >> > metadata
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > file
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > in
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > this
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > process.
> > > > > I've
> > > > > > > > > updated
> > > > > > > > > >>> the
> > > > > > > > > >>> > > KIP
> > > > > > > > > >>> > > > to
> > > > > > > > > >>> > > > >> > > > include
> > > > > > > > > >>> > > > >> > > > > > > this.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 15)
> I
> > > > > believe
> > > > > > > > Lucas
> > > > > > > > > >>> meant
> > > > > > > > > >>> > v1
> > > > > > > > > >>> > > > >> and v2
> > > > > > > > > >>> > > > >> > > > here.
> > > > > > > > > >>> > > > >> > > > > > He
> > > > > > > > > >>> > > > >> > > > > > > > was
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > referring
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> how
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > requests
> > > > > would
> > > > > > > > fall
> > > > > > > > > >>> under
> > > > > > > > > >>> > > > >> different
> > > > > > > > > >>> > > > >> > > IBP
> > > > > > > > > >>> > > > >> > > > > and
> > > > > > > > > >>> > > > >> > > > > > > > meant
> > > > > > > > > >>> > > > >> > > > > > > > >> > that
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > older
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > brokers
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > would
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > have
> > to
> > > > use
> > > > > the
> > > > > > > > > older
> > > > > > > > > >>> > > version
> > > > > > > > > >>> > > > of
> > > > > > > > > >>> > > > >> > the
> > > > > > > > > >>> > > > >> > > > > > request
> > > > > > > > > >>> > > > >> > > > > > > > and
> > > > > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > existing
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > topic
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > deletion
> > > > > > > process.
> > > > > > > > At
> > > > > > > > > >>> > first,
> > > > > > > > > >>> > > it
> > > > > > > > > >>> > > > >> > seemed
> > > > > > > > > >>> > > > >> > > > > like
> > > > > > > > > >>> > > > >> > > > > > > > tagged
> > > > > > > > > >>> > > > >> > > > > > > > >> > > fields
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > would
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > resolve
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > the
> > IBP
> > > > > issue.
> > > > > > > > > >>> However, we
> > > > > > > > > >>> > > may
> > > > > > > > > >>> > > > >> need
> > > > > > > > > >>> > > > >> > > IBP
> > > > > > > > > >>> > > > >> > > > > for
> > > > > > > > > >>> > > > >> > > > > > > > this
> > > > > > > > > >>> > > > >> > > > > > > > >> > > request
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > after
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > all
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > since
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > the
> > > > > controller
> > > > > > > > > >>> handles the
> > > > > > > > > >>> > > > topic
> > > > > > > > > >>> > > > >> > > > deletion
> > > > > > > > > >>> > > > >> > > > > > > > >> > differently
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> depending
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > on
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > IBP
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > version.
> > > > In
> > > > > an
> > > > > > > > older
> > > > > > > > > >>> > > version,
> > > > > > > > > >>> > > > we
> > > > > > > > > >>> > > > >> > > can't
> > > > > > > > > >>> > > > >> > > > > just
> > > > > > > > > >>> > > > >> > > > > > > > send
> > > > > > > > > >>> > > > >> > > > > > > > >> a
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > StopReplica
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > delete
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> topic
> > > > > > > immediately
> > > > > > > > > like
> > > > > > > > > >>> > we'd
> > > > > > > > > >>> > > > >> want to
> > > > > > > > > >>> > > > >> > > for
> > > > > > > > > >>> > > > >> > > > > > this
> > > > > > > > > >>> > > > >> > > > > > > > KIP.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > This
> > > makes
> > > > > me
> > > > > > > > wonder
> > > > > > > > > >>> if we
> > > > > > > > > >>> > > > want
> > > > > > > > > >>> > > > >> > > tagged
> > > > > > > > > >>> > > > >> > > > > > fields
> > > > > > > > > >>> > > > >> > > > > > > > on
> > > > > > > > > >>> > > > >> > > > > > > > >> all
> > > > > > > > > >>> > > > >> > > > > > > > >> > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> requests
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > after
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > all.
> > Let
> > > > me
> > > > > know
> > > > > > > > > your
> > > > > > > > > >>> > > > thoughts!
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > Justine
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > On
> > Tue,
> > > > Sep
> > > > > 15,
> > > > > > > > 2020
> > > > > > > > > >>> at
> > > > > > > > > >>> > 1:03
> > > > > > > > > >>> > > > PM
> > > > > > > > > >>> > > > >> > > Justine
> > > > > > > > > >>> > > > >> > > > > > > Olshan
> > > > > > > > > >>> > > > >> > > > > > > > <
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > jolshan@confluent.io
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> wrote:
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Hi
> > all,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Jun
> > > > > brought up
> > > > > > > a
> > > > > > > > > good
> > > > > > > > > >>> > point
> > > > > > > > > >>> > > > in
> > > > > > > > > >>> > > > >> his
> > > > > > > > > >>> > > > >> > > > last
> > > > > > > > > >>> > > > >> > > > > > > email
> > > > > > > > > >>> > > > >> > > > > > > > >> about
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > tagged
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > fields,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > and
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> I've
> > > > > updated
> > > > > > > the
> > > > > > > > > KIP
> > > > > > > > > >>> to
> > > > > > > > > >>> > > > reflect
> > > > > > > > > >>> > > > >> > that
> > > > > > > > > >>> > > > >> > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> changes to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > requests
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > responses
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> will
> > be
> > > > in
> > > > > the
> > > > > > > > form
> > > > > > > > > >>> of
> > > > > > > > > >>> > > tagged
> > > > > > > > > >>> > > > >> > fields
> > > > > > > > > >>> > > > >> > > > to
> > > > > > > > > >>> > > > >> > > > > > > avoid
> > > > > > > > > >>> > > > >> > > > > > > > >> > > changing
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > IBP.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> Jun:
> > I
> > > > > plan on
> > > > > > > > > >>> sending a
> > > > > > > > > >>> > > > >> followup
> > > > > > > > > >>> > > > >> > > > email
> > > > > > > > > >>> > > > >> > > > > to
> > > > > > > > > >>> > > > >> > > > > > > > >> address
> > > > > > > > > >>> > > > >> > > > > > > > >> > > some
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > of
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > other
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > points.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > Thanks,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > Justine
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> On
> > Mon,
> > > > > Sep 14,
> > > > > > > > > 2020
> > > > > > > > > >>> at
> > > > > > > > > >>> > > 4:25
> > > > > > > > > >>> > > > PM
> > > > > > > > > >>> > > > >> > Jun
> > > > > > > > > >>> > > > >> > > > Rao
> > > > > > > > > >>> > > > >> > > > > <
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > jun@confluent.io>
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > wrote:
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> Hi,
> > > > > Justine,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > Thanks
> > > > > for the
> > > > > > > > > >>> updated
> > > > > > > > > >>> > > KIP.
> > > > > > > > > >>> > > > A
> > > > > > > > > >>> > > > >> few
> > > > > > > > > >>> > > > >> > > > > > comments
> > > > > > > > > >>> > > > >> > > > > > > > >> below.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> 10.
> > > > > > > LeaderAndIsr
> > > > > > > > > >>> > Response:
> > > > > > > > > >>> > > > Do
> > > > > > > > > >>> > > > >> we
> > > > > > > > > >>> > > > >> > > need
> > > > > > > > > >>> > > > >> > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > topic
> > > > > > > > > >>> > > > >> > > > > > > > >> > > name?
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> 11.
> > > For
> > > > > the
> > > > > > > > > changed
> > > > > > > > > >>> > > > >> > > request/response,
> > > > > > > > > >>> > > > >> > > > > > other
> > > > > > > > > >>> > > > >> > > > > > > > >> than
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> LeaderAndIsr,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > UpdateMetadata,
> > > > > > > > > >>> > Metadata,
> > > > > > > > > >>> > > do
> > > > > > > > > >>> > > > >> we
> > > > > > > > > >>> > > > >> > > need
> > > > > > > > > >>> > > > >> > > > to
> > > > > > > > > >>> > > > >> > > > > > > > include
> > > > > > > > > >>> > > > >> > > > > > > > >> > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > topic
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> name?
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> 12.
> > It
> > > > > seems
> > > > > > > > that
> > > > > > > > > >>> > upgrades
> > > > > > > > > >>> > > > >> don't
> > > > > > > > > >>> > > > >> > > > > require
> > > > > > > > > >>> > > > >> > > > > > > IBP.
> > > > > > > > > >>> > > > >> > > > > > > > >> Does
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > that
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > mean
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > new
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > fields
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> in
> > all
> > > > the
> > > > > > > > > >>> > > request/response
> > > > > > > > > >>> > > > >> are
> > > > > > > > > >>> > > > >> > > added
> > > > > > > > > >>> > > > >> > > > > as
> > > > > > > > > >>> > > > >> > > > > > > > tagged
> > > > > > > > > >>> > > > >> > > > > > > > >> > > fields
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> without
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > bumping
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > up
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> the
> > > > > request
> > > > > > > > > >>> version? It
> > > > > > > > > >>> > > > would
> > > > > > > > > >>> > > > >> be
> > > > > > > > > >>> > > > >> > > > useful
> > > > > > > > > >>> > > > >> > > > > > to
> > > > > > > > > >>> > > > >> > > > > > > > make
> > > > > > > > > >>> > > > >> > > > > > > > >> > that
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > clear.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> 13.
> > > > > Partition
> > > > > > > > > >>> Metadata
> > > > > > > > > >>> > > file:
> > > > > > > > > >>> > > > >> Do
> > > > > > > > > >>> > > > >> > we
> > > > > > > > > >>> > > > >> > > > need
> > > > > > > > > >>> > > > >> > > > > > to
> > > > > > > > > >>> > > > >> > > > > > > > >> include
> > > > > > > > > >>> > > > >> > > > > > > > >> > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > topic
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > name
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > and
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > partition
> > > > > id
> > > > > > > > since
> > > > > > > > > >>> they
> > > > > > > > > >>> > > are
> > > > > > > > > >>> > > > >> > implied
> > > > > > > > > >>> > > > >> > > > in
> > > > > > > > > >>> > > > >> > > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > directory
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > name?
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> 14.
> > In
> > > > the
> > > > > > > JBOD
> > > > > > > > > >>> mode, we
> > > > > > > > > >>> > > > >> support
> > > > > > > > > >>> > > > >> > > > > moving a
> > > > > > > > > >>> > > > >> > > > > > > > >> > > partition's
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > data
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> from
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > one
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > disk
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > another.
> > > > > Will
> > > > > > > > the
> > > > > > > > > >>> new
> > > > > > > > > >>> > > > >> partition
> > > > > > > > > >>> > > > >> > > > > metadata
> > > > > > > > > >>> > > > >> > > > > > > file
> > > > > > > > > >>> > > > >> > > > > > > > >> be
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > copied
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> during
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > process?
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> 15.
> > > The
> > > > > KIP
> > > > > > > says
> > > > > > > > > >>> "Remove
> > > > > > > > > >>> > > > >> deleted
> > > > > > > > > >>> > > > >> > > > topics
> > > > > > > > > >>> > > > >> > > > > > > from
> > > > > > > > > >>> > > > >> > > > > > > > >> > > replicas
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > by
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > sending
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > StopReplicaRequest
> > > > > > > > > >>> V2
> > > > > > > > > >>> > for
> > > > > > > > > >>> > > > any
> > > > > > > > > >>> > > > >> > > topics
> > > > > > > > > >>> > > > >> > > > > > which
> > > > > > > > > >>> > > > >> > > > > > > do
> > > > > > > > > >>> > > > >> > > > > > > > >> not
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > contain
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > a
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > topic
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > ID,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > and
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> V3
> > for
> > > > any
> > > > > > > > topics
> > > > > > > > > >>> which
> > > > > > > > > >>> > do
> > > > > > > > > >>> > > > >> > contain
> > > > > > > > > >>> > > > >> > > a
> > > > > > > > > >>> > > > >> > > > > > topic
> > > > > > > > > >>> > > > >> > > > > > > > >> ID.".
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > However,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > it
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > seems
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > updated
> > > > > > > > controller
> > > > > > > > > >>> will
> > > > > > > > > >>> > > > create
> > > > > > > > > >>> > > > >> > all
> > > > > > > > > >>> > > > >> > > > > > missing
> > > > > > > > > >>> > > > >> > > > > > > > >> topic
> > > > > > > > > >>> > > > >> > > > > > > > >> > IDs
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > first
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > before
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > doing
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > other
> > > > > actions.
> > > > > > > > So,
> > > > > > > > > >>> is
> > > > > > > > > >>> > > > >> > > > > StopReplicaRequest
> > > > > > > > > >>> > > > >> > > > > > V2
> > > > > > > > > >>> > > > >> > > > > > > > >> > needed?
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> Jun
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> On
> > > Fri,
> > > > > Sep
> > > > > > > 11,
> > > > > > > > > >>> 2020 at
> > > > > > > > > >>> > > > 10:31
> > > > > > > > > >>> > > > >> AM
> > > > > > > > > >>> > > > >> > > John
> > > > > > > > > >>> > > > >> > > > > > > > Roesler <
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > vvcephei@apache.org
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > wrote:
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > Thanks,
> > > > > > > > Justine!
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > Your
> > > > > > > response
> > > > > > > > > >>> seems
> > > > > > > > > >>> > > > >> compelling
> > > > > > > > > >>> > > > >> > to
> > > > > > > > > >>> > > > >> > > > me.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > -John
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> On
> > > > Fri,
> > > > > > > > > >>> 2020-09-11 at
> > > > > > > > > >>> > > > 10:17
> > > > > > > > > >>> > > > >> > > -0700,
> > > > > > > > > >>> > > > >> > > > > > > Justine
> > > > > > > > > >>> > > > >> > > > > > > > >> > Olshan
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > wrote:
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > > > Hello
> > > > > all,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > > > > Thanks for
> > > > > > > > > >>> > continuing
> > > > > > > > > >>> > > > the
> > > > > > > > > >>> > > > >> > > > > > discussion! I
> > > > > > > > > >>> > > > >> > > > > > > > >> have a
> > > > > > > > > >>> > > > >> > > > > > > > >> > > few
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > responses
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > your
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > points.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > > Tom:
> > > > > You
> > > > > > > are
> > > > > > > > > >>> correct
> > > > > > > > > >>> > > in
> > > > > > > > > >>> > > > >> that
> > > > > > > > > >>> > > > >> > > this
> > > > > > > > > >>> > > > >> > > > > KIP
> > > > > > > > > >>> > > > >> > > > > > > has
> > > > > > > > > >>> > > > >> > > > > > > > >> not
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > mentioned
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > > > > > > > > >>> > DeleteTopicsRequest. I
> > > > > > > > > >>> > > > >> think
> > > > > > > > > >>> > > > >> > > that
> > > > > > > > > >>> > > > >> > > > > > this
> > > > > > > > > >>> > > > >> > > > > > > > >> would
> > > > > > > > > >>> > > > >> > > > > > > > >> > be
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > out
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > of
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > scope
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > for
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> now,
> > > but
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > > may
> > > > be
> > > > > > > > > something
> > > > > > > > > >>> > worth
> > > > > > > > > >>> > > > >> adding
> > > > > > > > > >>> > > > >> > > in
> > > > > > > > > >>> > > > >> > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> future.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > > > John:
> > > > > We
> > > > > > > did
> > > > > > > > > >>> > consider
> > > > > > > > > >>> > > > >> > sequence
> > > > > > > > > >>> > > > >> > > > ids,
> > > > > > > > > >>> > > > >> > > > > > but
> > > > > > > > > >>> > > > >> > > > > > > > >> there
> > > > > > > > > >>> > > > >> > > > > > > > >> > > are
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > a
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > few
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > reasons
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > favor
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > > > UUIDs.
> > > > > > > There
> > > > > > > > > are
> > > > > > > > > >>> > > several
> > > > > > > > > >>> > > > >> > cases
> > > > > > > > > >>> > > > >> > > > > where
> > > > > > > > > >>> > > > >> > > > > > > > topics
> > > > > > > > > >>> > > > >> > > > > > > > >> > from
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> different
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > clusters
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> may
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > > > > interact
> > > > > > > now
> > > > > > > > > >>> and in
> > > > > > > > > >>> > > the
> > > > > > > > > >>> > > > >> > future.
> > > > > > > > > >>> > > > >> > > > For
> > > > > > > > > >>> > > > >> > > > > > > > >> example,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > Mirror
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Maker 2
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > may
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > benefit
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > > from
> > > > > being
> > > > > > > > > able
> > > > > > > > > >>> to
> > > > > > > > > >>> > > > detect
> > > > > > > > > >>> > > > >> > when
> > > > > > > > > >>> > > > >> > > a
> > > > > > > > > >>> > > > >> > > > > > > cluster
> > > > > > > > > >>> > > > >> > > > > > > > >> being
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > mirrored
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> is
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > deleted
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> and
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > > > > recreated
> > > > > > > > and
> > > > > > > > > >>> > globally
> > > > > > > > > >>> > > > >> unique
> > > > > > > > > >>> > > > >> > > > > > > identifiers
> > > > > > > > > >>> > > > >> > > > > > > > >> > would
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > make
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > resolving
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > issues
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > > > easier
> > > > > > > than
> > > > > > > > > >>> sequence
> > > > > > > > > >>> > > IDs
> > > > > > > > > >>> > > > >> > which
> > > > > > > > > >>> > > > >> > > > may
> > > > > > > > > >>> > > > >> > > > > > > > collide
> > > > > > > > > >>> > > > >> > > > > > > > >> > > between
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > clusters.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> KIP-405
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > > > > (tiered
> > > > > > > > > storage)
> > > > > > > > > >>> > will
> > > > > > > > > >>> > > > also
> > > > > > > > > >>> > > > >> > > > benefit
> > > > > > > > > >>> > > > >> > > > > > from
> > > > > > > > > >>> > > > >> > > > > > > > >> > globally
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > unique
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> IDs
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > as
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > shared
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > > > > buckets
> > > > > > > may
> > > > > > > > be
> > > > > > > > > >>> used
> > > > > > > > > >>> > > > >> between
> > > > > > > > > >>> > > > >> > > > > clusters.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > > > > Globally
> > > > > > > > > unique
> > > > > > > > > >>> IDs
> > > > > > > > > >>> > > > would
> > > > > > > > > >>> > > > >> > also
> > > > > > > > > >>> > > > >> > > > make
> > > > > > > > > >>> > > > >> > > > > > > > >> > > functionality
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > like
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > moving
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > topics
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > > > > between
> > > > > > > > > >>> disparate
> > > > > > > > > >>> > > > clusters
> > > > > > > > > >>> > > > >> > > easier
> > > > > > > > > >>> > > > >> > > > > in
> > > > > > > > > >>> > > > >> > > > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > future,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> simplify
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > any
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > future
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > > > > > > > > implementations
> > > > > > > > > >>> of
> > > > > > > > > >>> > > > backups
> > > > > > > > > >>> > > > >> > and
> > > > > > > > > >>> > > > >> > > > > > > restores,
> > > > > > > > > >>> > > > >> > > > > > > > >> and
> > > > > > > > > >>> > > > >> > > > > > > > >> > > more.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > In
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > general,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > unique
> > > > IDs
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > > > would
> > > > > > > ensure
> > > > > > > > > >>> that
> > > > > > > > > >>> > the
> > > > > > > > > >>> > > > >> source
> > > > > > > > > >>> > > > >> > > > > cluster
> > > > > > > > > >>> > > > >> > > > > > > > >> topics do
> > > > > > > > > >>> > > > >> > > > > > > > >> > > not
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> conflict
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > with
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > > > > > > destination
> > > > > > > > > >>> cluster
> > > > > > > > > >>> > > > >> topics.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > If
> > > > we
> > > > > were
> > > > > > > > to
> > > > > > > > > >>> use
> > > > > > > > > >>> > > > sequence
> > > > > > > > > >>> > > > >> > ids,
> > > > > > > > > >>> > > > >> > > > we
> > > > > > > > > >>> > > > >> > > > > > > would
> > > > > > > > > >>> > > > >> > > > > > > > >> need
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> sufficiently
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > large
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > cluster
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > > ids
> > > > > to be
> > > > > > > > > stored
> > > > > > > > > >>> > with
> > > > > > > > > >>> > > > the
> > > > > > > > > >>> > > > >> > topic
> > > > > > > > > >>> > > > >> > > > > > > > >> identifiers or
> > > > > > > > > >>> > > > >> > > > > > > > >> > > we
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > run
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > risk
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > > > > > > collisions.
> > > > > > > > > This
> > > > > > > > > >>> > will
> > > > > > > > > >>> > > > >> give up
> > > > > > > > > >>> > > > >> > > any
> > > > > > > > > >>> > > > >> > > > > > > > >> advantage in
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> compactness
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > sequence
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > > > > numbers
> > > > > > > may
> > > > > > > > > >>> bring.
> > > > > > > > > >>> > > Given
> > > > > > > > > >>> > > > >> > these
> > > > > > > > > >>> > > > >> > > > > > > > advantages I
> > > > > > > > > >>> > > > >> > > > > > > > >> > > think
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > it
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> makes
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > sense
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> use
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > > > UUIDs.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > > > Gokul:
> > > > > > > This
> > > > > > > > is
> > > > > > > > > >>> an
> > > > > > > > > >>> > > > >> interesting
> > > > > > > > > >>> > > > >> > > > idea,
> > > > > > > > > >>> > > > >> > > > > > but
> > > > > > > > > >>> > > > >> > > > > > > > >> this
> > > > > > > > > >>> > > > >> > > > > > > > >> > is
> > > > > > > > > >>> > > > >> > > > > > > > >> > > a
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> breaking
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > change.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> Out
> > of
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > > > scope
> > > > > for
> > > > > > > > now,
> > > > > > > > > >>> but
> > > > > > > > > >>> > > maybe
> > > > > > > > > >>> > > > >> > worth
> > > > > > > > > >>> > > > >> > > > > > > discussing
> > > > > > > > > >>> > > > >> > > > > > > > >> in
> > > > > > > > > >>> > > > >> > > > > > > > >> > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > future.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > > Hope
> > > > > this
> > > > > > > > > >>> explains
> > > > > > > > > >>> > > some
> > > > > > > > > >>> > > > of
> > > > > > > > > >>> > > > >> > the
> > > > > > > > > >>> > > > >> > > > > > > decisions,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > > > > Justine
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > On
> > > > > Fri,
> > > > > > > Sep
> > > > > > > > > 11,
> > > > > > > > > >>> 2020
> > > > > > > > > >>> > > at
> > > > > > > > > >>> > > > >> 8:27
> > > > > > > > > >>> > > > >> > AM
> > > > > > > > > >>> > > > >> > > > > Gokul
> > > > > > > > > >>> > > > >> > > > > > > > >> Ramanan
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Subramanian <
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > > > > > > > > >>> > gokul2411s@gmail.com>
> > > > > > > > > >>> > > > >> wrote:
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > > Hi.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > > > Thanks
> > > > > > > for
> > > > > > > > > the
> > > > > > > > > >>> > KIP.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > > > Have you
> > > > > > > > > >>> thought
> > > > > > > > > >>> > > about
> > > > > > > > > >>> > > > >> > > whether
> > > > > > > > > >>> > > > >> > > > it
> > > > > > > > > >>> > > > >> > > > > > > makes
> > > > > > > > > >>> > > > >> > > > > > > > >> > sense
> > > > > > > > > >>> > > > >> > > > > > > > >> > > to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> support
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > authorizing a
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > > > > > principal
> > > > > > > > > for
> > > > > > > > > >>> a
> > > > > > > > > >>> > > topic
> > > > > > > > > >>> > > > ID
> > > > > > > > > >>> > > > >> > > rather
> > > > > > > > > >>> > > > >> > > > > > than
> > > > > > > > > >>> > > > >> > > > > > > a
> > > > > > > > > >>> > > > >> > > > > > > > >> topic
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > name
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > achieve
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > tighter
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > > > > > security?
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > Or
> > > > > is
> > > > > > > the
> > > > > > > > > >>> topic ID
> > > > > > > > > >>> > > > >> > > > fundamentally
> > > > > > > > > >>> > > > >> > > > > an
> > > > > > > > > >>> > > > >> > > > > > > > >> internal
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > detail
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > similar
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > epochs
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > used
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > > in a
> > > > > > > bunch
> > > > > > > > > of
> > > > > > > > > >>> > other
> > > > > > > > > >>> > > > >> places
> > > > > > > > > >>> > > > >> > in
> > > > > > > > > >>> > > > >> > > > > > Kafka?
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > > > Thanks.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > On
> > > > > Fri,
> > > > > > > > Sep
> > > > > > > > > >>> 11,
> > > > > > > > > >>> > 2020
> > > > > > > > > >>> > > > at
> > > > > > > > > >>> > > > >> > 4:06
> > > > > > > > > >>> > > > >> > > PM
> > > > > > > > > >>> > > > >> > > > > > John
> > > > > > > > > >>> > > > >> > > > > > > > >> > Roesler <
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > vvcephei@apache.org
> > > > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > wrote:
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > Hello
> > > > > > > > > >>> Justine,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > Thanks
> > > > > > > > for
> > > > > > > > > >>> the
> > > > > > > > > >>> > > KIP!
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > I
> > > > > > > happen
> > > > > > > > > to
> > > > > > > > > >>> have
> > > > > > > > > >>> > > > been
> > > > > > > > > >>> > > > >> > > > > confronted
> > > > > > > > > >>> > > > >> > > > > > > > >> recently
> > > > > > > > > >>> > > > >> > > > > > > > >> > > with
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> need
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > keep
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > track
> > > > of
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > a
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > large
> > > > > > > > > >>> number of
> > > > > > > > > >>> > > > >> topics as
> > > > > > > > > >>> > > > >> > > > > > compactly
> > > > > > > > > >>> > > > >> > > > > > > > as
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > possible. I
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> was
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > going
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> come
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> up
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > with
> > > > > > > > some
> > > > > > > > > >>> way to
> > > > > > > > > >>> > > > >> > dictionary
> > > > > > > > > >>> > > > >> > > > > > encode
> > > > > > > > > >>> > > > >> > > > > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > topic
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > names
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> as
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> integers,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> but
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > this
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > seems
> > > > > > > > much
> > > > > > > > > >>> > better!
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > > > > Apologies
> > > > > > > > > if
> > > > > > > > > >>> > this
> > > > > > > > > >>> > > > has
> > > > > > > > > >>> > > > >> > been
> > > > > > > > > >>> > > > >> > > > > raised
> > > > > > > > > >>> > > > >> > > > > > > > >> before,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > but
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > I’m
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > wondering
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > about
> > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > choice
> > > > > > > > of
> > > > > > > > > >>> UUID
> > > > > > > > > >>> > vs
> > > > > > > > > >>> > > > >> > sequence
> > > > > > > > > >>> > > > >> > > > > number
> > > > > > > > > >>> > > > >> > > > > > > for
> > > > > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > ids.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Typically,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > I’ve
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> seen
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > UUIDs
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > in two
> > > > > > > > > >>> > situations:
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > 1.
> > > > > > > When
> > > > > > > > > >>> > processes
> > > > > > > > > >>> > > > >> need to
> > > > > > > > > >>> > > > >> > > > > > generate
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > non-colliding
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> identifiers
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > without
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > > > > > >>> coordination.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > 2.
> > > > > > > When
> > > > > > > > > the
> > > > > > > > > >>> > > > identifier
> > > > > > > > > >>> > > > >> > > needs
> > > > > > > > > >>> > > > >> > > > to
> > > > > > > > > >>> > > > >> > > > > > be
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > “universally
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > unique”;
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > I.e.,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > > > > identifier
> > > > > > > > > >>> needs
> > > > > > > > > >>> > > to
> > > > > > > > > >>> > > > >> > > > distinguish
> > > > > > > > > >>> > > > >> > > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> entity
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > from
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > all
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > other
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > entities
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > that
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > could
> > > > > > > > ever
> > > > > > > > > >>> > exist.
> > > > > > > > > >>> > > > >> This is
> > > > > > > > > >>> > > > >> > > > > useful
> > > > > > > > > >>> > > > >> > > > > > in
> > > > > > > > > >>> > > > >> > > > > > > > >> cases
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > where
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > entities
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > from
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > all
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > kinds
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > of
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > > > systems
> > > > > > > > > get
> > > > > > > > > >>> > mixed
> > > > > > > > > >>> > > > >> > together,
> > > > > > > > > >>> > > > >> > > > > such
> > > > > > > > > >>> > > > >> > > > > > as
> > > > > > > > > >>> > > > >> > > > > > > > >> when
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > dumping
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> logs
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > from
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > all
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > processes
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > in
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > a
> > > > > > > > company
> > > > > > > > > >>> into a
> > > > > > > > > >>> > > > >> common
> > > > > > > > > >>> > > > >> > > > system.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > Maybe
> > > > > > > > I’m
> > > > > > > > > >>> being
> > > > > > > > > >>> > > > >> > > > short-sighted,
> > > > > > > > > >>> > > > >> > > > > > but
> > > > > > > > > >>> > > > >> > > > > > > it
> > > > > > > > > >>> > > > >> > > > > > > > >> > > doesn’t
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > seem
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> like
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > either
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > really
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > > > applies
> > > > > > > > > >>> here. It
> > > > > > > > > >>> > > > seems
> > > > > > > > > >>> > > > >> > like
> > > > > > > > > >>> > > > >> > > > the
> > > > > > > > > >>> > > > >> > > > > > > > brokers
> > > > > > > > > >>> > > > >> > > > > > > > >> > > could
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > and
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> would
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > achieve
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > consensus
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > when
> > > > > > > > > >>> creating a
> > > > > > > > > >>> > > > topic
> > > > > > > > > >>> > > > >> > > anyway,
> > > > > > > > > >>> > > > >> > > > > > which
> > > > > > > > > >>> > > > >> > > > > > > > is
> > > > > > > > > >>> > > > >> > > > > > > > >> all
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that’s
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > required
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > generate
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > > > > > >>> non-colliding
> > > > > > > > > >>> > > > sequence
> > > > > > > > > >>> > > > >> > ids.
> > > > > > > > > >>> > > > >> > > > For
> > > > > > > > > >>> > > > >> > > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > second,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > as
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > you
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > mention,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > we
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > could
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > > > assign
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > a
> > > > > UUID
> > > > > > > > to
> > > > > > > > > >>> the
> > > > > > > > > >>> > > > cluster
> > > > > > > > > >>> > > > >> as
> > > > > > > > > >>> > > > >> > a
> > > > > > > > > >>> > > > >> > > > > whole,
> > > > > > > > > >>> > > > >> > > > > > > > which
> > > > > > > > > >>> > > > >> > > > > > > > >> > > would
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > render
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > any
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> resource
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > scoped
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > the
> > > > > > > > broker
> > > > > > > > > >>> > > > universally
> > > > > > > > > >>> > > > >> > > unique
> > > > > > > > > >>> > > > >> > > > > as
> > > > > > > > > >>> > > > >> > > > > > > > well.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > The
> > > > > > > > > reason I
> > > > > > > > > >>> > > mention
> > > > > > > > > >>> > > > >> this
> > > > > > > > > >>> > > > >> > > is
> > > > > > > > > >>> > > > >> > > > > > that,
> > > > > > > > > >>> > > > >> > > > > > > > >> > although
> > > > > > > > > >>> > > > >> > > > > > > > >> > > a
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > UUID
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> is
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > way
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > more
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > compact
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > than
> > > > > > > > topic
> > > > > > > > > >>> > names,
> > > > > > > > > >>> > > > it’s
> > > > > > > > > >>> > > > >> > > still
> > > > > > > > > >>> > > > >> > > > 16
> > > > > > > > > >>> > > > >> > > > > > > > bytes.
> > > > > > > > > >>> > > > >> > > > > > > > >> In
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > contrast,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> a
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > 4-byte
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > integer
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > > > sequence
> > > > > > > > > id
> > > > > > > > > >>> > would
> > > > > > > > > >>> > > > give
> > > > > > > > > >>> > > > >> > us 4
> > > > > > > > > >>> > > > >> > > > > > billion
> > > > > > > > > >>> > > > >> > > > > > > > >> unique
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > topics
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> per
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > cluster,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > which
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > > > seems
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > like
> > > > > > > > > enough
> > > > > > > > > >>> ;)
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > > > > > Considering
> > > > > > > > > >>> the
> > > > > > > > > >>> > > > >> number of
> > > > > > > > > >>> > > > >> > > > > > different
> > > > > > > > > >>> > > > >> > > > > > > > >> times
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > these
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> topic
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > identifiers
> > > > > > > are
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > > sent
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > over
> > > > > > > the
> > > > > > > > > >>> wire or
> > > > > > > > > >>> > > > >> stored
> > > > > > > > > >>> > > > >> > in
> > > > > > > > > >>> > > > >> > > > > > memory,
> > > > > > > > > >>> > > > >> > > > > > > it
> > > > > > > > > >>> > > > >> > > > > > > > >> > seems
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > like
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > it
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > might
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > be
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > worth
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > > > > additional
> > > > > > > > > >>> 4x
> > > > > > > > > >>> > > space
> > > > > > > > > >>> > > > >> > > savings.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > What
> > > > > > > do
> > > > > > > > > you
> > > > > > > > > >>> > think
> > > > > > > > > >>> > > > >> about
> > > > > > > > > >>> > > > >> > > this?
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > > > Thanks,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > John
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > On
> > > > > > > Fri,
> > > > > > > > > Sep
> > > > > > > > > >>> 11,
> > > > > > > > > >>> > > > 2020,
> > > > > > > > > >>> > > > >> at
> > > > > > > > > >>> > > > >> > > > 03:20,
> > > > > > > > > >>> > > > >> > > > > > Tom
> > > > > > > > > >>> > > > >> > > > > > > > >> > Bentley
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > wrote:
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > Hi
> > > > > > > > > >>> Justine,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > This
> > > > > > > > > looks
> > > > > > > > > >>> > like
> > > > > > > > > >>> > > a
> > > > > > > > > >>> > > > >> very
> > > > > > > > > >>> > > > >> > > > > welcome
> > > > > > > > > >>> > > > >> > > > > > > > >> > > improvement.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Thanks!
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > > Maybe
> > > > > > > > I
> > > > > > > > > >>> missed
> > > > > > > > > >>> > > it,
> > > > > > > > > >>> > > > >> but
> > > > > > > > > >>> > > > >> > > the
> > > > > > > > > >>> > > > >> > > > > KIP
> > > > > > > > > >>> > > > >> > > > > > > > >> doesn't
> > > > > > > > > >>> > > > >> > > > > > > > >> > > seem
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > mention
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> changing
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > > > > >>> > > > DeleteTopicsRequest
> > > > > > > > > >>> > > > >> to
> > > > > > > > > >>> > > > >> > > > > identify
> > > > > > > > > >>> > > > >> > > > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > topic
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > using
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > an
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > id.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Maybe
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > that's
> > > > > out
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > of
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > > scope,
> > > > > > > > > but
> > > > > > > > > >>> > > > >> > > > > DeleteTopicsRequest
> > > > > > > > > >>> > > > >> > > > > > is
> > > > > > > > > >>> > > > >> > > > > > > > not
> > > > > > > > > >>> > > > >> > > > > > > > >> > > listed
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > among
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Future
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> Work
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > APIs
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > > > either.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > Kind
> > > > > > > > > >>> regards,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > Tom
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > On
> > > > > > > > Thu,
> > > > > > > > > >>> Sep
> > > > > > > > > >>> > 10,
> > > > > > > > > >>> > > > >> 2020 at
> > > > > > > > > >>> > > > >> > > > 3:59
> > > > > > > > > >>> > > > >> > > > > PM
> > > > > > > > > >>> > > > >> > > > > > > > >> Satish
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > Duggana <
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > > > > > > > >>> > > > >> satish.duggana@gmail.com
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > > wrote:
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > Thanks
> > > > > > > > > >>> > > > >> Lucas/Justine
> > > > > > > > > >>> > > > >> > > for
> > > > > > > > > >>> > > > >> > > > > the
> > > > > > > > > >>> > > > >> > > > > > > nice
> > > > > > > > > >>> > > > >> > > > > > > > >> KIP.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > It
> > > > > > > > has
> > > > > > > > > >>> > several
> > > > > > > > > >>> > > > >> > benefits
> > > > > > > > > >>> > > > >> > > > > which
> > > > > > > > > >>> > > > >> > > > > > > > also
> > > > > > > > > >>> > > > >> > > > > > > > >> > > include
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > simplifying
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > topic
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > deletion
> > > > > > > > > >>> > > process
> > > > > > > > > >>> > > > >> by
> > > > > > > > > >>> > > > >> > > > > > controller
> > > > > > > > > >>> > > > >> > > > > > > > and
> > > > > > > > > >>> > > > >> > > > > > > > >> > logs
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > cleanup
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> by
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > brokers
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > in
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > corner
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > cases.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > Best,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > Satish.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > On
> > > > > > > > > Wed,
> > > > > > > > > >>> Sep
> > > > > > > > > >>> > 9,
> > > > > > > > > >>> > > > >> 2020
> > > > > > > > > >>> > > > >> > at
> > > > > > > > > >>> > > > >> > > > > 10:07
> > > > > > > > > >>> > > > >> > > > > > PM
> > > > > > > > > >>> > > > >> > > > > > > > >> > Justine
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Olshan <
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > > > >>> jolshan@confluent.io
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > wrote:
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > > > Hello
> > > > > > > > > >>> all,
> > > > > > > > > >>> > > > it's
> > > > > > > > > >>> > > > >> > been
> > > > > > > > > >>> > > > >> > > > > > almost a
> > > > > > > > > >>> > > > >> > > > > > > > >> year!
> > > > > > > > > >>> > > > >> > > > > > > > >> > > I've
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > made
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > some
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> changes
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > this
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > KIP
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > and
> > > > > > > > > >>> hope to
> > > > > > > > > >>> > > > >> continue
> > > > > > > > > >>> > > > >> > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> discussion.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > > One
> > > > > > > > > >>> of the
> > > > > > > > > >>> > > > main
> > > > > > > > > >>> > > > >> > > changes
> > > > > > > > > >>> > > > >> > > > > > I've
> > > > > > > > > >>> > > > >> > > > > > > > >> added
> > > > > > > > > >>> > > > >> > > > > > > > >> > is
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > now
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > metadata
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > response
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > > will
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > include
> > > > > > > > > >>> the
> > > > > > > > > >>> > > > topic
> > > > > > > > > >>> > > > >> ID
> > > > > > > > > >>> > > > >> > > (as
> > > > > > > > > >>> > > > >> > > > > > Colin
> > > > > > > > > >>> > > > >> > > > > > > > >> > > suggested).
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Clients
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > can
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > obtain
> > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > > > topicID
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > of a
> > > > > > > > > >>> given
> > > > > > > > > >>> > > topic
> > > > > > > > > >>> > > > >> > > through
> > > > > > > > > >>> > > > >> > > > a
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > TopicDescription.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > The
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > topicId
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > will
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > also
> > > > be
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > included
> > > > > > > > > >>> > with
> > > > > > > > > >>> > > > the
> > > > > > > > > >>> > > > >> > > > > > > UpdateMetadata
> > > > > > > > > >>> > > > >> > > > > > > > >> > > request.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > > Let
> > > > > > > > > me
> > > > > > > > > >>> > know
> > > > > > > > > >>> > > > what
> > > > > > > > > >>> > > > >> > you
> > > > > > > > > >>> > > > >> > > > all
> > > > > > > > > >>> > > > >> > > > > > > think.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > > > Thank
> > > > > > > > > >>> you,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > > > >>> Justine
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > On
> > > > > > > > > >>> > > 2019/09/13
> > > > > > > > > >>> > > > >> > > 16:38:26,
> > > > > > > > > >>> > > > >> > > > > > > "Colin
> > > > > > > > > >>> > > > >> > > > > > > > >> > > McCabe" <
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > cmccabe@apache.org
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > wrote:
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > Hi
> > > > > > > > > >>> > Lucas,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > >>> Thanks
> > > > > > > > > >>> > for
> > > > > > > > > >>> > > > >> > tackling
> > > > > > > > > >>> > > > >> > > > > this.
> > > > > > > > > >>> > > > >> > > > > > > > >> Topic
> > > > > > > > > >>> > > > >> > > > > > > > >> > IDs
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > are a
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > great
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > idea,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> and
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > this
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > is
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > a
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > really
> > > > > > > > > >>> good
> > > > > > > > > >>> > > > >> writeup.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > For
> > > > > > > > > >>> > > > >> > > > > > > /brokers/topics/[topic],
> > > > > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > schema
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> version
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > should
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > be
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > bumped
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > version
> > > > > > > > > >>> 3,
> > > > > > > > > >>> > > > rather
> > > > > > > > > >>> > > > >> > than
> > > > > > > > > >>> > > > >> > > 2.
> > > > > > > > > >>> > > > >> > > > > > > > KIP-455
> > > > > > > > > >>> > > > >> > > > > > > > >> > > bumped
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > version
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > this
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > znode
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > 2
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > already
> > > > > > > > > >>> :)
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > >>> Given
> > > > > > > > > >>> > that
> > > > > > > > > >>> > > > >> we're
> > > > > > > > > >>> > > > >> > > > going
> > > > > > > > > >>> > > > >> > > > > to
> > > > > > > > > >>> > > > >> > > > > > > be
> > > > > > > > > >>> > > > >> > > > > > > > >> > seeing
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > these
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > things
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > as
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > strings
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> as
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > > lot
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > (in
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > logs,
> > > > > > > > > in
> > > > > > > > > >>> > > > >> ZooKeeper,
> > > > > > > > > >>> > > > >> > on
> > > > > > > > > >>> > > > >> > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > command-line,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > etc.),
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > does
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > it
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > make
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > sense
> > > > to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > use
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > base64
> > > > > > > > > >>> when
> > > > > > > > > >>> > > > >> > converting
> > > > > > > > > >>> > > > >> > > > them
> > > > > > > > > >>> > > > >> > > > > > to
> > > > > > > > > >>> > > > >> > > > > > > > >> > strings?
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > >>> Here is
> > > > > > > > > >>> > an
> > > > > > > > > >>> > > > >> > example
> > > > > > > > > >>> > > > >> > > of
> > > > > > > > > >>> > > > >> > > > > the
> > > > > > > > > >>> > > > >> > > > > > > hex
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> representation:
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > And
> > > > > > > > > >>> here
> > > > > > > > > >>> > > is
> > > > > > > > > >>> > > > an
> > > > > > > > > >>> > > > >> > > > example
> > > > > > > > > >>> > > > >> > > > > in
> > > > > > > > > >>> > > > >> > > > > > > > >> base64.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > >>> > > > >> > > > b8tRS7h4TJ2Vt43Dp85v2A
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > The
> > > > > > > > > >>> > base64
> > > > > > > > > >>> > > > >> > version
> > > > > > > > > >>> > > > >> > > > > saves
> > > > > > > > > >>> > > > >> > > > > > 15
> > > > > > > > > >>> > > > >> > > > > > > > >> > letters
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > (to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > be
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > fair,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > 4
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > those
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > were
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > dashes
> > > > > > > > > >>> that
> > > > > > > > > >>> > we
> > > > > > > > > >>> > > > >> could
> > > > > > > > > >>> > > > >> > > have
> > > > > > > > > >>> > > > >> > > > > > > elided
> > > > > > > > > >>> > > > >> > > > > > > > in
> > > > > > > > > >>> > > > >> > > > > > > > >> > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > hex
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > representation.)
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > >>> Another
> > > > > > > > > >>> > > > thing
> > > > > > > > > >>> > > > >> to
> > > > > > > > > >>> > > > >> > > > > consider
> > > > > > > > > >>> > > > >> > > > > > > is
> > > > > > > > > >>> > > > >> > > > > > > > >> that
> > > > > > > > > >>> > > > >> > > > > > > > >> > we
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > should
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > specify
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > that
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > >>> all-zeroes
> > > > > > > > > >>> > > UUID
> > > > > > > > > >>> > > > is
> > > > > > > > > >>> > > > >> > not
> > > > > > > > > >>> > > > >> > > a
> > > > > > > > > >>> > > > >> > > > > > valid
> > > > > > > > > >>> > > > >> > > > > > > > >> topic
> > > > > > > > > >>> > > > >> > > > > > > > >> > > UUID.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >  We
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > can't
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > use
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> null
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > for
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > > this
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > because
> > > > > > > > > >>> we
> > > > > > > > > >>> > > can't
> > > > > > > > > >>> > > > >> > pass a
> > > > > > > > > >>> > > > >> > > > > null
> > > > > > > > > >>> > > > >> > > > > > > UUID
> > > > > > > > > >>> > > > >> > > > > > > > >> over
> > > > > > > > > >>> > > > >> > > > > > > > >> > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > RPC
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > protocol
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > (there
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> is
> > > no
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > special
> > > > > > > > > >>> > > pattern
> > > > > > > > > >>> > > > >> for
> > > > > > > > > >>> > > > >> > > null,
> > > > > > > > > >>> > > > >> > > > > nor
> > > > > > > > > >>> > > > >> > > > > > > do
> > > > > > > > > >>> > > > >> > > > > > > > we
> > > > > > > > > >>> > > > >> > > > > > > > >> > want
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> waste
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > space
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > reserving
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > > such
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > a
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > >>> pattern.)
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > >>> Maybe I
> > > > > > > > > >>> > > > missed
> > > > > > > > > >>> > > > >> > it,
> > > > > > > > > >>> > > > >> > > > but
> > > > > > > > > >>> > > > >> > > > > > did
> > > > > > > > > >>> > > > >> > > > > > > > you
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > describe
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > "migration
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > of...
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > existing
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > topic[s]
> > > > > > > > > >>> > > without
> > > > > > > > > >>> > > > >> > topic
> > > > > > > > > >>> > > > >> > > > IDs"
> > > > > > > > > >>> > > > >> > > > > > in
> > > > > > > > > >>> > > > >> > > > > > > > >> detail
> > > > > > > > > >>> > > > >> > > > > > > > >> > in
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > any
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > section?
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > It
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > seems
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > like
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > when
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > the
> > > > > > > > > new
> > > > > > > > > >>> > > > controller
> > > > > > > > > >>> > > > >> > > > becomes
> > > > > > > > > >>> > > > >> > > > > > > > active,
> > > > > > > > > >>> > > > >> > > > > > > > >> it
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > should
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> just
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > generate
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > random
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > UUIDs
> > > > > > > > for
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > these,
> > > > > > > > > >>> and
> > > > > > > > > >>> > > write
> > > > > > > > > >>> > > > >> the
> > > > > > > > > >>> > > > >> > > > random
> > > > > > > > > >>> > > > >> > > > > > > UUIDs
> > > > > > > > > >>> > > > >> > > > > > > > >> back
> > > > > > > > > >>> > > > >> > > > > > > > >> > > to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > ZooKeeper.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > It
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > would
> > > be
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > > good
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > spell
> > > > > > > > > >>> that
> > > > > > > > > >>> > > out.
> > > > > > > > > >>> > > > >> We
> > > > > > > > > >>> > > > >> > > > should
> > > > > > > > > >>> > > > >> > > > > > make
> > > > > > > > > >>> > > > >> > > > > > > > it
> > > > > > > > > >>> > > > >> > > > > > > > >> > clear
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> this
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > happens
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > regardless
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > of
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > the
> > > > > > > > > >>> > > inter-broker
> > > > > > > > > >>> > > > >> > > protocol
> > > > > > > > > >>> > > > >> > > > > > > version
> > > > > > > > > >>> > > > >> > > > > > > > >> > (it's
> > > > > > > > > >>> > > > >> > > > > > > > >> > > a
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > compatible
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> change).
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > >>> > > > >> > > "LeaderAndIsrRequests
> > > > > > > > > >>> > > > >> > > > > > > > >> including an
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > is_every_partition
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > flag"
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > > > seems a
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > bit
> > > > > > > > > >>> wordy.
> > > > > > > > > >>> > > Can
> > > > > > > > > >>> > > > we
> > > > > > > > > >>> > > > >> > just
> > > > > > > > > >>> > > > >> > > > > call
> > > > > > > > > >>> > > > >> > > > > > > > these
> > > > > > > > > >>> > > > >> > > > > > > > >> > "full
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > > >>> LeaderAndIsrRequests"?
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > Then
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > RPC
> > > > > > > > > >>> field
> > > > > > > > > >>> > > could
> > > > > > > > > >>> > > > be
> > > > > > > > > >>> > > > >> > > named
> > > > > > > > > >>> > > > >> > > > > > > "full".
> > > > > > > > > >>> > > > >> > > > > > > > >> > Also,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > it
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > would
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > probably
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > be
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > better
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > for
> > > > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > RPC
> > > > > > > > > >>> field to
> > > > > > > > > >>> > > be
> > > > > > > > > >>> > > > an
> > > > > > > > > >>> > > > >> > enum
> > > > > > > > > >>> > > > >> > > > of
> > > > > > > > > >>> > > > >> > > > > {
> > > > > > > > > >>> > > > >> > > > > > > > >> > > UNSPECIFIED,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > INCREMENTAL,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > FULL
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> },
> > so
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > > that
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > we
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > can
> > > > > > > > > >>> cleanly
> > > > > > > > > >>> > > > handle
> > > > > > > > > >>> > > > >> > old
> > > > > > > > > >>> > > > >> > > > > > versions
> > > > > > > > > >>> > > > >> > > > > > > > (by
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > treating
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> them
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > as
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > UNSPECIFIED)
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > In
> > > > > > > > > >>> the
> > > > > > > > > >>> > > > >> > > > > > LeaderAndIsrRequest
> > > > > > > > > >>> > > > >> > > > > > > > >> > section,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > you
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> write
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > "A
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > final
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > deletion
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > event
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > will
> > > > > > > > > be
> > > > > > > > > >>> > > > secheduled
> > > > > > > > > >>> > > > >> > for
> > > > > > > > > >>> > > > >> > > X
> > > > > > > > > >>> > > > >> > > > ms
> > > > > > > > > >>> > > > >> > > > > > > after
> > > > > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > LeaderAndIsrRequest
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > was
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > first
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > >>> received..."
> > > > > > > > > >>> > > I
> > > > > > > > > >>> > > > >> guess
> > > > > > > > > >>> > > > >> > > > the X
> > > > > > > > > >>> > > > >> > > > > > > was a
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > placeholder
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> that
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > you
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > intended
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > > > replace
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > before
> > > > > > > > > >>> > > posting?
> > > > > > > > > >>> > > > :)
> > > > > > > > > >>> > > > >> > In
> > > > > > > > > >>> > > > >> > > > any
> > > > > > > > > >>> > > > >> > > > > > > case,
> > > > > > > > > >>> > > > >> > > > > > > > >> this
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > seems
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > like
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > kind
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > of
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > thing
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > > we'd
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > want a
> > > > > > > > > >>> > > > >> configuration
> > > > > > > > > >>> > > > >> > > for.
> > > > > > > > > >>> > > > >> > > > > > > Let's
> > > > > > > > > >>> > > > >> > > > > > > > >> > > describe
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > configuration
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> key
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > > > > somewhere
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > in
> > > > > > > > > this
> > > > > > > > > >>> KIP,
> > > > > > > > > >>> > > > >> > including
> > > > > > > > > >>> > > > >> > > > what
> > > > > > > > > >>> > > > >> > > > > > its
> > > > > > > > > >>> > > > >> > > > > > > > >> > default
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > value
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> is.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > We
> > > > > > > > > >>> > should
> > > > > > > > > >>> > > > >> > probably
> > > > > > > > > >>> > > > >> > > > also
> > > > > > > > > >>> > > > >> > > > > > > log a
> > > > > > > > > >>> > > > >> > > > > > > > >> > bunch
> > > > > > > > > >>> > > > >> > > > > > > > >> > > of
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> messages
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > at
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > WARN
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > level
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > > when
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > >>> something is
> > > > > > > > > >>> > > > >> > scheduled
> > > > > > > > > >>> > > > >> > > > for
> > > > > > > > > >>> > > > >> > > > > > > > >> deletion,
> > > > > > > > > >>> > > > >> > > > > > > > >> > as
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > well.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > (Maybe
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > this
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> was
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > > > assumed,
> > > > > > > > > but
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > it
> > > > > > > > > >>> would be
> > > > > > > > > >>> > > good
> > > > > > > > > >>> > > > >> to
> > > > > > > > > >>> > > > >> > > > mention
> > > > > > > > > >>> > > > >> > > > > > > it).
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > I
> > > > > > > > > >>> feel
> > > > > > > > > >>> > > like
> > > > > > > > > >>> > > > >> there
> > > > > > > > > >>> > > > >> > > > are a
> > > > > > > > > >>> > > > >> > > > > > few
> > > > > > > > > >>> > > > >> > > > > > > > >> > sections
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > should
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > be
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > moved
> > > to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > > > > "rejected
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > >>> > alternatives."
> > > > > > > > > >>> > > > >> For
> > > > > > > > > >>> > > > >> > > > > example,
> > > > > > > > > >>> > > > >> > > > > > in
> > > > > > > > > >>> > > > >> > > > > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > DeleteTopics
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > section,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > since
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > we're
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > not
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > going
> > > > > > > > > >>> to do
> > > > > > > > > >>> > > > >> option 1
> > > > > > > > > >>> > > > >> > or
> > > > > > > > > >>> > > > >> > > > 2,
> > > > > > > > > >>> > > > >> > > > > > > these
> > > > > > > > > >>> > > > >> > > > > > > > >> > should
> > > > > > > > > >>> > > > >> > > > > > > > >> > > be
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > moved
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > into
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > "rejected
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > >>> > alternatives,"
> > > > > > > > > >>> > > > >> > rather
> > > > > > > > > >>> > > > >> > > > than
> > > > > > > > > >>> > > > >> > > > > > > > >> appearing
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > inline.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Another
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > case
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> is
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > > > "Should
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > we
> > > > > > > > > >>> remove
> > > > > > > > > >>> > > topic
> > > > > > > > > >>> > > > >> name
> > > > > > > > > >>> > > > >> > > from
> > > > > > > > > >>> > > > >> > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> protocol
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > where
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > possible"
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > section.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > This
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > is
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > clearly
> > > > > > > > > >>> > > > >> discussing a
> > > > > > > > > >>> > > > >> > > > design
> > > > > > > > > >>> > > > >> > > > > > > > >> > alternative
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > that
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> we're
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > not
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > proposing
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > >>> implement:
> > > > > > > > > >>> > > > >> removing
> > > > > > > > > >>> > > > >> > the
> > > > > > > > > >>> > > > >> > > > > topic
> > > > > > > > > >>> > > > >> > > > > > > > name
> > > > > > > > > >>> > > > >> > > > > > > > >> > from
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > those
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > protocols.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > Is
> > > > > > > > > >>> it
> > > > > > > > > >>> > > really
> > > > > > > > > >>> > > > >> > > > necessary
> > > > > > > > > >>> > > > >> > > > > to
> > > > > > > > > >>> > > > >> > > > > > > > have
> > > > > > > > > >>> > > > >> > > > > > > > >> a
> > > > > > > > > >>> > > > >> > > > > > > > >> > new
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > > > >>> > > /admin/delete_topics_by_id
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > path
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > in
> > > > > > > > > >>> > ZooKeeper?
> > > > > > > > > >>> > > > It
> > > > > > > > > >>> > > > >> > seems
> > > > > > > > > >>> > > > >> > > > > like
> > > > > > > > > >>> > > > >> > > > > > we
> > > > > > > > > >>> > > > >> > > > > > > > >> don't
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > really
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> need
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > this.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > Whenever
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > there
> > > > > > > is
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > a
> > > > > > > > new
> > > > > > > > > >>> > > > controller,
> > > > > > > > > >>> > > > >> > we'll
> > > > > > > > > >>> > > > >> > > > > send
> > > > > > > > > >>> > > > >> > > > > > > out
> > > > > > > > > >>> > > > >> > > > > > > > >> full
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > LeaderAndIsrRequests
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > which
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > will
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > trigger
> > > > > > > > > >>> the
> > > > > > > > > >>> > > > stale
> > > > > > > > > >>> > > > >> > > topics
> > > > > > > > > >>> > > > >> > > > to
> > > > > > > > > >>> > > > >> > > > > > be
> > > > > > > > > >>> > > > >> > > > > > > > >> cleaned
> > > > > > > > > >>> > > > >> > > > > > > > >> > > up.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >  The
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > active
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > controller
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > > will
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > also
> > > > > > > > > >>> send
> > > > > > > > > >>> > the
> > > > > > > > > >>> > > > full
> > > > > > > > > >>> > > > >> > > > > > > > >> LeaderAndIsrRequest
> > > > > > > > > >>> > > > >> > > > > > > > >> > > to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> brokers
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > are
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> just
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > > > starting
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > up.
> > > > > > > > > >>> So we
> > > > > > > > > >>> > > > don't
> > > > > > > > > >>> > > > >> > > really
> > > > > > > > > >>> > > > >> > > > > > need
> > > > > > > > > >>> > > > >> > > > > > > > this
> > > > > > > > > >>> > > > >> > > > > > > > >> > kind
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > of
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > two-phase
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > commit
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > (send
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > > out
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > >>> > > > >> StopReplicasRequest,
> > > > > > > > > >>> > > > >> > > get
> > > > > > > > > >>> > > > >> > > > > ACKs
> > > > > > > > > >>> > > > >> > > > > > > > from
> > > > > > > > > >>> > > > >> > > > > > > > >> all
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > nodes,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > commit
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > by
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > removing
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > >>> > > > >> /admin/delete_topics
> > > > > > > > > >>> > > > >> > > > node)
> > > > > > > > > >>> > > > >> > > > > > any
> > > > > > > > > >>> > > > >> > > > > > > > >> more.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > You
> > > > > > > > > >>> > > mention
> > > > > > > > > >>> > > > >> that
> > > > > > > > > >>> > > > >> > > > > > > FetchRequest
> > > > > > > > > >>> > > > >> > > > > > > > >> will
> > > > > > > > > >>> > > > >> > > > > > > > >> > > now
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> include
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > UUID
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > avoid
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > > > issues
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > where
> > > > > > > > > >>> > requests
> > > > > > > > > >>> > > > are
> > > > > > > > > >>> > > > >> > made
> > > > > > > > > >>> > > > >> > > > to
> > > > > > > > > >>> > > > >> > > > > > > stale
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > partitions.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > However,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > adding a
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > UUID
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > >>> > > MetadataRequest
> > > > > > > > > >>> > > > is
> > > > > > > > > >>> > > > >> > > listed
> > > > > > > > > >>> > > > >> > > > > as
> > > > > > > > > >>> > > > >> > > > > > > > future
> > > > > > > > > >>> > > > >> > > > > > > > >> > > work,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > out
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > of
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > scope
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > for
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> this
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > KIP.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > How
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > will
> > > > > > > > > the
> > > > > > > > > >>> > > client
> > > > > > > > > >>> > > > >> learn
> > > > > > > > > >>> > > > >> > > > what
> > > > > > > > > >>> > > > >> > > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> topic
> > > > > > > > > >>> > > > >> > > > > > > > >> > > UUID
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > is,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > if
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> metadata
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > > > response
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > doesn't
> > > > > > > > > >>> > > include
> > > > > > > > > >>> > > > >> that
> > > > > > > > > >>> > > > >> > > > > > > information?
> > > > > > > > > >>> > > > >> > > > > > > > >> It
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > seems
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > like
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > adding
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> UUID
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > >>> > > MetadataResponse
> > > > > > > > > >>> > > > >> > would
> > > > > > > > > >>> > > > >> > > be
> > > > > > > > > >>> > > > >> > > > > an
> > > > > > > > > >>> > > > >> > > > > > > > >> > improvement
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > here
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> that
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > might
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > not
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> be
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > too
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > hard
> > > > > > > to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > make.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > >>> best,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > >>> Colin
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > On
> > > > > > > > > >>> Mon,
> > > > > > > > > >>> > > Sep
> > > > > > > > > >>> > > > 9,
> > > > > > > > > >>> > > > >> > > 2019,
> > > > > > > > > >>> > > > >> > > > at
> > > > > > > > > >>> > > > >> > > > > > > > 17:48,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > Ryanne
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Dolan
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > wrote:
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >>> Lucas,
> > > > > > > > > >>> > > > this
> > > > > > > > > >>> > > > >> > would
> > > > > > > > > >>> > > > >> > > > be
> > > > > > > > > >>> > > > >> > > > > > > great.
> > > > > > > > > >>> > > > >> > > > > > > > >> I've
> > > > > > > > > >>> > > > >> > > > > > > > >> > > run
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > into
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > issues
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > with
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > topics
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > being
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >>> > > > resurrected
> > > > > > > > > >>> > > > >> > > > > > accidentally,
> > > > > > > > > >>> > > > >> > > > > > > > >> since
> > > > > > > > > >>> > > > >> > > > > > > > >> > a
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > client
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > cannot
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > easily
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > > > > > distinguish
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > between
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > a
> > > > > > > > > >>> > > deleted
> > > > > > > > > >>> > > > >> topic
> > > > > > > > > >>> > > > >> > > > and a
> > > > > > > > > >>> > > > >> > > > > > new
> > > > > > > > > >>> > > > >> > > > > > > > >> topic
> > > > > > > > > >>> > > > >> > > > > > > > >> > > with
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> same
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > name.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I'd
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > need
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > ID
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >>> > > accessible
> > > > > > > > > >>> > > > >> from
> > > > > > > > > >>> > > > >> > > the
> > > > > > > > > >>> > > > >> > > > > > > client
> > > > > > > > > >>> > > > >> > > > > > > > to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > solve
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > that
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > issue,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > but
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> this
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > is a
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > good
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > first
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >>> step.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >>> Ryanne
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > On
> > > > > > > > > >>> > Wed,
> > > > > > > > > >>> > > > Sep
> > > > > > > > > >>> > > > >> 4,
> > > > > > > > > >>> > > > >> > > 2019
> > > > > > > > > >>> > > > >> > > > > at
> > > > > > > > > >>> > > > >> > > > > > > 1:41
> > > > > > > > > >>> > > > >> > > > > > > > >> PM
> > > > > > > > > >>> > > > >> > > > > > > > >> > > Lucas
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Bradstreet <
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > > > > > >>> > > lucas@confluent.io>
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > wrote:
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >>> Hi
> > > > > > > > > >>> > > all,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >>> I
> > > > > > > > > >>> > > would
> > > > > > > > > >>> > > > >> like
> > > > > > > > > >>> > > > >> > to
> > > > > > > > > >>> > > > >> > > > > kick
> > > > > > > > > >>> > > > >> > > > > > > off
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > discussion
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > of
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > KIP-516,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > an
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > > > > > >>> implementation
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > of
> > > > > > > > > topic
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >>> IDs
> > > > > > > > > >>> > > for
> > > > > > > > > >>> > > > >> > Kafka.
> > > > > > > > > >>> > > > >> > > > > Topic
> > > > > > > > > >>> > > > >> > > > > > > IDs
> > > > > > > > > >>> > > > >> > > > > > > > >> aim
> > > > > > > > > >>> > > > >> > > > > > > > >> > to
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > solve
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> topic
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > uniqueness
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > > > problems
> > > > > > > > > in
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > Kafka,
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >>> > where
> > > > > > > > > >>> > > > >> > referring
> > > > > > > > > >>> > > > >> > > > to
> > > > > > > > > >>> > > > >> > > > > a
> > > > > > > > > >>> > > > >> > > > > > > > topic
> > > > > > > > > >>> > > > >> > > > > > > > >> by
> > > > > > > > > >>> > > > >> > > > > > > > >> > > name
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> alone is
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > insufficient.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > > Such
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > cases
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >>> > > include
> > > > > > > > > >>> > > > >> when
> > > > > > > > > >>> > > > >> > a
> > > > > > > > > >>> > > > >> > > > > topic
> > > > > > > > > >>> > > > >> > > > > > > has
> > > > > > > > > >>> > > > >> > > > > > > > >> been
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > deleted
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> recreated
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> with
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > the
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > > same
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > name.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >>> > Unique
> > > > > > > > > >>> > > > >> > > > identifiers
> > > > > > > > > >>> > > > >> > > > > > will
> > > > > > > > > >>> > > > >> > > > > > > > >> help
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > simplify
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > improve
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > Kafka's
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > > > topic
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > deletion
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >>> > > process,
> > > > > > > > > >>> > > > >> as
> > > > > > > > > >>> > > > >> > > well
> > > > > > > > > >>> > > > >> > > > as
> > > > > > > > > >>> > > > >> > > > > > > > prevent
> > > > > > > > > >>> > > > >> > > > > > > > >> > > cases
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > where
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > brokers
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > may
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > > > > > > incorrectly
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > interact
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >>> with
> > > > > > > > > >>> > > > stale
> > > > > > > > > >>> > > > >> > > > versions
> > > > > > > > > >>> > > > >> > > > > > of
> > > > > > > > > >>> > > > >> > > > > > > > >> topics.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > > > >>> > > > >> > > > > > > >
> > > > > > > > > >>> > > > >> > > > > > >
> > > > > > > > > >>> > > > >> > > > > >
> > > > > > > > > >>> > > > >> > > > >
> > > > > > > > > >>> > > > >> > > >
> > > > > > > > > >>> > > > >> > >
> > > > > > > > > >>> > > > >> >
> > > > > > > > > >>> > > > >>
> > > > > > > > > >>> > > >
> > > > > > > > > >>> > >
> > > > > > > > > >>> >
> > > > > > > > > >>>
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >>> > > Looking
> > > > > > > > > >>> > > > >> > forward
> > > > > > > > > >>> > > > >> > > > to
> > > > > > > > > >>> > > > >> > > > > > your
> > > > > > > > > >>> > > > >> > > > > > > > >> > > thoughts.
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >>> > Lucas
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >
> > >
> > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > > > >>> > > > >> > > > > > > > >
> > > > > > > > > >>> > > > >> > > > > > > >
> > > > > > > > > >>> > > > >> > > > > > >
> > > > > > > > > >>> > > > >> > > > > >
> > > > > > > > > >>> > > > >> > > > >
> > > > > > > > > >>> > > > >> > > >
> > > > > > > > > >>> > > > >> > >
> > > > > > > > > >>> > > > >> >
> > > > > > > > > >>> > > > >>
> > > > > > > > > >>> > > > >
> > > > > > > > > >>> > > >
> > > > > > > > > >>> > >
> > > > > > > > > >>> >
> > > > > > > > > >>>
> > > > > > > > > >>
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Jun Rao <ju...@confluent.io>]

Hi, Justine,

The changes sound reasonable to me.

50. Should we make the topic name in the new MetadataRequest
and DeleteTopicsRequest nullable?

51. In the future work section, perhaps we could mention opportunities for
using topicId in other methods in AdminClient where topic name is used.

Thanks,

Jun

On Wed, Oct 7, 2020 at 2:44 PM Justine Olshan <jo...@confluent.io> wrote:

> Hi all,
>
> Another update on the KIP. I've added some information about how topic IDs
> will be supported in AdminClient
>
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-AdminClientSupport
>
>
> One specific decision I wanted to focus on is the inclusion of both topic
> names and topic Ids in metadata and deleteTopics requests.
>
> The idea is that when the request is received the topic id field is checked
> first, but if it contains the default (zero) uuid, then the topic name
> field will be used.
>
> In the case where ids are used, if the ids do not have a corresponding
> topic, the request will fail. In the case where both ids and names are
> included, the request will fail if the name does not match the id.
>
>
> Initially the id field will be used to support the admin client use cases
> as described in the KIP, but over time, we can transfer to using topic ids
> in all uses of the request.
>
>
> There will also be some support for returning the topic Id on the
> createtopics, deletetopics and metadata responses. I think it makes sense
> to include both the name and the id in the successful response for these
> cases.
>
>
> Let me know if you have any comments or concerns regarding the Admin Client
> changes.
>
> On Wed, Oct 7, 2020 at 11:05 AM Jun Rao <ju...@confluent.io> wrote:
>
> > Hi, Justine,
> >
> > I was just wondering if we expose topicId publicly, how would people use
> > it. One possibility is to use topicId in all those public interfaces
> where
> > we use topic name. I am not sure if the need is very strong though. So,
> > maybe we can list this as future considerations.
> >
> > Thanks,
> >
> > Jun
> >
> > On Wed, Oct 7, 2020 at 8:57 AM Justine Olshan <jo...@confluent.io>
> > wrote:
> >
> > > Hi all,
> > >
> > > After some careful consideration, I've decided that the
> TopicIdPartition
> > > will be out of scope for this KIP. After investigating what it will
> look
> > > like using it in only the Fetch Request, I found that I would need to
> > > modify several public APIs in the consumer and I would need to take
> > special
> > > care in many situations to handle the request correctly based on what
> > > version of the request was sent. For these reasons, I've added
> > > TopicIdPartition to potential future work along with an alternative to
> > > modify TopicPartition to include a topic ID field.
> > >
> > > Jun- I was also wondering about some of the details for supporting
> topic
> > > IDs for delete and describe topics in the admin client. Is the idea
> that
> > > instead of supplying topic names to describe or delete, we would
> supply a
> > > collection of ids? If that is the case, I'm wondering if it makes sense
> > to
> > > add topic Ids to the metadata and delete topics requests.
> > >
> > > Thanks,
> > > Justine
> > >
> > > On Mon, Oct 5, 2020 at 9:25 PM Satish Duggana <
> satish.duggana@gmail.com>
> > > wrote:
> > >
> > > > Hi Jun, Justine,
> > > > +1 to have TopicIdPartition in public packages. This is useful for
> > > > KIP-405 tiered storage too. We currently use TopicPartition for
> > > > pluggable log storage and metadata storage apis. We will be using
> > > > TopicIdPartition once the proposed changes in this KIP are merged.
> > > >
> > > > Thanks,
> > > > Satish.
> > > >
> > > > On Tue, Oct 6, 2020 at 5:21 AM Jun Rao <ju...@confluent.io> wrote:
> > > > >
> > > > > Hi, Justine,
> > > > >
> > > > > I guess we have to maintain the mapping between topic name and
> > topicId
> > > in
> > > > > both the client and the server. If TopicIdPartition is meant for
> > > internal
> > > > > usage only, we can create it under a package
> > > > > like org/apache/kafka/common/internal and we don't need to document
> > it
> > > in
> > > > > the KIP. If it's meant for external public usage (and potentially
> > > > internal
> > > > > usage too), we can create it in org/apache/kafka/common and need to
> > > > > document it in the KIP.
> > > > >
> > > > > Since topicId is now part of the topic metadata, perhaps it's
> useful
> > to
> > > > > expose it in TopicMetadataAndConfig too.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Jun
> > > > >
> > > > > On Mon, Oct 5, 2020 at 3:35 PM Justine Olshan <
> jolshan@confluent.io>
> > > > wrote:
> > > > >
> > > > > > Hi Jun,
> > > > > >
> > > > > > This KIP removes the topic name from many requests. A lot of
> > requests
> > > > keep
> > > > > > track of the partitions internally with a TopicPartition. So when
> > we
> > > > remove
> > > > > > the topic name, we can not differentiate unless we can also use
> > topic
> > > > ID.
> > > > > > The Fetch request relies heavily on this, but ListOffset,
> > > > > > OffsetsForLeaderEpoch, etc all use TopicPartition as a key value
> > for
> > > > > > internal maps. Digging deeper into Fetch, I've noticed that the
> > code
> > > > path
> > > > > > runs into some public APIs that use the TopicPartition. Some
> > examples
> > > > are
> > > > > > Cluster.java or RecordTooLargeException.
> > > > > >
> > > > > > A related issue is that in some places like
> > > > TopicAuthorizationException,
> > > > > > the topic name is required and without the name in the request,
> we
> > > > won't
> > > > > > know it.
> > > > > >
> > > > > > I guess an alternative would be to have some mapping from topic
> > name
> > > to
> > > > > > topic ID, but I would have to think about that more.
> > > > > >
> > > > > > The KIP mentions exposing TopicId in TopicDescription. (The font
> > was
> > > > > > weirdly colored, so I've made it easier to read.)
> > > > > >
> > > > > > Would the idea for the TopicMetadataAndConfig be that the topic
> ID
> > > > would be
> > > > > > returned as a result?
> > > > > > As for the other methods describe/deletetopics, it makes sense to
> > > > include
> > > > > > them at some point, but I'm not sure how far we should go in
> terms
> > of
> > > > > > supporting topic IDs everywhere.
> > > > > >
> > > > > > Thank you for continuing to look at the KIP!
> > > > > > Justine
> > > > > >
> > > > > >
> > > > > > On Mon, Oct 5, 2020 at 2:53 PM Jun Rao <ju...@confluent.io> wrote:
> > > > > >
> > > > > > > Hi, Justine,
> > > > > > >
> > > > > > > Adding a TopicIdPartition in the public interface may be
> useful.
> > > > Where
> > > > > > > would you expect it to be used?
> > > > > > >
> > > > > > > Related to this, since we are exposing topicId in the tool, it
> > > seems
> > > > that
> > > > > > > we may want to expose topicId in at least TopicDescription used
> > by
> > > > > > > AdminClient.describeTopics() and TopicMetadataAndConfig used in
> > > > > > > AdminClient.createTopics(Collection<NewTopic> newTopics). I am
> > also
> > > > not
> > > > > > > sure if we plan to expose topicId in other AdminClient methods
> in
> > > > this
> > > > > > KIP
> > > > > > > or in some future KIP. For example, it might be useful to
> support
> > > > topicId
> > > > > > > in the following methods.
> > > > > > >
> > > > > > >     DescribeTopicsResult describeTopics(Collection<String>
> > > > topicNames,
> > > > > > > DescribeTopicsOptions options);
> > > > > > >     DeleteTopicsResult deleteTopics(Collection<String> topics,
> > > > > > > DeleteTopicsOptions options);
> > > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > > > Jun
> > > > > > >
> > > > > > > On Mon, Oct 5, 2020 at 11:03 AM Justine Olshan <
> > > jolshan@confluent.io
> > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi all,
> > > > > > > >
> > > > > > > > I was thinking about this KIP some more. We are removing the
> > > topic
> > > > name
> > > > > > > > from many of these request types, and some of them (eg.
> Fetch)
> > > use
> > > > > > > > TopicPartition as keys in a map when handling the request. I
> > was
> > > > > > thinking
> > > > > > > > it would make sense to make some sort of TopicIdPartition
> with
> > > > topic
> > > > > > > name,
> > > > > > > > topic id, and partition fields (name and id would potentially
> > be
> > > > > > optional
> > > > > > > > types since different request versions will contain only one
> of
> > > the
> > > > > > two)
> > > > > > > I
> > > > > > > > was wondering if it made sense to include this as a public
> > facing
> > > > API
> > > > > > > > similar to TopicPartition. If so, I will add this class to
> the
> > > KIP
> > > > as
> > > > > > > well.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Justine
> > > > > > > >
> > > > > > > > On Mon, Oct 5, 2020 at 8:52 AM Justine Olshan <
> > > > jolshan@confluent.io>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi all,
> > > > > > > > > I've made a few minor changes to the UUID class. I've
> added a
> > > few
> > > > > > more
> > > > > > > > > methods to support UUIDs more generally, by including a
> > > > constructor
> > > > > > > and a
> > > > > > > > > method to generate random UUIDs. Both of these will be
> useful
> > > for
> > > > > > > > testing.
> > > > > > > > >
> > > > > > > > > Let me know if there are any other areas for discussion.
> > > > > > > > >
> > > > > > > > > Thank you,
> > > > > > > > > Justine
> > > > > > > > >
> > > > > > > > > On Thu, Oct 1, 2020 at 5:29 PM Justine Olshan <
> > > > jolshan@confluent.io>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > >> Hi Jun,
> > > > > > > > >> Thanks for looking at it again. Here's the new spec. (I
> > fixed
> > > > the
> > > > > > typo
> > > > > > > > in
> > > > > > > > >> it too.)
> > > > > > > > >> {"name": "id", "type": "string", "doc": option[UUID]}
> > > > > > > > >>
> > > > > > > > >> Justine
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > > >> On Thu, Oct 1, 2020 at 5:03 PM Jun Rao <ju...@confluent.io>
> > > > wrote:
> > > > > > > > >>
> > > > > > > > >>> Hi, Justine,
> > > > > > > > >>>
> > > > > > > > >>> Thanks for the update. The KIP looks good to me now.
> Just a
> > > > minor
> > > > > > > > comment
> > > > > > > > >>> below.
> > > > > > > > >>>
> > > > > > > > >>> 30. Perhaps "option[UUID]" can be put in the doc.
> > > > > > > > >>>
> > > > > > > > >>> Jun
> > > > > > > > >>>
> > > > > > > > >>> On Thu, Oct 1, 2020 at 3:28 PM Justine Olshan <
> > > > > > jolshan@confluent.io>
> > > > > > > > >>> wrote:
> > > > > > > > >>>
> > > > > > > > >>> > Hi Jun,
> > > > > > > > >>> > Thanks for the response!
> > > > > > > > >>> >
> > > > > > > > >>> > 30. I think I might have changed this in between. The
> > > current
> > > > > > > version
> > > > > > > > >>> > says:  {"name":
> > > > > > > > >>> > "id", "type": "option[UUID]"}, "doc": topic id}
> > > > > > > > >>> > I have switched to the option type to cover the
> migration
> > > > case
> > > > > > > where
> > > > > > > > a
> > > > > > > > >>> > TopicZNode does not yet have a topic ID.
> > > > > > > > >>> > I understand that due to json, this field is written
> as a
> > > > string,
> > > > > > > so
> > > > > > > > >>> if I
> > > > > > > > >>> > should move the "option[uuid]" to the doc field and the
> > > type
> > > > > > should
> > > > > > > > be
> > > > > > > > >>> > "string" please let me know.
> > > > > > > > >>> >
> > > > > > > > >>> > 40. I've added a definition for UUID.
> > > > > > > > >>> > 41,42. Fixed
> > > > > > > > >>> >
> > > > > > > > >>> > Thank you,
> > > > > > > > >>> > Justine
> > > > > > > > >>> >
> > > > > > > > >>> > On Wed, Sep 30, 2020 at 1:15 PM Jun Rao <
> > jun@confluent.io>
> > > > > > wrote:
> > > > > > > > >>> >
> > > > > > > > >>> > > Hi, Justine,
> > > > > > > > >>> > >
> > > > > > > > >>> > > Thanks for the summary. Just a few minor comments
> blow.
> > > > > > > > >>> > >
> > > > > > > > >>> > > 30.  {"name": "id", "type": "string", "doc": "version
> > > > id"}}:
> > > > > > The
> > > > > > > > doc
> > > > > > > > >>> > should
> > > > > > > > >>> > > say UUID. The issue seems unfixed.
> > > > > > > > >>> > >
> > > > > > > > >>> > > 40. Since UUID is public facing, could you include
> its
> > > > > > > definition?
> > > > > > > > >>> > >
> > > > > > > > >>> > > 41. StopReplicaResponse still includes the topic
> field.
> > > > > > > > >>> > >
> > > > > > > > >>> > > 42. "It is unnecessary to include the name of the
> topic
> > > in
> > > > the
> > > > > > > > >>> following
> > > > > > > > >>> > > Request/Response calls" It would be useful to include
> > all
> > > > > > > modified
> > > > > > > > >>> > requests
> > > > > > > > >>> > > (e.g. produce) in the list below.
> > > > > > > > >>> > >
> > > > > > > > >>> > > Thanks,
> > > > > > > > >>> > >
> > > > > > > > >>> > > Jun
> > > > > > > > >>> > >
> > > > > > > > >>> > >
> > > > > > > > >>> > > On Wed, Sep 30, 2020 at 10:17 AM Justine Olshan <
> > > > > > > > >>> jolshan@confluent.io>
> > > > > > > > >>> > > wrote:
> > > > > > > > >>> > >
> > > > > > > > >>> > > > Hello again,
> > > > > > > > >>> > > >
> > > > > > > > >>> > > > I've taken some time to discuss some of the
> remaining
> > > > points
> > > > > > > > >>> brought up
> > > > > > > > >>> > > by
> > > > > > > > >>> > > > the previous emails offline. Here are some of the
> > > > > > conclusions.
> > > > > > > > >>> > > >
> > > > > > > > >>> > > > 1. Directory Structure:
> > > > > > > > >>> > > > There was some talk about whether the directory
> > > structure
> > > > > > > should
> > > > > > > > be
> > > > > > > > >>> > > changed
> > > > > > > > >>> > > > to replace all topic names with topic IDs.
> > > > > > > > >>> > > > This will be a useful change, but will prevent
> > > > downgrades. It
> > > > > > > > will
> > > > > > > > >>> be
> > > > > > > > >>> > > best
> > > > > > > > >>> > > > to wait until a major release, likely alongside
> > KIP-500
> > > > > > changes
> > > > > > > > >>> that
> > > > > > > > >>> > will
> > > > > > > > >>> > > > prevent downgrades. I've updated the KIP to include
> > > this
> > > > > > change
> > > > > > > > >>> with
> > > > > > > > >>> > the
> > > > > > > > >>> > > > note about migration and deprecation.
> > > > > > > > >>> > > >
> > > > > > > > >>> > > >
> > > > > > > > >>> > >
> > > > > > > > >>> >
> > > > > > > > >>>
> > > > > > > >
> > > > > > >
> > > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-log.dirlayout
> > > > > > > > >>> > > >
> > > > > > > > >>> > > > 2. Partition Metadata file
> > > > > > > > >>> > > > There was some disagreement about the inclusion of
> > the
> > > > > > > partition
> > > > > > > > >>> > metadata
> > > > > > > > >>> > > > file.
> > > > > > > > >>> > > > This file will be needed to persist the topic ID,
> > > > especially
> > > > > > > > while
> > > > > > > > >>> we
> > > > > > > > >>> > > still
> > > > > > > > >>> > > > have the old directory structure. Even after the
> > > changes,
> > > > > > this
> > > > > > > > >>> file can
> > > > > > > > >>> > > be
> > > > > > > > >>> > > > useful for debugging and recovery.
> > > > > > > > >>> > > > Creating a single mapping file from topic names to
> > > topic
> > > > IDs
> > > > > > > was
> > > > > > > > >>> > > > considered, but ultimately scrapped as it would not
> > be
> > > as
> > > > > > easy
> > > > > > > to
> > > > > > > > >>> > > maintain.
> > > > > > > > >>> > > >
> > > > > > > > >>> > > >
> > > > > > > > >>> > >
> > > > > > > > >>> >
> > > > > > > > >>>
> > > > > > > >
> > > > > > >
> > > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PartitionMetadatafile
> > > > > > > > >>> > > >
> > > > > > > > >>> > > >
> > > > > > > > >>> > > >
> > > > > > > > >>> > >
> > > > > > > > >>> >
> > > > > > > > >>>
> > > > > > > >
> > > > > > >
> > > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PersistingTopicIDs
> > > > > > > > >>> > > >
> > > > > > > > >>> > > > 3. Produce Protocols
> > > > > > > > >>> > > > After some further discussion, this replacing the
> > topic
> > > > name
> > > > > > > with
> > > > > > > > >>> topic
> > > > > > > > >>> > > ID
> > > > > > > > >>> > > > in these protocols has been added to the KIP.
> > > > > > > > >>> > > > This will cut down on the size of the protocol.
> Since
> > > > changes
> > > > > > > to
> > > > > > > > >>> fetch
> > > > > > > > >>> > > are
> > > > > > > > >>> > > > included, it does make sense to update these
> > protocols.
> > > > > > > > >>> > > >
> > > > > > > > >>> > > >
> > > > > > > > >>> > >
> > > > > > > > >>> >
> > > > > > > > >>>
> > > > > > > >
> > > > > > >
> > > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-Produce
> > > > > > > > >>> > > >
> > > > > > > > >>> > > > 4. KIP-500 Compatibility
> > > > > > > > >>> > > > After some discussion with those involved with
> > KIP-500,
> > > > it
> > > > > > > seems
> > > > > > > > >>> best
> > > > > > > > >>> > to
> > > > > > > > >>> > > > use a sentinel topic ID for the metadata topic that
> > is
> > > > used
> > > > > > > > before
> > > > > > > > >>> the
> > > > > > > > >>> > > > first controller election.
> > > > > > > > >>> > > > We had an issue where this metadata topic may not
> be
> > > > assigned
> > > > > > > an
> > > > > > > > ID
> > > > > > > > >>> > > before
> > > > > > > > >>> > > > utilizing the Vote and Fetch protocols. It was
> > decided
> > > to
> > > > > > > > reserve a
> > > > > > > > >>> > > unique
> > > > > > > > >>> > > > ID for this topic to be used until the controller
> > could
> > > > give
> > > > > > > the
> > > > > > > > >>> topic
> > > > > > > > >>> > a
> > > > > > > > >>> > > > unique ID.
> > > > > > > > >>> > > > Having the topic keep the sentinel ID (not be
> > > reassigned
> > > > to a
> > > > > > > > >>> unique
> > > > > > > > >>> > ID)
> > > > > > > > >>> > > > was considered, but it seemed like a good idea to
> > leave
> > > > the
> > > > > > > > option
> > > > > > > > >>> open
> > > > > > > > >>> > > for
> > > > > > > > >>> > > > the metadata topic to have a unique ID in cases
> where
> > > it
> > > > > > would
> > > > > > > > >>> need to
> > > > > > > > >>> > be
> > > > > > > > >>> > > > differentiated from other clusters' metadata
> topics.
> > > (ex.
> > > > > > > tiered
> > > > > > > > >>> > storage)
> > > > > > > > >>> > > >
> > > > > > > > >>> > > >
> > > > > > > > >>> > >
> > > > > > > > >>> >
> > > > > > > > >>>
> > > > > > > >
> > > > > > >
> > > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-CompatibilitywithKIP-500
> > > > > > > > >>> > > >
> > > > > > > > >>> > > > I've also split up the KIP into sub-tasks on the
> > JIRA.
> > > > > > > Hopefully
> > > > > > > > >>> this
> > > > > > > > >>> > > will
> > > > > > > > >>> > > > provide a better idea about what tasks we have, and
> > > > > > eventually
> > > > > > > > >>> provide
> > > > > > > > >>> > a
> > > > > > > > >>> > > > place to see what's done and what is left.
> > > > > > > > >>> > > > If there is a task I am missing, please let me
> know!
> > > > > > > > >>> > > > https://issues.apache.org/jira/browse/KAFKA-8872
> > > > > > > > >>> > > >
> > > > > > > > >>> > > > Of course, these decisions are not set in stone,
> and
> > I
> > > > would
> > > > > > > love
> > > > > > > > >>> to
> > > > > > > > >>> > hear
> > > > > > > > >>> > > > any feedback.
> > > > > > > > >>> > > >
> > > > > > > > >>> > > > Thanks,
> > > > > > > > >>> > > > Justine
> > > > > > > > >>> > > >
> > > > > > > > >>> > > > On Mon, Sep 28, 2020 at 11:38 AM Justine Olshan <
> > > > > > > > >>> jolshan@confluent.io>
> > > > > > > > >>> > > > wrote:
> > > > > > > > >>> > > >
> > > > > > > > >>> > > > > Hello all,
> > > > > > > > >>> > > > >
> > > > > > > > >>> > > > > I just wanted to follow up on this discussion.
> Did
> > we
> > > > come
> > > > > > to
> > > > > > > > an
> > > > > > > > >>> > > > > understanding about the directory structure?
> > > > > > > > >>> > > > >
> > > > > > > > >>> > > > > I think the biggest question here is what is
> > > > acceptable to
> > > > > > > > leave
> > > > > > > > >>> out
> > > > > > > > >>> > > due
> > > > > > > > >>> > > > > to scope vs. what is considered to be too much
> tech
> > > > debt.
> > > > > > > > >>> > > > > This KIP is already pretty large with the number
> of
> > > > > > changes,
> > > > > > > > but
> > > > > > > > >>> it
> > > > > > > > >>> > > also
> > > > > > > > >>> > > > > makes sense to do things correctly, so I'd love
> to
> > > hear
> > > > > > > > >>> everyone's
> > > > > > > > >>> > > > thoughts.
> > > > > > > > >>> > > > >
> > > > > > > > >>> > > > > Thanks,
> > > > > > > > >>> > > > > Justine
> > > > > > > > >>> > > > >
> > > > > > > > >>> > > > > On Fri, Sep 25, 2020 at 8:19 AM Lucas Bradstreet
> <
> > > > > > > > >>> lucas@confluent.io
> > > > > > > > >>> > >
> > > > > > > > >>> > > > > wrote:
> > > > > > > > >>> > > > >
> > > > > > > > >>> > > > >> Hi Ismael,
> > > > > > > > >>> > > > >>
> > > > > > > > >>> > > > >> If you do not store it in a metadata file or in
> > the
> > > > > > > directory
> > > > > > > > >>> > > structure
> > > > > > > > >>> > > > >> would you then
> > > > > > > > >>> > > > >> require the LeaderAndIsrRequest following
> startup
> > to
> > > > give
> > > > > > > you
> > > > > > > > >>> some
> > > > > > > > >>> > > > notion
> > > > > > > > >>> > > > >> of
> > > > > > > > >>> > > > >> topic name in memory? We would still need this
> > > > information
> > > > > > > for
> > > > > > > > >>> the
> > > > > > > > >>> > > older
> > > > > > > > >>> > > > >> protocols, but
> > > > > > > > >>> > > > >> perhaps this is what's meant by tech debt.
> > > > > > > > >>> > > > >>
> > > > > > > > >>> > > > >> Once we're free of the old non-topicID requests
> > > then I
> > > > > > think
> > > > > > > > you
> > > > > > > > >>> > > > wouldn't
> > > > > > > > >>> > > > >> need to retain the topic name.
> > > > > > > > >>> > > > >> I think the ability to easily look up topic
> names
> > > > > > associated
> > > > > > > > >>> with
> > > > > > > > >>> > > > >> partition
> > > > > > > > >>> > > > >> directories would still be missed
> > > > > > > > >>> > > > >> when diagnosing issues, though maybe it wouldn't
> > be
> > > a
> > > > deal
> > > > > > > > >>> breaker
> > > > > > > > >>> > > with
> > > > > > > > >>> > > > >> the
> > > > > > > > >>> > > > >> right tooling.
> > > > > > > > >>> > > > >>
> > > > > > > > >>> > > > >> Thanks,
> > > > > > > > >>> > > > >>
> > > > > > > > >>> > > > >> Lucas
> > > > > > > > >>> > > > >>
> > > > > > > > >>> > > > >> On Fri, Sep 25, 2020 at 7:55 AM Ismael Juma <
> > > > > > > > ismael@juma.me.uk>
> > > > > > > > >>> > > wrote:
> > > > > > > > >>> > > > >>
> > > > > > > > >>> > > > >> > Hi Lucas,
> > > > > > > > >>> > > > >> >
> > > > > > > > >>> > > > >> > Why would you include the name and id? I think
> > > you'd
> > > > > > want
> > > > > > > to
> > > > > > > > >>> > remove
> > > > > > > > >>> > > > the
> > > > > > > > >>> > > > >> > name from the directory name right? Jason's
> > > > suggestion
> > > > > > was
> > > > > > > > >>> that if
> > > > > > > > >>> > > you
> > > > > > > > >>> > > > >> > remove the name from the directory, then why
> > would
> > > > you
> > > > > > > need
> > > > > > > > >>> the id
> > > > > > > > >>> > > > name
> > > > > > > > >>> > > > >> > mapping file?
> > > > > > > > >>> > > > >> >
> > > > > > > > >>> > > > >> > Ismael
> > > > > > > > >>> > > > >> >
> > > > > > > > >>> > > > >> > On Thu, Sep 24, 2020 at 4:24 PM Lucas
> > Bradstreet <
> > > > > > > > >>> > > lucas@confluent.io>
> > > > > > > > >>> > > > >> > wrote:
> > > > > > > > >>> > > > >> >
> > > > > > > > >>> > > > >> > > > 2. Part of the usage of the file is to
> have
> > > > > > persistent
> > > > > > > > >>> storage
> > > > > > > > >>> > > of
> > > > > > > > >>> > > > >> the
> > > > > > > > >>> > > > >> > > topic
> > > > > > > > >>> > > > >> > > ID and use it to compare with the ID
> supplied
> > in
> > > > the
> > > > > > > > >>> > LeaderAndIsr
> > > > > > > > >>> > > > >> > Request.
> > > > > > > > >>> > > > >> > > There is some discussion in the KIP about
> > > changes
> > > > to
> > > > > > the
> > > > > > > > >>> > directory
> > > > > > > > >>> > > > >> > > structure, but I believe directory changes
> > were
> > > > > > > considered
> > > > > > > > >>> to be
> > > > > > > > >>> > > out
> > > > > > > > >>> > > > >> of
> > > > > > > > >>> > > > >> > > scope when the KIP was written.
> > > > > > > > >>> > > > >> > >
> > > > > > > > >>> > > > >> > >
> > > > > > > > >>> > > > >> > > Yeah, I was hoping to get a better
> > understanding
> > > > of
> > > > > > why
> > > > > > > it
> > > > > > > > >>> was
> > > > > > > > >>> > > taken
> > > > > > > > >>> > > > >> out
> > > > > > > > >>> > > > >> > of
> > > > > > > > >>> > > > >> > > scope. Perhaps Lucas Bradstreet might have
> > more
> > > > > > insight
> > > > > > > > >>> about
> > > > > > > > >>> > the
> > > > > > > > >>> > > > >> > decision.
> > > > > > > > >>> > > > >> > > Basically my point is that we have to create
> > > > > > additional
> > > > > > > > >>> > > > infrastructure
> > > > > > > > >>> > > > >> > here
> > > > > > > > >>> > > > >> > > to support the name/id mapping, so I wanted
> to
> > > > > > > understand
> > > > > > > > >>> if we
> > > > > > > > >>> > > just
> > > > > > > > >>> > > > >> > > consider this a sort of tech debt. It would
> be
> > > > useful
> > > > > > to
> > > > > > > > >>> cover
> > > > > > > > >>> > how
> > > > > > > > >>> > > > we
> > > > > > > > >>> > > > >> > > handle the case when this file gets
> corrupted.
> > > > Seems
> > > > > > > like
> > > > > > > > we
> > > > > > > > >>> > just
> > > > > > > > >>> > > > >> have to
> > > > > > > > >>> > > > >> > > trust that it matches whatever the
> controller
> > > > tells us
> > > > > > > and
> > > > > > > > >>> > rewrite
> > > > > > > > >>> > > > it?
> > > > > > > > >>> > > > >> > >
> > > > > > > > >>> > > > >> > >
> > > > > > > > >>> > > > >> > > Hi Jason, Justine,
> > > > > > > > >>> > > > >> > >
> > > > > > > > >>> > > > >> > > My thought process is that we will likely
> need
> > > the
> > > > > > > > metadata
> > > > > > > > >>> file
> > > > > > > > >>> > > > >> whether
> > > > > > > > >>> > > > >> > we
> > > > > > > > >>> > > > >> > > rename the directories or not.
> > > > > > > > >>> > > > >> > > Linux supports filenames of up to 255 bytes
> > and
> > > > that
> > > > > > > would
> > > > > > > > >>> not
> > > > > > > > >>> > be
> > > > > > > > >>> > > > >> enough
> > > > > > > > >>> > > > >> > to
> > > > > > > > >>> > > > >> > > support a directory name
> > > > > > > > >>> > > > >> > >  including both the name and topic ID. Given
> > > that
> > > > > > fact,
> > > > > > > it
> > > > > > > > >>> > seemed
> > > > > > > > >>> > > > >> > > reasonable to add the metadata file
> > > > > > > > >>> > > > >> > > and leave the directory rename until some
> time
> > > in
> > > > the
> > > > > > > > future
> > > > > > > > >>> > > > (possibly
> > > > > > > > >>> > > > >> > > never).
> > > > > > > > >>> > > > >> > >
> > > > > > > > >>> > > > >> > > If the file does get corrupted I think
> you're
> > > > right.
> > > > > > We
> > > > > > > > >>> would
> > > > > > > > >>> > > either
> > > > > > > > >>> > > > >> have
> > > > > > > > >>> > > > >> > > to trust it matches what the controller
> tells
> > us
> > > > > > > > >>> > > > >> > >  or error out and let an administrator
> resolve
> > > it
> > > > by
> > > > > > > > >>> checking
> > > > > > > > >>> > > across
> > > > > > > > >>> > > > >> > > replicas for consistency.
> > > > > > > > >>> > > > >> > >
> > > > > > > > >>> > > > >> > > Lucas
> > > > > > > > >>> > > > >> > >
> > > > > > > > >>> > > > >> > >
> > > > > > > > >>> > > > >> > > On Thu, Sep 24, 2020 at 3:41 PM Jason
> > Gustafson
> > > <
> > > > > > > > >>> > > jason@confluent.io
> > > > > > > > >>> > > > >
> > > > > > > > >>> > > > >> > > wrote:
> > > > > > > > >>> > > > >> > >
> > > > > > > > >>> > > > >> > > > Thanks Justine. Responses below:
> > > > > > > > >>> > > > >> > > >
> > > > > > > > >>> > > > >> > > > > 1. Yes, the directory will still be
> based
> > on
> > > > the
> > > > > > > topic
> > > > > > > > >>> > names.
> > > > > > > > >>> > > > >> > > > LeaderAndIsrRequest is one of the few
> > requests
> > > > that
> > > > > > > will
> > > > > > > > >>> still
> > > > > > > > >>> > > > >> contain
> > > > > > > > >>> > > > >> > > the
> > > > > > > > >>> > > > >> > > > topic name. So I think we have this
> covered.
> > > > Sorry
> > > > > > for
> > > > > > > > >>> > > confusion.
> > > > > > > > >>> > > > >> > > >
> > > > > > > > >>> > > > >> > > > Ah, you're right. My eyes passed right
> over
> > > the
> > > > > > field.
> > > > > > > > >>> > > > >> > > >
> > > > > > > > >>> > > > >> > > > > 2. Part of the usage of the file is to
> > have
> > > > > > > persistent
> > > > > > > > >>> > storage
> > > > > > > > >>> > > > of
> > > > > > > > >>> > > > >> the
> > > > > > > > >>> > > > >> > > > topic
> > > > > > > > >>> > > > >> > > > ID and use it to compare with the ID
> > supplied
> > > > in the
> > > > > > > > >>> > > LeaderAndIsr
> > > > > > > > >>> > > > >> > > Request.
> > > > > > > > >>> > > > >> > > > There is some discussion in the KIP about
> > > > changes to
> > > > > > > the
> > > > > > > > >>> > > directory
> > > > > > > > >>> > > > >> > > > structure, but I believe directory changes
> > > were
> > > > > > > > >>> considered to
> > > > > > > > >>> > be
> > > > > > > > >>> > > > >> out of
> > > > > > > > >>> > > > >> > > > scope when the KIP was written.
> > > > > > > > >>> > > > >> > > >
> > > > > > > > >>> > > > >> > > > Yeah, I was hoping to get a better
> > > > understanding of
> > > > > > > why
> > > > > > > > >>> it was
> > > > > > > > >>> > > > taken
> > > > > > > > >>> > > > >> > out
> > > > > > > > >>> > > > >> > > of
> > > > > > > > >>> > > > >> > > > scope. Perhaps Lucas Bradstreet might have
> > > more
> > > > > > > insight
> > > > > > > > >>> about
> > > > > > > > >>> > > the
> > > > > > > > >>> > > > >> > > decision.
> > > > > > > > >>> > > > >> > > > Basically my point is that we have to
> create
> > > > > > > additional
> > > > > > > > >>> > > > >> infrastructure
> > > > > > > > >>> > > > >> > > here
> > > > > > > > >>> > > > >> > > > to support the name/id mapping, so I
> wanted
> > to
> > > > > > > > understand
> > > > > > > > >>> if
> > > > > > > > >>> > we
> > > > > > > > >>> > > > just
> > > > > > > > >>> > > > >> > > > consider this a sort of tech debt. It
> would
> > be
> > > > > > useful
> > > > > > > to
> > > > > > > > >>> cover
> > > > > > > > >>> > > how
> > > > > > > > >>> > > > >> we
> > > > > > > > >>> > > > >> > > > handle the case when this file gets
> > corrupted.
> > > > Seems
> > > > > > > > like
> > > > > > > > >>> we
> > > > > > > > >>> > > just
> > > > > > > > >>> > > > >> have
> > > > > > > > >>> > > > >> > to
> > > > > > > > >>> > > > >> > > > trust that it matches whatever the
> > controller
> > > > tells
> > > > > > us
> > > > > > > > and
> > > > > > > > >>> > > rewrite
> > > > > > > > >>> > > > >> it?
> > > > > > > > >>> > > > >> > > >
> > > > > > > > >>> > > > >> > > > > 3. I think this is a good point, but I
> > > again I
> > > > > > > wonder
> > > > > > > > >>> about
> > > > > > > > >>> > > the
> > > > > > > > >>> > > > >> scope
> > > > > > > > >>> > > > >> > > of
> > > > > > > > >>> > > > >> > > > the KIP. Most of the changes mentioned in
> > the
> > > > KIP
> > > > > > are
> > > > > > > > for
> > > > > > > > >>> > > > supporting
> > > > > > > > >>> > > > >> > > topic
> > > > > > > > >>> > > > >> > > > deletion and I believe that is why the
> > produce
> > > > > > request
> > > > > > > > was
> > > > > > > > >>> > > listed
> > > > > > > > >>> > > > >> under
> > > > > > > > >>> > > > >> > > > future work.
> > > > > > > > >>> > > > >> > > >
> > > > > > > > >>> > > > >> > > > That's fair. I brought it up since `Fetch`
> > is
> > > > > > already
> > > > > > > > >>> > included.
> > > > > > > > >>> > > If
> > > > > > > > >>> > > > >> > we've
> > > > > > > > >>> > > > >> > > > got `Metadata` and `Fetch`, seems we may
> as
> > > > well do
> > > > > > > > >>> `Produce`
> > > > > > > > >>> > > and
> > > > > > > > >>> > > > >> save
> > > > > > > > >>> > > > >> > an
> > > > > > > > >>> > > > >> > > > extra kip. No strong objection though if
> you
> > > > want to
> > > > > > > > >>> leave it
> > > > > > > > >>> > > out.
> > > > > > > > >>> > > > >> > > >
> > > > > > > > >>> > > > >> > > >
> > > > > > > > >>> > > > >> > > > -Jason
> > > > > > > > >>> > > > >> > > >
> > > > > > > > >>> > > > >> > > >
> > > > > > > > >>> > > > >> > > > On Thu, Sep 24, 2020 at 3:26 PM Justine
> > > Olshan <
> > > > > > > > >>> > > > >> jolshan@confluent.io>
> > > > > > > > >>> > > > >> > > > wrote:
> > > > > > > > >>> > > > >> > > >
> > > > > > > > >>> > > > >> > > > > Hi Jason,
> > > > > > > > >>> > > > >> > > > >
> > > > > > > > >>> > > > >> > > > > Thanks for your comments.
> > > > > > > > >>> > > > >> > > > >
> > > > > > > > >>> > > > >> > > > > 1. Yes, the directory will still be
> based
> > on
> > > > the
> > > > > > > topic
> > > > > > > > >>> > names.
> > > > > > > > >>> > > > >> > > > > LeaderAndIsrRequest is one of the few
> > > requests
> > > > > > that
> > > > > > > > will
> > > > > > > > >>> > still
> > > > > > > > >>> > > > >> > contain
> > > > > > > > >>> > > > >> > > > the
> > > > > > > > >>> > > > >> > > > > topic name. So I think we have this
> > covered.
> > > > Sorry
> > > > > > > for
> > > > > > > > >>> > > > confusion.
> > > > > > > > >>> > > > >> > > > >
> > > > > > > > >>> > > > >> > > > > 2. Part of the usage of the file is to
> > have
> > > > > > > persistent
> > > > > > > > >>> > storage
> > > > > > > > >>> > > > of
> > > > > > > > >>> > > > >> the
> > > > > > > > >>> > > > >> > > > topic
> > > > > > > > >>> > > > >> > > > > ID and use it to compare with the ID
> > > supplied
> > > > in
> > > > > > the
> > > > > > > > >>> > > > LeaderAndIsr
> > > > > > > > >>> > > > >> > > > Request.
> > > > > > > > >>> > > > >> > > > > There is some discussion in the KIP
> about
> > > > changes
> > > > > > to
> > > > > > > > the
> > > > > > > > >>> > > > directory
> > > > > > > > >>> > > > >> > > > > structure, but I believe directory
> changes
> > > > were
> > > > > > > > >>> considered
> > > > > > > > >>> > to
> > > > > > > > >>> > > be
> > > > > > > > >>> > > > >> out
> > > > > > > > >>> > > > >> > of
> > > > > > > > >>> > > > >> > > > > scope when the KIP was written.
> > > > > > > > >>> > > > >> > > > >
> > > > > > > > >>> > > > >> > > > > 3. I think this is a good point, but I
> > > again I
> > > > > > > wonder
> > > > > > > > >>> about
> > > > > > > > >>> > > the
> > > > > > > > >>> > > > >> scope
> > > > > > > > >>> > > > >> > > of
> > > > > > > > >>> > > > >> > > > > the KIP. Most of the changes mentioned
> in
> > > the
> > > > KIP
> > > > > > > are
> > > > > > > > >>> for
> > > > > > > > >>> > > > >> supporting
> > > > > > > > >>> > > > >> > > > topic
> > > > > > > > >>> > > > >> > > > > deletion and I believe that is why the
> > > produce
> > > > > > > request
> > > > > > > > >>> was
> > > > > > > > >>> > > > listed
> > > > > > > > >>> > > > >> > under
> > > > > > > > >>> > > > >> > > > > future work.
> > > > > > > > >>> > > > >> > > > >
> > > > > > > > >>> > > > >> > > > > 4. This sounds like it might be a good
> > > > solution,
> > > > > > > but I
> > > > > > > > >>> will
> > > > > > > > >>> > > need
> > > > > > > > >>> > > > >> to
> > > > > > > > >>> > > > >> > > > discuss
> > > > > > > > >>> > > > >> > > > > more with KIP-500 folks to get the
> details
> > > > right.
> > > > > > > > >>> > > > >> > > > >
> > > > > > > > >>> > > > >> > > > > Thanks,
> > > > > > > > >>> > > > >> > > > > Justine
> > > > > > > > >>> > > > >> > > > >
> > > > > > > > >>> > > > >> > > > > On Thu, Sep 24, 2020 at 12:30 PM Jason
> > > > Gustafson <
> > > > > > > > >>> > > > >> jason@confluent.io
> > > > > > > > >>> > > > >> > >
> > > > > > > > >>> > > > >> > > > > wrote:
> > > > > > > > >>> > > > >> > > > >
> > > > > > > > >>> > > > >> > > > > > Hi Justine,
> > > > > > > > >>> > > > >> > > > > >
> > > > > > > > >>> > > > >> > > > > > Thanks for picking up this work. I
> have
> > a
> > > > few
> > > > > > > > >>> > > > >> questions/comments:
> > > > > > > > >>> > > > >> > > > > >
> > > > > > > > >>> > > > >> > > > > > 1. It sounds like the directory
> > structure
> > > is
> > > > > > still
> > > > > > > > >>> going
> > > > > > > > >>> > to
> > > > > > > > >>> > > be
> > > > > > > > >>> > > > >> > based
> > > > > > > > >>> > > > >> > > on
> > > > > > > > >>> > > > >> > > > > > topic names. Do I have that right? One
> > > > > > > complication
> > > > > > > > is
> > > > > > > > >>> > that
> > > > > > > > >>> > > > the
> > > > > > > > >>> > > > >> > > > > > LeaderAndIsr request does not include
> > the
> > > > topic
> > > > > > > name
> > > > > > > > >>> any
> > > > > > > > >>> > > > longer.
> > > > > > > > >>> > > > >> > This
> > > > > > > > >>> > > > >> > > > > means
> > > > > > > > >>> > > > >> > > > > > that a replica must wait for the
> > > > UpdateMetadata
> > > > > > > > >>> request to
> > > > > > > > >>> > > > >> arrive
> > > > > > > > >>> > > > >> > > with
> > > > > > > > >>> > > > >> > > > > the
> > > > > > > > >>> > > > >> > > > > > topic name mapping before it can
> create
> > > the
> > > > > > > > >>> directory. I
> > > > > > > > >>> > > > wonder
> > > > > > > > >>> > > > >> if
> > > > > > > > >>> > > > >> > it
> > > > > > > > >>> > > > >> > > > > would
> > > > > > > > >>> > > > >> > > > > > be simpler to avoid assumptions on the
> > > > ordering
> > > > > > of
> > > > > > > > >>> > > > >> UpdateMetadata
> > > > > > > > >>> > > > >> > and
> > > > > > > > >>> > > > >> > > > let
> > > > > > > > >>> > > > >> > > > > > LeaderAndIsr include the topic name as
> > > well.
> > > > > > Feels
> > > > > > > > >>> like we
> > > > > > > > >>> > > are
> > > > > > > > >>> > > > >> not
> > > > > > > > >>> > > > >> > > > saving
> > > > > > > > >>> > > > >> > > > > > that much by excluding it.
> > > > > > > > >>> > > > >> > > > > >
> > > > > > > > >>> > > > >> > > > > > 2. On a related note, it seems that
> the
> > > > reason
> > > > > > we
> > > > > > > > >>> have the
> > > > > > > > >>> > > > >> > partition
> > > > > > > > >>> > > > >> > > > > > metadata file is because we are not
> > > > changing the
> > > > > > > > >>> directory
> > > > > > > > >>> > > > >> > structure.
> > > > > > > > >>> > > > >> > > > We
> > > > > > > > >>> > > > >> > > > > > need it so that we remember which
> > > > directories
> > > > > > map
> > > > > > > to
> > > > > > > > >>> which
> > > > > > > > >>> > > > topic
> > > > > > > > >>> > > > >> > id.
> > > > > > > > >>> > > > >> > > I
> > > > > > > > >>> > > > >> > > > > > think it would be useful to add some
> > > detail
> > > > > > about
> > > > > > > > why
> > > > > > > > >>> > > changing
> > > > > > > > >>> > > > >> the
> > > > > > > > >>> > > > >> > > > > > directory layout was rejected.
> Basically
> > > > trying
> > > > > > to
> > > > > > > > >>> > > understand
> > > > > > > > >>> > > > >> how
> > > > > > > > >>> > > > >> > > much
> > > > > > > > >>> > > > >> > > > > > effort we are saving by skipping this
> > step
> > > > if we
> > > > > > > > have
> > > > > > > > >>> to
> > > > > > > > >>> > > > >> implement
> > > > > > > > >>> > > > >> > > this
> > > > > > > > >>> > > > >> > > > > > additional bookkeeping. One concern I
> > have
> > > > for
> > > > > > > > >>> example is
> > > > > > > > >>> > > that
> > > > > > > > >>> > > > >> the
> > > > > > > > >>> > > > >> > > > > > partition metadata file gets corrupt
> and
> > > > then
> > > > > > all
> > > > > > > > >>> bets are
> > > > > > > > >>> > > off
> > > > > > > > >>> > > > >> as
> > > > > > > > >>> > > > >> > far
> > > > > > > > >>> > > > >> > > > as
> > > > > > > > >>> > > > >> > > > > > topic consistency.
> > > > > > > > >>> > > > >> > > > > >
> > > > > > > > >>> > > > >> > > > > > 3. Is it worth adding support now for
> > > topic
> > > > ids
> > > > > > in
> > > > > > > > the
> > > > > > > > >>> > > > `Produce`
> > > > > > > > >>> > > > >> > > > request
> > > > > > > > >>> > > > >> > > > > > now? Overhead is mentioned as one of
> the
> > > > > > > motivations
> > > > > > > > >>> and
> > > > > > > > >>> > the
> > > > > > > > >>> > > > >> best
> > > > > > > > >>> > > > >> > > APIs
> > > > > > > > >>> > > > >> > > > to
> > > > > > > > >>> > > > >> > > > > > get savings from are `Produce` and
> > > `Fetch`.
> > > > > > > > >>> > > > >> > > > > >
> > > > > > > > >>> > > > >> > > > > > 4. When it comes to bootstrapping the
> > > > metadata
> > > > > > > topic
> > > > > > > > >>> with
> > > > > > > > >>> > > > >> respect
> > > > > > > > >>> > > > >> > to
> > > > > > > > >>> > > > >> > > > > > KIP-500, one suggestion would be to
> > > reserve
> > > > a
> > > > > > > > sentinel
> > > > > > > > >>> > topic
> > > > > > > > >>> > > > ID
> > > > > > > > >>> > > > >> > which
> > > > > > > > >>> > > > >> > > > can
> > > > > > > > >>> > > > >> > > > > > be used initially by new brokers. When
> > the
> > > > first
> > > > > > > > >>> > controller
> > > > > > > > >>> > > is
> > > > > > > > >>> > > > >> > > elected,
> > > > > > > > >>> > > > >> > > > > it
> > > > > > > > >>> > > > >> > > > > > can assign a topicId to the metadata
> > > topic.
> > > > This
> > > > > > > is
> > > > > > > > a
> > > > > > > > >>> bit
> > > > > > > > >>> > > > >> similar
> > > > > > > > >>> > > > >> > to
> > > > > > > > >>> > > > >> > > > how
> > > > > > > > >>> > > > >> > > > > we
> > > > > > > > >>> > > > >> > > > > > were planning to generate the
> clusterId.
> > > > > > > > >>> > > > >> > > > > >
> > > > > > > > >>> > > > >> > > > > > Thanks,
> > > > > > > > >>> > > > >> > > > > > Jason
> > > > > > > > >>> > > > >> > > > > >
> > > > > > > > >>> > > > >> > > > > >
> > > > > > > > >>> > > > >> > > > > > On Thu, Sep 24, 2020 at 11:10 AM Jun
> > Rao <
> > > > > > > > >>> > jun@confluent.io>
> > > > > > > > >>> > > > >> wrote:
> > > > > > > > >>> > > > >> > > > > >
> > > > > > > > >>> > > > >> > > > > > > Hi, Justine,
> > > > > > > > >>> > > > >> > > > > > >
> > > > > > > > >>> > > > >> > > > > > > Thanks for the updated KIP. A few
> more
> > > > > > comments
> > > > > > > > >>> below.
> > > > > > > > >>> > > > >> > > > > > >
> > > > > > > > >>> > > > >> > > > > > > 30.  {"name": "id", "type":
> "string",
> > > > "doc":
> > > > > > > > >>> "version
> > > > > > > > >>> > > id"}}:
> > > > > > > > >>> > > > >> The
> > > > > > > > >>> > > > >> > > doc
> > > > > > > > >>> > > > >> > > > > > should
> > > > > > > > >>> > > > >> > > > > > > say UUID.
> > > > > > > > >>> > > > >> > > > > > >
> > > > > > > > >>> > > > >> > > > > > > 31. LeaderAndIsrResponse v5 and
> > > > > > > > StopReplicaResponse
> > > > > > > > >>> v4 :
> > > > > > > > >>> > > It
> > > > > > > > >>> > > > >> seems
> > > > > > > > >>> > > > >> > > > there
> > > > > > > > >>> > > > >> > > > > > is
> > > > > > > > >>> > > > >> > > > > > > no need to add topic_id at
> partitions
> > > > level.
> > > > > > > > >>> > > > >> > > > > > >
> > > > > > > > >>> > > > >> > > > > > > 32. Regarding partition metadata
> file.
> > > > Perhaps
> > > > > > > the
> > > > > > > > >>> key
> > > > > > > > >>> > can
> > > > > > > > >>> > > > be
> > > > > > > > >>> > > > >> a
> > > > > > > > >>> > > > >> > > > single
> > > > > > > > >>> > > > >> > > > > > > word, sth like the following.
> > > > > > > > >>> > > > >> > > > > > > version: 0
> > > > > > > > >>> > > > >> > > > > > > topic_id:
> > > > 46bdb63f-9e8d-4a38-bf7b-ee4eb2a794e4
> > > > > > > > >>> > > > >> > > > > > >
> > > > > > > > >>> > > > >> > > > > > > 33. Another tricky thing that I
> > realized
> > > > is
> > > > > > how
> > > > > > > to
> > > > > > > > >>> > support
> > > > > > > > >>> > > > the
> > > > > > > > >>> > > > >> > > > metadata
> > > > > > > > >>> > > > >> > > > > > > topic introduced in KIP-595. It's a
> > bit
> > > > tricky
> > > > > > > to
> > > > > > > > >>> > assign a
> > > > > > > > >>> > > > >> UUID
> > > > > > > > >>> > > > >> > to
> > > > > > > > >>> > > > >> > > > the
> > > > > > > > >>> > > > >> > > > > > > metadata topic since we have a
> > > > chicken-and-egg
> > > > > > > > >>> problem.
> > > > > > > > >>> > > The
> > > > > > > > >>> > > > >> > > > controller
> > > > > > > > >>> > > > >> > > > > > > needs to persist the UUID in the
> > > metadata
> > > > > > topic
> > > > > > > in
> > > > > > > > >>> order
> > > > > > > > >>> > > to
> > > > > > > > >>> > > > >> > assign
> > > > > > > > >>> > > > >> > > > one
> > > > > > > > >>> > > > >> > > > > > > successfully, but the metadata topic
> > is
> > > > needed
> > > > > > > to
> > > > > > > > >>> elect
> > > > > > > > >>> > a
> > > > > > > > >>> > > > >> > > controller
> > > > > > > > >>> > > > >> > > > > > > (KIP-631). So, this probably needs a
> > bit
> > > > more
> > > > > > > > >>> thought.
> > > > > > > > >>> > > > >> > > > > > >
> > > > > > > > >>> > > > >> > > > > > > Jun
> > > > > > > > >>> > > > >> > > > > > >
> > > > > > > > >>> > > > >> > > > > > > On Thu, Sep 24, 2020 at 3:04 AM
> Ismael
> > > > Juma <
> > > > > > > > >>> > > > >> ismael@juma.me.uk>
> > > > > > > > >>> > > > >> > > > wrote:
> > > > > > > > >>> > > > >> > > > > > >
> > > > > > > > >>> > > > >> > > > > > > > Also, can we provide more details
> on
> > > > how the
> > > > > > > > >>> Partition
> > > > > > > > >>> > > > >> Metadata
> > > > > > > > >>> > > > >> > > > file
> > > > > > > > >>> > > > >> > > > > > will
> > > > > > > > >>> > > > >> > > > > > > > be used?
> > > > > > > > >>> > > > >> > > > > > > >
> > > > > > > > >>> > > > >> > > > > > > > Ismael
> > > > > > > > >>> > > > >> > > > > > > >
> > > > > > > > >>> > > > >> > > > > > > > On Thu, Sep 24, 2020 at 3:01 AM
> > Ismael
> > > > Juma
> > > > > > <
> > > > > > > > >>> > > > >> ismael@juma.me.uk
> > > > > > > > >>> > > > >> > >
> > > > > > > > >>> > > > >> > > > > wrote:
> > > > > > > > >>> > > > >> > > > > > > >
> > > > > > > > >>> > > > >> > > > > > > > > Hi Justine,
> > > > > > > > >>> > > > >> > > > > > > > >
> > > > > > > > >>> > > > >> > > > > > > > > I think we need to update the
> > > > "Rejected
> > > > > > > > >>> > Alternatives"
> > > > > > > > >>> > > > >> section
> > > > > > > > >>> > > > >> > > to
> > > > > > > > >>> > > > >> > > > > take
> > > > > > > > >>> > > > >> > > > > > > > into
> > > > > > > > >>> > > > >> > > > > > > > > account that the proposal now
> > > removes
> > > > the
> > > > > > > > topic
> > > > > > > > >>> name
> > > > > > > > >>> > > > from
> > > > > > > > >>> > > > >> the
> > > > > > > > >>> > > > >> > > > fetch
> > > > > > > > >>> > > > >> > > > > > > > > request. Also, if we are
> removing
> > it
> > > > from
> > > > > > > the
> > > > > > > > >>> Fetch
> > > > > > > > >>> > > > >> request,
> > > > > > > > >>> > > > >> > > does
> > > > > > > > >>> > > > >> > > > > it
> > > > > > > > >>> > > > >> > > > > > > make
> > > > > > > > >>> > > > >> > > > > > > > > sense not to remove it from
> > similar
> > > > > > requests
> > > > > > > > >>> like
> > > > > > > > >>> > > > >> > > > > ListOffsetRequest?
> > > > > > > > >>> > > > >> > > > > > > > >
> > > > > > > > >>> > > > >> > > > > > > > > Ismael
> > > > > > > > >>> > > > >> > > > > > > > >
> > > > > > > > >>> > > > >> > > > > > > > > On Thu, Sep 24, 2020 at 2:46 AM
> > > David
> > > > > > Jacot
> > > > > > > <
> > > > > > > > >>> > > > >> > > djacot@confluent.io
> > > > > > > > >>> > > > >> > > > >
> > > > > > > > >>> > > > >> > > > > > > wrote:
> > > > > > > > >>> > > > >> > > > > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> Hi Justine,
> > > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > > >>> > > > >> > > > > > > > >> Thanks for the KIP. I finally
> had
> > > > time to
> > > > > > > > read
> > > > > > > > >>> it
> > > > > > > > >>> > :).
> > > > > > > > >>> > > > It
> > > > > > > > >>> > > > >> is
> > > > > > > > >>> > > > >> > a
> > > > > > > > >>> > > > >> > > > > great
> > > > > > > > >>> > > > >> > > > > > > > >> improvement.
> > > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > > >>> > > > >> > > > > > > > >> I have a few
> comments/questions:
> > > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > > >>> > > > >> > > > > > > > >> 1. It seems that the schema of
> > the
> > > > > > > > >>> > StopReplicaRequest
> > > > > > > > >>> > > > is
> > > > > > > > >>> > > > >> > > > slightly
> > > > > > > > >>> > > > >> > > > > > > > >> outdated.
> > > > > > > > >>> > > > >> > > > > > > > >> We
> > > > > > > > >>> > > > >> > > > > > > > >> did some changes as part of
> > > KIP-570.
> > > > V3
> > > > > > is
> > > > > > > > >>> already
> > > > > > > > >>> > > > >> organized
> > > > > > > > >>> > > > >> > > by
> > > > > > > > >>> > > > >> > > > > > > topics.
> > > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > > >>> > > > >> > > > > > > > >> 2. I just want to make sure
> that
> > I
> > > > > > > understand
> > > > > > > > >>> the
> > > > > > > > >>> > > > >> > > reconciliation
> > > > > > > > >>> > > > >> > > > > > > > >> logic correctly. When
> > > > > > > > >>> > > > >> > > > > > > > >> an "INCREMENTAL" LeaderAndIsr
> > > > Request is
> > > > > > > > >>> received,
> > > > > > > > >>> > > the
> > > > > > > > >>> > > > >> > broker
> > > > > > > > >>> > > > >> > > > will
> > > > > > > > >>> > > > >> > > > > > > also
> > > > > > > > >>> > > > >> > > > > > > > >> reconcile
> > > > > > > > >>> > > > >> > > > > > > > >> when the local uuid does not
> > match
> > > > the
> > > > > > uuid
> > > > > > > > in
> > > > > > > > >>> the
> > > > > > > > >>> > > > >> request,
> > > > > > > > >>> > > > >> > > > right?
> > > > > > > > >>> > > > >> > > > > > In
> > > > > > > > >>> > > > >> > > > > > > > this
> > > > > > > > >>> > > > >> > > > > > > > >> case, the
> > > > > > > > >>> > > > >> > > > > > > > >> local log is staged for
> deletion.
> > > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > > >>> > > > >> > > > > > > > >> 3. In the documentation of the
> `
> > > > > > > > >>> > > > >> delete.stale.topic.delay.ms
> > > > > > > > >>> > > > >> > `
> > > > > > > > >>> > > > >> > > > > > config,
> > > > > > > > >>> > > > >> > > > > > > it
> > > > > > > > >>> > > > >> > > > > > > > >> says "When a
> > > > > > > > >>> > > > >> > > > > > > > >> FULL LeaderAndIsrRequest is
> > > > received..."
> > > > > > > but
> > > > > > > > I
> > > > > > > > >>> > > suppose
> > > > > > > > >>> > > > it
> > > > > > > > >>> > > > >> > > > applies
> > > > > > > > >>> > > > >> > > > > to
> > > > > > > > >>> > > > >> > > > > > > > both
> > > > > > > > >>> > > > >> > > > > > > > >> types.
> > > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > > >>> > > > >> > > > > > > > >> Best,
> > > > > > > > >>> > > > >> > > > > > > > >> David
> > > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > > >>> > > > >> > > > > > > > >> On Thu, Sep 24, 2020 at 1:40 AM
> > > > Justine
> > > > > > > > Olshan
> > > > > > > > >>> <
> > > > > > > > >>> > > > >> > > > > > jolshan@confluent.io>
> > > > > > > > >>> > > > >> > > > > > > > >> wrote:
> > > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > > >>> > > > >> > > > > > > > >> > Hi Jun,
> > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > >>> > > > >> > > > > > > > >> > Thanks for the comments. I
> > > > apologize
> > > > > > for
> > > > > > > > >>> some of
> > > > > > > > >>> > > the
> > > > > > > > >>> > > > >> typos
> > > > > > > > >>> > > > >> > > and
> > > > > > > > >>> > > > >> > > > > > > > >> confusion.
> > > > > > > > >>> > > > >> > > > > > > > >> > I’ve updated the KIP to fix
> > some
> > > > of the
> > > > > > > > >>> issues
> > > > > > > > >>> > you
> > > > > > > > >>> > > > >> > > mentioned.
> > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > >>> > > > >> > > > > > > > >> > 20.2 I’ve changed the type to
> > > > String.
> > > > > > > > >>> > > > >> > > > > > > > >> > 20.1/3 I’ve updated the
> > > TopicZNode
> > > > to
> > > > > > fix
> > > > > > > > >>> > > formatting
> > > > > > > > >>> > > > >> and
> > > > > > > > >>> > > > >> > > > reflect
> > > > > > > > >>> > > > >> > > > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> latest
> > > > > > > > >>> > > > >> > > > > > > > >> > version before this change.
> > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > >>> > > > >> > > > > > > > >> > 21. You are correct and I’ve
> > > > removed
> > > > > > this
> > > > > > > > >>> line.
> > > > > > > > >>> > > I’ve
> > > > > > > > >>> > > > >> also
> > > > > > > > >>> > > > >> > > > added
> > > > > > > > >>> > > > >> > > > > a
> > > > > > > > >>> > > > >> > > > > > > line
> > > > > > > > >>> > > > >> > > > > > > > >> > mentioning an IBP bump is
> > > > necessary for
> > > > > > > > >>> migration
> > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > >>> > > > >> > > > > > > > >> > 22. I think the wording was
> > > > unclear but
> > > > > > > > your
> > > > > > > > >>> > > summary
> > > > > > > > >>> > > > is
> > > > > > > > >>> > > > >> > what
> > > > > > > > >>> > > > >> > > > was
> > > > > > > > >>> > > > >> > > > > > > > >> intended
> > > > > > > > >>> > > > >> > > > > > > > >> > by this line. I’ve updated to
> > > make
> > > > this
> > > > > > > > point
> > > > > > > > >>> > more
> > > > > > > > >>> > > > >> clear.
> > > > > > > > >>> > > > >> > > > > “Remove
> > > > > > > > >>> > > > >> > > > > > > > >> deleted
> > > > > > > > >>> > > > >> > > > > > > > >> > topics from replicas by
> sending
> > > > > > > > >>> > StopReplicaRequest
> > > > > > > > >>> > > V3
> > > > > > > > >>> > > > >> > before
> > > > > > > > >>> > > > >> > > > the
> > > > > > > > >>> > > > >> > > > > > IBP
> > > > > > > > >>> > > > >> > > > > > > > >> bump
> > > > > > > > >>> > > > >> > > > > > > > >> > using the old logic, and
> using
> > V4
> > > > and
> > > > > > the
> > > > > > > > new
> > > > > > > > >>> > logic
> > > > > > > > >>> > > > >> with
> > > > > > > > >>> > > > >> > > topic
> > > > > > > > >>> > > > >> > > > > IDs
> > > > > > > > >>> > > > >> > > > > > > > after
> > > > > > > > >>> > > > >> > > > > > > > >> > the IBP bump.”
> > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > >>> > > > >> > > > > > > > >> > 23. I’ve removed the
> > unspecified
> > > > type
> > > > > > > from
> > > > > > > > >>> the
> > > > > > > > >>> > KIP
> > > > > > > > >>> > > > and
> > > > > > > > >>> > > > >> > > mention
> > > > > > > > >>> > > > >> > > > > > that
> > > > > > > > >>> > > > >> > > > > > > > IBP
> > > > > > > > >>> > > > >> > > > > > > > >> > will be used to handle this
> > > > request.
> > > > > > “IBP
> > > > > > > > >>> will be
> > > > > > > > >>> > > > used
> > > > > > > > >>> > > > >> to
> > > > > > > > >>> > > > >> > > > > > determine
> > > > > > > > >>> > > > >> > > > > > > > >> whether
> > > > > > > > >>> > > > >> > > > > > > > >> > this new form of the request
> > will
> > > > be
> > > > > > > used.
> > > > > > > > >>> For
> > > > > > > > >>> > > older
> > > > > > > > >>> > > > >> > > requests,
> > > > > > > > >>> > > > >> > > > > we
> > > > > > > > >>> > > > >> > > > > > > will
> > > > > > > > >>> > > > >> > > > > > > > >> > ignore this field and default
> > to
> > > > > > previous
> > > > > > > > >>> > > behavior.”
> > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > >>> > > > >> > > > > > > > >> > 24. I’ve fixed this typo.
> > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > >>> > > > >> > > > > > > > >> > 25. I've added a topics at a
> > > higher
> > > > > > level
> > > > > > > > for
> > > > > > > > >>> > > > >> > > > > LeaderAndIsrResponse
> > > > > > > > >>> > > > >> > > > > > > v5,
> > > > > > > > >>> > > > >> > > > > > > > >> > StopReplicaResponse v4. I've
> > also
> > > > > > changed
> > > > > > > > >>> > > > >> > StopReplicaRequest
> > > > > > > > >>> > > > >> > > > v4
> > > > > > > > >>> > > > >> > > > > to
> > > > > > > > >>> > > > >> > > > > > > > have
> > > > > > > > >>> > > > >> > > > > > > > >> > topics at a higher level.
> > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > >>> > > > >> > > > > > > > >> > 26. I’ve updated
> > > > > > > > >>> forgotten_topics_data--added the
> > > > > > > > >>> > > > >> topic ID
> > > > > > > > >>> > > > >> > > and
> > > > > > > > >>> > > > >> > > > > > > removed
> > > > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > > > >>> > > > >> > > > > > > > >> > topic name
> > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > >>> > > > >> > > > > > > > >> > 27. I’ve decided on plain
> text,
> > > and
> > > > > > I’ve
> > > > > > > > >>> added an
> > > > > > > > >>> > > > >> example.
> > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > >>> > > > >> > > > > > > > >> > 28. I’ve added this idea to
> > > future
> > > > > > work.
> > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > >>> > > > >> > > > > > > > >> > Thanks again for taking a
> look,
> > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > >>> > > > >> > > > > > > > >> > Justine
> > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > >>> > > > >> > > > > > > > >> > On Wed, Sep 23, 2020 at 10:28
> > AM
> > > > Jun
> > > > > > Rao
> > > > > > > <
> > > > > > > > >>> > > > >> > jun@confluent.io>
> > > > > > > > >>> > > > >> > > > > > wrote:
> > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > >>> > > > >> > > > > > > > >> > > Hi, Justine,
> > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > Thanks for the response.
> Made
> > > > another
> > > > > > > > >>> pass. A
> > > > > > > > >>> > few
> > > > > > > > >>> > > > >> more
> > > > > > > > >>> > > > >> > > > > comments
> > > > > > > > >>> > > > >> > > > > > > > below.
> > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > 20. znode schema:
> > > > > > > > >>> > > > >> > > > > > > > >> > > 20.1 It seems that {"name":
> > > > > > "version",
> > > > > > > > >>> "type":
> > > > > > > > >>> > > > "int",
> > > > > > > > >>> > > > >> > > "id":
> > > > > > > > >>> > > > >> > > > > > > "UUID",
> > > > > > > > >>> > > > >> > > > > > > > >> > "doc":
> > > > > > > > >>> > > > >> > > > > > > > >> > > "version id"} should be
> > > {"name":
> > > > > > > > "version",
> > > > > > > > >>> > > "type":
> > > > > > > > >>> > > > >> > > "int"},
> > > > > > > > >>> > > > >> > > > > > > {"name":
> > > > > > > > >>> > > > >> > > > > > > > >> > "id",
> > > > > > > > >>> > > > >> > > > > > > > >> > > "type": "UUID", "doc":
> > "version
> > > > id"}.
> > > > > > > > >>> > > > >> > > > > > > > >> > > 20.2 The znode format is
> JSON
> > > > which
> > > > > > > > doesn't
> > > > > > > > >>> > have
> > > > > > > > >>> > > > UUID
> > > > > > > > >>> > > > >> > > type.
> > > > > > > > >>> > > > >> > > > So
> > > > > > > > >>> > > > >> > > > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> type
> > > > > > > > >>> > > > >> > > > > > > > >> > > probably should be string?
> > > > > > > > >>> > > > >> > > > > > > > >> > > 20.3 Also, the existing
> > format
> > > > used
> > > > > > > seems
> > > > > > > > >>> > > outdated.
> > > > > > > > >>> > > > >> It
> > > > > > > > >>> > > > >> > > > should
> > > > > > > > >>> > > > >> > > > > > have
> > > > > > > > >>> > > > >> > > > > > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > following format.
> > > > > > > > >>> > > > >> > > > > > > > >> > >     Json.encodeAsBytes(Map(
> > > > > > > > >>> > > > >> > > > > > > > >> > >       "version" -> 2,
> > > > > > > > >>> > > > >> > > > > > > > >> > >       "partitions" ->
> > > > > > > > >>> > > replicaAssignmentJson.asJava,
> > > > > > > > >>> > > > >> > > > > > > > >> > >       "adding_replicas" ->
> > > > > > > > >>> > > > >> > > > > addingReplicasAssignmentJson.asJava,
> > > > > > > > >>> > > > >> > > > > > > > >> > >       "removing_replicas"
> ->
> > > > > > > > >>> > > > >> > > > > > removingReplicasAssignmentJson.asJava
> > > > > > > > >>> > > > >> > > > > > > > >> > >     ).asJava)
> > > > > > > > >>> > > > >> > > > > > > > >> > >   }
> > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > 21. Migration: The KIP says
> > > "The
> > > > > > > > migration
> > > > > > > > >>> > > process
> > > > > > > > >>> > > > >> can
> > > > > > > > >>> > > > >> > > take
> > > > > > > > >>> > > > >> > > > > > place
> > > > > > > > >>> > > > >> > > > > > > > >> without
> > > > > > > > >>> > > > >> > > > > > > > >> > > an inter-broker protocol
> > bump,
> > > > as the
> > > > > > > > >>> format
> > > > > > > > >>> > > stored
> > > > > > > > >>> > > > >> in
> > > > > > > > >>> > > > >> > > > > > > > >> > > /brokers/topics/[topic]
> will
> > be
> > > > > > > > compatible
> > > > > > > > >>> with
> > > > > > > > >>> > > > older
> > > > > > > > >>> > > > >> > > broker
> > > > > > > > >>> > > > >> > > > > > > > >> versions."
> > > > > > > > >>> > > > >> > > > > > > > >> > > However, since we are
> bumping
> > > up
> > > > the
> > > > > > > > >>> version of
> > > > > > > > >>> > > > >> > > inter-broker
> > > > > > > > >>> > > > >> > > > > > > > >> requests, it
> > > > > > > > >>> > > > >> > > > > > > > >> > > seems that we need to use
> IBP
> > > for
> > > > > > > > >>> migration.
> > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > 22. The KIP says "Remove
> > > deleted
> > > > > > topics
> > > > > > > > >>> from
> > > > > > > > >>> > > > >> replicas by
> > > > > > > > >>> > > > >> > > > > sending
> > > > > > > > >>> > > > >> > > > > > > > >> > > StopReplicaRequest V3 for
> any
> > > > topics
> > > > > > > > which
> > > > > > > > >>> do
> > > > > > > > >>> > not
> > > > > > > > >>> > > > >> > contain
> > > > > > > > >>> > > > >> > > a
> > > > > > > > >>> > > > >> > > > > > topic
> > > > > > > > >>> > > > >> > > > > > > > ID,
> > > > > > > > >>> > > > >> > > > > > > > >> and
> > > > > > > > >>> > > > >> > > > > > > > >> > > V4 for any topics which do
> > > > contain a
> > > > > > > > topic
> > > > > > > > >>> > ID.".
> > > > > > > > >>> > > > >> > However,
> > > > > > > > >>> > > > >> > > if
> > > > > > > > >>> > > > >> > > > > we
> > > > > > > > >>> > > > >> > > > > > > use
> > > > > > > > >>> > > > >> > > > > > > > >> IBP,
> > > > > > > > >>> > > > >> > > > > > > > >> > it
> > > > > > > > >>> > > > >> > > > > > > > >> > > seems that the controller
> > will
> > > > either
> > > > > > > > send
> > > > > > > > >>> > > > >> > > > StopReplicaRequest
> > > > > > > > >>> > > > >> > > > > V3
> > > > > > > > >>> > > > >> > > > > > > > >> > > or StopReplicaRequest V4,
> but
> > > > never
> > > > > > > mixed
> > > > > > > > >>> V3
> > > > > > > > >>> > and
> > > > > > > > >>> > > V4
> > > > > > > > >>> > > > >> for
> > > > > > > > >>> > > > >> > > > > > different
> > > > > > > > >>> > > > >> > > > > > > > >> topics.
> > > > > > > > >>> > > > >> > > > > > > > >> > > Basically, before the IBP
> > bump,
> > > > V3
> > > > > > will
> > > > > > > > be
> > > > > > > > >>> > used.
> > > > > > > > >>> > > > >> After
> > > > > > > > >>> > > > >> > the
> > > > > > > > >>> > > > >> > > > IBP
> > > > > > > > >>> > > > >> > > > > > > bump,
> > > > > > > > >>> > > > >> > > > > > > > >> > > topicId will be created and
> > V4
> > > > will
> > > > > > be
> > > > > > > > >>> used.
> > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > 23. Given that we depend on
> > > IBP,
> > > > do
> > > > > > we
> > > > > > > > >>> still
> > > > > > > > >>> > need
> > > > > > > > >>> > > > "0
> > > > > > > > >>> > > > >> > > > > > UNSPECIFIED"
> > > > > > > > >>> > > > >> > > > > > > > >> > > in LeaderAndIsr?
> > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > 24. LeaderAndIsrResponse
> v5 :
> > > It
> > > > > > still
> > > > > > > > has
> > > > > > > > >>> the
> > > > > > > > >>> > > > topic
> > > > > > > > >>> > > > >> > > field.
> > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > 25. LeaderAndIsrResponse
> v5,
> > > > > > > > >>> > StopReplicaResponse
> > > > > > > > >>> > > > v4:
> > > > > > > > >>> > > > >> > Could
> > > > > > > > >>> > > > >> > > > we
> > > > > > > > >>> > > > >> > > > > > use
> > > > > > > > >>> > > > >> > > > > > > > this
> > > > > > > > >>> > > > >> > > > > > > > >> > > opportunity to organize the
> > > > response
> > > > > > > in 2
> > > > > > > > >>> > levels,
> > > > > > > > >>> > > > >> first
> > > > > > > > >>> > > > >> > by
> > > > > > > > >>> > > > >> > > > > > topic,
> > > > > > > > >>> > > > >> > > > > > > > >> then by
> > > > > > > > >>> > > > >> > > > > > > > >> > > partition, as most other
> > > > > > > > >>> requests/responses?
> > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > 26. FetchRequest v13 :
> Should
> > > > > > > > >>> > > forgotten_topics_data
> > > > > > > > >>> > > > >> use
> > > > > > > > >>> > > > >> > > > > topicId
> > > > > > > > >>> > > > >> > > > > > > too?
> > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > 27. "This file can either
> be
> > > > plain
> > > > > > text
> > > > > > > > >>> > > (key/value
> > > > > > > > >>> > > > >> > pairs)
> > > > > > > > >>> > > > >> > > or
> > > > > > > > >>> > > > >> > > > > > > JSON."
> > > > > > > > >>> > > > >> > > > > > > > >> Have
> > > > > > > > >>> > > > >> > > > > > > > >> > we
> > > > > > > > >>> > > > >> > > > > > > > >> > > decided which one to use?
> > Also,
> > > > it
> > > > > > > would
> > > > > > > > be
> > > > > > > > >>> > > helpful
> > > > > > > > >>> > > > >> to
> > > > > > > > >>> > > > >> > > > provide
> > > > > > > > >>> > > > >> > > > > > an
> > > > > > > > >>> > > > >> > > > > > > > >> > example.
> > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > 28. Future improvement:
> > Another
> > > > > > future
> > > > > > > > >>> > > improvement
> > > > > > > > >>> > > > >> > > > opportunity
> > > > > > > > >>> > > > >> > > > > > is
> > > > > > > > >>> > > > >> > > > > > > to
> > > > > > > > >>> > > > >> > > > > > > > >> use
> > > > > > > > >>> > > > >> > > > > > > > >> > > topicId in
> > > > > > > > >>> GroupMetadataManager.offsetCommitKey
> > > > > > > > >>> > > in
> > > > > > > > >>> > > > >> the
> > > > > > > > >>> > > > >> > > > > > > offset_commit
> > > > > > > > >>> > > > >> > > > > > > > >> > topic.
> > > > > > > > >>> > > > >> > > > > > > > >> > > This may save some space.
> > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > Thanks,
> > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > Jun
> > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > On Wed, Sep 23, 2020 at
> 8:50
> > AM
> > > > > > Justine
> > > > > > > > >>> Olshan
> > > > > > > > >>> > <
> > > > > > > > >>> > > > >> > > > > > > > jolshan@confluent.io>
> > > > > > > > >>> > > > >> > > > > > > > >> > > wrote:
> > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > Hi Tom,
> > > > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > Thanks for the comment. I
> > > think
> > > > > > this
> > > > > > > > is a
> > > > > > > > >>> > > really
> > > > > > > > >>> > > > >> good
> > > > > > > > >>> > > > >> > > idea
> > > > > > > > >>> > > > >> > > > > and
> > > > > > > > >>> > > > >> > > > > > > it
> > > > > > > > >>> > > > >> > > > > > > > >> has
> > > > > > > > >>> > > > >> > > > > > > > >> > > been
> > > > > > > > >>> > > > >> > > > > > > > >> > > > added to the KIP under
> the
> > > > newly
> > > > > > > added
> > > > > > > > >>> > tooling
> > > > > > > > >>> > > > >> > section.
> > > > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > Thanks again,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > Justine
> > > > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > On Wed, Sep 23, 2020 at
> > 3:17
> > > > AM Tom
> > > > > > > > >>> Bentley <
> > > > > > > > >>> > > > >> > > > > > > tbentley@redhat.com>
> > > > > > > > >>> > > > >> > > > > > > > >> > wrote:
> > > > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > Hi Justine,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > I know you started the
> > vote
> > > > > > thread,
> > > > > > > > >>> but on
> > > > > > > > >>> > > > >> > re-reading
> > > > > > > > >>> > > > >> > > > the
> > > > > > > > >>> > > > >> > > > > > KIP
> > > > > > > > >>> > > > >> > > > > > > I
> > > > > > > > >>> > > > >> > > > > > > > >> > noticed
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > that although the topic
> > id
> > > is
> > > > > > > > included
> > > > > > > > >>> in
> > > > > > > > >>> > the
> > > > > > > > >>> > > > >> > > > > > MetadataResponse
> > > > > > > > >>> > > > >> > > > > > > > >> it's
> > > > > > > > >>> > > > >> > > > > > > > >> > not
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > surfaced in the output
> > from
> > > > > > > > >>> > `kafka-topics.sh
> > > > > > > > >>> > > > >> > > > --describe`.
> > > > > > > > >>> > > > >> > > > > > > Maybe
> > > > > > > > >>> > > > >> > > > > > > > >> that
> > > > > > > > >>> > > > >> > > > > > > > >> > > was
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > intentional because ids
> > are
> > > > > > > > >>> intentionally
> > > > > > > > >>> > not
> > > > > > > > >>> > > > >> really
> > > > > > > > >>> > > > >> > > > > > something
> > > > > > > > >>> > > > >> > > > > > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > user
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > should care deeply
> about,
> > > > but it
> > > > > > > > would
> > > > > > > > >>> also
> > > > > > > > >>> > > > make
> > > > > > > > >>> > > > >> > life
> > > > > > > > >>> > > > >> > > > > harder
> > > > > > > > >>> > > > >> > > > > > > for
> > > > > > > > >>> > > > >> > > > > > > > >> > anyone
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > debugging Kafka and
> this
> > > > would
> > > > > > > likely
> > > > > > > > >>> get
> > > > > > > > >>> > > worse
> > > > > > > > >>> > > > >> the
> > > > > > > > >>> > > > >> > > more
> > > > > > > > >>> > > > >> > > > > > topic
> > > > > > > > >>> > > > >> > > > > > > > ids
> > > > > > > > >>> > > > >> > > > > > > > >> > got
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > rolled out across the
> > > > protocols,
> > > > > > > > >>> clients
> > > > > > > > >>> > etc.
> > > > > > > > >>> > > > It
> > > > > > > > >>> > > > >> > seems
> > > > > > > > >>> > > > >> > > > > > likely
> > > > > > > > >>> > > > >> > > > > > > > that
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > `kafka-topics.sh` will
> > > > eventually
> > > > > > > > need
> > > > > > > > >>> the
> > > > > > > > >>> > > > >> ability
> > > > > > > > >>> > > > >> > to
> > > > > > > > >>> > > > >> > > > show
> > > > > > > > >>> > > > >> > > > > > the
> > > > > > > > >>> > > > >> > > > > > > > id
> > > > > > > > >>> > > > >> > > > > > > > >> of
> > > > > > > > >>> > > > >> > > > > > > > >> > a
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > topic and perhaps find
> a
> > > > topic
> > > > > > name
> > > > > > > > >>> given
> > > > > > > > >>> > an
> > > > > > > > >>> > > > id.
> > > > > > > > >>> > > > >> Is
> > > > > > > > >>> > > > >> > > > there
> > > > > > > > >>> > > > >> > > > > > any
> > > > > > > > >>> > > > >> > > > > > > > >> reason
> > > > > > > > >>> > > > >> > > > > > > > >> > > not
> > > > > > > > >>> > > > >> > > > > > > > >> > > > to
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > implement that in this
> > KIP?
> > > > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > Many thanks,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > Tom
> > > > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > On Mon, Sep 21, 2020 at
> > > 9:54
> > > > PM
> > > > > > > > Justine
> > > > > > > > >>> > > Olshan
> > > > > > > > >>> > > > <
> > > > > > > > >>> > > > >> > > > > > > > >> jolshan@confluent.io
> > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > wrote:
> > > > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Hi all,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > After thinking about
> > it,
> > > > I've
> > > > > > > > >>> decided to
> > > > > > > > >>> > > > remove
> > > > > > > > >>> > > > >> > the
> > > > > > > > >>> > > > >> > > > > topic
> > > > > > > > >>> > > > >> > > > > > > name
> > > > > > > > >>> > > > >> > > > > > > > >> from
> > > > > > > > >>> > > > >> > > > > > > > >> > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Fetch Request and
> > > Response
> > > > > > after
> > > > > > > > all.
> > > > > > > > >>> > Since
> > > > > > > > >>> > > > >> there
> > > > > > > > >>> > > > >> > > are
> > > > > > > > >>> > > > >> > > > so
> > > > > > > > >>> > > > >> > > > > > > many
> > > > > > > > >>> > > > >> > > > > > > > of
> > > > > > > > >>> > > > >> > > > > > > > >> > > these
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > requests per second,
> it
> > > is
> > > > > > worth
> > > > > > > > >>> removing
> > > > > > > > >>> > > the
> > > > > > > > >>> > > > >> > extra
> > > > > > > > >>> > > > >> > > > > > > > information.
> > > > > > > > >>> > > > >> > > > > > > > >> > I've
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > updated the KIP to
> > > reflect
> > > > this
> > > > > > > > >>> change.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Please let me know if
> > > > there is
> > > > > > > > >>> anything
> > > > > > > > >>> > > else
> > > > > > > > >>> > > > we
> > > > > > > > >>> > > > >> > > should
> > > > > > > > >>> > > > >> > > > > > > discuss
> > > > > > > > >>> > > > >> > > > > > > > >> > before
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > voting.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Thank you,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Justine
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > On Fri, Sep 18, 2020
> at
> > > > 9:46 AM
> > > > > > > > >>> Justine
> > > > > > > > >>> > > > Olshan
> > > > > > > > >>> > > > >> <
> > > > > > > > >>> > > > >> > > > > > > > >> > jolshan@confluent.io
> > > > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > wrote:
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > Hi Jun,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > I see what you are
> > > > saying.
> > > > > > For
> > > > > > > > now
> > > > > > > > >>> we
> > > > > > > > >>> > can
> > > > > > > > >>> > > > >> remove
> > > > > > > > >>> > > > >> > > the
> > > > > > > > >>> > > > >> > > > > > extra
> > > > > > > > >>> > > > >> > > > > > > > >> > > > information.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > I'll leave the
> option
> > > to
> > > > add
> > > > > > > more
> > > > > > > > >>> > fields
> > > > > > > > >>> > > to
> > > > > > > > >>> > > > >> the
> > > > > > > > >>> > > > >> > > file
> > > > > > > > >>> > > > >> > > > > in
> > > > > > > > >>> > > > >> > > > > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > future.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > The
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > KIP
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > has been updated to
> > > > reflect
> > > > > > > this
> > > > > > > > >>> > change.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > Thanks,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > Justine
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > > On Fri, Sep 18,
> 2020
> > at
> > > > 8:46
> > > > > > AM
> > > > > > > > Jun
> > > > > > > > >>> > Rao <
> > > > > > > > >>> > > > >> > > > > > jun@confluent.io
> > > > > > > > >>> > > > >> > > > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > wrote:
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Hi, Justine,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Thanks for the
> > reply.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> 13. If the log
> > > > directory is
> > > > > > > the
> > > > > > > > >>> source
> > > > > > > > >>> > > of
> > > > > > > > >>> > > > >> > truth,
> > > > > > > > >>> > > > >> > > it
> > > > > > > > >>> > > > >> > > > > > means
> > > > > > > > >>> > > > >> > > > > > > > >> that
> > > > > > > > >>> > > > >> > > > > > > > >> > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> redundant info in
> > the
> > > > > > metadata
> > > > > > > > >>> file
> > > > > > > > >>> > will
> > > > > > > > >>> > > > be
> > > > > > > > >>> > > > >> > > > ignored.
> > > > > > > > >>> > > > >> > > > > > Then
> > > > > > > > >>> > > > >> > > > > > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > question
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > is
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> why do we need to
> > put
> > > > the
> > > > > > > > >>> redundant
> > > > > > > > >>> > info
> > > > > > > > >>> > > > in
> > > > > > > > >>> > > > >> the
> > > > > > > > >>> > > > >> > > > > > metadata
> > > > > > > > >>> > > > >> > > > > > > > file
> > > > > > > > >>> > > > >> > > > > > > > >> > now?
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Thanks,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Jun
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> On Thu, Sep 17,
> 2020
> > > at
> > > > 5:07
> > > > > > > PM
> > > > > > > > >>> > Justine
> > > > > > > > >>> > > > >> Olshan
> > > > > > > > >>> > > > >> > <
> > > > > > > > >>> > > > >> > > > > > > > >> > > > jolshan@confluent.io>
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> wrote:
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Hi Jun,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Thanks for the
> > quick
> > > > > > > response!
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > 12. I've decided
> > to
> > > > bump
> > > > > > up
> > > > > > > > the
> > > > > > > > >>> > > versions
> > > > > > > > >>> > > > >> on
> > > > > > > > >>> > > > >> > the
> > > > > > > > >>> > > > >> > > > > > > requests
> > > > > > > > >>> > > > >> > > > > > > > >> and
> > > > > > > > >>> > > > >> > > > > > > > >> > > > updated
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > KIP. I think
> it's
> > > > good we
> > > > > > > > >>> thoroughly
> > > > > > > > >>> > > > >> > discussed
> > > > > > > > >>> > > > >> > > > the
> > > > > > > > >>> > > > >> > > > > > > > options
> > > > > > > > >>> > > > >> > > > > > > > >> > here,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > so
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > we
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> know
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > we made a good
> > > > choice. :)
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > 13. This is an
> > > > interesting
> > > > > > > > >>> > situation.
> > > > > > > > >>> > > I
> > > > > > > > >>> > > > >> think
> > > > > > > > >>> > > > >> > > if
> > > > > > > > >>> > > > >> > > > > this
> > > > > > > > >>> > > > >> > > > > > > > does
> > > > > > > > >>> > > > >> > > > > > > > >> > occur
> > > > > > > > >>> > > > >> > > > > > > > >> > > > we
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> should
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > give a warning.
> I
> > > > agree
> > > > > > that
> > > > > > > > >>> it's
> > > > > > > > >>> > hard
> > > > > > > > >>> > > > to
> > > > > > > > >>> > > > >> > know
> > > > > > > > >>> > > > >> > > > the
> > > > > > > > >>> > > > >> > > > > > > source
> > > > > > > > >>> > > > >> > > > > > > > >> of
> > > > > > > > >>> > > > >> > > > > > > > >> > > truth
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > for
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> sure
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > since the
> > directory
> > > > or the
> > > > > > > > file
> > > > > > > > >>> > could
> > > > > > > > >>> > > be
> > > > > > > > >>> > > > >> > > manually
> > > > > > > > >>> > > > >> > > > > > > > >> modified. I
> > > > > > > > >>> > > > >> > > > > > > > >> > > > guess
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > directory could
> be
> > > > used as
> > > > > > > the
> > > > > > > > >>> > source
> > > > > > > > >>> > > of
> > > > > > > > >>> > > > >> > truth.
> > > > > > > > >>> > > > >> > > > To
> > > > > > > > >>> > > > >> > > > > be
> > > > > > > > >>> > > > >> > > > > > > > >> honest,
> > > > > > > > >>> > > > >> > > > > > > > >> > > I'm
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > not
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > really sure what
> > > > happens
> > > > > > in
> > > > > > > > >>> kafka
> > > > > > > > >>> > when
> > > > > > > > >>> > > > the
> > > > > > > > >>> > > > >> > log
> > > > > > > > >>> > > > >> > > > > > > directory
> > > > > > > > >>> > > > >> > > > > > > > is
> > > > > > > > >>> > > > >> > > > > > > > >> > > > renamed
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > manually in
> such a
> > > > way.
> > > > > > I'm
> > > > > > > > also
> > > > > > > > >>> > > > >> wondering if
> > > > > > > > >>> > > > >> > > the
> > > > > > > > >>> > > > >> > > > > > > > >> situation is
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> recoverable
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > in this
> scenario.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Thanks,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Justine
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > On Thu, Sep 17,
> > 2020
> > > > at
> > > > > > 4:28
> > > > > > > > PM
> > > > > > > > >>> Jun
> > > > > > > > >>> > > Rao
> > > > > > > > >>> > > > <
> > > > > > > > >>> > > > >> > > > > > > > jun@confluent.io>
> > > > > > > > >>> > > > >> > > > > > > > >> > > wrote:
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Hi, Justine,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Thanks for the
> > > > reply.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > 12. I don't
> > have a
> > > > > > strong
> > > > > > > > >>> > preference
> > > > > > > > >>> > > > >> > either.
> > > > > > > > >>> > > > >> > > > > > However,
> > > > > > > > >>> > > > >> > > > > > > > if
> > > > > > > > >>> > > > >> > > > > > > > >> we
> > > > > > > > >>> > > > >> > > > > > > > >> > > need
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > IBP
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > anyway, maybe
> > it's
> > > > > > easier
> > > > > > > to
> > > > > > > > >>> just
> > > > > > > > >>> > > bump
> > > > > > > > >>> > > > >> up
> > > > > > > > >>> > > > >> > the
> > > > > > > > >>> > > > >> > > > > > version
> > > > > > > > >>> > > > >> > > > > > > > for
> > > > > > > > >>> > > > >> > > > > > > > >> > all
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > inter
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > broker
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > requests and
> add
> > > the
> > > > > > topic
> > > > > > > > id
> > > > > > > > >>> > field
> > > > > > > > >>> > > > as a
> > > > > > > > >>> > > > >> > > > regular
> > > > > > > > >>> > > > >> > > > > > > > field. A
> > > > > > > > >>> > > > >> > > > > > > > >> > > > regular
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> field
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > is
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > a bit more
> > concise
> > > > in
> > > > > > wire
> > > > > > > > >>> > transfer
> > > > > > > > >>> > > > >> than a
> > > > > > > > >>> > > > >> > > > > flexible
> > > > > > > > >>> > > > >> > > > > > > > >> field.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > 13. The
> > confusion
> > > > that I
> > > > > > > was
> > > > > > > > >>> > > referring
> > > > > > > > >>> > > > >> to
> > > > > > > > >>> > > > >> > is
> > > > > > > > >>> > > > >> > > > > > between
> > > > > > > > >>> > > > >> > > > > > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > topic
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > name
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > partition
> number
> > > > between
> > > > > > > the
> > > > > > > > >>> log
> > > > > > > > >>> > dir
> > > > > > > > >>> > > > and
> > > > > > > > >>> > > > >> > the
> > > > > > > > >>> > > > >> > > > > > metadata
> > > > > > > > >>> > > > >> > > > > > > > >> file.
> > > > > > > > >>> > > > >> > > > > > > > >> > > For
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> example,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > if
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > the log dir is
> > > > topicA-1
> > > > > > > and
> > > > > > > > >>> the
> > > > > > > > >>> > > > metadata
> > > > > > > > >>> > > > >> > file
> > > > > > > > >>> > > > >> > > > in
> > > > > > > > >>> > > > >> > > > > it
> > > > > > > > >>> > > > >> > > > > > > has
> > > > > > > > >>> > > > >> > > > > > > > >> > topicB
> > > > > > > > >>> > > > >> > > > > > > > >> > > > and
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > partition 0
> (say
> > > > due to
> > > > > > a
> > > > > > > > bug
> > > > > > > > >>> or
> > > > > > > > >>> > > > manual
> > > > > > > > >>> > > > >> > > > > > > modification),
> > > > > > > > >>> > > > >> > > > > > > > >> which
> > > > > > > > >>> > > > >> > > > > > > > >> > > one
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > do
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > we
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > use
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > as the source
> of
> > > > truth?
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Jun
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > On Thu, Sep
> 17,
> > > > 2020 at
> > > > > > > 3:43
> > > > > > > > >>> PM
> > > > > > > > >>> > > > Justine
> > > > > > > > >>> > > > >> > > Olshan
> > > > > > > > >>> > > > >> > > > <
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > jolshan@confluent.io
> >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > wrote:
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > Hi Jun,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > Thanks for
> the
> > > > > > comments.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > 12. I bumped
> > the
> > > > > > > > >>> > > LeaderAndIsrRequest
> > > > > > > > >>> > > > >> > > because
> > > > > > > > >>> > > > >> > > > I
> > > > > > > > >>> > > > >> > > > > > > > removed
> > > > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > topic
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> name
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > field
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > in the
> > response.
> > > > It
> > > > > > may
> > > > > > > be
> > > > > > > > >>> > > possible
> > > > > > > > >>> > > > to
> > > > > > > > >>> > > > >> > > avoid
> > > > > > > > >>> > > > >> > > > > > > bumping
> > > > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > version
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > without
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > that change,
> > but
> > > > I may
> > > > > > > be
> > > > > > > > >>> > missing
> > > > > > > > >>> > > > >> > > something.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > I believe
> > > > StopReplica
> > > > > > is
> > > > > > > > >>> > actually
> > > > > > > > >>> > > on
> > > > > > > > >>> > > > >> > > version
> > > > > > > > >>> > > > >> > > > 3
> > > > > > > > >>> > > > >> > > > > > now,
> > > > > > > > >>> > > > >> > > > > > > > but
> > > > > > > > >>> > > > >> > > > > > > > >> > > > because
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > version 2
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > is
> flexible, I
> > > > kept
> > > > > > that
> > > > > > > > >>> listed
> > > > > > > > >>> > as
> > > > > > > > >>> > > > >> > version
> > > > > > > > >>> > > > >> > > 2
> > > > > > > > >>> > > > >> > > > on
> > > > > > > > >>> > > > >> > > > > > the
> > > > > > > > >>> > > > >> > > > > > > > KIP
> > > > > > > > >>> > > > >> > > > > > > > >> > > page.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> However,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > you
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > may be right
> > in
> > > > that
> > > > > > we
> > > > > > > > may
> > > > > > > > >>> need
> > > > > > > > >>> > > to
> > > > > > > > >>> > > > >> bump
> > > > > > > > >>> > > > >> > > the
> > > > > > > > >>> > > > >> > > > > > > version
> > > > > > > > >>> > > > >> > > > > > > > on
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > StopReplica
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> to
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > deal
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > with
> deletion
> > > > > > > differently
> > > > > > > > as
> > > > > > > > >>> > > > mentioned
> > > > > > > > >>> > > > >> > > > above. I
> > > > > > > > >>> > > > >> > > > > > > don't
> > > > > > > > >>> > > > >> > > > > > > > >> know
> > > > > > > > >>> > > > >> > > > > > > > >> > > if
> > > > > > > > >>> > > > >> > > > > > > > >> > > > I
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> have a
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > big
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > preference
> > over
> > > > used
> > > > > > > > tagged
> > > > > > > > >>> > fields
> > > > > > > > >>> > > > or
> > > > > > > > >>> > > > >> > not.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > 13. I was
> > > thinking
> > > > > > that
> > > > > > > in
> > > > > > > > >>> the
> > > > > > > > >>> > > case
> > > > > > > > >>> > > > >> where
> > > > > > > > >>> > > > >> > > the
> > > > > > > > >>> > > > >> > > > > > file
> > > > > > > > >>> > > > >> > > > > > > > and
> > > > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > request
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > topic
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > ids don't
> > match,
> > > > it
> > > > > > > means
> > > > > > > > >>> that
> > > > > > > > >>> > the
> > > > > > > > >>> > > > >> > broker's
> > > > > > > > >>> > > > >> > > > > > > topic/the
> > > > > > > > >>> > > > >> > > > > > > > >> one
> > > > > > > > >>> > > > >> > > > > > > > >> > in
> > > > > > > > >>> > > > >> > > > > > > > >> > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> file
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > has
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > been
> deleted.
> > In
> > > > that
> > > > > > > > case,
> > > > > > > > >>> we
> > > > > > > > >>> > > would
> > > > > > > > >>> > > > >> need
> > > > > > > > >>> > > > >> > > to
> > > > > > > > >>> > > > >> > > > > > delete
> > > > > > > > >>> > > > >> > > > > > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > old
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > topic
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > start
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > receiving
> the
> > > new
> > > > > > > version.
> > > > > > > > >>> If
> > > > > > > > >>> > the
> > > > > > > > >>> > > > >> topic
> > > > > > > > >>> > > > >> > > name
> > > > > > > > >>> > > > >> > > > > were
> > > > > > > > >>> > > > >> > > > > > > to
> > > > > > > > >>> > > > >> > > > > > > > >> > change,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > but
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > ids
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > still
> matched,
> > > the
> > > > > > file
> > > > > > > > >>> would
> > > > > > > > >>> > also
> > > > > > > > >>> > > > >> need
> > > > > > > > >>> > > > >> > to
> > > > > > > > >>> > > > >> > > > > > update.
> > > > > > > > >>> > > > >> > > > > > > > Am I
> > > > > > > > >>> > > > >> > > > > > > > >> > > > missing
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > a
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> case
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > where the
> file
> > > > would
> > > > > > be
> > > > > > > > >>> correct
> > > > > > > > >>> > > and
> > > > > > > > >>> > > > >> not
> > > > > > > > >>> > > > >> > the
> > > > > > > > >>> > > > >> > > > > > > request?
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > Thanks,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > Justine
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > On Thu, Sep
> > 17,
> > > > 2020
> > > > > > at
> > > > > > > > >>> 3:18 PM
> > > > > > > > >>> > > Jun
> > > > > > > > >>> > > > >> Rao <
> > > > > > > > >>> > > > >> > > > > > > > >> jun@confluent.io
> > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > wrote:
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Hi,
> Justine,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Thanks for
> > the
> > > > > > reply.
> > > > > > > A
> > > > > > > > >>> couple
> > > > > > > > >>> > > of
> > > > > > > > >>> > > > >> more
> > > > > > > > >>> > > > >> > > > > comments
> > > > > > > > >>> > > > >> > > > > > > > >> below.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > 12.
> > ListOffset
> > > > and
> > > > > > > > >>> > > OffsetForLeader
> > > > > > > > >>> > > > >> > > > currently
> > > > > > > > >>> > > > >> > > > > > > don't
> > > > > > > > >>> > > > >> > > > > > > > >> > support
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> flexible
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > fields.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > So, we
> have
> > to
> > > > bump
> > > > > > up
> > > > > > > > the
> > > > > > > > >>> > > version
> > > > > > > > >>> > > > >> > number
> > > > > > > > >>> > > > >> > > > and
> > > > > > > > >>> > > > >> > > > > > use
> > > > > > > > >>> > > > >> > > > > > > > >> IBP at
> > > > > > > > >>> > > > >> > > > > > > > >> > > > least
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > for
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > these
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > two
> > requests.
> > > > Note
> > > > > > > that
> > > > > > > > it
> > > > > > > > >>> > seems
> > > > > > > > >>> > > > >> 2.7.0
> > > > > > > > >>> > > > >> > > will
> > > > > > > > >>> > > > >> > > > > > > require
> > > > > > > > >>> > > > >> > > > > > > > >> IBP
> > > > > > > > >>> > > > >> > > > > > > > >> > > > anyway
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > because
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > of
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > changes in
> > > > > > > KAFKA-10435.
> > > > > > > > >>> Also,
> > > > > > > > >>> > it
> > > > > > > > >>> > > > >> seems
> > > > > > > > >>> > > > >> > > that
> > > > > > > > >>> > > > >> > > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> version
> > > > > > > > >>> > > > >> > > > > > > > >> > > for
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > LeaderAndIsrRequest
> > > > > > > and
> > > > > > > > >>> > > > StopReplica
> > > > > > > > >>> > > > >> are
> > > > > > > > >>> > > > >> > > > > bumped
> > > > > > > > >>> > > > >> > > > > > > even
> > > > > > > > >>> > > > >> > > > > > > > >> > though
> > > > > > > > >>> > > > >> > > > > > > > >> > > > we
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > only
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > added
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > a
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > tagged
> > field.
> > > > But
> > > > > > > since
> > > > > > > > >>> IBP is
> > > > > > > > >>> > > > >> needed
> > > > > > > > >>> > > > >> > > > anyway,
> > > > > > > > >>> > > > >> > > > > > we
> > > > > > > > >>> > > > >> > > > > > > > may
> > > > > > > > >>> > > > >> > > > > > > > >> > want
> > > > > > > > >>> > > > >> > > > > > > > >> > > to
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> revisit
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > overall
> > tagged
> > > > field
> > > > > > > > >>> choice.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > 13. The
> only
> > > > > > downside
> > > > > > > is
> > > > > > > > >>> the
> > > > > > > > >>> > > > >> potential
> > > > > > > > >>> > > > >> > > > > > confusion
> > > > > > > > >>> > > > >> > > > > > > on
> > > > > > > > >>> > > > >> > > > > > > > >> > which
> > > > > > > > >>> > > > >> > > > > > > > >> > > > one
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > is
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > source
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > of truth
> if
> > > they
> > > > > > don't
> > > > > > > > >>> match.
> > > > > > > > >>> > > > >> Another
> > > > > > > > >>> > > > >> > > > option
> > > > > > > > >>> > > > >> > > > > is
> > > > > > > > >>> > > > >> > > > > > > to
> > > > > > > > >>> > > > >> > > > > > > > >> > include
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > those
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > fields
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > in
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> metadata
> > > > file
> > > > > > when
> > > > > > > > we
> > > > > > > > >>> > > actually
> > > > > > > > >>> > > > >> > change
> > > > > > > > >>> > > > >> > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> directory
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > structure.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Thanks,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Jun
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > On Thu,
> Sep
> > > 17,
> > > > 2020
> > > > > > > at
> > > > > > > > >>> 2:01
> > > > > > > > >>> > PM
> > > > > > > > >>> > > > >> Justine
> > > > > > > > >>> > > > >> > > > > Olshan
> > > > > > > > >>> > > > >> > > > > > <
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > jolshan@confluent.io
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > wrote:
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > Hello
> all,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I've
> > thought
> > > > some
> > > > > > > more
> > > > > > > > >>> about
> > > > > > > > >>> > > > >> removing
> > > > > > > > >>> > > > >> > > the
> > > > > > > > >>> > > > >> > > > > > topic
> > > > > > > > >>> > > > >> > > > > > > > >> name
> > > > > > > > >>> > > > >> > > > > > > > >> > > field
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > from
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > some
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > of
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> requests.
> > On
> > > > > > closer
> > > > > > > > >>> > inspection
> > > > > > > > >>> > > > of
> > > > > > > > >>> > > > >> the
> > > > > > > > >>> > > > >> > > > > > > > >> > > requests/responses,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > it
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> seems
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > internal
> > > > changes
> > > > > > > would
> > > > > > > > >>> be
> > > > > > > > >>> > much
> > > > > > > > >>> > > > >> larger
> > > > > > > > >>> > > > >> > > > than
> > > > > > > > >>> > > > >> > > > > I
> > > > > > > > >>> > > > >> > > > > > > > >> expected.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > Some
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > protocols
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > involve
> > > > clients,
> > > > > > so
> > > > > > > > they
> > > > > > > > >>> > would
> > > > > > > > >>> > > > >> > require
> > > > > > > > >>> > > > >> > > > > > changes
> > > > > > > > >>> > > > >> > > > > > > > too.
> > > > > > > > >>> > > > >> > > > > > > > >> > I'm
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > thinking
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > that
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > for
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > now,
> > > removing
> > > > the
> > > > > > > > topic
> > > > > > > > >>> name
> > > > > > > > >>> > > > from
> > > > > > > > >>> > > > >> > these
> > > > > > > > >>> > > > >> > > > > > > requests
> > > > > > > > >>> > > > >> > > > > > > > >> and
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > responses
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> are
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > out
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > scope.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I have
> > > > decided to
> > > > > > > just
> > > > > > > > >>> keep
> > > > > > > > >>> > > the
> > > > > > > > >>> > > > >> > change
> > > > > > > > >>> > > > >> > > > > > > > >> > > > LeaderAndIsrResponse
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > to
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > remove
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > topic
> > name,
> > > > and
> > > > > > have
> > > > > > > > >>> updated
> > > > > > > > >>> > > the
> > > > > > > > >>> > > > >> KIP
> > > > > > > > >>> > > > >> > to
> > > > > > > > >>> > > > >> > > > > > reflect
> > > > > > > > >>> > > > >> > > > > > > > >> this
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > change. I
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> have
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > also
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> mentioned
> > > the
> > > > > > other
> > > > > > > > >>> requests
> > > > > > > > >>> > > and
> > > > > > > > >>> > > > >> > > > responses
> > > > > > > > >>> > > > >> > > > > in
> > > > > > > > >>> > > > >> > > > > > > > >> future
> > > > > > > > >>> > > > >> > > > > > > > >> > > work.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I'm
> hoping
> > > to
> > > > > > start
> > > > > > > > the
> > > > > > > > >>> > voting
> > > > > > > > >>> > > > >> > process
> > > > > > > > >>> > > > >> > > > > soon,
> > > > > > > > >>> > > > >> > > > > > so
> > > > > > > > >>> > > > >> > > > > > > > >> let me
> > > > > > > > >>> > > > >> > > > > > > > >> > > > know
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > if
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > there
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > is
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > anything
> > > else
> > > > we
> > > > > > > > should
> > > > > > > > >>> > > discuss.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > Thank
> you,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > Justine
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > On Tue,
> > Sep
> > > > 15,
> > > > > > 2020
> > > > > > > > at
> > > > > > > > >>> 3:57
> > > > > > > > >>> > > PM
> > > > > > > > >>> > > > >> > Justine
> > > > > > > > >>> > > > >> > > > > > Olshan
> > > > > > > > >>> > > > >> > > > > > > <
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > jolshan@confluent.io
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > wrote:
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > Hello
> > > again,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > To
> > follow
> > > > up on
> > > > > > > some
> > > > > > > > >>> of
> > > > > > > > >>> > the
> > > > > > > > >>> > > > >> other
> > > > > > > > >>> > > > >> > > > > comments:
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 10/11)
> > We
> > > > can
> > > > > > > remove
> > > > > > > > >>> the
> > > > > > > > >>> > > topic
> > > > > > > > >>> > > > >> name
> > > > > > > > >>> > > > >> > > > from
> > > > > > > > >>> > > > >> > > > > > > these
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > requests/responses,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > and
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > that
> > means
> > > > we
> > > > > > will
> > > > > > > > >>> just
> > > > > > > > >>> > have
> > > > > > > > >>> > > > to
> > > > > > > > >>> > > > >> > make
> > > > > > > > >>> > > > >> > > a
> > > > > > > > >>> > > > >> > > > > few
> > > > > > > > >>> > > > >> > > > > > > > >> internal
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > changes
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > to
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > make
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > partitions
> > > > > > > > accessible
> > > > > > > > >>> by
> > > > > > > > >>> > > topic
> > > > > > > > >>> > > > >> id
> > > > > > > > >>> > > > >> > and
> > > > > > > > >>> > > > >> > > > > > > > partition.
> > > > > > > > >>> > > > >> > > > > > > > >> I
> > > > > > > > >>> > > > >> > > > > > > > >> > can
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > update
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > KIP
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > remove
> > > them
> > > > > > unless
> > > > > > > > >>> anyone
> > > > > > > > >>> > > > thinks
> > > > > > > > >>> > > > >> > they
> > > > > > > > >>> > > > >> > > > > > should
> > > > > > > > >>> > > > >> > > > > > > > >> stay.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 12)
> > > > Addressed in
> > > > > > > the
> > > > > > > > >>> > > previous
> > > > > > > > >>> > > > >> > email.
> > > > > > > > >>> > > > >> > > > I've
> > > > > > > > >>> > > > >> > > > > > > > updated
> > > > > > > > >>> > > > >> > > > > > > > >> > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > KIP
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > to
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > include
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > tagged
> > > > fields
> > > > > > for
> > > > > > > > the
> > > > > > > > >>> > > requests
> > > > > > > > >>> > > > >> and
> > > > > > > > >>> > > > >> > > > > > responses.
> > > > > > > > >>> > > > >> > > > > > > > >> (More
> > > > > > > > >>> > > > >> > > > > > > > >> > on
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > that
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > below)
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 13) I
> > > think
> > > > part
> > > > > > > of
> > > > > > > > >>> the
> > > > > > > > >>> > idea
> > > > > > > > >>> > > > for
> > > > > > > > >>> > > > >> > > > > including
> > > > > > > > >>> > > > >> > > > > > > this
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > information
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> is to
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > prepare
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > for
> > future
> > > > > > > changes.
> > > > > > > > >>> > Perhaps
> > > > > > > > >>> > > > the
> > > > > > > > >>> > > > >> > > > directory
> > > > > > > > >>> > > > >> > > > > > > > >> structure
> > > > > > > > >>> > > > >> > > > > > > > >> > > > might
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> change
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > from
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > >>> topicName_partitionNumber
> > > > > > > > >>> > to
> > > > > > > > >>> > > > >> > > something
> > > > > > > > >>> > > > >> > > > > like
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > topicID_partitionNumber.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Then
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > it
> would
> > > be
> > > > > > useful
> > > > > > > > to
> > > > > > > > >>> have
> > > > > > > > >>> > > the
> > > > > > > > >>> > > > >> > topic
> > > > > > > > >>> > > > >> > > > name
> > > > > > > > >>> > > > >> > > > > > in
> > > > > > > > >>> > > > >> > > > > > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > file
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > since
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > it
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > would
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > not
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > be
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > in the
> > > > directory
> > > > > > > > >>> > structure.
> > > > > > > > >>> > > > >> > > Supporting
> > > > > > > > >>> > > > >> > > > > > topic
> > > > > > > > >>> > > > >> > > > > > > > >> renames
> > > > > > > > >>> > > > >> > > > > > > > >> > > > might
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > be
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > easier
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > if
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > other
> > > > fields are
> > > > > > > > >>> included.
> > > > > > > > >>> > > > Would
> > > > > > > > >>> > > > >> > > there
> > > > > > > > >>> > > > >> > > > be
> > > > > > > > >>> > > > >> > > > > > any
> > > > > > > > >>> > > > >> > > > > > > > >> > > downsides
> > > > > > > > >>> > > > >> > > > > > > > >> > > > to
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > including
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > this
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > information?
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 14)
> > Yes,
> > > we
> > > > > > would
> > > > > > > > >>> need to
> > > > > > > > >>> > > > copy
> > > > > > > > >>> > > > >> the
> > > > > > > > >>> > > > >> > > > > > partition
> > > > > > > > >>> > > > >> > > > > > > > >> > metadata
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > file
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > in
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > this
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> process.
> > > > I've
> > > > > > > > updated
> > > > > > > > >>> the
> > > > > > > > >>> > > KIP
> > > > > > > > >>> > > > to
> > > > > > > > >>> > > > >> > > > include
> > > > > > > > >>> > > > >> > > > > > > this.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 15) I
> > > > believe
> > > > > > > Lucas
> > > > > > > > >>> meant
> > > > > > > > >>> > v1
> > > > > > > > >>> > > > >> and v2
> > > > > > > > >>> > > > >> > > > here.
> > > > > > > > >>> > > > >> > > > > > He
> > > > > > > > >>> > > > >> > > > > > > > was
> > > > > > > > >>> > > > >> > > > > > > > >> > > > referring
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > to
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> how
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> requests
> > > > would
> > > > > > > fall
> > > > > > > > >>> under
> > > > > > > > >>> > > > >> different
> > > > > > > > >>> > > > >> > > IBP
> > > > > > > > >>> > > > >> > > > > and
> > > > > > > > >>> > > > >> > > > > > > > meant
> > > > > > > > >>> > > > >> > > > > > > > >> > that
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > older
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > brokers
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > would
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > have
> to
> > > use
> > > > the
> > > > > > > > older
> > > > > > > > >>> > > version
> > > > > > > > >>> > > > of
> > > > > > > > >>> > > > >> > the
> > > > > > > > >>> > > > >> > > > > > request
> > > > > > > > >>> > > > >> > > > > > > > and
> > > > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > existing
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > topic
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> deletion
> > > > > > process.
> > > > > > > At
> > > > > > > > >>> > first,
> > > > > > > > >>> > > it
> > > > > > > > >>> > > > >> > seemed
> > > > > > > > >>> > > > >> > > > > like
> > > > > > > > >>> > > > >> > > > > > > > tagged
> > > > > > > > >>> > > > >> > > > > > > > >> > > fields
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > would
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > resolve
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > the
> IBP
> > > > issue.
> > > > > > > > >>> However, we
> > > > > > > > >>> > > may
> > > > > > > > >>> > > > >> need
> > > > > > > > >>> > > > >> > > IBP
> > > > > > > > >>> > > > >> > > > > for
> > > > > > > > >>> > > > >> > > > > > > > this
> > > > > > > > >>> > > > >> > > > > > > > >> > > request
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > after
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > all
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > since
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > the
> > > > controller
> > > > > > > > >>> handles the
> > > > > > > > >>> > > > topic
> > > > > > > > >>> > > > >> > > > deletion
> > > > > > > > >>> > > > >> > > > > > > > >> > differently
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> depending
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > on
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > IBP
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> version.
> > > In
> > > > an
> > > > > > > older
> > > > > > > > >>> > > version,
> > > > > > > > >>> > > > we
> > > > > > > > >>> > > > >> > > can't
> > > > > > > > >>> > > > >> > > > > just
> > > > > > > > >>> > > > >> > > > > > > > send
> > > > > > > > >>> > > > >> > > > > > > > >> a
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > StopReplica
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > delete
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > topic
> > > > > > immediately
> > > > > > > > like
> > > > > > > > >>> > we'd
> > > > > > > > >>> > > > >> want to
> > > > > > > > >>> > > > >> > > for
> > > > > > > > >>> > > > >> > > > > > this
> > > > > > > > >>> > > > >> > > > > > > > KIP.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > This
> > makes
> > > > me
> > > > > > > wonder
> > > > > > > > >>> if we
> > > > > > > > >>> > > > want
> > > > > > > > >>> > > > >> > > tagged
> > > > > > > > >>> > > > >> > > > > > fields
> > > > > > > > >>> > > > >> > > > > > > > on
> > > > > > > > >>> > > > >> > > > > > > > >> all
> > > > > > > > >>> > > > >> > > > > > > > >> > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> requests
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > after
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > all.
> Let
> > > me
> > > > know
> > > > > > > > your
> > > > > > > > >>> > > > thoughts!
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> Justine
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > On
> Tue,
> > > Sep
> > > > 15,
> > > > > > > 2020
> > > > > > > > >>> at
> > > > > > > > >>> > 1:03
> > > > > > > > >>> > > > PM
> > > > > > > > >>> > > > >> > > Justine
> > > > > > > > >>> > > > >> > > > > > > Olshan
> > > > > > > > >>> > > > >> > > > > > > > <
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > jolshan@confluent.io
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > wrote:
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Hi
> all,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Jun
> > > > brought up
> > > > > > a
> > > > > > > > good
> > > > > > > > >>> > point
> > > > > > > > >>> > > > in
> > > > > > > > >>> > > > >> his
> > > > > > > > >>> > > > >> > > > last
> > > > > > > > >>> > > > >> > > > > > > email
> > > > > > > > >>> > > > >> > > > > > > > >> about
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > tagged
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > fields,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > and
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> I've
> > > > updated
> > > > > > the
> > > > > > > > KIP
> > > > > > > > >>> to
> > > > > > > > >>> > > > reflect
> > > > > > > > >>> > > > >> > that
> > > > > > > > >>> > > > >> > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> changes to
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > requests
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> responses
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> will
> be
> > > in
> > > > the
> > > > > > > form
> > > > > > > > >>> of
> > > > > > > > >>> > > tagged
> > > > > > > > >>> > > > >> > fields
> > > > > > > > >>> > > > >> > > > to
> > > > > > > > >>> > > > >> > > > > > > avoid
> > > > > > > > >>> > > > >> > > > > > > > >> > > changing
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > IBP.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Jun:
> I
> > > > plan on
> > > > > > > > >>> sending a
> > > > > > > > >>> > > > >> followup
> > > > > > > > >>> > > > >> > > > email
> > > > > > > > >>> > > > >> > > > > to
> > > > > > > > >>> > > > >> > > > > > > > >> address
> > > > > > > > >>> > > > >> > > > > > > > >> > > some
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > of
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > other
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> points.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> Thanks,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> Justine
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> On
> Mon,
> > > > Sep 14,
> > > > > > > > 2020
> > > > > > > > >>> at
> > > > > > > > >>> > > 4:25
> > > > > > > > >>> > > > PM
> > > > > > > > >>> > > > >> > Jun
> > > > > > > > >>> > > > >> > > > Rao
> > > > > > > > >>> > > > >> > > > > <
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > jun@confluent.io>
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > wrote:
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Hi,
> > > > Justine,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> Thanks
> > > > for the
> > > > > > > > >>> updated
> > > > > > > > >>> > > KIP.
> > > > > > > > >>> > > > A
> > > > > > > > >>> > > > >> few
> > > > > > > > >>> > > > >> > > > > > comments
> > > > > > > > >>> > > > >> > > > > > > > >> below.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 10.
> > > > > > LeaderAndIsr
> > > > > > > > >>> > Response:
> > > > > > > > >>> > > > Do
> > > > > > > > >>> > > > >> we
> > > > > > > > >>> > > > >> > > need
> > > > > > > > >>> > > > >> > > > > the
> > > > > > > > >>> > > > >> > > > > > > > topic
> > > > > > > > >>> > > > >> > > > > > > > >> > > name?
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 11.
> > For
> > > > the
> > > > > > > > changed
> > > > > > > > >>> > > > >> > > request/response,
> > > > > > > > >>> > > > >> > > > > > other
> > > > > > > > >>> > > > >> > > > > > > > >> than
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> LeaderAndIsr,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > UpdateMetadata,
> > > > > > > > >>> > Metadata,
> > > > > > > > >>> > > do
> > > > > > > > >>> > > > >> we
> > > > > > > > >>> > > > >> > > need
> > > > > > > > >>> > > > >> > > > to
> > > > > > > > >>> > > > >> > > > > > > > include
> > > > > > > > >>> > > > >> > > > > > > > >> > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > topic
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> name?
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 12.
> It
> > > > seems
> > > > > > > that
> > > > > > > > >>> > upgrades
> > > > > > > > >>> > > > >> don't
> > > > > > > > >>> > > > >> > > > > require
> > > > > > > > >>> > > > >> > > > > > > IBP.
> > > > > > > > >>> > > > >> > > > > > > > >> Does
> > > > > > > > >>> > > > >> > > > > > > > >> > > > that
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > mean
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > new
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> fields
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> in
> all
> > > the
> > > > > > > > >>> > > request/response
> > > > > > > > >>> > > > >> are
> > > > > > > > >>> > > > >> > > added
> > > > > > > > >>> > > > >> > > > > as
> > > > > > > > >>> > > > >> > > > > > > > tagged
> > > > > > > > >>> > > > >> > > > > > > > >> > > fields
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> without
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > bumping
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > up
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> the
> > > > request
> > > > > > > > >>> version? It
> > > > > > > > >>> > > > would
> > > > > > > > >>> > > > >> be
> > > > > > > > >>> > > > >> > > > useful
> > > > > > > > >>> > > > >> > > > > > to
> > > > > > > > >>> > > > >> > > > > > > > make
> > > > > > > > >>> > > > >> > > > > > > > >> > that
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > clear.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 13.
> > > > Partition
> > > > > > > > >>> Metadata
> > > > > > > > >>> > > file:
> > > > > > > > >>> > > > >> Do
> > > > > > > > >>> > > > >> > we
> > > > > > > > >>> > > > >> > > > need
> > > > > > > > >>> > > > >> > > > > > to
> > > > > > > > >>> > > > >> > > > > > > > >> include
> > > > > > > > >>> > > > >> > > > > > > > >> > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > topic
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > name
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > and
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > partition
> > > > id
> > > > > > > since
> > > > > > > > >>> they
> > > > > > > > >>> > > are
> > > > > > > > >>> > > > >> > implied
> > > > > > > > >>> > > > >> > > > in
> > > > > > > > >>> > > > >> > > > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > directory
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > name?
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 14.
> In
> > > the
> > > > > > JBOD
> > > > > > > > >>> mode, we
> > > > > > > > >>> > > > >> support
> > > > > > > > >>> > > > >> > > > > moving a
> > > > > > > > >>> > > > >> > > > > > > > >> > > partition's
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > data
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> from
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > one
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > disk
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> to
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > another.
> > > > Will
> > > > > > > the
> > > > > > > > >>> new
> > > > > > > > >>> > > > >> partition
> > > > > > > > >>> > > > >> > > > > metadata
> > > > > > > > >>> > > > >> > > > > > > file
> > > > > > > > >>> > > > >> > > > > > > > >> be
> > > > > > > > >>> > > > >> > > > > > > > >> > > > copied
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> during
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > process?
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 15.
> > The
> > > > KIP
> > > > > > says
> > > > > > > > >>> "Remove
> > > > > > > > >>> > > > >> deleted
> > > > > > > > >>> > > > >> > > > topics
> > > > > > > > >>> > > > >> > > > > > > from
> > > > > > > > >>> > > > >> > > > > > > > >> > > replicas
> > > > > > > > >>> > > > >> > > > > > > > >> > > > by
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > sending
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > StopReplicaRequest
> > > > > > > > >>> V2
> > > > > > > > >>> > for
> > > > > > > > >>> > > > any
> > > > > > > > >>> > > > >> > > topics
> > > > > > > > >>> > > > >> > > > > > which
> > > > > > > > >>> > > > >> > > > > > > do
> > > > > > > > >>> > > > >> > > > > > > > >> not
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > contain
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > a
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > topic
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > ID,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > and
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> V3
> for
> > > any
> > > > > > > topics
> > > > > > > > >>> which
> > > > > > > > >>> > do
> > > > > > > > >>> > > > >> > contain
> > > > > > > > >>> > > > >> > > a
> > > > > > > > >>> > > > >> > > > > > topic
> > > > > > > > >>> > > > >> > > > > > > > >> ID.".
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > However,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > it
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > seems
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > updated
> > > > > > > controller
> > > > > > > > >>> will
> > > > > > > > >>> > > > create
> > > > > > > > >>> > > > >> > all
> > > > > > > > >>> > > > >> > > > > > missing
> > > > > > > > >>> > > > >> > > > > > > > >> topic
> > > > > > > > >>> > > > >> > > > > > > > >> > IDs
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > first
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > before
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > doing
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> other
> > > > actions.
> > > > > > > So,
> > > > > > > > >>> is
> > > > > > > > >>> > > > >> > > > > StopReplicaRequest
> > > > > > > > >>> > > > >> > > > > > V2
> > > > > > > > >>> > > > >> > > > > > > > >> > needed?
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Jun
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> On
> > Fri,
> > > > Sep
> > > > > > 11,
> > > > > > > > >>> 2020 at
> > > > > > > > >>> > > > 10:31
> > > > > > > > >>> > > > >> AM
> > > > > > > > >>> > > > >> > > John
> > > > > > > > >>> > > > >> > > > > > > > Roesler <
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > vvcephei@apache.org
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> wrote:
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > Thanks,
> > > > > > > Justine!
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> Your
> > > > > > response
> > > > > > > > >>> seems
> > > > > > > > >>> > > > >> compelling
> > > > > > > > >>> > > > >> > to
> > > > > > > > >>> > > > >> > > > me.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > -John
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > On
> > > Fri,
> > > > > > > > >>> 2020-09-11 at
> > > > > > > > >>> > > > 10:17
> > > > > > > > >>> > > > >> > > -0700,
> > > > > > > > >>> > > > >> > > > > > > Justine
> > > > > > > > >>> > > > >> > > > > > > > >> > Olshan
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > wrote:
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > Hello
> > > > all,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > Thanks for
> > > > > > > > >>> > continuing
> > > > > > > > >>> > > > the
> > > > > > > > >>> > > > >> > > > > > discussion! I
> > > > > > > > >>> > > > >> > > > > > > > >> have a
> > > > > > > > >>> > > > >> > > > > > > > >> > > few
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > responses
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > to
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > your
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > points.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > Tom:
> > > > You
> > > > > > are
> > > > > > > > >>> correct
> > > > > > > > >>> > > in
> > > > > > > > >>> > > > >> that
> > > > > > > > >>> > > > >> > > this
> > > > > > > > >>> > > > >> > > > > KIP
> > > > > > > > >>> > > > >> > > > > > > has
> > > > > > > > >>> > > > >> > > > > > > > >> not
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > mentioned
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > > >>> > DeleteTopicsRequest. I
> > > > > > > > >>> > > > >> think
> > > > > > > > >>> > > > >> > > that
> > > > > > > > >>> > > > >> > > > > > this
> > > > > > > > >>> > > > >> > > > > > > > >> would
> > > > > > > > >>> > > > >> > > > > > > > >> > be
> > > > > > > > >>> > > > >> > > > > > > > >> > > > out
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > of
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > scope
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > for
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> now,
> > but
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > may
> > > be
> > > > > > > > something
> > > > > > > > >>> > worth
> > > > > > > > >>> > > > >> adding
> > > > > > > > >>> > > > >> > > in
> > > > > > > > >>> > > > >> > > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> future.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > John:
> > > > We
> > > > > > did
> > > > > > > > >>> > consider
> > > > > > > > >>> > > > >> > sequence
> > > > > > > > >>> > > > >> > > > ids,
> > > > > > > > >>> > > > >> > > > > > but
> > > > > > > > >>> > > > >> > > > > > > > >> there
> > > > > > > > >>> > > > >> > > > > > > > >> > > are
> > > > > > > > >>> > > > >> > > > > > > > >> > > > a
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > few
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > reasons
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> favor
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > UUIDs.
> > > > > > There
> > > > > > > > are
> > > > > > > > >>> > > several
> > > > > > > > >>> > > > >> > cases
> > > > > > > > >>> > > > >> > > > > where
> > > > > > > > >>> > > > >> > > > > > > > topics
> > > > > > > > >>> > > > >> > > > > > > > >> > from
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> different
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > clusters
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> may
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > interact
> > > > > > now
> > > > > > > > >>> and in
> > > > > > > > >>> > > the
> > > > > > > > >>> > > > >> > future.
> > > > > > > > >>> > > > >> > > > For
> > > > > > > > >>> > > > >> > > > > > > > >> example,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > Mirror
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Maker 2
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > may
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > benefit
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > from
> > > > being
> > > > > > > > able
> > > > > > > > >>> to
> > > > > > > > >>> > > > detect
> > > > > > > > >>> > > > >> > when
> > > > > > > > >>> > > > >> > > a
> > > > > > > > >>> > > > >> > > > > > > cluster
> > > > > > > > >>> > > > >> > > > > > > > >> being
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > mirrored
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> is
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > deleted
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> and
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > recreated
> > > > > > > and
> > > > > > > > >>> > globally
> > > > > > > > >>> > > > >> unique
> > > > > > > > >>> > > > >> > > > > > > identifiers
> > > > > > > > >>> > > > >> > > > > > > > >> > would
> > > > > > > > >>> > > > >> > > > > > > > >> > > > make
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > resolving
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > issues
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > easier
> > > > > > than
> > > > > > > > >>> sequence
> > > > > > > > >>> > > IDs
> > > > > > > > >>> > > > >> > which
> > > > > > > > >>> > > > >> > > > may
> > > > > > > > >>> > > > >> > > > > > > > collide
> > > > > > > > >>> > > > >> > > > > > > > >> > > between
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > clusters.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > KIP-405
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > (tiered
> > > > > > > > storage)
> > > > > > > > >>> > will
> > > > > > > > >>> > > > also
> > > > > > > > >>> > > > >> > > > benefit
> > > > > > > > >>> > > > >> > > > > > from
> > > > > > > > >>> > > > >> > > > > > > > >> > globally
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > unique
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> IDs
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > as
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > shared
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > buckets
> > > > > > may
> > > > > > > be
> > > > > > > > >>> used
> > > > > > > > >>> > > > >> between
> > > > > > > > >>> > > > >> > > > > clusters.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > Globally
> > > > > > > > unique
> > > > > > > > >>> IDs
> > > > > > > > >>> > > > would
> > > > > > > > >>> > > > >> > also
> > > > > > > > >>> > > > >> > > > make
> > > > > > > > >>> > > > >> > > > > > > > >> > > functionality
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > like
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > moving
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > topics
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > between
> > > > > > > > >>> disparate
> > > > > > > > >>> > > > clusters
> > > > > > > > >>> > > > >> > > easier
> > > > > > > > >>> > > > >> > > > > in
> > > > > > > > >>> > > > >> > > > > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > future,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> simplify
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > any
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > future
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > > implementations
> > > > > > > > >>> of
> > > > > > > > >>> > > > backups
> > > > > > > > >>> > > > >> > and
> > > > > > > > >>> > > > >> > > > > > > restores,
> > > > > > > > >>> > > > >> > > > > > > > >> and
> > > > > > > > >>> > > > >> > > > > > > > >> > > more.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > In
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > general,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> unique
> > > IDs
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > would
> > > > > > ensure
> > > > > > > > >>> that
> > > > > > > > >>> > the
> > > > > > > > >>> > > > >> source
> > > > > > > > >>> > > > >> > > > > cluster
> > > > > > > > >>> > > > >> > > > > > > > >> topics do
> > > > > > > > >>> > > > >> > > > > > > > >> > > not
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> conflict
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > with
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > destination
> > > > > > > > >>> cluster
> > > > > > > > >>> > > > >> topics.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> If
> > > we
> > > > were
> > > > > > > to
> > > > > > > > >>> use
> > > > > > > > >>> > > > sequence
> > > > > > > > >>> > > > >> > ids,
> > > > > > > > >>> > > > >> > > > we
> > > > > > > > >>> > > > >> > > > > > > would
> > > > > > > > >>> > > > >> > > > > > > > >> need
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> sufficiently
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > large
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > cluster
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > ids
> > > > to be
> > > > > > > > stored
> > > > > > > > >>> > with
> > > > > > > > >>> > > > the
> > > > > > > > >>> > > > >> > topic
> > > > > > > > >>> > > > >> > > > > > > > >> identifiers or
> > > > > > > > >>> > > > >> > > > > > > > >> > > we
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > run
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > risk
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > collisions.
> > > > > > > > This
> > > > > > > > >>> > will
> > > > > > > > >>> > > > >> give up
> > > > > > > > >>> > > > >> > > any
> > > > > > > > >>> > > > >> > > > > > > > >> advantage in
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> compactness
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > sequence
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > numbers
> > > > > > may
> > > > > > > > >>> bring.
> > > > > > > > >>> > > Given
> > > > > > > > >>> > > > >> > these
> > > > > > > > >>> > > > >> > > > > > > > advantages I
> > > > > > > > >>> > > > >> > > > > > > > >> > > think
> > > > > > > > >>> > > > >> > > > > > > > >> > > > it
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> makes
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > sense
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> use
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > UUIDs.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > Gokul:
> > > > > > This
> > > > > > > is
> > > > > > > > >>> an
> > > > > > > > >>> > > > >> interesting
> > > > > > > > >>> > > > >> > > > idea,
> > > > > > > > >>> > > > >> > > > > > but
> > > > > > > > >>> > > > >> > > > > > > > >> this
> > > > > > > > >>> > > > >> > > > > > > > >> > is
> > > > > > > > >>> > > > >> > > > > > > > >> > > a
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> breaking
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > change.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Out
> of
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > scope
> > > > for
> > > > > > > now,
> > > > > > > > >>> but
> > > > > > > > >>> > > maybe
> > > > > > > > >>> > > > >> > worth
> > > > > > > > >>> > > > >> > > > > > > discussing
> > > > > > > > >>> > > > >> > > > > > > > >> in
> > > > > > > > >>> > > > >> > > > > > > > >> > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > future.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > Hope
> > > > this
> > > > > > > > >>> explains
> > > > > > > > >>> > > some
> > > > > > > > >>> > > > of
> > > > > > > > >>> > > > >> > the
> > > > > > > > >>> > > > >> > > > > > > decisions,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > Justine
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> On
> > > > Fri,
> > > > > > Sep
> > > > > > > > 11,
> > > > > > > > >>> 2020
> > > > > > > > >>> > > at
> > > > > > > > >>> > > > >> 8:27
> > > > > > > > >>> > > > >> > AM
> > > > > > > > >>> > > > >> > > > > Gokul
> > > > > > > > >>> > > > >> > > > > > > > >> Ramanan
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Subramanian <
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > > >>> > gokul2411s@gmail.com>
> > > > > > > > >>> > > > >> wrote:
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > > Hi.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > > > Thanks
> > > > > > for
> > > > > > > > the
> > > > > > > > >>> > KIP.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > > > Have you
> > > > > > > > >>> thought
> > > > > > > > >>> > > about
> > > > > > > > >>> > > > >> > > whether
> > > > > > > > >>> > > > >> > > > it
> > > > > > > > >>> > > > >> > > > > > > makes
> > > > > > > > >>> > > > >> > > > > > > > >> > sense
> > > > > > > > >>> > > > >> > > > > > > > >> > > to
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> support
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > authorizing a
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > > > > > principal
> > > > > > > > for
> > > > > > > > >>> a
> > > > > > > > >>> > > topic
> > > > > > > > >>> > > > ID
> > > > > > > > >>> > > > >> > > rather
> > > > > > > > >>> > > > >> > > > > > than
> > > > > > > > >>> > > > >> > > > > > > a
> > > > > > > > >>> > > > >> > > > > > > > >> topic
> > > > > > > > >>> > > > >> > > > > > > > >> > > > name
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > to
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > achieve
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > tighter
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > > > > > security?
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > Or
> > > > is
> > > > > > the
> > > > > > > > >>> topic ID
> > > > > > > > >>> > > > >> > > > fundamentally
> > > > > > > > >>> > > > >> > > > > an
> > > > > > > > >>> > > > >> > > > > > > > >> internal
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > detail
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > similar
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > to
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> epochs
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> used
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > > in a
> > > > > > bunch
> > > > > > > > of
> > > > > > > > >>> > other
> > > > > > > > >>> > > > >> places
> > > > > > > > >>> > > > >> > in
> > > > > > > > >>> > > > >> > > > > > Kafka?
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > > > Thanks.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > On
> > > > Fri,
> > > > > > > Sep
> > > > > > > > >>> 11,
> > > > > > > > >>> > 2020
> > > > > > > > >>> > > > at
> > > > > > > > >>> > > > >> > 4:06
> > > > > > > > >>> > > > >> > > PM
> > > > > > > > >>> > > > >> > > > > > John
> > > > > > > > >>> > > > >> > > > > > > > >> > Roesler <
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > vvcephei@apache.org
> > > > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > wrote:
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > Hello
> > > > > > > > >>> Justine,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > Thanks
> > > > > > > for
> > > > > > > > >>> the
> > > > > > > > >>> > > KIP!
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > I
> > > > > > happen
> > > > > > > > to
> > > > > > > > >>> have
> > > > > > > > >>> > > > been
> > > > > > > > >>> > > > >> > > > > confronted
> > > > > > > > >>> > > > >> > > > > > > > >> recently
> > > > > > > > >>> > > > >> > > > > > > > >> > > with
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> need
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > to
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > keep
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > track
> > > of
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > a
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > large
> > > > > > > > >>> number of
> > > > > > > > >>> > > > >> topics as
> > > > > > > > >>> > > > >> > > > > > compactly
> > > > > > > > >>> > > > >> > > > > > > > as
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > possible. I
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> was
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > going
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> come
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > up
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > with
> > > > > > > some
> > > > > > > > >>> way to
> > > > > > > > >>> > > > >> > dictionary
> > > > > > > > >>> > > > >> > > > > > encode
> > > > > > > > >>> > > > >> > > > > > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > topic
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > names
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> as
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > integers,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> but
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> this
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > seems
> > > > > > > much
> > > > > > > > >>> > better!
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > > > > Apologies
> > > > > > > > if
> > > > > > > > >>> > this
> > > > > > > > >>> > > > has
> > > > > > > > >>> > > > >> > been
> > > > > > > > >>> > > > >> > > > > raised
> > > > > > > > >>> > > > >> > > > > > > > >> before,
> > > > > > > > >>> > > > >> > > > > > > > >> > > but
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > I’m
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > wondering
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> about
> > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > choice
> > > > > > > of
> > > > > > > > >>> UUID
> > > > > > > > >>> > vs
> > > > > > > > >>> > > > >> > sequence
> > > > > > > > >>> > > > >> > > > > number
> > > > > > > > >>> > > > >> > > > > > > for
> > > > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > > > >>> > > > >> > > > > > > > >> > > ids.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Typically,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > I’ve
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> seen
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > UUIDs
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > in two
> > > > > > > > >>> > situations:
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > 1.
> > > > > > When
> > > > > > > > >>> > processes
> > > > > > > > >>> > > > >> need to
> > > > > > > > >>> > > > >> > > > > > generate
> > > > > > > > >>> > > > >> > > > > > > > >> > > > non-colliding
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > identifiers
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > without
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > > > > > >>> coordination.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > 2.
> > > > > > When
> > > > > > > > the
> > > > > > > > >>> > > > identifier
> > > > > > > > >>> > > > >> > > needs
> > > > > > > > >>> > > > >> > > > to
> > > > > > > > >>> > > > >> > > > > > be
> > > > > > > > >>> > > > >> > > > > > > > >> > > > “universally
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > unique”;
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > I.e.,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > > > > identifier
> > > > > > > > >>> needs
> > > > > > > > >>> > > to
> > > > > > > > >>> > > > >> > > > distinguish
> > > > > > > > >>> > > > >> > > > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> entity
> > > > > > > > >>> > > > >> > > > > > > > >> > > > from
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > all
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > other
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > entities
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> that
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > could
> > > > > > > ever
> > > > > > > > >>> > exist.
> > > > > > > > >>> > > > >> This is
> > > > > > > > >>> > > > >> > > > > useful
> > > > > > > > >>> > > > >> > > > > > in
> > > > > > > > >>> > > > >> > > > > > > > >> cases
> > > > > > > > >>> > > > >> > > > > > > > >> > > > where
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > entities
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > from
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > all
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > kinds
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > of
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > > > systems
> > > > > > > > get
> > > > > > > > >>> > mixed
> > > > > > > > >>> > > > >> > together,
> > > > > > > > >>> > > > >> > > > > such
> > > > > > > > >>> > > > >> > > > > > as
> > > > > > > > >>> > > > >> > > > > > > > >> when
> > > > > > > > >>> > > > >> > > > > > > > >> > > > dumping
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> logs
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > from
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > all
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > processes
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > in
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > a
> > > > > > > company
> > > > > > > > >>> into a
> > > > > > > > >>> > > > >> common
> > > > > > > > >>> > > > >> > > > system.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > Maybe
> > > > > > > I’m
> > > > > > > > >>> being
> > > > > > > > >>> > > > >> > > > short-sighted,
> > > > > > > > >>> > > > >> > > > > > but
> > > > > > > > >>> > > > >> > > > > > > it
> > > > > > > > >>> > > > >> > > > > > > > >> > > doesn’t
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > seem
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> like
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > either
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> really
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > > > applies
> > > > > > > > >>> here. It
> > > > > > > > >>> > > > seems
> > > > > > > > >>> > > > >> > like
> > > > > > > > >>> > > > >> > > > the
> > > > > > > > >>> > > > >> > > > > > > > brokers
> > > > > > > > >>> > > > >> > > > > > > > >> > > could
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > and
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> would
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > achieve
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > consensus
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > when
> > > > > > > > >>> creating a
> > > > > > > > >>> > > > topic
> > > > > > > > >>> > > > >> > > anyway,
> > > > > > > > >>> > > > >> > > > > > which
> > > > > > > > >>> > > > >> > > > > > > > is
> > > > > > > > >>> > > > >> > > > > > > > >> all
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > that’s
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > required
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > to
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > generate
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > > > > > >>> non-colliding
> > > > > > > > >>> > > > sequence
> > > > > > > > >>> > > > >> > ids.
> > > > > > > > >>> > > > >> > > > For
> > > > > > > > >>> > > > >> > > > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > second,
> > > > > > > > >>> > > > >> > > > > > > > >> > > as
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > you
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > mention,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > we
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> could
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > > > assign
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > a
> > > > UUID
> > > > > > > to
> > > > > > > > >>> the
> > > > > > > > >>> > > > cluster
> > > > > > > > >>> > > > >> as
> > > > > > > > >>> > > > >> > a
> > > > > > > > >>> > > > >> > > > > whole,
> > > > > > > > >>> > > > >> > > > > > > > which
> > > > > > > > >>> > > > >> > > > > > > > >> > > would
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > render
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > any
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > resource
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > scoped
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > to
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > the
> > > > > > > broker
> > > > > > > > >>> > > > universally
> > > > > > > > >>> > > > >> > > unique
> > > > > > > > >>> > > > >> > > > > as
> > > > > > > > >>> > > > >> > > > > > > > well.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > The
> > > > > > > > reason I
> > > > > > > > >>> > > mention
> > > > > > > > >>> > > > >> this
> > > > > > > > >>> > > > >> > > is
> > > > > > > > >>> > > > >> > > > > > that,
> > > > > > > > >>> > > > >> > > > > > > > >> > although
> > > > > > > > >>> > > > >> > > > > > > > >> > > a
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > UUID
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> is
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > way
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > more
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > compact
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > than
> > > > > > > topic
> > > > > > > > >>> > names,
> > > > > > > > >>> > > > it’s
> > > > > > > > >>> > > > >> > > still
> > > > > > > > >>> > > > >> > > > 16
> > > > > > > > >>> > > > >> > > > > > > > bytes.
> > > > > > > > >>> > > > >> > > > > > > > >> In
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > contrast,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> a
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > 4-byte
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > integer
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > > > sequence
> > > > > > > > id
> > > > > > > > >>> > would
> > > > > > > > >>> > > > give
> > > > > > > > >>> > > > >> > us 4
> > > > > > > > >>> > > > >> > > > > > billion
> > > > > > > > >>> > > > >> > > > > > > > >> unique
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > topics
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> per
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > cluster,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> which
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > > > seems
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > like
> > > > > > > > enough
> > > > > > > > >>> ;)
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > > > > > Considering
> > > > > > > > >>> the
> > > > > > > > >>> > > > >> number of
> > > > > > > > >>> > > > >> > > > > > different
> > > > > > > > >>> > > > >> > > > > > > > >> times
> > > > > > > > >>> > > > >> > > > > > > > >> > > > these
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> topic
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > identifiers
> > > > > > are
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > > sent
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > over
> > > > > > the
> > > > > > > > >>> wire or
> > > > > > > > >>> > > > >> stored
> > > > > > > > >>> > > > >> > in
> > > > > > > > >>> > > > >> > > > > > memory,
> > > > > > > > >>> > > > >> > > > > > > it
> > > > > > > > >>> > > > >> > > > > > > > >> > seems
> > > > > > > > >>> > > > >> > > > > > > > >> > > > like
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > it
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > might
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > be
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> worth
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > > > > additional
> > > > > > > > >>> 4x
> > > > > > > > >>> > > space
> > > > > > > > >>> > > > >> > > savings.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > What
> > > > > > do
> > > > > > > > you
> > > > > > > > >>> > think
> > > > > > > > >>> > > > >> about
> > > > > > > > >>> > > > >> > > this?
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > > > Thanks,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > John
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > On
> > > > > > Fri,
> > > > > > > > Sep
> > > > > > > > >>> 11,
> > > > > > > > >>> > > > 2020,
> > > > > > > > >>> > > > >> at
> > > > > > > > >>> > > > >> > > > 03:20,
> > > > > > > > >>> > > > >> > > > > > Tom
> > > > > > > > >>> > > > >> > > > > > > > >> > Bentley
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > wrote:
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > Hi
> > > > > > > > >>> Justine,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > This
> > > > > > > > looks
> > > > > > > > >>> > like
> > > > > > > > >>> > > a
> > > > > > > > >>> > > > >> very
> > > > > > > > >>> > > > >> > > > > welcome
> > > > > > > > >>> > > > >> > > > > > > > >> > > improvement.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Thanks!
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > > Maybe
> > > > > > > I
> > > > > > > > >>> missed
> > > > > > > > >>> > > it,
> > > > > > > > >>> > > > >> but
> > > > > > > > >>> > > > >> > > the
> > > > > > > > >>> > > > >> > > > > KIP
> > > > > > > > >>> > > > >> > > > > > > > >> doesn't
> > > > > > > > >>> > > > >> > > > > > > > >> > > seem
> > > > > > > > >>> > > > >> > > > > > > > >> > > > to
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > mention
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > changing
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > > > > >>> > > > DeleteTopicsRequest
> > > > > > > > >>> > > > >> to
> > > > > > > > >>> > > > >> > > > > identify
> > > > > > > > >>> > > > >> > > > > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > topic
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > using
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > an
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > id.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Maybe
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > that's
> > > > out
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > of
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > > scope,
> > > > > > > > but
> > > > > > > > >>> > > > >> > > > > DeleteTopicsRequest
> > > > > > > > >>> > > > >> > > > > > is
> > > > > > > > >>> > > > >> > > > > > > > not
> > > > > > > > >>> > > > >> > > > > > > > >> > > listed
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > among
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Future
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Work
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> APIs
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > > > either.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > Kind
> > > > > > > > >>> regards,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > Tom
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > On
> > > > > > > Thu,
> > > > > > > > >>> Sep
> > > > > > > > >>> > 10,
> > > > > > > > >>> > > > >> 2020 at
> > > > > > > > >>> > > > >> > > > 3:59
> > > > > > > > >>> > > > >> > > > > PM
> > > > > > > > >>> > > > >> > > > > > > > >> Satish
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > Duggana <
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > > > > > > > >>> > > > >> satish.duggana@gmail.com
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > > wrote:
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > Thanks
> > > > > > > > >>> > > > >> Lucas/Justine
> > > > > > > > >>> > > > >> > > for
> > > > > > > > >>> > > > >> > > > > the
> > > > > > > > >>> > > > >> > > > > > > nice
> > > > > > > > >>> > > > >> > > > > > > > >> KIP.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > It
> > > > > > > has
> > > > > > > > >>> > several
> > > > > > > > >>> > > > >> > benefits
> > > > > > > > >>> > > > >> > > > > which
> > > > > > > > >>> > > > >> > > > > > > > also
> > > > > > > > >>> > > > >> > > > > > > > >> > > include
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > simplifying
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> topic
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > > deletion
> > > > > > > > >>> > > process
> > > > > > > > >>> > > > >> by
> > > > > > > > >>> > > > >> > > > > > controller
> > > > > > > > >>> > > > >> > > > > > > > and
> > > > > > > > >>> > > > >> > > > > > > > >> > logs
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > cleanup
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> by
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > brokers
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > in
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > corner
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > cases.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > Best,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > > Satish.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > On
> > > > > > > > Wed,
> > > > > > > > >>> Sep
> > > > > > > > >>> > 9,
> > > > > > > > >>> > > > >> 2020
> > > > > > > > >>> > > > >> > at
> > > > > > > > >>> > > > >> > > > > 10:07
> > > > > > > > >>> > > > >> > > > > > PM
> > > > > > > > >>> > > > >> > > > > > > > >> > Justine
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Olshan <
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > > >>> jolshan@confluent.io
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > wrote:
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > > > Hello
> > > > > > > > >>> all,
> > > > > > > > >>> > > > it's
> > > > > > > > >>> > > > >> > been
> > > > > > > > >>> > > > >> > > > > > almost a
> > > > > > > > >>> > > > >> > > > > > > > >> year!
> > > > > > > > >>> > > > >> > > > > > > > >> > > I've
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > made
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > some
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > changes
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> to
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> this
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > KIP
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > and
> > > > > > > > >>> hope to
> > > > > > > > >>> > > > >> continue
> > > > > > > > >>> > > > >> > > the
> > > > > > > > >>> > > > >> > > > > > > > >> discussion.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > > One
> > > > > > > > >>> of the
> > > > > > > > >>> > > > main
> > > > > > > > >>> > > > >> > > changes
> > > > > > > > >>> > > > >> > > > > > I've
> > > > > > > > >>> > > > >> > > > > > > > >> added
> > > > > > > > >>> > > > >> > > > > > > > >> > is
> > > > > > > > >>> > > > >> > > > > > > > >> > > > now
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > metadata
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > response
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > > will
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > > include
> > > > > > > > >>> the
> > > > > > > > >>> > > > topic
> > > > > > > > >>> > > > >> ID
> > > > > > > > >>> > > > >> > > (as
> > > > > > > > >>> > > > >> > > > > > Colin
> > > > > > > > >>> > > > >> > > > > > > > >> > > suggested).
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Clients
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > can
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> obtain
> > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > > > topicID
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > of a
> > > > > > > > >>> given
> > > > > > > > >>> > > topic
> > > > > > > > >>> > > > >> > > through
> > > > > > > > >>> > > > >> > > > a
> > > > > > > > >>> > > > >> > > > > > > > >> > > > TopicDescription.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > The
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > topicId
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > will
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> also
> > > be
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > > included
> > > > > > > > >>> > with
> > > > > > > > >>> > > > the
> > > > > > > > >>> > > > >> > > > > > > UpdateMetadata
> > > > > > > > >>> > > > >> > > > > > > > >> > > request.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > > Let
> > > > > > > > me
> > > > > > > > >>> > know
> > > > > > > > >>> > > > what
> > > > > > > > >>> > > > >> > you
> > > > > > > > >>> > > > >> > > > all
> > > > > > > > >>> > > > >> > > > > > > think.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > > > Thank
> > > > > > > > >>> you,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > > > >>> Justine
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > On
> > > > > > > > >>> > > 2019/09/13
> > > > > > > > >>> > > > >> > > 16:38:26,
> > > > > > > > >>> > > > >> > > > > > > "Colin
> > > > > > > > >>> > > > >> > > > > > > > >> > > McCabe" <
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > cmccabe@apache.org
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > wrote:
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > Hi
> > > > > > > > >>> > Lucas,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > > >>> Thanks
> > > > > > > > >>> > for
> > > > > > > > >>> > > > >> > tackling
> > > > > > > > >>> > > > >> > > > > this.
> > > > > > > > >>> > > > >> > > > > > > > >> Topic
> > > > > > > > >>> > > > >> > > > > > > > >> > IDs
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > are a
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > great
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > idea,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> and
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> this
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > is
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > a
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > really
> > > > > > > > >>> good
> > > > > > > > >>> > > > >> writeup.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > > For
> > > > > > > > >>> > > > >> > > > > > > /brokers/topics/[topic],
> > > > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > schema
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> version
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > should
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > be
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > bumped
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > to
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > > version
> > > > > > > > >>> 3,
> > > > > > > > >>> > > > rather
> > > > > > > > >>> > > > >> > than
> > > > > > > > >>> > > > >> > > 2.
> > > > > > > > >>> > > > >> > > > > > > > KIP-455
> > > > > > > > >>> > > > >> > > > > > > > >> > > bumped
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > version
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > this
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > znode
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > to
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > 2
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > > already
> > > > > > > > >>> :)
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > > >>> Given
> > > > > > > > >>> > that
> > > > > > > > >>> > > > >> we're
> > > > > > > > >>> > > > >> > > > going
> > > > > > > > >>> > > > >> > > > > to
> > > > > > > > >>> > > > >> > > > > > > be
> > > > > > > > >>> > > > >> > > > > > > > >> > seeing
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > these
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > things
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > as
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > strings
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > as
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > > lot
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > (in
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > logs,
> > > > > > > > in
> > > > > > > > >>> > > > >> ZooKeeper,
> > > > > > > > >>> > > > >> > on
> > > > > > > > >>> > > > >> > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > command-line,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > etc.),
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > does
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > it
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > make
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > sense
> > > to
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > use
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > base64
> > > > > > > > >>> when
> > > > > > > > >>> > > > >> > converting
> > > > > > > > >>> > > > >> > > > them
> > > > > > > > >>> > > > >> > > > > > to
> > > > > > > > >>> > > > >> > > > > > > > >> > strings?
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > > >>> Here is
> > > > > > > > >>> > an
> > > > > > > > >>> > > > >> > example
> > > > > > > > >>> > > > >> > > of
> > > > > > > > >>> > > > >> > > > > the
> > > > > > > > >>> > > > >> > > > > > > hex
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> representation:
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > > And
> > > > > > > > >>> here
> > > > > > > > >>> > > is
> > > > > > > > >>> > > > an
> > > > > > > > >>> > > > >> > > > example
> > > > > > > > >>> > > > >> > > > > in
> > > > > > > > >>> > > > >> > > > > > > > >> base64.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > > >>> > > > >> > > > b8tRS7h4TJ2Vt43Dp85v2A
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > > The
> > > > > > > > >>> > base64
> > > > > > > > >>> > > > >> > version
> > > > > > > > >>> > > > >> > > > > saves
> > > > > > > > >>> > > > >> > > > > > 15
> > > > > > > > >>> > > > >> > > > > > > > >> > letters
> > > > > > > > >>> > > > >> > > > > > > > >> > > > (to
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > be
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > fair,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > 4
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> those
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> were
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > dashes
> > > > > > > > >>> that
> > > > > > > > >>> > we
> > > > > > > > >>> > > > >> could
> > > > > > > > >>> > > > >> > > have
> > > > > > > > >>> > > > >> > > > > > > elided
> > > > > > > > >>> > > > >> > > > > > > > in
> > > > > > > > >>> > > > >> > > > > > > > >> > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > hex
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > representation.)
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > > >>> Another
> > > > > > > > >>> > > > thing
> > > > > > > > >>> > > > >> to
> > > > > > > > >>> > > > >> > > > > consider
> > > > > > > > >>> > > > >> > > > > > > is
> > > > > > > > >>> > > > >> > > > > > > > >> that
> > > > > > > > >>> > > > >> > > > > > > > >> > we
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > should
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > specify
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > that
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > > >>> all-zeroes
> > > > > > > > >>> > > UUID
> > > > > > > > >>> > > > is
> > > > > > > > >>> > > > >> > not
> > > > > > > > >>> > > > >> > > a
> > > > > > > > >>> > > > >> > > > > > valid
> > > > > > > > >>> > > > >> > > > > > > > >> topic
> > > > > > > > >>> > > > >> > > > > > > > >> > > UUID.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > >  We
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > can't
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > use
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> null
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> for
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > > this
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > > because
> > > > > > > > >>> we
> > > > > > > > >>> > > can't
> > > > > > > > >>> > > > >> > pass a
> > > > > > > > >>> > > > >> > > > > null
> > > > > > > > >>> > > > >> > > > > > > UUID
> > > > > > > > >>> > > > >> > > > > > > > >> over
> > > > > > > > >>> > > > >> > > > > > > > >> > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > RPC
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > protocol
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> (there
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > is
> > no
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > > special
> > > > > > > > >>> > > pattern
> > > > > > > > >>> > > > >> for
> > > > > > > > >>> > > > >> > > null,
> > > > > > > > >>> > > > >> > > > > nor
> > > > > > > > >>> > > > >> > > > > > > do
> > > > > > > > >>> > > > >> > > > > > > > we
> > > > > > > > >>> > > > >> > > > > > > > >> > want
> > > > > > > > >>> > > > >> > > > > > > > >> > > > to
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> waste
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > space
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > reserving
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > > such
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > a
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > > >>> pattern.)
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > > >>> Maybe I
> > > > > > > > >>> > > > missed
> > > > > > > > >>> > > > >> > it,
> > > > > > > > >>> > > > >> > > > but
> > > > > > > > >>> > > > >> > > > > > did
> > > > > > > > >>> > > > >> > > > > > > > you
> > > > > > > > >>> > > > >> > > > > > > > >> > > > describe
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > "migration
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > of...
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > existing
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > > topic[s]
> > > > > > > > >>> > > without
> > > > > > > > >>> > > > >> > topic
> > > > > > > > >>> > > > >> > > > IDs"
> > > > > > > > >>> > > > >> > > > > > in
> > > > > > > > >>> > > > >> > > > > > > > >> detail
> > > > > > > > >>> > > > >> > > > > > > > >> > in
> > > > > > > > >>> > > > >> > > > > > > > >> > > > any
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > section?
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > It
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> seems
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> like
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > when
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > the
> > > > > > > > new
> > > > > > > > >>> > > > controller
> > > > > > > > >>> > > > >> > > > becomes
> > > > > > > > >>> > > > >> > > > > > > > active,
> > > > > > > > >>> > > > >> > > > > > > > >> it
> > > > > > > > >>> > > > >> > > > > > > > >> > > > should
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> just
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > generate
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> random
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > UUIDs
> > > > > > > for
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > these,
> > > > > > > > >>> and
> > > > > > > > >>> > > write
> > > > > > > > >>> > > > >> the
> > > > > > > > >>> > > > >> > > > random
> > > > > > > > >>> > > > >> > > > > > > UUIDs
> > > > > > > > >>> > > > >> > > > > > > > >> back
> > > > > > > > >>> > > > >> > > > > > > > >> > > to
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > ZooKeeper.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > It
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> would
> > be
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > > good
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > to
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > spell
> > > > > > > > >>> that
> > > > > > > > >>> > > out.
> > > > > > > > >>> > > > >> We
> > > > > > > > >>> > > > >> > > > should
> > > > > > > > >>> > > > >> > > > > > make
> > > > > > > > >>> > > > >> > > > > > > > it
> > > > > > > > >>> > > > >> > > > > > > > >> > clear
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > that
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> this
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > happens
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > regardless
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > of
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > the
> > > > > > > > >>> > > inter-broker
> > > > > > > > >>> > > > >> > > protocol
> > > > > > > > >>> > > > >> > > > > > > version
> > > > > > > > >>> > > > >> > > > > > > > >> > (it's
> > > > > > > > >>> > > > >> > > > > > > > >> > > a
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > compatible
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > change).
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > > >>> > > > >> > > "LeaderAndIsrRequests
> > > > > > > > >>> > > > >> > > > > > > > >> including an
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > is_every_partition
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> flag"
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > > > seems a
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > bit
> > > > > > > > >>> wordy.
> > > > > > > > >>> > > Can
> > > > > > > > >>> > > > we
> > > > > > > > >>> > > > >> > just
> > > > > > > > >>> > > > >> > > > > call
> > > > > > > > >>> > > > >> > > > > > > > these
> > > > > > > > >>> > > > >> > > > > > > > >> > "full
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > >>> LeaderAndIsrRequests"?
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> Then
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > RPC
> > > > > > > > >>> field
> > > > > > > > >>> > > could
> > > > > > > > >>> > > > be
> > > > > > > > >>> > > > >> > > named
> > > > > > > > >>> > > > >> > > > > > > "full".
> > > > > > > > >>> > > > >> > > > > > > > >> > Also,
> > > > > > > > >>> > > > >> > > > > > > > >> > > it
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > would
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > probably
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > be
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > better
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > for
> > > > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > RPC
> > > > > > > > >>> field to
> > > > > > > > >>> > > be
> > > > > > > > >>> > > > an
> > > > > > > > >>> > > > >> > enum
> > > > > > > > >>> > > > >> > > > of
> > > > > > > > >>> > > > >> > > > > {
> > > > > > > > >>> > > > >> > > > > > > > >> > > UNSPECIFIED,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > INCREMENTAL,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > FULL
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> },
> so
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > > that
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > we
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > can
> > > > > > > > >>> cleanly
> > > > > > > > >>> > > > handle
> > > > > > > > >>> > > > >> > old
> > > > > > > > >>> > > > >> > > > > > versions
> > > > > > > > >>> > > > >> > > > > > > > (by
> > > > > > > > >>> > > > >> > > > > > > > >> > > > treating
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> them
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > as
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > UNSPECIFIED)
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > In
> > > > > > > > >>> the
> > > > > > > > >>> > > > >> > > > > > LeaderAndIsrRequest
> > > > > > > > >>> > > > >> > > > > > > > >> > section,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > you
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> write
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > "A
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > final
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > deletion
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > event
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > will
> > > > > > > > be
> > > > > > > > >>> > > > secheduled
> > > > > > > > >>> > > > >> > for
> > > > > > > > >>> > > > >> > > X
> > > > > > > > >>> > > > >> > > > ms
> > > > > > > > >>> > > > >> > > > > > > after
> > > > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > LeaderAndIsrRequest
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > was
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > first
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > > >>> received..."
> > > > > > > > >>> > > I
> > > > > > > > >>> > > > >> guess
> > > > > > > > >>> > > > >> > > > the X
> > > > > > > > >>> > > > >> > > > > > > was a
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > placeholder
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> that
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > you
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > intended
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > > > replace
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > before
> > > > > > > > >>> > > posting?
> > > > > > > > >>> > > > :)
> > > > > > > > >>> > > > >> > In
> > > > > > > > >>> > > > >> > > > any
> > > > > > > > >>> > > > >> > > > > > > case,
> > > > > > > > >>> > > > >> > > > > > > > >> this
> > > > > > > > >>> > > > >> > > > > > > > >> > > > seems
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > like
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > kind
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > of
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > thing
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > > we'd
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > want a
> > > > > > > > >>> > > > >> configuration
> > > > > > > > >>> > > > >> > > for.
> > > > > > > > >>> > > > >> > > > > > > Let's
> > > > > > > > >>> > > > >> > > > > > > > >> > > describe
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > that
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > configuration
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> key
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > > > > somewhere
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > in
> > > > > > > > this
> > > > > > > > >>> KIP,
> > > > > > > > >>> > > > >> > including
> > > > > > > > >>> > > > >> > > > what
> > > > > > > > >>> > > > >> > > > > > its
> > > > > > > > >>> > > > >> > > > > > > > >> > default
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > value
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> is.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > We
> > > > > > > > >>> > should
> > > > > > > > >>> > > > >> > probably
> > > > > > > > >>> > > > >> > > > also
> > > > > > > > >>> > > > >> > > > > > > log a
> > > > > > > > >>> > > > >> > > > > > > > >> > bunch
> > > > > > > > >>> > > > >> > > > > > > > >> > > of
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> messages
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > at
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > WARN
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> level
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > > when
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > > >>> something is
> > > > > > > > >>> > > > >> > scheduled
> > > > > > > > >>> > > > >> > > > for
> > > > > > > > >>> > > > >> > > > > > > > >> deletion,
> > > > > > > > >>> > > > >> > > > > > > > >> > as
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > well.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > (Maybe
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > this
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> was
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > > > assumed,
> > > > > > > > but
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > it
> > > > > > > > >>> would be
> > > > > > > > >>> > > good
> > > > > > > > >>> > > > >> to
> > > > > > > > >>> > > > >> > > > mention
> > > > > > > > >>> > > > >> > > > > > > it).
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > I
> > > > > > > > >>> feel
> > > > > > > > >>> > > like
> > > > > > > > >>> > > > >> there
> > > > > > > > >>> > > > >> > > > are a
> > > > > > > > >>> > > > >> > > > > > few
> > > > > > > > >>> > > > >> > > > > > > > >> > sections
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > that
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > should
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > be
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> moved
> > to
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > > > > "rejected
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > > >>> > alternatives."
> > > > > > > > >>> > > > >> For
> > > > > > > > >>> > > > >> > > > > example,
> > > > > > > > >>> > > > >> > > > > > in
> > > > > > > > >>> > > > >> > > > > > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > DeleteTopics
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > section,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> since
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > we're
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > not
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > going
> > > > > > > > >>> to do
> > > > > > > > >>> > > > >> option 1
> > > > > > > > >>> > > > >> > or
> > > > > > > > >>> > > > >> > > > 2,
> > > > > > > > >>> > > > >> > > > > > > these
> > > > > > > > >>> > > > >> > > > > > > > >> > should
> > > > > > > > >>> > > > >> > > > > > > > >> > > be
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > moved
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > into
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > "rejected
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > > >>> > alternatives,"
> > > > > > > > >>> > > > >> > rather
> > > > > > > > >>> > > > >> > > > than
> > > > > > > > >>> > > > >> > > > > > > > >> appearing
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > inline.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Another
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > case
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> is
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > > > "Should
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > we
> > > > > > > > >>> remove
> > > > > > > > >>> > > topic
> > > > > > > > >>> > > > >> name
> > > > > > > > >>> > > > >> > > from
> > > > > > > > >>> > > > >> > > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> protocol
> > > > > > > > >>> > > > >> > > > > > > > >> > > > where
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > possible"
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > section.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> This
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > is
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > > clearly
> > > > > > > > >>> > > > >> discussing a
> > > > > > > > >>> > > > >> > > > design
> > > > > > > > >>> > > > >> > > > > > > > >> > alternative
> > > > > > > > >>> > > > >> > > > > > > > >> > > > that
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> we're
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > not
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > proposing
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > > >>> implement:
> > > > > > > > >>> > > > >> removing
> > > > > > > > >>> > > > >> > the
> > > > > > > > >>> > > > >> > > > > topic
> > > > > > > > >>> > > > >> > > > > > > > name
> > > > > > > > >>> > > > >> > > > > > > > >> > from
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > those
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > protocols.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > Is
> > > > > > > > >>> it
> > > > > > > > >>> > > really
> > > > > > > > >>> > > > >> > > > necessary
> > > > > > > > >>> > > > >> > > > > to
> > > > > > > > >>> > > > >> > > > > > > > have
> > > > > > > > >>> > > > >> > > > > > > > >> a
> > > > > > > > >>> > > > >> > > > > > > > >> > new
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > > >>> > > /admin/delete_topics_by_id
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > path
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > in
> > > > > > > > >>> > ZooKeeper?
> > > > > > > > >>> > > > It
> > > > > > > > >>> > > > >> > seems
> > > > > > > > >>> > > > >> > > > > like
> > > > > > > > >>> > > > >> > > > > > we
> > > > > > > > >>> > > > >> > > > > > > > >> don't
> > > > > > > > >>> > > > >> > > > > > > > >> > > > really
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> need
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > this.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > Whenever
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > there
> > > > > > is
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > a
> > > > > > > new
> > > > > > > > >>> > > > controller,
> > > > > > > > >>> > > > >> > we'll
> > > > > > > > >>> > > > >> > > > > send
> > > > > > > > >>> > > > >> > > > > > > out
> > > > > > > > >>> > > > >> > > > > > > > >> full
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > LeaderAndIsrRequests
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> which
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> will
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > > trigger
> > > > > > > > >>> the
> > > > > > > > >>> > > > stale
> > > > > > > > >>> > > > >> > > topics
> > > > > > > > >>> > > > >> > > > to
> > > > > > > > >>> > > > >> > > > > > be
> > > > > > > > >>> > > > >> > > > > > > > >> cleaned
> > > > > > > > >>> > > > >> > > > > > > > >> > > up.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > >  The
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > active
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > controller
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > > will
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > also
> > > > > > > > >>> send
> > > > > > > > >>> > the
> > > > > > > > >>> > > > full
> > > > > > > > >>> > > > >> > > > > > > > >> LeaderAndIsrRequest
> > > > > > > > >>> > > > >> > > > > > > > >> > > to
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> brokers
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > are
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> just
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > > > starting
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > up.
> > > > > > > > >>> So we
> > > > > > > > >>> > > > don't
> > > > > > > > >>> > > > >> > > really
> > > > > > > > >>> > > > >> > > > > > need
> > > > > > > > >>> > > > >> > > > > > > > this
> > > > > > > > >>> > > > >> > > > > > > > >> > kind
> > > > > > > > >>> > > > >> > > > > > > > >> > > > of
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > two-phase
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > commit
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > (send
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > > out
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > > >>> > > > >> StopReplicasRequest,
> > > > > > > > >>> > > > >> > > get
> > > > > > > > >>> > > > >> > > > > ACKs
> > > > > > > > >>> > > > >> > > > > > > > from
> > > > > > > > >>> > > > >> > > > > > > > >> all
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > nodes,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > commit
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > by
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > removing
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > > >>> > > > >> /admin/delete_topics
> > > > > > > > >>> > > > >> > > > node)
> > > > > > > > >>> > > > >> > > > > > any
> > > > > > > > >>> > > > >> > > > > > > > >> more.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > > You
> > > > > > > > >>> > > mention
> > > > > > > > >>> > > > >> that
> > > > > > > > >>> > > > >> > > > > > > FetchRequest
> > > > > > > > >>> > > > >> > > > > > > > >> will
> > > > > > > > >>> > > > >> > > > > > > > >> > > now
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> include
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > UUID
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> avoid
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > > > issues
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > where
> > > > > > > > >>> > requests
> > > > > > > > >>> > > > are
> > > > > > > > >>> > > > >> > made
> > > > > > > > >>> > > > >> > > > to
> > > > > > > > >>> > > > >> > > > > > > stale
> > > > > > > > >>> > > > >> > > > > > > > >> > > > partitions.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > However,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > adding a
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> UUID
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > to
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > > >>> > > MetadataRequest
> > > > > > > > >>> > > > is
> > > > > > > > >>> > > > >> > > listed
> > > > > > > > >>> > > > >> > > > > as
> > > > > > > > >>> > > > >> > > > > > > > future
> > > > > > > > >>> > > > >> > > > > > > > >> > > work,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > out
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > of
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > scope
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > for
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> this
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> KIP.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > How
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > will
> > > > > > > > the
> > > > > > > > >>> > > client
> > > > > > > > >>> > > > >> learn
> > > > > > > > >>> > > > >> > > > what
> > > > > > > > >>> > > > >> > > > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> topic
> > > > > > > > >>> > > > >> > > > > > > > >> > > UUID
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > is,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > if
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > metadata
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > > > response
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > > doesn't
> > > > > > > > >>> > > include
> > > > > > > > >>> > > > >> that
> > > > > > > > >>> > > > >> > > > > > > information?
> > > > > > > > >>> > > > >> > > > > > > > >> It
> > > > > > > > >>> > > > >> > > > > > > > >> > > > seems
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > like
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > adding
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> UUID
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > > >>> > > MetadataResponse
> > > > > > > > >>> > > > >> > would
> > > > > > > > >>> > > > >> > > be
> > > > > > > > >>> > > > >> > > > > an
> > > > > > > > >>> > > > >> > > > > > > > >> > improvement
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > here
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> that
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > might
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > not
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> be
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> too
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > hard
> > > > > > to
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > make.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > > >>> best,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > > >>> Colin
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > On
> > > > > > > > >>> Mon,
> > > > > > > > >>> > > Sep
> > > > > > > > >>> > > > 9,
> > > > > > > > >>> > > > >> > > 2019,
> > > > > > > > >>> > > > >> > > > at
> > > > > > > > >>> > > > >> > > > > > > > 17:48,
> > > > > > > > >>> > > > >> > > > > > > > >> > > Ryanne
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > Dolan
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > wrote:
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >>> Lucas,
> > > > > > > > >>> > > > this
> > > > > > > > >>> > > > >> > would
> > > > > > > > >>> > > > >> > > > be
> > > > > > > > >>> > > > >> > > > > > > great.
> > > > > > > > >>> > > > >> > > > > > > > >> I've
> > > > > > > > >>> > > > >> > > > > > > > >> > > run
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > into
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > issues
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > with
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > topics
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > being
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >>> > > > resurrected
> > > > > > > > >>> > > > >> > > > > > accidentally,
> > > > > > > > >>> > > > >> > > > > > > > >> since
> > > > > > > > >>> > > > >> > > > > > > > >> > a
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > client
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > cannot
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > easily
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > > > > > distinguish
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > > between
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > a
> > > > > > > > >>> > > deleted
> > > > > > > > >>> > > > >> topic
> > > > > > > > >>> > > > >> > > > and a
> > > > > > > > >>> > > > >> > > > > > new
> > > > > > > > >>> > > > >> > > > > > > > >> topic
> > > > > > > > >>> > > > >> > > > > > > > >> > > with
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> same
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > name.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I'd
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> need
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > > the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > ID
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >>> > > accessible
> > > > > > > > >>> > > > >> from
> > > > > > > > >>> > > > >> > > the
> > > > > > > > >>> > > > >> > > > > > > client
> > > > > > > > >>> > > > >> > > > > > > > to
> > > > > > > > >>> > > > >> > > > > > > > >> > > solve
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > that
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > issue,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > but
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> this
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> is a
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > good
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > first
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >>> step.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >>> Ryanne
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > On
> > > > > > > > >>> > Wed,
> > > > > > > > >>> > > > Sep
> > > > > > > > >>> > > > >> 4,
> > > > > > > > >>> > > > >> > > 2019
> > > > > > > > >>> > > > >> > > > > at
> > > > > > > > >>> > > > >> > > > > > > 1:41
> > > > > > > > >>> > > > >> > > > > > > > >> PM
> > > > > > > > >>> > > > >> > > > > > > > >> > > Lucas
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Bradstreet <
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > > > > > >>> > > lucas@confluent.io>
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > wrote:
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > >>> Hi
> > > > > > > > >>> > > all,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > >>> I
> > > > > > > > >>> > > would
> > > > > > > > >>> > > > >> like
> > > > > > > > >>> > > > >> > to
> > > > > > > > >>> > > > >> > > > > kick
> > > > > > > > >>> > > > >> > > > > > > off
> > > > > > > > >>> > > > >> > > > > > > > >> > > > discussion
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > of
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > KIP-516,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > an
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > > > > > >>> implementation
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > of
> > > > > > > > topic
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > >>> IDs
> > > > > > > > >>> > > for
> > > > > > > > >>> > > > >> > Kafka.
> > > > > > > > >>> > > > >> > > > > Topic
> > > > > > > > >>> > > > >> > > > > > > IDs
> > > > > > > > >>> > > > >> > > > > > > > >> aim
> > > > > > > > >>> > > > >> > > > > > > > >> > to
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > solve
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> topic
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > uniqueness
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > > > problems
> > > > > > > > in
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > Kafka,
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > >>> > where
> > > > > > > > >>> > > > >> > referring
> > > > > > > > >>> > > > >> > > > to
> > > > > > > > >>> > > > >> > > > > a
> > > > > > > > >>> > > > >> > > > > > > > topic
> > > > > > > > >>> > > > >> > > > > > > > >> by
> > > > > > > > >>> > > > >> > > > > > > > >> > > name
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> alone is
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > insufficient.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > > Such
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > cases
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > >>> > > include
> > > > > > > > >>> > > > >> when
> > > > > > > > >>> > > > >> > a
> > > > > > > > >>> > > > >> > > > > topic
> > > > > > > > >>> > > > >> > > > > > > has
> > > > > > > > >>> > > > >> > > > > > > > >> been
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > deleted
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > recreated
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> with
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> the
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > > same
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > name.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > >>> > Unique
> > > > > > > > >>> > > > >> > > > identifiers
> > > > > > > > >>> > > > >> > > > > > will
> > > > > > > > >>> > > > >> > > > > > > > >> help
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > simplify
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > improve
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > Kafka's
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > > > topic
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > > deletion
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > >>> > > process,
> > > > > > > > >>> > > > >> as
> > > > > > > > >>> > > > >> > > well
> > > > > > > > >>> > > > >> > > > as
> > > > > > > > >>> > > > >> > > > > > > > prevent
> > > > > > > > >>> > > > >> > > > > > > > >> > > cases
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > where
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > brokers
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > may
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > > > > > > incorrectly
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > >
> > > > > > > > interact
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > >>> with
> > > > > > > > >>> > > > stale
> > > > > > > > >>> > > > >> > > > versions
> > > > > > > > >>> > > > >> > > > > > of
> > > > > > > > >>> > > > >> > > > > > > > >> topics.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > > >>> > > > >> > > > > > > >
> > > > > > > > >>> > > > >> > > > > > >
> > > > > > > > >>> > > > >> > > > > >
> > > > > > > > >>> > > > >> > > > >
> > > > > > > > >>> > > > >> > > >
> > > > > > > > >>> > > > >> > >
> > > > > > > > >>> > > > >> >
> > > > > > > > >>> > > > >>
> > > > > > > > >>> > > >
> > > > > > > > >>> > >
> > > > > > > > >>> >
> > > > > > > > >>>
> > > > > > > >
> > > > > > >
> > > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > >>> > > Looking
> > > > > > > > >>> > > > >> > forward
> > > > > > > > >>> > > > >> > > > to
> > > > > > > > >>> > > > >> > > > > > your
> > > > > > > > >>> > > > >> > > > > > > > >> > > thoughts.
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > >>> > Lucas
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > > >>> > > > >> > > > > > > > >
> > > > > > > > >>> > > > >> > > > > > > >
> > > > > > > > >>> > > > >> > > > > > >
> > > > > > > > >>> > > > >> > > > > >
> > > > > > > > >>> > > > >> > > > >
> > > > > > > > >>> > > > >> > > >
> > > > > > > > >>> > > > >> > >
> > > > > > > > >>> > > > >> >
> > > > > > > > >>> > > > >>
> > > > > > > > >>> > > > >
> > > > > > > > >>> > > >
> > > > > > > > >>> > >
> > > > > > > > >>> >
> > > > > > > > >>>
> > > > > > > > >>
> > > > > > > >
> > > > > > >
> > > > > >
> > > >
> > > >
> > >
> >
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Justine Olshan <jo...@confluent.io>]

Hi all,

Another update on the KIP. I've added some information about how topic IDs
will be supported in AdminClient

https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-AdminClientSupport


One specific decision I wanted to focus on is the inclusion of both topic
names and topic Ids in metadata and deleteTopics requests.

The idea is that when the request is received the topic id field is checked
first, but if it contains the default (zero) uuid, then the topic name
field will be used.

In the case where ids are used, if the ids do not have a corresponding
topic, the request will fail. In the case where both ids and names are
included, the request will fail if the name does not match the id.


Initially the id field will be used to support the admin client use cases
as described in the KIP, but over time, we can transfer to using topic ids
in all uses of the request.


There will also be some support for returning the topic Id on the
createtopics, deletetopics and metadata responses. I think it makes sense
to include both the name and the id in the successful response for these
cases.


Let me know if you have any comments or concerns regarding the Admin Client
changes.

On Wed, Oct 7, 2020 at 11:05 AM Jun Rao <ju...@confluent.io> wrote:

> Hi, Justine,
>
> I was just wondering if we expose topicId publicly, how would people use
> it. One possibility is to use topicId in all those public interfaces where
> we use topic name. I am not sure if the need is very strong though. So,
> maybe we can list this as future considerations.
>
> Thanks,
>
> Jun
>
> On Wed, Oct 7, 2020 at 8:57 AM Justine Olshan <jo...@confluent.io>
> wrote:
>
> > Hi all,
> >
> > After some careful consideration, I've decided that the TopicIdPartition
> > will be out of scope for this KIP. After investigating what it will look
> > like using it in only the Fetch Request, I found that I would need to
> > modify several public APIs in the consumer and I would need to take
> special
> > care in many situations to handle the request correctly based on what
> > version of the request was sent. For these reasons, I've added
> > TopicIdPartition to potential future work along with an alternative to
> > modify TopicPartition to include a topic ID field.
> >
> > Jun- I was also wondering about some of the details for supporting topic
> > IDs for delete and describe topics in the admin client. Is the idea that
> > instead of supplying topic names to describe or delete, we would supply a
> > collection of ids? If that is the case, I'm wondering if it makes sense
> to
> > add topic Ids to the metadata and delete topics requests.
> >
> > Thanks,
> > Justine
> >
> > On Mon, Oct 5, 2020 at 9:25 PM Satish Duggana <sa...@gmail.com>
> > wrote:
> >
> > > Hi Jun, Justine,
> > > +1 to have TopicIdPartition in public packages. This is useful for
> > > KIP-405 tiered storage too. We currently use TopicPartition for
> > > pluggable log storage and metadata storage apis. We will be using
> > > TopicIdPartition once the proposed changes in this KIP are merged.
> > >
> > > Thanks,
> > > Satish.
> > >
> > > On Tue, Oct 6, 2020 at 5:21 AM Jun Rao <ju...@confluent.io> wrote:
> > > >
> > > > Hi, Justine,
> > > >
> > > > I guess we have to maintain the mapping between topic name and
> topicId
> > in
> > > > both the client and the server. If TopicIdPartition is meant for
> > internal
> > > > usage only, we can create it under a package
> > > > like org/apache/kafka/common/internal and we don't need to document
> it
> > in
> > > > the KIP. If it's meant for external public usage (and potentially
> > > internal
> > > > usage too), we can create it in org/apache/kafka/common and need to
> > > > document it in the KIP.
> > > >
> > > > Since topicId is now part of the topic metadata, perhaps it's useful
> to
> > > > expose it in TopicMetadataAndConfig too.
> > > >
> > > > Thanks,
> > > >
> > > > Jun
> > > >
> > > > On Mon, Oct 5, 2020 at 3:35 PM Justine Olshan <jo...@confluent.io>
> > > wrote:
> > > >
> > > > > Hi Jun,
> > > > >
> > > > > This KIP removes the topic name from many requests. A lot of
> requests
> > > keep
> > > > > track of the partitions internally with a TopicPartition. So when
> we
> > > remove
> > > > > the topic name, we can not differentiate unless we can also use
> topic
> > > ID.
> > > > > The Fetch request relies heavily on this, but ListOffset,
> > > > > OffsetsForLeaderEpoch, etc all use TopicPartition as a key value
> for
> > > > > internal maps. Digging deeper into Fetch, I've noticed that the
> code
> > > path
> > > > > runs into some public APIs that use the TopicPartition. Some
> examples
> > > are
> > > > > Cluster.java or RecordTooLargeException.
> > > > >
> > > > > A related issue is that in some places like
> > > TopicAuthorizationException,
> > > > > the topic name is required and without the name in the request, we
> > > won't
> > > > > know it.
> > > > >
> > > > > I guess an alternative would be to have some mapping from topic
> name
> > to
> > > > > topic ID, but I would have to think about that more.
> > > > >
> > > > > The KIP mentions exposing TopicId in TopicDescription. (The font
> was
> > > > > weirdly colored, so I've made it easier to read.)
> > > > >
> > > > > Would the idea for the TopicMetadataAndConfig be that the topic ID
> > > would be
> > > > > returned as a result?
> > > > > As for the other methods describe/deletetopics, it makes sense to
> > > include
> > > > > them at some point, but I'm not sure how far we should go in terms
> of
> > > > > supporting topic IDs everywhere.
> > > > >
> > > > > Thank you for continuing to look at the KIP!
> > > > > Justine
> > > > >
> > > > >
> > > > > On Mon, Oct 5, 2020 at 2:53 PM Jun Rao <ju...@confluent.io> wrote:
> > > > >
> > > > > > Hi, Justine,
> > > > > >
> > > > > > Adding a TopicIdPartition in the public interface may be useful.
> > > Where
> > > > > > would you expect it to be used?
> > > > > >
> > > > > > Related to this, since we are exposing topicId in the tool, it
> > seems
> > > that
> > > > > > we may want to expose topicId in at least TopicDescription used
> by
> > > > > > AdminClient.describeTopics() and TopicMetadataAndConfig used in
> > > > > > AdminClient.createTopics(Collection<NewTopic> newTopics). I am
> also
> > > not
> > > > > > sure if we plan to expose topicId in other AdminClient methods in
> > > this
> > > > > KIP
> > > > > > or in some future KIP. For example, it might be useful to support
> > > topicId
> > > > > > in the following methods.
> > > > > >
> > > > > >     DescribeTopicsResult describeTopics(Collection<String>
> > > topicNames,
> > > > > > DescribeTopicsOptions options);
> > > > > >     DeleteTopicsResult deleteTopics(Collection<String> topics,
> > > > > > DeleteTopicsOptions options);
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Jun
> > > > > >
> > > > > > On Mon, Oct 5, 2020 at 11:03 AM Justine Olshan <
> > jolshan@confluent.io
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hi all,
> > > > > > >
> > > > > > > I was thinking about this KIP some more. We are removing the
> > topic
> > > name
> > > > > > > from many of these request types, and some of them (eg. Fetch)
> > use
> > > > > > > TopicPartition as keys in a map when handling the request. I
> was
> > > > > thinking
> > > > > > > it would make sense to make some sort of TopicIdPartition with
> > > topic
> > > > > > name,
> > > > > > > topic id, and partition fields (name and id would potentially
> be
> > > > > optional
> > > > > > > types since different request versions will contain only one of
> > the
> > > > > two)
> > > > > > I
> > > > > > > was wondering if it made sense to include this as a public
> facing
> > > API
> > > > > > > similar to TopicPartition. If so, I will add this class to the
> > KIP
> > > as
> > > > > > well.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Justine
> > > > > > >
> > > > > > > On Mon, Oct 5, 2020 at 8:52 AM Justine Olshan <
> > > jolshan@confluent.io>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi all,
> > > > > > > > I've made a few minor changes to the UUID class. I've added a
> > few
> > > > > more
> > > > > > > > methods to support UUIDs more generally, by including a
> > > constructor
> > > > > > and a
> > > > > > > > method to generate random UUIDs. Both of these will be useful
> > for
> > > > > > > testing.
> > > > > > > >
> > > > > > > > Let me know if there are any other areas for discussion.
> > > > > > > >
> > > > > > > > Thank you,
> > > > > > > > Justine
> > > > > > > >
> > > > > > > > On Thu, Oct 1, 2020 at 5:29 PM Justine Olshan <
> > > jolshan@confluent.io>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > >> Hi Jun,
> > > > > > > >> Thanks for looking at it again. Here's the new spec. (I
> fixed
> > > the
> > > > > typo
> > > > > > > in
> > > > > > > >> it too.)
> > > > > > > >> {"name": "id", "type": "string", "doc": option[UUID]}
> > > > > > > >>
> > > > > > > >> Justine
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> On Thu, Oct 1, 2020 at 5:03 PM Jun Rao <ju...@confluent.io>
> > > wrote:
> > > > > > > >>
> > > > > > > >>> Hi, Justine,
> > > > > > > >>>
> > > > > > > >>> Thanks for the update. The KIP looks good to me now. Just a
> > > minor
> > > > > > > comment
> > > > > > > >>> below.
> > > > > > > >>>
> > > > > > > >>> 30. Perhaps "option[UUID]" can be put in the doc.
> > > > > > > >>>
> > > > > > > >>> Jun
> > > > > > > >>>
> > > > > > > >>> On Thu, Oct 1, 2020 at 3:28 PM Justine Olshan <
> > > > > jolshan@confluent.io>
> > > > > > > >>> wrote:
> > > > > > > >>>
> > > > > > > >>> > Hi Jun,
> > > > > > > >>> > Thanks for the response!
> > > > > > > >>> >
> > > > > > > >>> > 30. I think I might have changed this in between. The
> > current
> > > > > > version
> > > > > > > >>> > says:  {"name":
> > > > > > > >>> > "id", "type": "option[UUID]"}, "doc": topic id}
> > > > > > > >>> > I have switched to the option type to cover the migration
> > > case
> > > > > > where
> > > > > > > a
> > > > > > > >>> > TopicZNode does not yet have a topic ID.
> > > > > > > >>> > I understand that due to json, this field is written as a
> > > string,
> > > > > > so
> > > > > > > >>> if I
> > > > > > > >>> > should move the "option[uuid]" to the doc field and the
> > type
> > > > > should
> > > > > > > be
> > > > > > > >>> > "string" please let me know.
> > > > > > > >>> >
> > > > > > > >>> > 40. I've added a definition for UUID.
> > > > > > > >>> > 41,42. Fixed
> > > > > > > >>> >
> > > > > > > >>> > Thank you,
> > > > > > > >>> > Justine
> > > > > > > >>> >
> > > > > > > >>> > On Wed, Sep 30, 2020 at 1:15 PM Jun Rao <
> jun@confluent.io>
> > > > > wrote:
> > > > > > > >>> >
> > > > > > > >>> > > Hi, Justine,
> > > > > > > >>> > >
> > > > > > > >>> > > Thanks for the summary. Just a few minor comments blow.
> > > > > > > >>> > >
> > > > > > > >>> > > 30.  {"name": "id", "type": "string", "doc": "version
> > > id"}}:
> > > > > The
> > > > > > > doc
> > > > > > > >>> > should
> > > > > > > >>> > > say UUID. The issue seems unfixed.
> > > > > > > >>> > >
> > > > > > > >>> > > 40. Since UUID is public facing, could you include its
> > > > > > definition?
> > > > > > > >>> > >
> > > > > > > >>> > > 41. StopReplicaResponse still includes the topic field.
> > > > > > > >>> > >
> > > > > > > >>> > > 42. "It is unnecessary to include the name of the topic
> > in
> > > the
> > > > > > > >>> following
> > > > > > > >>> > > Request/Response calls" It would be useful to include
> all
> > > > > > modified
> > > > > > > >>> > requests
> > > > > > > >>> > > (e.g. produce) in the list below.
> > > > > > > >>> > >
> > > > > > > >>> > > Thanks,
> > > > > > > >>> > >
> > > > > > > >>> > > Jun
> > > > > > > >>> > >
> > > > > > > >>> > >
> > > > > > > >>> > > On Wed, Sep 30, 2020 at 10:17 AM Justine Olshan <
> > > > > > > >>> jolshan@confluent.io>
> > > > > > > >>> > > wrote:
> > > > > > > >>> > >
> > > > > > > >>> > > > Hello again,
> > > > > > > >>> > > >
> > > > > > > >>> > > > I've taken some time to discuss some of the remaining
> > > points
> > > > > > > >>> brought up
> > > > > > > >>> > > by
> > > > > > > >>> > > > the previous emails offline. Here are some of the
> > > > > conclusions.
> > > > > > > >>> > > >
> > > > > > > >>> > > > 1. Directory Structure:
> > > > > > > >>> > > > There was some talk about whether the directory
> > structure
> > > > > > should
> > > > > > > be
> > > > > > > >>> > > changed
> > > > > > > >>> > > > to replace all topic names with topic IDs.
> > > > > > > >>> > > > This will be a useful change, but will prevent
> > > downgrades. It
> > > > > > > will
> > > > > > > >>> be
> > > > > > > >>> > > best
> > > > > > > >>> > > > to wait until a major release, likely alongside
> KIP-500
> > > > > changes
> > > > > > > >>> that
> > > > > > > >>> > will
> > > > > > > >>> > > > prevent downgrades. I've updated the KIP to include
> > this
> > > > > change
> > > > > > > >>> with
> > > > > > > >>> > the
> > > > > > > >>> > > > note about migration and deprecation.
> > > > > > > >>> > > >
> > > > > > > >>> > > >
> > > > > > > >>> > >
> > > > > > > >>> >
> > > > > > > >>>
> > > > > > >
> > > > > >
> > > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-log.dirlayout
> > > > > > > >>> > > >
> > > > > > > >>> > > > 2. Partition Metadata file
> > > > > > > >>> > > > There was some disagreement about the inclusion of
> the
> > > > > > partition
> > > > > > > >>> > metadata
> > > > > > > >>> > > > file.
> > > > > > > >>> > > > This file will be needed to persist the topic ID,
> > > especially
> > > > > > > while
> > > > > > > >>> we
> > > > > > > >>> > > still
> > > > > > > >>> > > > have the old directory structure. Even after the
> > changes,
> > > > > this
> > > > > > > >>> file can
> > > > > > > >>> > > be
> > > > > > > >>> > > > useful for debugging and recovery.
> > > > > > > >>> > > > Creating a single mapping file from topic names to
> > topic
> > > IDs
> > > > > > was
> > > > > > > >>> > > > considered, but ultimately scrapped as it would not
> be
> > as
> > > > > easy
> > > > > > to
> > > > > > > >>> > > maintain.
> > > > > > > >>> > > >
> > > > > > > >>> > > >
> > > > > > > >>> > >
> > > > > > > >>> >
> > > > > > > >>>
> > > > > > >
> > > > > >
> > > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PartitionMetadatafile
> > > > > > > >>> > > >
> > > > > > > >>> > > >
> > > > > > > >>> > > >
> > > > > > > >>> > >
> > > > > > > >>> >
> > > > > > > >>>
> > > > > > >
> > > > > >
> > > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PersistingTopicIDs
> > > > > > > >>> > > >
> > > > > > > >>> > > > 3. Produce Protocols
> > > > > > > >>> > > > After some further discussion, this replacing the
> topic
> > > name
> > > > > > with
> > > > > > > >>> topic
> > > > > > > >>> > > ID
> > > > > > > >>> > > > in these protocols has been added to the KIP.
> > > > > > > >>> > > > This will cut down on the size of the protocol. Since
> > > changes
> > > > > > to
> > > > > > > >>> fetch
> > > > > > > >>> > > are
> > > > > > > >>> > > > included, it does make sense to update these
> protocols.
> > > > > > > >>> > > >
> > > > > > > >>> > > >
> > > > > > > >>> > >
> > > > > > > >>> >
> > > > > > > >>>
> > > > > > >
> > > > > >
> > > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-Produce
> > > > > > > >>> > > >
> > > > > > > >>> > > > 4. KIP-500 Compatibility
> > > > > > > >>> > > > After some discussion with those involved with
> KIP-500,
> > > it
> > > > > > seems
> > > > > > > >>> best
> > > > > > > >>> > to
> > > > > > > >>> > > > use a sentinel topic ID for the metadata topic that
> is
> > > used
> > > > > > > before
> > > > > > > >>> the
> > > > > > > >>> > > > first controller election.
> > > > > > > >>> > > > We had an issue where this metadata topic may not be
> > > assigned
> > > > > > an
> > > > > > > ID
> > > > > > > >>> > > before
> > > > > > > >>> > > > utilizing the Vote and Fetch protocols. It was
> decided
> > to
> > > > > > > reserve a
> > > > > > > >>> > > unique
> > > > > > > >>> > > > ID for this topic to be used until the controller
> could
> > > give
> > > > > > the
> > > > > > > >>> topic
> > > > > > > >>> > a
> > > > > > > >>> > > > unique ID.
> > > > > > > >>> > > > Having the topic keep the sentinel ID (not be
> > reassigned
> > > to a
> > > > > > > >>> unique
> > > > > > > >>> > ID)
> > > > > > > >>> > > > was considered, but it seemed like a good idea to
> leave
> > > the
> > > > > > > option
> > > > > > > >>> open
> > > > > > > >>> > > for
> > > > > > > >>> > > > the metadata topic to have a unique ID in cases where
> > it
> > > > > would
> > > > > > > >>> need to
> > > > > > > >>> > be
> > > > > > > >>> > > > differentiated from other clusters' metadata topics.
> > (ex.
> > > > > > tiered
> > > > > > > >>> > storage)
> > > > > > > >>> > > >
> > > > > > > >>> > > >
> > > > > > > >>> > >
> > > > > > > >>> >
> > > > > > > >>>
> > > > > > >
> > > > > >
> > > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-CompatibilitywithKIP-500
> > > > > > > >>> > > >
> > > > > > > >>> > > > I've also split up the KIP into sub-tasks on the
> JIRA.
> > > > > > Hopefully
> > > > > > > >>> this
> > > > > > > >>> > > will
> > > > > > > >>> > > > provide a better idea about what tasks we have, and
> > > > > eventually
> > > > > > > >>> provide
> > > > > > > >>> > a
> > > > > > > >>> > > > place to see what's done and what is left.
> > > > > > > >>> > > > If there is a task I am missing, please let me know!
> > > > > > > >>> > > > https://issues.apache.org/jira/browse/KAFKA-8872
> > > > > > > >>> > > >
> > > > > > > >>> > > > Of course, these decisions are not set in stone, and
> I
> > > would
> > > > > > love
> > > > > > > >>> to
> > > > > > > >>> > hear
> > > > > > > >>> > > > any feedback.
> > > > > > > >>> > > >
> > > > > > > >>> > > > Thanks,
> > > > > > > >>> > > > Justine
> > > > > > > >>> > > >
> > > > > > > >>> > > > On Mon, Sep 28, 2020 at 11:38 AM Justine Olshan <
> > > > > > > >>> jolshan@confluent.io>
> > > > > > > >>> > > > wrote:
> > > > > > > >>> > > >
> > > > > > > >>> > > > > Hello all,
> > > > > > > >>> > > > >
> > > > > > > >>> > > > > I just wanted to follow up on this discussion. Did
> we
> > > come
> > > > > to
> > > > > > > an
> > > > > > > >>> > > > > understanding about the directory structure?
> > > > > > > >>> > > > >
> > > > > > > >>> > > > > I think the biggest question here is what is
> > > acceptable to
> > > > > > > leave
> > > > > > > >>> out
> > > > > > > >>> > > due
> > > > > > > >>> > > > > to scope vs. what is considered to be too much tech
> > > debt.
> > > > > > > >>> > > > > This KIP is already pretty large with the number of
> > > > > changes,
> > > > > > > but
> > > > > > > >>> it
> > > > > > > >>> > > also
> > > > > > > >>> > > > > makes sense to do things correctly, so I'd love to
> > hear
> > > > > > > >>> everyone's
> > > > > > > >>> > > > thoughts.
> > > > > > > >>> > > > >
> > > > > > > >>> > > > > Thanks,
> > > > > > > >>> > > > > Justine
> > > > > > > >>> > > > >
> > > > > > > >>> > > > > On Fri, Sep 25, 2020 at 8:19 AM Lucas Bradstreet <
> > > > > > > >>> lucas@confluent.io
> > > > > > > >>> > >
> > > > > > > >>> > > > > wrote:
> > > > > > > >>> > > > >
> > > > > > > >>> > > > >> Hi Ismael,
> > > > > > > >>> > > > >>
> > > > > > > >>> > > > >> If you do not store it in a metadata file or in
> the
> > > > > > directory
> > > > > > > >>> > > structure
> > > > > > > >>> > > > >> would you then
> > > > > > > >>> > > > >> require the LeaderAndIsrRequest following startup
> to
> > > give
> > > > > > you
> > > > > > > >>> some
> > > > > > > >>> > > > notion
> > > > > > > >>> > > > >> of
> > > > > > > >>> > > > >> topic name in memory? We would still need this
> > > information
> > > > > > for
> > > > > > > >>> the
> > > > > > > >>> > > older
> > > > > > > >>> > > > >> protocols, but
> > > > > > > >>> > > > >> perhaps this is what's meant by tech debt.
> > > > > > > >>> > > > >>
> > > > > > > >>> > > > >> Once we're free of the old non-topicID requests
> > then I
> > > > > think
> > > > > > > you
> > > > > > > >>> > > > wouldn't
> > > > > > > >>> > > > >> need to retain the topic name.
> > > > > > > >>> > > > >> I think the ability to easily look up topic names
> > > > > associated
> > > > > > > >>> with
> > > > > > > >>> > > > >> partition
> > > > > > > >>> > > > >> directories would still be missed
> > > > > > > >>> > > > >> when diagnosing issues, though maybe it wouldn't
> be
> > a
> > > deal
> > > > > > > >>> breaker
> > > > > > > >>> > > with
> > > > > > > >>> > > > >> the
> > > > > > > >>> > > > >> right tooling.
> > > > > > > >>> > > > >>
> > > > > > > >>> > > > >> Thanks,
> > > > > > > >>> > > > >>
> > > > > > > >>> > > > >> Lucas
> > > > > > > >>> > > > >>
> > > > > > > >>> > > > >> On Fri, Sep 25, 2020 at 7:55 AM Ismael Juma <
> > > > > > > ismael@juma.me.uk>
> > > > > > > >>> > > wrote:
> > > > > > > >>> > > > >>
> > > > > > > >>> > > > >> > Hi Lucas,
> > > > > > > >>> > > > >> >
> > > > > > > >>> > > > >> > Why would you include the name and id? I think
> > you'd
> > > > > want
> > > > > > to
> > > > > > > >>> > remove
> > > > > > > >>> > > > the
> > > > > > > >>> > > > >> > name from the directory name right? Jason's
> > > suggestion
> > > > > was
> > > > > > > >>> that if
> > > > > > > >>> > > you
> > > > > > > >>> > > > >> > remove the name from the directory, then why
> would
> > > you
> > > > > > need
> > > > > > > >>> the id
> > > > > > > >>> > > > name
> > > > > > > >>> > > > >> > mapping file?
> > > > > > > >>> > > > >> >
> > > > > > > >>> > > > >> > Ismael
> > > > > > > >>> > > > >> >
> > > > > > > >>> > > > >> > On Thu, Sep 24, 2020 at 4:24 PM Lucas
> Bradstreet <
> > > > > > > >>> > > lucas@confluent.io>
> > > > > > > >>> > > > >> > wrote:
> > > > > > > >>> > > > >> >
> > > > > > > >>> > > > >> > > > 2. Part of the usage of the file is to have
> > > > > persistent
> > > > > > > >>> storage
> > > > > > > >>> > > of
> > > > > > > >>> > > > >> the
> > > > > > > >>> > > > >> > > topic
> > > > > > > >>> > > > >> > > ID and use it to compare with the ID supplied
> in
> > > the
> > > > > > > >>> > LeaderAndIsr
> > > > > > > >>> > > > >> > Request.
> > > > > > > >>> > > > >> > > There is some discussion in the KIP about
> > changes
> > > to
> > > > > the
> > > > > > > >>> > directory
> > > > > > > >>> > > > >> > > structure, but I believe directory changes
> were
> > > > > > considered
> > > > > > > >>> to be
> > > > > > > >>> > > out
> > > > > > > >>> > > > >> of
> > > > > > > >>> > > > >> > > scope when the KIP was written.
> > > > > > > >>> > > > >> > >
> > > > > > > >>> > > > >> > >
> > > > > > > >>> > > > >> > > Yeah, I was hoping to get a better
> understanding
> > > of
> > > > > why
> > > > > > it
> > > > > > > >>> was
> > > > > > > >>> > > taken
> > > > > > > >>> > > > >> out
> > > > > > > >>> > > > >> > of
> > > > > > > >>> > > > >> > > scope. Perhaps Lucas Bradstreet might have
> more
> > > > > insight
> > > > > > > >>> about
> > > > > > > >>> > the
> > > > > > > >>> > > > >> > decision.
> > > > > > > >>> > > > >> > > Basically my point is that we have to create
> > > > > additional
> > > > > > > >>> > > > infrastructure
> > > > > > > >>> > > > >> > here
> > > > > > > >>> > > > >> > > to support the name/id mapping, so I wanted to
> > > > > > understand
> > > > > > > >>> if we
> > > > > > > >>> > > just
> > > > > > > >>> > > > >> > > consider this a sort of tech debt. It would be
> > > useful
> > > > > to
> > > > > > > >>> cover
> > > > > > > >>> > how
> > > > > > > >>> > > > we
> > > > > > > >>> > > > >> > > handle the case when this file gets corrupted.
> > > Seems
> > > > > > like
> > > > > > > we
> > > > > > > >>> > just
> > > > > > > >>> > > > >> have to
> > > > > > > >>> > > > >> > > trust that it matches whatever the controller
> > > tells us
> > > > > > and
> > > > > > > >>> > rewrite
> > > > > > > >>> > > > it?
> > > > > > > >>> > > > >> > >
> > > > > > > >>> > > > >> > >
> > > > > > > >>> > > > >> > > Hi Jason, Justine,
> > > > > > > >>> > > > >> > >
> > > > > > > >>> > > > >> > > My thought process is that we will likely need
> > the
> > > > > > > metadata
> > > > > > > >>> file
> > > > > > > >>> > > > >> whether
> > > > > > > >>> > > > >> > we
> > > > > > > >>> > > > >> > > rename the directories or not.
> > > > > > > >>> > > > >> > > Linux supports filenames of up to 255 bytes
> and
> > > that
> > > > > > would
> > > > > > > >>> not
> > > > > > > >>> > be
> > > > > > > >>> > > > >> enough
> > > > > > > >>> > > > >> > to
> > > > > > > >>> > > > >> > > support a directory name
> > > > > > > >>> > > > >> > >  including both the name and topic ID. Given
> > that
> > > > > fact,
> > > > > > it
> > > > > > > >>> > seemed
> > > > > > > >>> > > > >> > > reasonable to add the metadata file
> > > > > > > >>> > > > >> > > and leave the directory rename until some time
> > in
> > > the
> > > > > > > future
> > > > > > > >>> > > > (possibly
> > > > > > > >>> > > > >> > > never).
> > > > > > > >>> > > > >> > >
> > > > > > > >>> > > > >> > > If the file does get corrupted I think you're
> > > right.
> > > > > We
> > > > > > > >>> would
> > > > > > > >>> > > either
> > > > > > > >>> > > > >> have
> > > > > > > >>> > > > >> > > to trust it matches what the controller tells
> us
> > > > > > > >>> > > > >> > >  or error out and let an administrator resolve
> > it
> > > by
> > > > > > > >>> checking
> > > > > > > >>> > > across
> > > > > > > >>> > > > >> > > replicas for consistency.
> > > > > > > >>> > > > >> > >
> > > > > > > >>> > > > >> > > Lucas
> > > > > > > >>> > > > >> > >
> > > > > > > >>> > > > >> > >
> > > > > > > >>> > > > >> > > On Thu, Sep 24, 2020 at 3:41 PM Jason
> Gustafson
> > <
> > > > > > > >>> > > jason@confluent.io
> > > > > > > >>> > > > >
> > > > > > > >>> > > > >> > > wrote:
> > > > > > > >>> > > > >> > >
> > > > > > > >>> > > > >> > > > Thanks Justine. Responses below:
> > > > > > > >>> > > > >> > > >
> > > > > > > >>> > > > >> > > > > 1. Yes, the directory will still be based
> on
> > > the
> > > > > > topic
> > > > > > > >>> > names.
> > > > > > > >>> > > > >> > > > LeaderAndIsrRequest is one of the few
> requests
> > > that
> > > > > > will
> > > > > > > >>> still
> > > > > > > >>> > > > >> contain
> > > > > > > >>> > > > >> > > the
> > > > > > > >>> > > > >> > > > topic name. So I think we have this covered.
> > > Sorry
> > > > > for
> > > > > > > >>> > > confusion.
> > > > > > > >>> > > > >> > > >
> > > > > > > >>> > > > >> > > > Ah, you're right. My eyes passed right over
> > the
> > > > > field.
> > > > > > > >>> > > > >> > > >
> > > > > > > >>> > > > >> > > > > 2. Part of the usage of the file is to
> have
> > > > > > persistent
> > > > > > > >>> > storage
> > > > > > > >>> > > > of
> > > > > > > >>> > > > >> the
> > > > > > > >>> > > > >> > > > topic
> > > > > > > >>> > > > >> > > > ID and use it to compare with the ID
> supplied
> > > in the
> > > > > > > >>> > > LeaderAndIsr
> > > > > > > >>> > > > >> > > Request.
> > > > > > > >>> > > > >> > > > There is some discussion in the KIP about
> > > changes to
> > > > > > the
> > > > > > > >>> > > directory
> > > > > > > >>> > > > >> > > > structure, but I believe directory changes
> > were
> > > > > > > >>> considered to
> > > > > > > >>> > be
> > > > > > > >>> > > > >> out of
> > > > > > > >>> > > > >> > > > scope when the KIP was written.
> > > > > > > >>> > > > >> > > >
> > > > > > > >>> > > > >> > > > Yeah, I was hoping to get a better
> > > understanding of
> > > > > > why
> > > > > > > >>> it was
> > > > > > > >>> > > > taken
> > > > > > > >>> > > > >> > out
> > > > > > > >>> > > > >> > > of
> > > > > > > >>> > > > >> > > > scope. Perhaps Lucas Bradstreet might have
> > more
> > > > > > insight
> > > > > > > >>> about
> > > > > > > >>> > > the
> > > > > > > >>> > > > >> > > decision.
> > > > > > > >>> > > > >> > > > Basically my point is that we have to create
> > > > > > additional
> > > > > > > >>> > > > >> infrastructure
> > > > > > > >>> > > > >> > > here
> > > > > > > >>> > > > >> > > > to support the name/id mapping, so I wanted
> to
> > > > > > > understand
> > > > > > > >>> if
> > > > > > > >>> > we
> > > > > > > >>> > > > just
> > > > > > > >>> > > > >> > > > consider this a sort of tech debt. It would
> be
> > > > > useful
> > > > > > to
> > > > > > > >>> cover
> > > > > > > >>> > > how
> > > > > > > >>> > > > >> we
> > > > > > > >>> > > > >> > > > handle the case when this file gets
> corrupted.
> > > Seems
> > > > > > > like
> > > > > > > >>> we
> > > > > > > >>> > > just
> > > > > > > >>> > > > >> have
> > > > > > > >>> > > > >> > to
> > > > > > > >>> > > > >> > > > trust that it matches whatever the
> controller
> > > tells
> > > > > us
> > > > > > > and
> > > > > > > >>> > > rewrite
> > > > > > > >>> > > > >> it?
> > > > > > > >>> > > > >> > > >
> > > > > > > >>> > > > >> > > > > 3. I think this is a good point, but I
> > again I
> > > > > > wonder
> > > > > > > >>> about
> > > > > > > >>> > > the
> > > > > > > >>> > > > >> scope
> > > > > > > >>> > > > >> > > of
> > > > > > > >>> > > > >> > > > the KIP. Most of the changes mentioned in
> the
> > > KIP
> > > > > are
> > > > > > > for
> > > > > > > >>> > > > supporting
> > > > > > > >>> > > > >> > > topic
> > > > > > > >>> > > > >> > > > deletion and I believe that is why the
> produce
> > > > > request
> > > > > > > was
> > > > > > > >>> > > listed
> > > > > > > >>> > > > >> under
> > > > > > > >>> > > > >> > > > future work.
> > > > > > > >>> > > > >> > > >
> > > > > > > >>> > > > >> > > > That's fair. I brought it up since `Fetch`
> is
> > > > > already
> > > > > > > >>> > included.
> > > > > > > >>> > > If
> > > > > > > >>> > > > >> > we've
> > > > > > > >>> > > > >> > > > got `Metadata` and `Fetch`, seems we may as
> > > well do
> > > > > > > >>> `Produce`
> > > > > > > >>> > > and
> > > > > > > >>> > > > >> save
> > > > > > > >>> > > > >> > an
> > > > > > > >>> > > > >> > > > extra kip. No strong objection though if you
> > > want to
> > > > > > > >>> leave it
> > > > > > > >>> > > out.
> > > > > > > >>> > > > >> > > >
> > > > > > > >>> > > > >> > > >
> > > > > > > >>> > > > >> > > > -Jason
> > > > > > > >>> > > > >> > > >
> > > > > > > >>> > > > >> > > >
> > > > > > > >>> > > > >> > > > On Thu, Sep 24, 2020 at 3:26 PM Justine
> > Olshan <
> > > > > > > >>> > > > >> jolshan@confluent.io>
> > > > > > > >>> > > > >> > > > wrote:
> > > > > > > >>> > > > >> > > >
> > > > > > > >>> > > > >> > > > > Hi Jason,
> > > > > > > >>> > > > >> > > > >
> > > > > > > >>> > > > >> > > > > Thanks for your comments.
> > > > > > > >>> > > > >> > > > >
> > > > > > > >>> > > > >> > > > > 1. Yes, the directory will still be based
> on
> > > the
> > > > > > topic
> > > > > > > >>> > names.
> > > > > > > >>> > > > >> > > > > LeaderAndIsrRequest is one of the few
> > requests
> > > > > that
> > > > > > > will
> > > > > > > >>> > still
> > > > > > > >>> > > > >> > contain
> > > > > > > >>> > > > >> > > > the
> > > > > > > >>> > > > >> > > > > topic name. So I think we have this
> covered.
> > > Sorry
> > > > > > for
> > > > > > > >>> > > > confusion.
> > > > > > > >>> > > > >> > > > >
> > > > > > > >>> > > > >> > > > > 2. Part of the usage of the file is to
> have
> > > > > > persistent
> > > > > > > >>> > storage
> > > > > > > >>> > > > of
> > > > > > > >>> > > > >> the
> > > > > > > >>> > > > >> > > > topic
> > > > > > > >>> > > > >> > > > > ID and use it to compare with the ID
> > supplied
> > > in
> > > > > the
> > > > > > > >>> > > > LeaderAndIsr
> > > > > > > >>> > > > >> > > > Request.
> > > > > > > >>> > > > >> > > > > There is some discussion in the KIP about
> > > changes
> > > > > to
> > > > > > > the
> > > > > > > >>> > > > directory
> > > > > > > >>> > > > >> > > > > structure, but I believe directory changes
> > > were
> > > > > > > >>> considered
> > > > > > > >>> > to
> > > > > > > >>> > > be
> > > > > > > >>> > > > >> out
> > > > > > > >>> > > > >> > of
> > > > > > > >>> > > > >> > > > > scope when the KIP was written.
> > > > > > > >>> > > > >> > > > >
> > > > > > > >>> > > > >> > > > > 3. I think this is a good point, but I
> > again I
> > > > > > wonder
> > > > > > > >>> about
> > > > > > > >>> > > the
> > > > > > > >>> > > > >> scope
> > > > > > > >>> > > > >> > > of
> > > > > > > >>> > > > >> > > > > the KIP. Most of the changes mentioned in
> > the
> > > KIP
> > > > > > are
> > > > > > > >>> for
> > > > > > > >>> > > > >> supporting
> > > > > > > >>> > > > >> > > > topic
> > > > > > > >>> > > > >> > > > > deletion and I believe that is why the
> > produce
> > > > > > request
> > > > > > > >>> was
> > > > > > > >>> > > > listed
> > > > > > > >>> > > > >> > under
> > > > > > > >>> > > > >> > > > > future work.
> > > > > > > >>> > > > >> > > > >
> > > > > > > >>> > > > >> > > > > 4. This sounds like it might be a good
> > > solution,
> > > > > > but I
> > > > > > > >>> will
> > > > > > > >>> > > need
> > > > > > > >>> > > > >> to
> > > > > > > >>> > > > >> > > > discuss
> > > > > > > >>> > > > >> > > > > more with KIP-500 folks to get the details
> > > right.
> > > > > > > >>> > > > >> > > > >
> > > > > > > >>> > > > >> > > > > Thanks,
> > > > > > > >>> > > > >> > > > > Justine
> > > > > > > >>> > > > >> > > > >
> > > > > > > >>> > > > >> > > > > On Thu, Sep 24, 2020 at 12:30 PM Jason
> > > Gustafson <
> > > > > > > >>> > > > >> jason@confluent.io
> > > > > > > >>> > > > >> > >
> > > > > > > >>> > > > >> > > > > wrote:
> > > > > > > >>> > > > >> > > > >
> > > > > > > >>> > > > >> > > > > > Hi Justine,
> > > > > > > >>> > > > >> > > > > >
> > > > > > > >>> > > > >> > > > > > Thanks for picking up this work. I have
> a
> > > few
> > > > > > > >>> > > > >> questions/comments:
> > > > > > > >>> > > > >> > > > > >
> > > > > > > >>> > > > >> > > > > > 1. It sounds like the directory
> structure
> > is
> > > > > still
> > > > > > > >>> going
> > > > > > > >>> > to
> > > > > > > >>> > > be
> > > > > > > >>> > > > >> > based
> > > > > > > >>> > > > >> > > on
> > > > > > > >>> > > > >> > > > > > topic names. Do I have that right? One
> > > > > > complication
> > > > > > > is
> > > > > > > >>> > that
> > > > > > > >>> > > > the
> > > > > > > >>> > > > >> > > > > > LeaderAndIsr request does not include
> the
> > > topic
> > > > > > name
> > > > > > > >>> any
> > > > > > > >>> > > > longer.
> > > > > > > >>> > > > >> > This
> > > > > > > >>> > > > >> > > > > means
> > > > > > > >>> > > > >> > > > > > that a replica must wait for the
> > > UpdateMetadata
> > > > > > > >>> request to
> > > > > > > >>> > > > >> arrive
> > > > > > > >>> > > > >> > > with
> > > > > > > >>> > > > >> > > > > the
> > > > > > > >>> > > > >> > > > > > topic name mapping before it can create
> > the
> > > > > > > >>> directory. I
> > > > > > > >>> > > > wonder
> > > > > > > >>> > > > >> if
> > > > > > > >>> > > > >> > it
> > > > > > > >>> > > > >> > > > > would
> > > > > > > >>> > > > >> > > > > > be simpler to avoid assumptions on the
> > > ordering
> > > > > of
> > > > > > > >>> > > > >> UpdateMetadata
> > > > > > > >>> > > > >> > and
> > > > > > > >>> > > > >> > > > let
> > > > > > > >>> > > > >> > > > > > LeaderAndIsr include the topic name as
> > well.
> > > > > Feels
> > > > > > > >>> like we
> > > > > > > >>> > > are
> > > > > > > >>> > > > >> not
> > > > > > > >>> > > > >> > > > saving
> > > > > > > >>> > > > >> > > > > > that much by excluding it.
> > > > > > > >>> > > > >> > > > > >
> > > > > > > >>> > > > >> > > > > > 2. On a related note, it seems that the
> > > reason
> > > > > we
> > > > > > > >>> have the
> > > > > > > >>> > > > >> > partition
> > > > > > > >>> > > > >> > > > > > metadata file is because we are not
> > > changing the
> > > > > > > >>> directory
> > > > > > > >>> > > > >> > structure.
> > > > > > > >>> > > > >> > > > We
> > > > > > > >>> > > > >> > > > > > need it so that we remember which
> > > directories
> > > > > map
> > > > > > to
> > > > > > > >>> which
> > > > > > > >>> > > > topic
> > > > > > > >>> > > > >> > id.
> > > > > > > >>> > > > >> > > I
> > > > > > > >>> > > > >> > > > > > think it would be useful to add some
> > detail
> > > > > about
> > > > > > > why
> > > > > > > >>> > > changing
> > > > > > > >>> > > > >> the
> > > > > > > >>> > > > >> > > > > > directory layout was rejected. Basically
> > > trying
> > > > > to
> > > > > > > >>> > > understand
> > > > > > > >>> > > > >> how
> > > > > > > >>> > > > >> > > much
> > > > > > > >>> > > > >> > > > > > effort we are saving by skipping this
> step
> > > if we
> > > > > > > have
> > > > > > > >>> to
> > > > > > > >>> > > > >> implement
> > > > > > > >>> > > > >> > > this
> > > > > > > >>> > > > >> > > > > > additional bookkeeping. One concern I
> have
> > > for
> > > > > > > >>> example is
> > > > > > > >>> > > that
> > > > > > > >>> > > > >> the
> > > > > > > >>> > > > >> > > > > > partition metadata file gets corrupt and
> > > then
> > > > > all
> > > > > > > >>> bets are
> > > > > > > >>> > > off
> > > > > > > >>> > > > >> as
> > > > > > > >>> > > > >> > far
> > > > > > > >>> > > > >> > > > as
> > > > > > > >>> > > > >> > > > > > topic consistency.
> > > > > > > >>> > > > >> > > > > >
> > > > > > > >>> > > > >> > > > > > 3. Is it worth adding support now for
> > topic
> > > ids
> > > > > in
> > > > > > > the
> > > > > > > >>> > > > `Produce`
> > > > > > > >>> > > > >> > > > request
> > > > > > > >>> > > > >> > > > > > now? Overhead is mentioned as one of the
> > > > > > motivations
> > > > > > > >>> and
> > > > > > > >>> > the
> > > > > > > >>> > > > >> best
> > > > > > > >>> > > > >> > > APIs
> > > > > > > >>> > > > >> > > > to
> > > > > > > >>> > > > >> > > > > > get savings from are `Produce` and
> > `Fetch`.
> > > > > > > >>> > > > >> > > > > >
> > > > > > > >>> > > > >> > > > > > 4. When it comes to bootstrapping the
> > > metadata
> > > > > > topic
> > > > > > > >>> with
> > > > > > > >>> > > > >> respect
> > > > > > > >>> > > > >> > to
> > > > > > > >>> > > > >> > > > > > KIP-500, one suggestion would be to
> > reserve
> > > a
> > > > > > > sentinel
> > > > > > > >>> > topic
> > > > > > > >>> > > > ID
> > > > > > > >>> > > > >> > which
> > > > > > > >>> > > > >> > > > can
> > > > > > > >>> > > > >> > > > > > be used initially by new brokers. When
> the
> > > first
> > > > > > > >>> > controller
> > > > > > > >>> > > is
> > > > > > > >>> > > > >> > > elected,
> > > > > > > >>> > > > >> > > > > it
> > > > > > > >>> > > > >> > > > > > can assign a topicId to the metadata
> > topic.
> > > This
> > > > > > is
> > > > > > > a
> > > > > > > >>> bit
> > > > > > > >>> > > > >> similar
> > > > > > > >>> > > > >> > to
> > > > > > > >>> > > > >> > > > how
> > > > > > > >>> > > > >> > > > > we
> > > > > > > >>> > > > >> > > > > > were planning to generate the clusterId.
> > > > > > > >>> > > > >> > > > > >
> > > > > > > >>> > > > >> > > > > > Thanks,
> > > > > > > >>> > > > >> > > > > > Jason
> > > > > > > >>> > > > >> > > > > >
> > > > > > > >>> > > > >> > > > > >
> > > > > > > >>> > > > >> > > > > > On Thu, Sep 24, 2020 at 11:10 AM Jun
> Rao <
> > > > > > > >>> > jun@confluent.io>
> > > > > > > >>> > > > >> wrote:
> > > > > > > >>> > > > >> > > > > >
> > > > > > > >>> > > > >> > > > > > > Hi, Justine,
> > > > > > > >>> > > > >> > > > > > >
> > > > > > > >>> > > > >> > > > > > > Thanks for the updated KIP. A few more
> > > > > comments
> > > > > > > >>> below.
> > > > > > > >>> > > > >> > > > > > >
> > > > > > > >>> > > > >> > > > > > > 30.  {"name": "id", "type": "string",
> > > "doc":
> > > > > > > >>> "version
> > > > > > > >>> > > id"}}:
> > > > > > > >>> > > > >> The
> > > > > > > >>> > > > >> > > doc
> > > > > > > >>> > > > >> > > > > > should
> > > > > > > >>> > > > >> > > > > > > say UUID.
> > > > > > > >>> > > > >> > > > > > >
> > > > > > > >>> > > > >> > > > > > > 31. LeaderAndIsrResponse v5 and
> > > > > > > StopReplicaResponse
> > > > > > > >>> v4 :
> > > > > > > >>> > > It
> > > > > > > >>> > > > >> seems
> > > > > > > >>> > > > >> > > > there
> > > > > > > >>> > > > >> > > > > > is
> > > > > > > >>> > > > >> > > > > > > no need to add topic_id at partitions
> > > level.
> > > > > > > >>> > > > >> > > > > > >
> > > > > > > >>> > > > >> > > > > > > 32. Regarding partition metadata file.
> > > Perhaps
> > > > > > the
> > > > > > > >>> key
> > > > > > > >>> > can
> > > > > > > >>> > > > be
> > > > > > > >>> > > > >> a
> > > > > > > >>> > > > >> > > > single
> > > > > > > >>> > > > >> > > > > > > word, sth like the following.
> > > > > > > >>> > > > >> > > > > > > version: 0
> > > > > > > >>> > > > >> > > > > > > topic_id:
> > > 46bdb63f-9e8d-4a38-bf7b-ee4eb2a794e4
> > > > > > > >>> > > > >> > > > > > >
> > > > > > > >>> > > > >> > > > > > > 33. Another tricky thing that I
> realized
> > > is
> > > > > how
> > > > > > to
> > > > > > > >>> > support
> > > > > > > >>> > > > the
> > > > > > > >>> > > > >> > > > metadata
> > > > > > > >>> > > > >> > > > > > > topic introduced in KIP-595. It's a
> bit
> > > tricky
> > > > > > to
> > > > > > > >>> > assign a
> > > > > > > >>> > > > >> UUID
> > > > > > > >>> > > > >> > to
> > > > > > > >>> > > > >> > > > the
> > > > > > > >>> > > > >> > > > > > > metadata topic since we have a
> > > chicken-and-egg
> > > > > > > >>> problem.
> > > > > > > >>> > > The
> > > > > > > >>> > > > >> > > > controller
> > > > > > > >>> > > > >> > > > > > > needs to persist the UUID in the
> > metadata
> > > > > topic
> > > > > > in
> > > > > > > >>> order
> > > > > > > >>> > > to
> > > > > > > >>> > > > >> > assign
> > > > > > > >>> > > > >> > > > one
> > > > > > > >>> > > > >> > > > > > > successfully, but the metadata topic
> is
> > > needed
> > > > > > to
> > > > > > > >>> elect
> > > > > > > >>> > a
> > > > > > > >>> > > > >> > > controller
> > > > > > > >>> > > > >> > > > > > > (KIP-631). So, this probably needs a
> bit
> > > more
> > > > > > > >>> thought.
> > > > > > > >>> > > > >> > > > > > >
> > > > > > > >>> > > > >> > > > > > > Jun
> > > > > > > >>> > > > >> > > > > > >
> > > > > > > >>> > > > >> > > > > > > On Thu, Sep 24, 2020 at 3:04 AM Ismael
> > > Juma <
> > > > > > > >>> > > > >> ismael@juma.me.uk>
> > > > > > > >>> > > > >> > > > wrote:
> > > > > > > >>> > > > >> > > > > > >
> > > > > > > >>> > > > >> > > > > > > > Also, can we provide more details on
> > > how the
> > > > > > > >>> Partition
> > > > > > > >>> > > > >> Metadata
> > > > > > > >>> > > > >> > > > file
> > > > > > > >>> > > > >> > > > > > will
> > > > > > > >>> > > > >> > > > > > > > be used?
> > > > > > > >>> > > > >> > > > > > > >
> > > > > > > >>> > > > >> > > > > > > > Ismael
> > > > > > > >>> > > > >> > > > > > > >
> > > > > > > >>> > > > >> > > > > > > > On Thu, Sep 24, 2020 at 3:01 AM
> Ismael
> > > Juma
> > > > > <
> > > > > > > >>> > > > >> ismael@juma.me.uk
> > > > > > > >>> > > > >> > >
> > > > > > > >>> > > > >> > > > > wrote:
> > > > > > > >>> > > > >> > > > > > > >
> > > > > > > >>> > > > >> > > > > > > > > Hi Justine,
> > > > > > > >>> > > > >> > > > > > > > >
> > > > > > > >>> > > > >> > > > > > > > > I think we need to update the
> > > "Rejected
> > > > > > > >>> > Alternatives"
> > > > > > > >>> > > > >> section
> > > > > > > >>> > > > >> > > to
> > > > > > > >>> > > > >> > > > > take
> > > > > > > >>> > > > >> > > > > > > > into
> > > > > > > >>> > > > >> > > > > > > > > account that the proposal now
> > removes
> > > the
> > > > > > > topic
> > > > > > > >>> name
> > > > > > > >>> > > > from
> > > > > > > >>> > > > >> the
> > > > > > > >>> > > > >> > > > fetch
> > > > > > > >>> > > > >> > > > > > > > > request. Also, if we are removing
> it
> > > from
> > > > > > the
> > > > > > > >>> Fetch
> > > > > > > >>> > > > >> request,
> > > > > > > >>> > > > >> > > does
> > > > > > > >>> > > > >> > > > > it
> > > > > > > >>> > > > >> > > > > > > make
> > > > > > > >>> > > > >> > > > > > > > > sense not to remove it from
> similar
> > > > > requests
> > > > > > > >>> like
> > > > > > > >>> > > > >> > > > > ListOffsetRequest?
> > > > > > > >>> > > > >> > > > > > > > >
> > > > > > > >>> > > > >> > > > > > > > > Ismael
> > > > > > > >>> > > > >> > > > > > > > >
> > > > > > > >>> > > > >> > > > > > > > > On Thu, Sep 24, 2020 at 2:46 AM
> > David
> > > > > Jacot
> > > > > > <
> > > > > > > >>> > > > >> > > djacot@confluent.io
> > > > > > > >>> > > > >> > > > >
> > > > > > > >>> > > > >> > > > > > > wrote:
> > > > > > > >>> > > > >> > > > > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> Hi Justine,
> > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > >>> > > > >> > > > > > > > >> Thanks for the KIP. I finally had
> > > time to
> > > > > > > read
> > > > > > > >>> it
> > > > > > > >>> > :).
> > > > > > > >>> > > > It
> > > > > > > >>> > > > >> is
> > > > > > > >>> > > > >> > a
> > > > > > > >>> > > > >> > > > > great
> > > > > > > >>> > > > >> > > > > > > > >> improvement.
> > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > >>> > > > >> > > > > > > > >> I have a few comments/questions:
> > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > >>> > > > >> > > > > > > > >> 1. It seems that the schema of
> the
> > > > > > > >>> > StopReplicaRequest
> > > > > > > >>> > > > is
> > > > > > > >>> > > > >> > > > slightly
> > > > > > > >>> > > > >> > > > > > > > >> outdated.
> > > > > > > >>> > > > >> > > > > > > > >> We
> > > > > > > >>> > > > >> > > > > > > > >> did some changes as part of
> > KIP-570.
> > > V3
> > > > > is
> > > > > > > >>> already
> > > > > > > >>> > > > >> organized
> > > > > > > >>> > > > >> > > by
> > > > > > > >>> > > > >> > > > > > > topics.
> > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > >>> > > > >> > > > > > > > >> 2. I just want to make sure that
> I
> > > > > > understand
> > > > > > > >>> the
> > > > > > > >>> > > > >> > > reconciliation
> > > > > > > >>> > > > >> > > > > > > > >> logic correctly. When
> > > > > > > >>> > > > >> > > > > > > > >> an "INCREMENTAL" LeaderAndIsr
> > > Request is
> > > > > > > >>> received,
> > > > > > > >>> > > the
> > > > > > > >>> > > > >> > broker
> > > > > > > >>> > > > >> > > > will
> > > > > > > >>> > > > >> > > > > > > also
> > > > > > > >>> > > > >> > > > > > > > >> reconcile
> > > > > > > >>> > > > >> > > > > > > > >> when the local uuid does not
> match
> > > the
> > > > > uuid
> > > > > > > in
> > > > > > > >>> the
> > > > > > > >>> > > > >> request,
> > > > > > > >>> > > > >> > > > right?
> > > > > > > >>> > > > >> > > > > > In
> > > > > > > >>> > > > >> > > > > > > > this
> > > > > > > >>> > > > >> > > > > > > > >> case, the
> > > > > > > >>> > > > >> > > > > > > > >> local log is staged for deletion.
> > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > >>> > > > >> > > > > > > > >> 3. In the documentation of the `
> > > > > > > >>> > > > >> delete.stale.topic.delay.ms
> > > > > > > >>> > > > >> > `
> > > > > > > >>> > > > >> > > > > > config,
> > > > > > > >>> > > > >> > > > > > > it
> > > > > > > >>> > > > >> > > > > > > > >> says "When a
> > > > > > > >>> > > > >> > > > > > > > >> FULL LeaderAndIsrRequest is
> > > received..."
> > > > > > but
> > > > > > > I
> > > > > > > >>> > > suppose
> > > > > > > >>> > > > it
> > > > > > > >>> > > > >> > > > applies
> > > > > > > >>> > > > >> > > > > to
> > > > > > > >>> > > > >> > > > > > > > both
> > > > > > > >>> > > > >> > > > > > > > >> types.
> > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > >>> > > > >> > > > > > > > >> Best,
> > > > > > > >>> > > > >> > > > > > > > >> David
> > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > >>> > > > >> > > > > > > > >> On Thu, Sep 24, 2020 at 1:40 AM
> > > Justine
> > > > > > > Olshan
> > > > > > > >>> <
> > > > > > > >>> > > > >> > > > > > jolshan@confluent.io>
> > > > > > > >>> > > > >> > > > > > > > >> wrote:
> > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > >>> > > > >> > > > > > > > >> > Hi Jun,
> > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > >>> > > > >> > > > > > > > >> > Thanks for the comments. I
> > > apologize
> > > > > for
> > > > > > > >>> some of
> > > > > > > >>> > > the
> > > > > > > >>> > > > >> typos
> > > > > > > >>> > > > >> > > and
> > > > > > > >>> > > > >> > > > > > > > >> confusion.
> > > > > > > >>> > > > >> > > > > > > > >> > I’ve updated the KIP to fix
> some
> > > of the
> > > > > > > >>> issues
> > > > > > > >>> > you
> > > > > > > >>> > > > >> > > mentioned.
> > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > >>> > > > >> > > > > > > > >> > 20.2 I’ve changed the type to
> > > String.
> > > > > > > >>> > > > >> > > > > > > > >> > 20.1/3 I’ve updated the
> > TopicZNode
> > > to
> > > > > fix
> > > > > > > >>> > > formatting
> > > > > > > >>> > > > >> and
> > > > > > > >>> > > > >> > > > reflect
> > > > > > > >>> > > > >> > > > > > the
> > > > > > > >>> > > > >> > > > > > > > >> latest
> > > > > > > >>> > > > >> > > > > > > > >> > version before this change.
> > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > >>> > > > >> > > > > > > > >> > 21. You are correct and I’ve
> > > removed
> > > > > this
> > > > > > > >>> line.
> > > > > > > >>> > > I’ve
> > > > > > > >>> > > > >> also
> > > > > > > >>> > > > >> > > > added
> > > > > > > >>> > > > >> > > > > a
> > > > > > > >>> > > > >> > > > > > > line
> > > > > > > >>> > > > >> > > > > > > > >> > mentioning an IBP bump is
> > > necessary for
> > > > > > > >>> migration
> > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > >>> > > > >> > > > > > > > >> > 22. I think the wording was
> > > unclear but
> > > > > > > your
> > > > > > > >>> > > summary
> > > > > > > >>> > > > is
> > > > > > > >>> > > > >> > what
> > > > > > > >>> > > > >> > > > was
> > > > > > > >>> > > > >> > > > > > > > >> intended
> > > > > > > >>> > > > >> > > > > > > > >> > by this line. I’ve updated to
> > make
> > > this
> > > > > > > point
> > > > > > > >>> > more
> > > > > > > >>> > > > >> clear.
> > > > > > > >>> > > > >> > > > > “Remove
> > > > > > > >>> > > > >> > > > > > > > >> deleted
> > > > > > > >>> > > > >> > > > > > > > >> > topics from replicas by sending
> > > > > > > >>> > StopReplicaRequest
> > > > > > > >>> > > V3
> > > > > > > >>> > > > >> > before
> > > > > > > >>> > > > >> > > > the
> > > > > > > >>> > > > >> > > > > > IBP
> > > > > > > >>> > > > >> > > > > > > > >> bump
> > > > > > > >>> > > > >> > > > > > > > >> > using the old logic, and using
> V4
> > > and
> > > > > the
> > > > > > > new
> > > > > > > >>> > logic
> > > > > > > >>> > > > >> with
> > > > > > > >>> > > > >> > > topic
> > > > > > > >>> > > > >> > > > > IDs
> > > > > > > >>> > > > >> > > > > > > > after
> > > > > > > >>> > > > >> > > > > > > > >> > the IBP bump.”
> > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > >>> > > > >> > > > > > > > >> > 23. I’ve removed the
> unspecified
> > > type
> > > > > > from
> > > > > > > >>> the
> > > > > > > >>> > KIP
> > > > > > > >>> > > > and
> > > > > > > >>> > > > >> > > mention
> > > > > > > >>> > > > >> > > > > > that
> > > > > > > >>> > > > >> > > > > > > > IBP
> > > > > > > >>> > > > >> > > > > > > > >> > will be used to handle this
> > > request.
> > > > > “IBP
> > > > > > > >>> will be
> > > > > > > >>> > > > used
> > > > > > > >>> > > > >> to
> > > > > > > >>> > > > >> > > > > > determine
> > > > > > > >>> > > > >> > > > > > > > >> whether
> > > > > > > >>> > > > >> > > > > > > > >> > this new form of the request
> will
> > > be
> > > > > > used.
> > > > > > > >>> For
> > > > > > > >>> > > older
> > > > > > > >>> > > > >> > > requests,
> > > > > > > >>> > > > >> > > > > we
> > > > > > > >>> > > > >> > > > > > > will
> > > > > > > >>> > > > >> > > > > > > > >> > ignore this field and default
> to
> > > > > previous
> > > > > > > >>> > > behavior.”
> > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > >>> > > > >> > > > > > > > >> > 24. I’ve fixed this typo.
> > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > >>> > > > >> > > > > > > > >> > 25. I've added a topics at a
> > higher
> > > > > level
> > > > > > > for
> > > > > > > >>> > > > >> > > > > LeaderAndIsrResponse
> > > > > > > >>> > > > >> > > > > > > v5,
> > > > > > > >>> > > > >> > > > > > > > >> > StopReplicaResponse v4. I've
> also
> > > > > changed
> > > > > > > >>> > > > >> > StopReplicaRequest
> > > > > > > >>> > > > >> > > > v4
> > > > > > > >>> > > > >> > > > > to
> > > > > > > >>> > > > >> > > > > > > > have
> > > > > > > >>> > > > >> > > > > > > > >> > topics at a higher level.
> > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > >>> > > > >> > > > > > > > >> > 26. I’ve updated
> > > > > > > >>> forgotten_topics_data--added the
> > > > > > > >>> > > > >> topic ID
> > > > > > > >>> > > > >> > > and
> > > > > > > >>> > > > >> > > > > > > removed
> > > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > > >>> > > > >> > > > > > > > >> > topic name
> > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > >>> > > > >> > > > > > > > >> > 27. I’ve decided on plain text,
> > and
> > > > > I’ve
> > > > > > > >>> added an
> > > > > > > >>> > > > >> example.
> > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > >>> > > > >> > > > > > > > >> > 28. I’ve added this idea to
> > future
> > > > > work.
> > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > >>> > > > >> > > > > > > > >> > Thanks again for taking a look,
> > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > >>> > > > >> > > > > > > > >> > Justine
> > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > >>> > > > >> > > > > > > > >> > On Wed, Sep 23, 2020 at 10:28
> AM
> > > Jun
> > > > > Rao
> > > > > > <
> > > > > > > >>> > > > >> > jun@confluent.io>
> > > > > > > >>> > > > >> > > > > > wrote:
> > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > >>> > > > >> > > > > > > > >> > > Hi, Justine,
> > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > Thanks for the response. Made
> > > another
> > > > > > > >>> pass. A
> > > > > > > >>> > few
> > > > > > > >>> > > > >> more
> > > > > > > >>> > > > >> > > > > comments
> > > > > > > >>> > > > >> > > > > > > > below.
> > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > 20. znode schema:
> > > > > > > >>> > > > >> > > > > > > > >> > > 20.1 It seems that {"name":
> > > > > "version",
> > > > > > > >>> "type":
> > > > > > > >>> > > > "int",
> > > > > > > >>> > > > >> > > "id":
> > > > > > > >>> > > > >> > > > > > > "UUID",
> > > > > > > >>> > > > >> > > > > > > > >> > "doc":
> > > > > > > >>> > > > >> > > > > > > > >> > > "version id"} should be
> > {"name":
> > > > > > > "version",
> > > > > > > >>> > > "type":
> > > > > > > >>> > > > >> > > "int"},
> > > > > > > >>> > > > >> > > > > > > {"name":
> > > > > > > >>> > > > >> > > > > > > > >> > "id",
> > > > > > > >>> > > > >> > > > > > > > >> > > "type": "UUID", "doc":
> "version
> > > id"}.
> > > > > > > >>> > > > >> > > > > > > > >> > > 20.2 The znode format is JSON
> > > which
> > > > > > > doesn't
> > > > > > > >>> > have
> > > > > > > >>> > > > UUID
> > > > > > > >>> > > > >> > > type.
> > > > > > > >>> > > > >> > > > So
> > > > > > > >>> > > > >> > > > > > the
> > > > > > > >>> > > > >> > > > > > > > >> type
> > > > > > > >>> > > > >> > > > > > > > >> > > probably should be string?
> > > > > > > >>> > > > >> > > > > > > > >> > > 20.3 Also, the existing
> format
> > > used
> > > > > > seems
> > > > > > > >>> > > outdated.
> > > > > > > >>> > > > >> It
> > > > > > > >>> > > > >> > > > should
> > > > > > > >>> > > > >> > > > > > have
> > > > > > > >>> > > > >> > > > > > > > the
> > > > > > > >>> > > > >> > > > > > > > >> > > following format.
> > > > > > > >>> > > > >> > > > > > > > >> > >     Json.encodeAsBytes(Map(
> > > > > > > >>> > > > >> > > > > > > > >> > >       "version" -> 2,
> > > > > > > >>> > > > >> > > > > > > > >> > >       "partitions" ->
> > > > > > > >>> > > replicaAssignmentJson.asJava,
> > > > > > > >>> > > > >> > > > > > > > >> > >       "adding_replicas" ->
> > > > > > > >>> > > > >> > > > > addingReplicasAssignmentJson.asJava,
> > > > > > > >>> > > > >> > > > > > > > >> > >       "removing_replicas" ->
> > > > > > > >>> > > > >> > > > > > removingReplicasAssignmentJson.asJava
> > > > > > > >>> > > > >> > > > > > > > >> > >     ).asJava)
> > > > > > > >>> > > > >> > > > > > > > >> > >   }
> > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > 21. Migration: The KIP says
> > "The
> > > > > > > migration
> > > > > > > >>> > > process
> > > > > > > >>> > > > >> can
> > > > > > > >>> > > > >> > > take
> > > > > > > >>> > > > >> > > > > > place
> > > > > > > >>> > > > >> > > > > > > > >> without
> > > > > > > >>> > > > >> > > > > > > > >> > > an inter-broker protocol
> bump,
> > > as the
> > > > > > > >>> format
> > > > > > > >>> > > stored
> > > > > > > >>> > > > >> in
> > > > > > > >>> > > > >> > > > > > > > >> > > /brokers/topics/[topic] will
> be
> > > > > > > compatible
> > > > > > > >>> with
> > > > > > > >>> > > > older
> > > > > > > >>> > > > >> > > broker
> > > > > > > >>> > > > >> > > > > > > > >> versions."
> > > > > > > >>> > > > >> > > > > > > > >> > > However, since we are bumping
> > up
> > > the
> > > > > > > >>> version of
> > > > > > > >>> > > > >> > > inter-broker
> > > > > > > >>> > > > >> > > > > > > > >> requests, it
> > > > > > > >>> > > > >> > > > > > > > >> > > seems that we need to use IBP
> > for
> > > > > > > >>> migration.
> > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > 22. The KIP says "Remove
> > deleted
> > > > > topics
> > > > > > > >>> from
> > > > > > > >>> > > > >> replicas by
> > > > > > > >>> > > > >> > > > > sending
> > > > > > > >>> > > > >> > > > > > > > >> > > StopReplicaRequest V3 for any
> > > topics
> > > > > > > which
> > > > > > > >>> do
> > > > > > > >>> > not
> > > > > > > >>> > > > >> > contain
> > > > > > > >>> > > > >> > > a
> > > > > > > >>> > > > >> > > > > > topic
> > > > > > > >>> > > > >> > > > > > > > ID,
> > > > > > > >>> > > > >> > > > > > > > >> and
> > > > > > > >>> > > > >> > > > > > > > >> > > V4 for any topics which do
> > > contain a
> > > > > > > topic
> > > > > > > >>> > ID.".
> > > > > > > >>> > > > >> > However,
> > > > > > > >>> > > > >> > > if
> > > > > > > >>> > > > >> > > > > we
> > > > > > > >>> > > > >> > > > > > > use
> > > > > > > >>> > > > >> > > > > > > > >> IBP,
> > > > > > > >>> > > > >> > > > > > > > >> > it
> > > > > > > >>> > > > >> > > > > > > > >> > > seems that the controller
> will
> > > either
> > > > > > > send
> > > > > > > >>> > > > >> > > > StopReplicaRequest
> > > > > > > >>> > > > >> > > > > V3
> > > > > > > >>> > > > >> > > > > > > > >> > > or StopReplicaRequest V4, but
> > > never
> > > > > > mixed
> > > > > > > >>> V3
> > > > > > > >>> > and
> > > > > > > >>> > > V4
> > > > > > > >>> > > > >> for
> > > > > > > >>> > > > >> > > > > > different
> > > > > > > >>> > > > >> > > > > > > > >> topics.
> > > > > > > >>> > > > >> > > > > > > > >> > > Basically, before the IBP
> bump,
> > > V3
> > > > > will
> > > > > > > be
> > > > > > > >>> > used.
> > > > > > > >>> > > > >> After
> > > > > > > >>> > > > >> > the
> > > > > > > >>> > > > >> > > > IBP
> > > > > > > >>> > > > >> > > > > > > bump,
> > > > > > > >>> > > > >> > > > > > > > >> > > topicId will be created and
> V4
> > > will
> > > > > be
> > > > > > > >>> used.
> > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > 23. Given that we depend on
> > IBP,
> > > do
> > > > > we
> > > > > > > >>> still
> > > > > > > >>> > need
> > > > > > > >>> > > > "0
> > > > > > > >>> > > > >> > > > > > UNSPECIFIED"
> > > > > > > >>> > > > >> > > > > > > > >> > > in LeaderAndIsr?
> > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > 24. LeaderAndIsrResponse v5 :
> > It
> > > > > still
> > > > > > > has
> > > > > > > >>> the
> > > > > > > >>> > > > topic
> > > > > > > >>> > > > >> > > field.
> > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > 25. LeaderAndIsrResponse v5,
> > > > > > > >>> > StopReplicaResponse
> > > > > > > >>> > > > v4:
> > > > > > > >>> > > > >> > Could
> > > > > > > >>> > > > >> > > > we
> > > > > > > >>> > > > >> > > > > > use
> > > > > > > >>> > > > >> > > > > > > > this
> > > > > > > >>> > > > >> > > > > > > > >> > > opportunity to organize the
> > > response
> > > > > > in 2
> > > > > > > >>> > levels,
> > > > > > > >>> > > > >> first
> > > > > > > >>> > > > >> > by
> > > > > > > >>> > > > >> > > > > > topic,
> > > > > > > >>> > > > >> > > > > > > > >> then by
> > > > > > > >>> > > > >> > > > > > > > >> > > partition, as most other
> > > > > > > >>> requests/responses?
> > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > 26. FetchRequest v13 : Should
> > > > > > > >>> > > forgotten_topics_data
> > > > > > > >>> > > > >> use
> > > > > > > >>> > > > >> > > > > topicId
> > > > > > > >>> > > > >> > > > > > > too?
> > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > 27. "This file can either be
> > > plain
> > > > > text
> > > > > > > >>> > > (key/value
> > > > > > > >>> > > > >> > pairs)
> > > > > > > >>> > > > >> > > or
> > > > > > > >>> > > > >> > > > > > > JSON."
> > > > > > > >>> > > > >> > > > > > > > >> Have
> > > > > > > >>> > > > >> > > > > > > > >> > we
> > > > > > > >>> > > > >> > > > > > > > >> > > decided which one to use?
> Also,
> > > it
> > > > > > would
> > > > > > > be
> > > > > > > >>> > > helpful
> > > > > > > >>> > > > >> to
> > > > > > > >>> > > > >> > > > provide
> > > > > > > >>> > > > >> > > > > > an
> > > > > > > >>> > > > >> > > > > > > > >> > example.
> > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > 28. Future improvement:
> Another
> > > > > future
> > > > > > > >>> > > improvement
> > > > > > > >>> > > > >> > > > opportunity
> > > > > > > >>> > > > >> > > > > > is
> > > > > > > >>> > > > >> > > > > > > to
> > > > > > > >>> > > > >> > > > > > > > >> use
> > > > > > > >>> > > > >> > > > > > > > >> > > topicId in
> > > > > > > >>> GroupMetadataManager.offsetCommitKey
> > > > > > > >>> > > in
> > > > > > > >>> > > > >> the
> > > > > > > >>> > > > >> > > > > > > offset_commit
> > > > > > > >>> > > > >> > > > > > > > >> > topic.
> > > > > > > >>> > > > >> > > > > > > > >> > > This may save some space.
> > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > Thanks,
> > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > Jun
> > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > On Wed, Sep 23, 2020 at 8:50
> AM
> > > > > Justine
> > > > > > > >>> Olshan
> > > > > > > >>> > <
> > > > > > > >>> > > > >> > > > > > > > jolshan@confluent.io>
> > > > > > > >>> > > > >> > > > > > > > >> > > wrote:
> > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > Hi Tom,
> > > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > Thanks for the comment. I
> > think
> > > > > this
> > > > > > > is a
> > > > > > > >>> > > really
> > > > > > > >>> > > > >> good
> > > > > > > >>> > > > >> > > idea
> > > > > > > >>> > > > >> > > > > and
> > > > > > > >>> > > > >> > > > > > > it
> > > > > > > >>> > > > >> > > > > > > > >> has
> > > > > > > >>> > > > >> > > > > > > > >> > > been
> > > > > > > >>> > > > >> > > > > > > > >> > > > added to the KIP under the
> > > newly
> > > > > > added
> > > > > > > >>> > tooling
> > > > > > > >>> > > > >> > section.
> > > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > Thanks again,
> > > > > > > >>> > > > >> > > > > > > > >> > > > Justine
> > > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > On Wed, Sep 23, 2020 at
> 3:17
> > > AM Tom
> > > > > > > >>> Bentley <
> > > > > > > >>> > > > >> > > > > > > tbentley@redhat.com>
> > > > > > > >>> > > > >> > > > > > > > >> > wrote:
> > > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > Hi Justine,
> > > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > I know you started the
> vote
> > > > > thread,
> > > > > > > >>> but on
> > > > > > > >>> > > > >> > re-reading
> > > > > > > >>> > > > >> > > > the
> > > > > > > >>> > > > >> > > > > > KIP
> > > > > > > >>> > > > >> > > > > > > I
> > > > > > > >>> > > > >> > > > > > > > >> > noticed
> > > > > > > >>> > > > >> > > > > > > > >> > > > > that although the topic
> id
> > is
> > > > > > > included
> > > > > > > >>> in
> > > > > > > >>> > the
> > > > > > > >>> > > > >> > > > > > MetadataResponse
> > > > > > > >>> > > > >> > > > > > > > >> it's
> > > > > > > >>> > > > >> > > > > > > > >> > not
> > > > > > > >>> > > > >> > > > > > > > >> > > > > surfaced in the output
> from
> > > > > > > >>> > `kafka-topics.sh
> > > > > > > >>> > > > >> > > > --describe`.
> > > > > > > >>> > > > >> > > > > > > Maybe
> > > > > > > >>> > > > >> > > > > > > > >> that
> > > > > > > >>> > > > >> > > > > > > > >> > > was
> > > > > > > >>> > > > >> > > > > > > > >> > > > > intentional because ids
> are
> > > > > > > >>> intentionally
> > > > > > > >>> > not
> > > > > > > >>> > > > >> really
> > > > > > > >>> > > > >> > > > > > something
> > > > > > > >>> > > > >> > > > > > > > the
> > > > > > > >>> > > > >> > > > > > > > >> > user
> > > > > > > >>> > > > >> > > > > > > > >> > > > > should care deeply about,
> > > but it
> > > > > > > would
> > > > > > > >>> also
> > > > > > > >>> > > > make
> > > > > > > >>> > > > >> > life
> > > > > > > >>> > > > >> > > > > harder
> > > > > > > >>> > > > >> > > > > > > for
> > > > > > > >>> > > > >> > > > > > > > >> > anyone
> > > > > > > >>> > > > >> > > > > > > > >> > > > > debugging Kafka and this
> > > would
> > > > > > likely
> > > > > > > >>> get
> > > > > > > >>> > > worse
> > > > > > > >>> > > > >> the
> > > > > > > >>> > > > >> > > more
> > > > > > > >>> > > > >> > > > > > topic
> > > > > > > >>> > > > >> > > > > > > > ids
> > > > > > > >>> > > > >> > > > > > > > >> > got
> > > > > > > >>> > > > >> > > > > > > > >> > > > > rolled out across the
> > > protocols,
> > > > > > > >>> clients
> > > > > > > >>> > etc.
> > > > > > > >>> > > > It
> > > > > > > >>> > > > >> > seems
> > > > > > > >>> > > > >> > > > > > likely
> > > > > > > >>> > > > >> > > > > > > > that
> > > > > > > >>> > > > >> > > > > > > > >> > > > > `kafka-topics.sh` will
> > > eventually
> > > > > > > need
> > > > > > > >>> the
> > > > > > > >>> > > > >> ability
> > > > > > > >>> > > > >> > to
> > > > > > > >>> > > > >> > > > show
> > > > > > > >>> > > > >> > > > > > the
> > > > > > > >>> > > > >> > > > > > > > id
> > > > > > > >>> > > > >> > > > > > > > >> of
> > > > > > > >>> > > > >> > > > > > > > >> > a
> > > > > > > >>> > > > >> > > > > > > > >> > > > > topic and perhaps find a
> > > topic
> > > > > name
> > > > > > > >>> given
> > > > > > > >>> > an
> > > > > > > >>> > > > id.
> > > > > > > >>> > > > >> Is
> > > > > > > >>> > > > >> > > > there
> > > > > > > >>> > > > >> > > > > > any
> > > > > > > >>> > > > >> > > > > > > > >> reason
> > > > > > > >>> > > > >> > > > > > > > >> > > not
> > > > > > > >>> > > > >> > > > > > > > >> > > > to
> > > > > > > >>> > > > >> > > > > > > > >> > > > > implement that in this
> KIP?
> > > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > Many thanks,
> > > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > Tom
> > > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > On Mon, Sep 21, 2020 at
> > 9:54
> > > PM
> > > > > > > Justine
> > > > > > > >>> > > Olshan
> > > > > > > >>> > > > <
> > > > > > > >>> > > > >> > > > > > > > >> jolshan@confluent.io
> > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > wrote:
> > > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > Hi all,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > After thinking about
> it,
> > > I've
> > > > > > > >>> decided to
> > > > > > > >>> > > > remove
> > > > > > > >>> > > > >> > the
> > > > > > > >>> > > > >> > > > > topic
> > > > > > > >>> > > > >> > > > > > > name
> > > > > > > >>> > > > >> > > > > > > > >> from
> > > > > > > >>> > > > >> > > > > > > > >> > > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > Fetch Request and
> > Response
> > > > > after
> > > > > > > all.
> > > > > > > >>> > Since
> > > > > > > >>> > > > >> there
> > > > > > > >>> > > > >> > > are
> > > > > > > >>> > > > >> > > > so
> > > > > > > >>> > > > >> > > > > > > many
> > > > > > > >>> > > > >> > > > > > > > of
> > > > > > > >>> > > > >> > > > > > > > >> > > these
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > requests per second, it
> > is
> > > > > worth
> > > > > > > >>> removing
> > > > > > > >>> > > the
> > > > > > > >>> > > > >> > extra
> > > > > > > >>> > > > >> > > > > > > > information.
> > > > > > > >>> > > > >> > > > > > > > >> > I've
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > updated the KIP to
> > reflect
> > > this
> > > > > > > >>> change.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > Please let me know if
> > > there is
> > > > > > > >>> anything
> > > > > > > >>> > > else
> > > > > > > >>> > > > we
> > > > > > > >>> > > > >> > > should
> > > > > > > >>> > > > >> > > > > > > discuss
> > > > > > > >>> > > > >> > > > > > > > >> > before
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > voting.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > Thank you,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > Justine
> > > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > On Fri, Sep 18, 2020 at
> > > 9:46 AM
> > > > > > > >>> Justine
> > > > > > > >>> > > > Olshan
> > > > > > > >>> > > > >> <
> > > > > > > >>> > > > >> > > > > > > > >> > jolshan@confluent.io
> > > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > wrote:
> > > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > > Hi Jun,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > > I see what you are
> > > saying.
> > > > > For
> > > > > > > now
> > > > > > > >>> we
> > > > > > > >>> > can
> > > > > > > >>> > > > >> remove
> > > > > > > >>> > > > >> > > the
> > > > > > > >>> > > > >> > > > > > extra
> > > > > > > >>> > > > >> > > > > > > > >> > > > information.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > > I'll leave the option
> > to
> > > add
> > > > > > more
> > > > > > > >>> > fields
> > > > > > > >>> > > to
> > > > > > > >>> > > > >> the
> > > > > > > >>> > > > >> > > file
> > > > > > > >>> > > > >> > > > > in
> > > > > > > >>> > > > >> > > > > > > the
> > > > > > > >>> > > > >> > > > > > > > >> > future.
> > > > > > > >>> > > > >> > > > > > > > >> > > > The
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > KIP
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > > has been updated to
> > > reflect
> > > > > > this
> > > > > > > >>> > change.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > > Thanks,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > > Justine
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > > On Fri, Sep 18, 2020
> at
> > > 8:46
> > > > > AM
> > > > > > > Jun
> > > > > > > >>> > Rao <
> > > > > > > >>> > > > >> > > > > > jun@confluent.io
> > > > > > > >>> > > > >> > > > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > wrote:
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Hi, Justine,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Thanks for the
> reply.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> 13. If the log
> > > directory is
> > > > > > the
> > > > > > > >>> source
> > > > > > > >>> > > of
> > > > > > > >>> > > > >> > truth,
> > > > > > > >>> > > > >> > > it
> > > > > > > >>> > > > >> > > > > > means
> > > > > > > >>> > > > >> > > > > > > > >> that
> > > > > > > >>> > > > >> > > > > > > > >> > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> redundant info in
> the
> > > > > metadata
> > > > > > > >>> file
> > > > > > > >>> > will
> > > > > > > >>> > > > be
> > > > > > > >>> > > > >> > > > ignored.
> > > > > > > >>> > > > >> > > > > > Then
> > > > > > > >>> > > > >> > > > > > > > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > question
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > is
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> why do we need to
> put
> > > the
> > > > > > > >>> redundant
> > > > > > > >>> > info
> > > > > > > >>> > > > in
> > > > > > > >>> > > > >> the
> > > > > > > >>> > > > >> > > > > > metadata
> > > > > > > >>> > > > >> > > > > > > > file
> > > > > > > >>> > > > >> > > > > > > > >> > now?
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Thanks,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Jun
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> On Thu, Sep 17, 2020
> > at
> > > 5:07
> > > > > > PM
> > > > > > > >>> > Justine
> > > > > > > >>> > > > >> Olshan
> > > > > > > >>> > > > >> > <
> > > > > > > >>> > > > >> > > > > > > > >> > > > jolshan@confluent.io>
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> wrote:
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Hi Jun,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Thanks for the
> quick
> > > > > > response!
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > 12. I've decided
> to
> > > bump
> > > > > up
> > > > > > > the
> > > > > > > >>> > > versions
> > > > > > > >>> > > > >> on
> > > > > > > >>> > > > >> > the
> > > > > > > >>> > > > >> > > > > > > requests
> > > > > > > >>> > > > >> > > > > > > > >> and
> > > > > > > >>> > > > >> > > > > > > > >> > > > updated
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > KIP. I think it's
> > > good we
> > > > > > > >>> thoroughly
> > > > > > > >>> > > > >> > discussed
> > > > > > > >>> > > > >> > > > the
> > > > > > > >>> > > > >> > > > > > > > options
> > > > > > > >>> > > > >> > > > > > > > >> > here,
> > > > > > > >>> > > > >> > > > > > > > >> > > > so
> > > > > > > >>> > > > >> > > > > > > > >> > > > > we
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> know
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > we made a good
> > > choice. :)
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > 13. This is an
> > > interesting
> > > > > > > >>> > situation.
> > > > > > > >>> > > I
> > > > > > > >>> > > > >> think
> > > > > > > >>> > > > >> > > if
> > > > > > > >>> > > > >> > > > > this
> > > > > > > >>> > > > >> > > > > > > > does
> > > > > > > >>> > > > >> > > > > > > > >> > occur
> > > > > > > >>> > > > >> > > > > > > > >> > > > we
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> should
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > give a warning. I
> > > agree
> > > > > that
> > > > > > > >>> it's
> > > > > > > >>> > hard
> > > > > > > >>> > > > to
> > > > > > > >>> > > > >> > know
> > > > > > > >>> > > > >> > > > the
> > > > > > > >>> > > > >> > > > > > > source
> > > > > > > >>> > > > >> > > > > > > > >> of
> > > > > > > >>> > > > >> > > > > > > > >> > > truth
> > > > > > > >>> > > > >> > > > > > > > >> > > > > for
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> sure
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > since the
> directory
> > > or the
> > > > > > > file
> > > > > > > >>> > could
> > > > > > > >>> > > be
> > > > > > > >>> > > > >> > > manually
> > > > > > > >>> > > > >> > > > > > > > >> modified. I
> > > > > > > >>> > > > >> > > > > > > > >> > > > guess
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > directory could be
> > > used as
> > > > > > the
> > > > > > > >>> > source
> > > > > > > >>> > > of
> > > > > > > >>> > > > >> > truth.
> > > > > > > >>> > > > >> > > > To
> > > > > > > >>> > > > >> > > > > be
> > > > > > > >>> > > > >> > > > > > > > >> honest,
> > > > > > > >>> > > > >> > > > > > > > >> > > I'm
> > > > > > > >>> > > > >> > > > > > > > >> > > > > not
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > really sure what
> > > happens
> > > > > in
> > > > > > > >>> kafka
> > > > > > > >>> > when
> > > > > > > >>> > > > the
> > > > > > > >>> > > > >> > log
> > > > > > > >>> > > > >> > > > > > > directory
> > > > > > > >>> > > > >> > > > > > > > is
> > > > > > > >>> > > > >> > > > > > > > >> > > > renamed
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > manually in such a
> > > way.
> > > > > I'm
> > > > > > > also
> > > > > > > >>> > > > >> wondering if
> > > > > > > >>> > > > >> > > the
> > > > > > > >>> > > > >> > > > > > > > >> situation is
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> recoverable
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > in this scenario.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Thanks,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Justine
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > On Thu, Sep 17,
> 2020
> > > at
> > > > > 4:28
> > > > > > > PM
> > > > > > > >>> Jun
> > > > > > > >>> > > Rao
> > > > > > > >>> > > > <
> > > > > > > >>> > > > >> > > > > > > > jun@confluent.io>
> > > > > > > >>> > > > >> > > > > > > > >> > > wrote:
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Hi, Justine,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Thanks for the
> > > reply.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > 12. I don't
> have a
> > > > > strong
> > > > > > > >>> > preference
> > > > > > > >>> > > > >> > either.
> > > > > > > >>> > > > >> > > > > > However,
> > > > > > > >>> > > > >> > > > > > > > if
> > > > > > > >>> > > > >> > > > > > > > >> we
> > > > > > > >>> > > > >> > > > > > > > >> > > need
> > > > > > > >>> > > > >> > > > > > > > >> > > > > IBP
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > anyway, maybe
> it's
> > > > > easier
> > > > > > to
> > > > > > > >>> just
> > > > > > > >>> > > bump
> > > > > > > >>> > > > >> up
> > > > > > > >>> > > > >> > the
> > > > > > > >>> > > > >> > > > > > version
> > > > > > > >>> > > > >> > > > > > > > for
> > > > > > > >>> > > > >> > > > > > > > >> > all
> > > > > > > >>> > > > >> > > > > > > > >> > > > > inter
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > broker
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > requests and add
> > the
> > > > > topic
> > > > > > > id
> > > > > > > >>> > field
> > > > > > > >>> > > > as a
> > > > > > > >>> > > > >> > > > regular
> > > > > > > >>> > > > >> > > > > > > > field. A
> > > > > > > >>> > > > >> > > > > > > > >> > > > regular
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> field
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > is
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > a bit more
> concise
> > > in
> > > > > wire
> > > > > > > >>> > transfer
> > > > > > > >>> > > > >> than a
> > > > > > > >>> > > > >> > > > > flexible
> > > > > > > >>> > > > >> > > > > > > > >> field.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > 13. The
> confusion
> > > that I
> > > > > > was
> > > > > > > >>> > > referring
> > > > > > > >>> > > > >> to
> > > > > > > >>> > > > >> > is
> > > > > > > >>> > > > >> > > > > > between
> > > > > > > >>> > > > >> > > > > > > > the
> > > > > > > >>> > > > >> > > > > > > > >> > topic
> > > > > > > >>> > > > >> > > > > > > > >> > > > > name
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > partition number
> > > between
> > > > > > the
> > > > > > > >>> log
> > > > > > > >>> > dir
> > > > > > > >>> > > > and
> > > > > > > >>> > > > >> > the
> > > > > > > >>> > > > >> > > > > > metadata
> > > > > > > >>> > > > >> > > > > > > > >> file.
> > > > > > > >>> > > > >> > > > > > > > >> > > For
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> example,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > if
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > the log dir is
> > > topicA-1
> > > > > > and
> > > > > > > >>> the
> > > > > > > >>> > > > metadata
> > > > > > > >>> > > > >> > file
> > > > > > > >>> > > > >> > > > in
> > > > > > > >>> > > > >> > > > > it
> > > > > > > >>> > > > >> > > > > > > has
> > > > > > > >>> > > > >> > > > > > > > >> > topicB
> > > > > > > >>> > > > >> > > > > > > > >> > > > and
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > partition 0 (say
> > > due to
> > > > > a
> > > > > > > bug
> > > > > > > >>> or
> > > > > > > >>> > > > manual
> > > > > > > >>> > > > >> > > > > > > modification),
> > > > > > > >>> > > > >> > > > > > > > >> which
> > > > > > > >>> > > > >> > > > > > > > >> > > one
> > > > > > > >>> > > > >> > > > > > > > >> > > > > do
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > we
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > use
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > as the source of
> > > truth?
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Jun
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > On Thu, Sep 17,
> > > 2020 at
> > > > > > 3:43
> > > > > > > >>> PM
> > > > > > > >>> > > > Justine
> > > > > > > >>> > > > >> > > Olshan
> > > > > > > >>> > > > >> > > > <
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > jolshan@confluent.io>
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > wrote:
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > Hi Jun,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > Thanks for the
> > > > > comments.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > 12. I bumped
> the
> > > > > > > >>> > > LeaderAndIsrRequest
> > > > > > > >>> > > > >> > > because
> > > > > > > >>> > > > >> > > > I
> > > > > > > >>> > > > >> > > > > > > > removed
> > > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > > >>> > > > >> > > > > > > > >> > > > topic
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> name
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > field
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > in the
> response.
> > > It
> > > > > may
> > > > > > be
> > > > > > > >>> > > possible
> > > > > > > >>> > > > to
> > > > > > > >>> > > > >> > > avoid
> > > > > > > >>> > > > >> > > > > > > bumping
> > > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > > >>> > > > >> > > > > > > > >> > > > version
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > without
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > that change,
> but
> > > I may
> > > > > > be
> > > > > > > >>> > missing
> > > > > > > >>> > > > >> > > something.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > I believe
> > > StopReplica
> > > > > is
> > > > > > > >>> > actually
> > > > > > > >>> > > on
> > > > > > > >>> > > > >> > > version
> > > > > > > >>> > > > >> > > > 3
> > > > > > > >>> > > > >> > > > > > now,
> > > > > > > >>> > > > >> > > > > > > > but
> > > > > > > >>> > > > >> > > > > > > > >> > > > because
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > version 2
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > is flexible, I
> > > kept
> > > > > that
> > > > > > > >>> listed
> > > > > > > >>> > as
> > > > > > > >>> > > > >> > version
> > > > > > > >>> > > > >> > > 2
> > > > > > > >>> > > > >> > > > on
> > > > > > > >>> > > > >> > > > > > the
> > > > > > > >>> > > > >> > > > > > > > KIP
> > > > > > > >>> > > > >> > > > > > > > >> > > page.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> However,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > you
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > may be right
> in
> > > that
> > > > > we
> > > > > > > may
> > > > > > > >>> need
> > > > > > > >>> > > to
> > > > > > > >>> > > > >> bump
> > > > > > > >>> > > > >> > > the
> > > > > > > >>> > > > >> > > > > > > version
> > > > > > > >>> > > > >> > > > > > > > on
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > StopReplica
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> to
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > deal
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > with deletion
> > > > > > differently
> > > > > > > as
> > > > > > > >>> > > > mentioned
> > > > > > > >>> > > > >> > > > above. I
> > > > > > > >>> > > > >> > > > > > > don't
> > > > > > > >>> > > > >> > > > > > > > >> know
> > > > > > > >>> > > > >> > > > > > > > >> > > if
> > > > > > > >>> > > > >> > > > > > > > >> > > > I
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> have a
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > big
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > preference
> over
> > > used
> > > > > > > tagged
> > > > > > > >>> > fields
> > > > > > > >>> > > > or
> > > > > > > >>> > > > >> > not.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > 13. I was
> > thinking
> > > > > that
> > > > > > in
> > > > > > > >>> the
> > > > > > > >>> > > case
> > > > > > > >>> > > > >> where
> > > > > > > >>> > > > >> > > the
> > > > > > > >>> > > > >> > > > > > file
> > > > > > > >>> > > > >> > > > > > > > and
> > > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > request
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > topic
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > ids don't
> match,
> > > it
> > > > > > means
> > > > > > > >>> that
> > > > > > > >>> > the
> > > > > > > >>> > > > >> > broker's
> > > > > > > >>> > > > >> > > > > > > topic/the
> > > > > > > >>> > > > >> > > > > > > > >> one
> > > > > > > >>> > > > >> > > > > > > > >> > in
> > > > > > > >>> > > > >> > > > > > > > >> > > > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> file
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > has
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > been deleted.
> In
> > > that
> > > > > > > case,
> > > > > > > >>> we
> > > > > > > >>> > > would
> > > > > > > >>> > > > >> need
> > > > > > > >>> > > > >> > > to
> > > > > > > >>> > > > >> > > > > > delete
> > > > > > > >>> > > > >> > > > > > > > the
> > > > > > > >>> > > > >> > > > > > > > >> > old
> > > > > > > >>> > > > >> > > > > > > > >> > > > > topic
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > start
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > receiving the
> > new
> > > > > > version.
> > > > > > > >>> If
> > > > > > > >>> > the
> > > > > > > >>> > > > >> topic
> > > > > > > >>> > > > >> > > name
> > > > > > > >>> > > > >> > > > > were
> > > > > > > >>> > > > >> > > > > > > to
> > > > > > > >>> > > > >> > > > > > > > >> > change,
> > > > > > > >>> > > > >> > > > > > > > >> > > > but
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > ids
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > still matched,
> > the
> > > > > file
> > > > > > > >>> would
> > > > > > > >>> > also
> > > > > > > >>> > > > >> need
> > > > > > > >>> > > > >> > to
> > > > > > > >>> > > > >> > > > > > update.
> > > > > > > >>> > > > >> > > > > > > > Am I
> > > > > > > >>> > > > >> > > > > > > > >> > > > missing
> > > > > > > >>> > > > >> > > > > > > > >> > > > > a
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> case
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > where the file
> > > would
> > > > > be
> > > > > > > >>> correct
> > > > > > > >>> > > and
> > > > > > > >>> > > > >> not
> > > > > > > >>> > > > >> > the
> > > > > > > >>> > > > >> > > > > > > request?
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > Thanks,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > Justine
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > On Thu, Sep
> 17,
> > > 2020
> > > > > at
> > > > > > > >>> 3:18 PM
> > > > > > > >>> > > Jun
> > > > > > > >>> > > > >> Rao <
> > > > > > > >>> > > > >> > > > > > > > >> jun@confluent.io
> > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > wrote:
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Hi, Justine,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Thanks for
> the
> > > > > reply.
> > > > > > A
> > > > > > > >>> couple
> > > > > > > >>> > > of
> > > > > > > >>> > > > >> more
> > > > > > > >>> > > > >> > > > > comments
> > > > > > > >>> > > > >> > > > > > > > >> below.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > 12.
> ListOffset
> > > and
> > > > > > > >>> > > OffsetForLeader
> > > > > > > >>> > > > >> > > > currently
> > > > > > > >>> > > > >> > > > > > > don't
> > > > > > > >>> > > > >> > > > > > > > >> > support
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> flexible
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > fields.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > So, we have
> to
> > > bump
> > > > > up
> > > > > > > the
> > > > > > > >>> > > version
> > > > > > > >>> > > > >> > number
> > > > > > > >>> > > > >> > > > and
> > > > > > > >>> > > > >> > > > > > use
> > > > > > > >>> > > > >> > > > > > > > >> IBP at
> > > > > > > >>> > > > >> > > > > > > > >> > > > least
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > for
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > these
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > two
> requests.
> > > Note
> > > > > > that
> > > > > > > it
> > > > > > > >>> > seems
> > > > > > > >>> > > > >> 2.7.0
> > > > > > > >>> > > > >> > > will
> > > > > > > >>> > > > >> > > > > > > require
> > > > > > > >>> > > > >> > > > > > > > >> IBP
> > > > > > > >>> > > > >> > > > > > > > >> > > > anyway
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > because
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > of
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > changes in
> > > > > > KAFKA-10435.
> > > > > > > >>> Also,
> > > > > > > >>> > it
> > > > > > > >>> > > > >> seems
> > > > > > > >>> > > > >> > > that
> > > > > > > >>> > > > >> > > > > the
> > > > > > > >>> > > > >> > > > > > > > >> version
> > > > > > > >>> > > > >> > > > > > > > >> > > for
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > LeaderAndIsrRequest
> > > > > > and
> > > > > > > >>> > > > StopReplica
> > > > > > > >>> > > > >> are
> > > > > > > >>> > > > >> > > > > bumped
> > > > > > > >>> > > > >> > > > > > > even
> > > > > > > >>> > > > >> > > > > > > > >> > though
> > > > > > > >>> > > > >> > > > > > > > >> > > > we
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > only
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > added
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > a
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > tagged
> field.
> > > But
> > > > > > since
> > > > > > > >>> IBP is
> > > > > > > >>> > > > >> needed
> > > > > > > >>> > > > >> > > > anyway,
> > > > > > > >>> > > > >> > > > > > we
> > > > > > > >>> > > > >> > > > > > > > may
> > > > > > > >>> > > > >> > > > > > > > >> > want
> > > > > > > >>> > > > >> > > > > > > > >> > > to
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> revisit
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > overall
> tagged
> > > field
> > > > > > > >>> choice.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > 13. The only
> > > > > downside
> > > > > > is
> > > > > > > >>> the
> > > > > > > >>> > > > >> potential
> > > > > > > >>> > > > >> > > > > > confusion
> > > > > > > >>> > > > >> > > > > > > on
> > > > > > > >>> > > > >> > > > > > > > >> > which
> > > > > > > >>> > > > >> > > > > > > > >> > > > one
> > > > > > > >>> > > > >> > > > > > > > >> > > > > is
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > source
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > of truth if
> > they
> > > > > don't
> > > > > > > >>> match.
> > > > > > > >>> > > > >> Another
> > > > > > > >>> > > > >> > > > option
> > > > > > > >>> > > > >> > > > > is
> > > > > > > >>> > > > >> > > > > > > to
> > > > > > > >>> > > > >> > > > > > > > >> > include
> > > > > > > >>> > > > >> > > > > > > > >> > > > > those
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > fields
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > in
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the metadata
> > > file
> > > > > when
> > > > > > > we
> > > > > > > >>> > > actually
> > > > > > > >>> > > > >> > change
> > > > > > > >>> > > > >> > > > the
> > > > > > > >>> > > > >> > > > > > > > >> directory
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > structure.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Thanks,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Jun
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > On Thu, Sep
> > 17,
> > > 2020
> > > > > > at
> > > > > > > >>> 2:01
> > > > > > > >>> > PM
> > > > > > > >>> > > > >> Justine
> > > > > > > >>> > > > >> > > > > Olshan
> > > > > > > >>> > > > >> > > > > > <
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> jolshan@confluent.io
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > wrote:
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > Hello all,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I've
> thought
> > > some
> > > > > > more
> > > > > > > >>> about
> > > > > > > >>> > > > >> removing
> > > > > > > >>> > > > >> > > the
> > > > > > > >>> > > > >> > > > > > topic
> > > > > > > >>> > > > >> > > > > > > > >> name
> > > > > > > >>> > > > >> > > > > > > > >> > > field
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > from
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > some
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > of
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > requests.
> On
> > > > > closer
> > > > > > > >>> > inspection
> > > > > > > >>> > > > of
> > > > > > > >>> > > > >> the
> > > > > > > >>> > > > >> > > > > > > > >> > > requests/responses,
> > > > > > > >>> > > > >> > > > > > > > >> > > > it
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> seems
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > internal
> > > changes
> > > > > > would
> > > > > > > >>> be
> > > > > > > >>> > much
> > > > > > > >>> > > > >> larger
> > > > > > > >>> > > > >> > > > than
> > > > > > > >>> > > > >> > > > > I
> > > > > > > >>> > > > >> > > > > > > > >> expected.
> > > > > > > >>> > > > >> > > > > > > > >> > > > Some
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > protocols
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > involve
> > > clients,
> > > > > so
> > > > > > > they
> > > > > > > >>> > would
> > > > > > > >>> > > > >> > require
> > > > > > > >>> > > > >> > > > > > changes
> > > > > > > >>> > > > >> > > > > > > > too.
> > > > > > > >>> > > > >> > > > > > > > >> > I'm
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > thinking
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > that
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > for
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > now,
> > removing
> > > the
> > > > > > > topic
> > > > > > > >>> name
> > > > > > > >>> > > > from
> > > > > > > >>> > > > >> > these
> > > > > > > >>> > > > >> > > > > > > requests
> > > > > > > >>> > > > >> > > > > > > > >> and
> > > > > > > >>> > > > >> > > > > > > > >> > > > > responses
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> are
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > out
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > scope.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I have
> > > decided to
> > > > > > just
> > > > > > > >>> keep
> > > > > > > >>> > > the
> > > > > > > >>> > > > >> > change
> > > > > > > >>> > > > >> > > > > > > > >> > > > LeaderAndIsrResponse
> > > > > > > >>> > > > >> > > > > > > > >> > > > > to
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > remove
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > topic
> name,
> > > and
> > > > > have
> > > > > > > >>> updated
> > > > > > > >>> > > the
> > > > > > > >>> > > > >> KIP
> > > > > > > >>> > > > >> > to
> > > > > > > >>> > > > >> > > > > > reflect
> > > > > > > >>> > > > >> > > > > > > > >> this
> > > > > > > >>> > > > >> > > > > > > > >> > > > > change. I
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> have
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > also
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > mentioned
> > the
> > > > > other
> > > > > > > >>> requests
> > > > > > > >>> > > and
> > > > > > > >>> > > > >> > > > responses
> > > > > > > >>> > > > >> > > > > in
> > > > > > > >>> > > > >> > > > > > > > >> future
> > > > > > > >>> > > > >> > > > > > > > >> > > work.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I'm hoping
> > to
> > > > > start
> > > > > > > the
> > > > > > > >>> > voting
> > > > > > > >>> > > > >> > process
> > > > > > > >>> > > > >> > > > > soon,
> > > > > > > >>> > > > >> > > > > > so
> > > > > > > >>> > > > >> > > > > > > > >> let me
> > > > > > > >>> > > > >> > > > > > > > >> > > > know
> > > > > > > >>> > > > >> > > > > > > > >> > > > > if
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > there
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > is
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > anything
> > else
> > > we
> > > > > > > should
> > > > > > > >>> > > discuss.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > Thank you,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > Justine
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > On Tue,
> Sep
> > > 15,
> > > > > 2020
> > > > > > > at
> > > > > > > >>> 3:57
> > > > > > > >>> > > PM
> > > > > > > >>> > > > >> > Justine
> > > > > > > >>> > > > >> > > > > > Olshan
> > > > > > > >>> > > > >> > > > > > > <
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > jolshan@confluent.io
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > wrote:
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > Hello
> > again,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > To
> follow
> > > up on
> > > > > > some
> > > > > > > >>> of
> > > > > > > >>> > the
> > > > > > > >>> > > > >> other
> > > > > > > >>> > > > >> > > > > comments:
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 10/11)
> We
> > > can
> > > > > > remove
> > > > > > > >>> the
> > > > > > > >>> > > topic
> > > > > > > >>> > > > >> name
> > > > > > > >>> > > > >> > > > from
> > > > > > > >>> > > > >> > > > > > > these
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> requests/responses,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > and
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > that
> means
> > > we
> > > > > will
> > > > > > > >>> just
> > > > > > > >>> > have
> > > > > > > >>> > > > to
> > > > > > > >>> > > > >> > make
> > > > > > > >>> > > > >> > > a
> > > > > > > >>> > > > >> > > > > few
> > > > > > > >>> > > > >> > > > > > > > >> internal
> > > > > > > >>> > > > >> > > > > > > > >> > > > > changes
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > to
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > make
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> partitions
> > > > > > > accessible
> > > > > > > >>> by
> > > > > > > >>> > > topic
> > > > > > > >>> > > > >> id
> > > > > > > >>> > > > >> > and
> > > > > > > >>> > > > >> > > > > > > > partition.
> > > > > > > >>> > > > >> > > > > > > > >> I
> > > > > > > >>> > > > >> > > > > > > > >> > can
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > update
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > KIP
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > remove
> > them
> > > > > unless
> > > > > > > >>> anyone
> > > > > > > >>> > > > thinks
> > > > > > > >>> > > > >> > they
> > > > > > > >>> > > > >> > > > > > should
> > > > > > > >>> > > > >> > > > > > > > >> stay.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 12)
> > > Addressed in
> > > > > > the
> > > > > > > >>> > > previous
> > > > > > > >>> > > > >> > email.
> > > > > > > >>> > > > >> > > > I've
> > > > > > > >>> > > > >> > > > > > > > updated
> > > > > > > >>> > > > >> > > > > > > > >> > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > KIP
> > > > > > > >>> > > > >> > > > > > > > >> > > > > to
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > include
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > tagged
> > > fields
> > > > > for
> > > > > > > the
> > > > > > > >>> > > requests
> > > > > > > >>> > > > >> and
> > > > > > > >>> > > > >> > > > > > responses.
> > > > > > > >>> > > > >> > > > > > > > >> (More
> > > > > > > >>> > > > >> > > > > > > > >> > on
> > > > > > > >>> > > > >> > > > > > > > >> > > > > that
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > below)
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 13) I
> > think
> > > part
> > > > > > of
> > > > > > > >>> the
> > > > > > > >>> > idea
> > > > > > > >>> > > > for
> > > > > > > >>> > > > >> > > > > including
> > > > > > > >>> > > > >> > > > > > > this
> > > > > > > >>> > > > >> > > > > > > > >> > > > > information
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> is to
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > prepare
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > for
> future
> > > > > > changes.
> > > > > > > >>> > Perhaps
> > > > > > > >>> > > > the
> > > > > > > >>> > > > >> > > > directory
> > > > > > > >>> > > > >> > > > > > > > >> structure
> > > > > > > >>> > > > >> > > > > > > > >> > > > might
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> change
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > from
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > >>> topicName_partitionNumber
> > > > > > > >>> > to
> > > > > > > >>> > > > >> > > something
> > > > > > > >>> > > > >> > > > > like
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > topicID_partitionNumber.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Then
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > it would
> > be
> > > > > useful
> > > > > > > to
> > > > > > > >>> have
> > > > > > > >>> > > the
> > > > > > > >>> > > > >> > topic
> > > > > > > >>> > > > >> > > > name
> > > > > > > >>> > > > >> > > > > > in
> > > > > > > >>> > > > >> > > > > > > > the
> > > > > > > >>> > > > >> > > > > > > > >> > file
> > > > > > > >>> > > > >> > > > > > > > >> > > > > since
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > it
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > would
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > not
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > be
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > in the
> > > directory
> > > > > > > >>> > structure.
> > > > > > > >>> > > > >> > > Supporting
> > > > > > > >>> > > > >> > > > > > topic
> > > > > > > >>> > > > >> > > > > > > > >> renames
> > > > > > > >>> > > > >> > > > > > > > >> > > > might
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > be
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > easier
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > if
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > other
> > > fields are
> > > > > > > >>> included.
> > > > > > > >>> > > > Would
> > > > > > > >>> > > > >> > > there
> > > > > > > >>> > > > >> > > > be
> > > > > > > >>> > > > >> > > > > > any
> > > > > > > >>> > > > >> > > > > > > > >> > > downsides
> > > > > > > >>> > > > >> > > > > > > > >> > > > to
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > including
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > this
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > information?
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 14)
> Yes,
> > we
> > > > > would
> > > > > > > >>> need to
> > > > > > > >>> > > > copy
> > > > > > > >>> > > > >> the
> > > > > > > >>> > > > >> > > > > > partition
> > > > > > > >>> > > > >> > > > > > > > >> > metadata
> > > > > > > >>> > > > >> > > > > > > > >> > > > > file
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > in
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > this
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > process.
> > > I've
> > > > > > > updated
> > > > > > > >>> the
> > > > > > > >>> > > KIP
> > > > > > > >>> > > > to
> > > > > > > >>> > > > >> > > > include
> > > > > > > >>> > > > >> > > > > > > this.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 15) I
> > > believe
> > > > > > Lucas
> > > > > > > >>> meant
> > > > > > > >>> > v1
> > > > > > > >>> > > > >> and v2
> > > > > > > >>> > > > >> > > > here.
> > > > > > > >>> > > > >> > > > > > He
> > > > > > > >>> > > > >> > > > > > > > was
> > > > > > > >>> > > > >> > > > > > > > >> > > > referring
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > to
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> how
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > requests
> > > would
> > > > > > fall
> > > > > > > >>> under
> > > > > > > >>> > > > >> different
> > > > > > > >>> > > > >> > > IBP
> > > > > > > >>> > > > >> > > > > and
> > > > > > > >>> > > > >> > > > > > > > meant
> > > > > > > >>> > > > >> > > > > > > > >> > that
> > > > > > > >>> > > > >> > > > > > > > >> > > > > older
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > brokers
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > would
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > have to
> > use
> > > the
> > > > > > > older
> > > > > > > >>> > > version
> > > > > > > >>> > > > of
> > > > > > > >>> > > > >> > the
> > > > > > > >>> > > > >> > > > > > request
> > > > > > > >>> > > > >> > > > > > > > and
> > > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > existing
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > topic
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > deletion
> > > > > process.
> > > > > > At
> > > > > > > >>> > first,
> > > > > > > >>> > > it
> > > > > > > >>> > > > >> > seemed
> > > > > > > >>> > > > >> > > > > like
> > > > > > > >>> > > > >> > > > > > > > tagged
> > > > > > > >>> > > > >> > > > > > > > >> > > fields
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > would
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > resolve
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > the IBP
> > > issue.
> > > > > > > >>> However, we
> > > > > > > >>> > > may
> > > > > > > >>> > > > >> need
> > > > > > > >>> > > > >> > > IBP
> > > > > > > >>> > > > >> > > > > for
> > > > > > > >>> > > > >> > > > > > > > this
> > > > > > > >>> > > > >> > > > > > > > >> > > request
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > after
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > all
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > since
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > the
> > > controller
> > > > > > > >>> handles the
> > > > > > > >>> > > > topic
> > > > > > > >>> > > > >> > > > deletion
> > > > > > > >>> > > > >> > > > > > > > >> > differently
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> depending
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > on
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > IBP
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > version.
> > In
> > > an
> > > > > > older
> > > > > > > >>> > > version,
> > > > > > > >>> > > > we
> > > > > > > >>> > > > >> > > can't
> > > > > > > >>> > > > >> > > > > just
> > > > > > > >>> > > > >> > > > > > > > send
> > > > > > > >>> > > > >> > > > > > > > >> a
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > StopReplica
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > delete
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > topic
> > > > > immediately
> > > > > > > like
> > > > > > > >>> > we'd
> > > > > > > >>> > > > >> want to
> > > > > > > >>> > > > >> > > for
> > > > > > > >>> > > > >> > > > > > this
> > > > > > > >>> > > > >> > > > > > > > KIP.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > This
> makes
> > > me
> > > > > > wonder
> > > > > > > >>> if we
> > > > > > > >>> > > > want
> > > > > > > >>> > > > >> > > tagged
> > > > > > > >>> > > > >> > > > > > fields
> > > > > > > >>> > > > >> > > > > > > > on
> > > > > > > >>> > > > >> > > > > > > > >> all
> > > > > > > >>> > > > >> > > > > > > > >> > > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> requests
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > after
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > all. Let
> > me
> > > know
> > > > > > > your
> > > > > > > >>> > > > thoughts!
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > Justine
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > On Tue,
> > Sep
> > > 15,
> > > > > > 2020
> > > > > > > >>> at
> > > > > > > >>> > 1:03
> > > > > > > >>> > > > PM
> > > > > > > >>> > > > >> > > Justine
> > > > > > > >>> > > > >> > > > > > > Olshan
> > > > > > > >>> > > > >> > > > > > > > <
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > jolshan@confluent.io
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > wrote:
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Hi all,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Jun
> > > brought up
> > > > > a
> > > > > > > good
> > > > > > > >>> > point
> > > > > > > >>> > > > in
> > > > > > > >>> > > > >> his
> > > > > > > >>> > > > >> > > > last
> > > > > > > >>> > > > >> > > > > > > email
> > > > > > > >>> > > > >> > > > > > > > >> about
> > > > > > > >>> > > > >> > > > > > > > >> > > > > tagged
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > fields,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > and
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> I've
> > > updated
> > > > > the
> > > > > > > KIP
> > > > > > > >>> to
> > > > > > > >>> > > > reflect
> > > > > > > >>> > > > >> > that
> > > > > > > >>> > > > >> > > > the
> > > > > > > >>> > > > >> > > > > > > > >> changes to
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > requests
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > responses
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> will be
> > in
> > > the
> > > > > > form
> > > > > > > >>> of
> > > > > > > >>> > > tagged
> > > > > > > >>> > > > >> > fields
> > > > > > > >>> > > > >> > > > to
> > > > > > > >>> > > > >> > > > > > > avoid
> > > > > > > >>> > > > >> > > > > > > > >> > > changing
> > > > > > > >>> > > > >> > > > > > > > >> > > > > IBP.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Jun: I
> > > plan on
> > > > > > > >>> sending a
> > > > > > > >>> > > > >> followup
> > > > > > > >>> > > > >> > > > email
> > > > > > > >>> > > > >> > > > > to
> > > > > > > >>> > > > >> > > > > > > > >> address
> > > > > > > >>> > > > >> > > > > > > > >> > > some
> > > > > > > >>> > > > >> > > > > > > > >> > > > > of
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > other
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> points.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Thanks,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Justine
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> On Mon,
> > > Sep 14,
> > > > > > > 2020
> > > > > > > >>> at
> > > > > > > >>> > > 4:25
> > > > > > > >>> > > > PM
> > > > > > > >>> > > > >> > Jun
> > > > > > > >>> > > > >> > > > Rao
> > > > > > > >>> > > > >> > > > > <
> > > > > > > >>> > > > >> > > > > > > > >> > > > > jun@confluent.io>
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > wrote:
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Hi,
> > > Justine,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Thanks
> > > for the
> > > > > > > >>> updated
> > > > > > > >>> > > KIP.
> > > > > > > >>> > > > A
> > > > > > > >>> > > > >> few
> > > > > > > >>> > > > >> > > > > > comments
> > > > > > > >>> > > > >> > > > > > > > >> below.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 10.
> > > > > LeaderAndIsr
> > > > > > > >>> > Response:
> > > > > > > >>> > > > Do
> > > > > > > >>> > > > >> we
> > > > > > > >>> > > > >> > > need
> > > > > > > >>> > > > >> > > > > the
> > > > > > > >>> > > > >> > > > > > > > topic
> > > > > > > >>> > > > >> > > > > > > > >> > > name?
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 11.
> For
> > > the
> > > > > > > changed
> > > > > > > >>> > > > >> > > request/response,
> > > > > > > >>> > > > >> > > > > > other
> > > > > > > >>> > > > >> > > > > > > > >> than
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> LeaderAndIsr,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > UpdateMetadata,
> > > > > > > >>> > Metadata,
> > > > > > > >>> > > do
> > > > > > > >>> > > > >> we
> > > > > > > >>> > > > >> > > need
> > > > > > > >>> > > > >> > > > to
> > > > > > > >>> > > > >> > > > > > > > include
> > > > > > > >>> > > > >> > > > > > > > >> > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > topic
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> name?
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 12. It
> > > seems
> > > > > > that
> > > > > > > >>> > upgrades
> > > > > > > >>> > > > >> don't
> > > > > > > >>> > > > >> > > > > require
> > > > > > > >>> > > > >> > > > > > > IBP.
> > > > > > > >>> > > > >> > > > > > > > >> Does
> > > > > > > >>> > > > >> > > > > > > > >> > > > that
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > mean
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > new
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> fields
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> in all
> > the
> > > > > > > >>> > > request/response
> > > > > > > >>> > > > >> are
> > > > > > > >>> > > > >> > > added
> > > > > > > >>> > > > >> > > > > as
> > > > > > > >>> > > > >> > > > > > > > tagged
> > > > > > > >>> > > > >> > > > > > > > >> > > fields
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> without
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > bumping
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > up
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> the
> > > request
> > > > > > > >>> version? It
> > > > > > > >>> > > > would
> > > > > > > >>> > > > >> be
> > > > > > > >>> > > > >> > > > useful
> > > > > > > >>> > > > >> > > > > > to
> > > > > > > >>> > > > >> > > > > > > > make
> > > > > > > >>> > > > >> > > > > > > > >> > that
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > clear.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 13.
> > > Partition
> > > > > > > >>> Metadata
> > > > > > > >>> > > file:
> > > > > > > >>> > > > >> Do
> > > > > > > >>> > > > >> > we
> > > > > > > >>> > > > >> > > > need
> > > > > > > >>> > > > >> > > > > > to
> > > > > > > >>> > > > >> > > > > > > > >> include
> > > > > > > >>> > > > >> > > > > > > > >> > > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > topic
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > name
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > and
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > partition
> > > id
> > > > > > since
> > > > > > > >>> they
> > > > > > > >>> > > are
> > > > > > > >>> > > > >> > implied
> > > > > > > >>> > > > >> > > > in
> > > > > > > >>> > > > >> > > > > > the
> > > > > > > >>> > > > >> > > > > > > > >> > directory
> > > > > > > >>> > > > >> > > > > > > > >> > > > > name?
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 14. In
> > the
> > > > > JBOD
> > > > > > > >>> mode, we
> > > > > > > >>> > > > >> support
> > > > > > > >>> > > > >> > > > > moving a
> > > > > > > >>> > > > >> > > > > > > > >> > > partition's
> > > > > > > >>> > > > >> > > > > > > > >> > > > > data
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> from
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > one
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > disk
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> to
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> another.
> > > Will
> > > > > > the
> > > > > > > >>> new
> > > > > > > >>> > > > >> partition
> > > > > > > >>> > > > >> > > > > metadata
> > > > > > > >>> > > > >> > > > > > > file
> > > > > > > >>> > > > >> > > > > > > > >> be
> > > > > > > >>> > > > >> > > > > > > > >> > > > copied
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> during
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> process?
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 15.
> The
> > > KIP
> > > > > says
> > > > > > > >>> "Remove
> > > > > > > >>> > > > >> deleted
> > > > > > > >>> > > > >> > > > topics
> > > > > > > >>> > > > >> > > > > > > from
> > > > > > > >>> > > > >> > > > > > > > >> > > replicas
> > > > > > > >>> > > > >> > > > > > > > >> > > > by
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > sending
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > StopReplicaRequest
> > > > > > > >>> V2
> > > > > > > >>> > for
> > > > > > > >>> > > > any
> > > > > > > >>> > > > >> > > topics
> > > > > > > >>> > > > >> > > > > > which
> > > > > > > >>> > > > >> > > > > > > do
> > > > > > > >>> > > > >> > > > > > > > >> not
> > > > > > > >>> > > > >> > > > > > > > >> > > > > contain
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > a
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > topic
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > ID,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > and
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> V3 for
> > any
> > > > > > topics
> > > > > > > >>> which
> > > > > > > >>> > do
> > > > > > > >>> > > > >> > contain
> > > > > > > >>> > > > >> > > a
> > > > > > > >>> > > > >> > > > > > topic
> > > > > > > >>> > > > >> > > > > > > > >> ID.".
> > > > > > > >>> > > > >> > > > > > > > >> > > > > However,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > it
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > seems
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> updated
> > > > > > controller
> > > > > > > >>> will
> > > > > > > >>> > > > create
> > > > > > > >>> > > > >> > all
> > > > > > > >>> > > > >> > > > > > missing
> > > > > > > >>> > > > >> > > > > > > > >> topic
> > > > > > > >>> > > > >> > > > > > > > >> > IDs
> > > > > > > >>> > > > >> > > > > > > > >> > > > > first
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > before
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > doing
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> other
> > > actions.
> > > > > > So,
> > > > > > > >>> is
> > > > > > > >>> > > > >> > > > > StopReplicaRequest
> > > > > > > >>> > > > >> > > > > > V2
> > > > > > > >>> > > > >> > > > > > > > >> > needed?
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Jun
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> On
> Fri,
> > > Sep
> > > > > 11,
> > > > > > > >>> 2020 at
> > > > > > > >>> > > > 10:31
> > > > > > > >>> > > > >> AM
> > > > > > > >>> > > > >> > > John
> > > > > > > >>> > > > >> > > > > > > > Roesler <
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > vvcephei@apache.org
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> wrote:
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > Thanks,
> > > > > > Justine!
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > Your
> > > > > response
> > > > > > > >>> seems
> > > > > > > >>> > > > >> compelling
> > > > > > > >>> > > > >> > to
> > > > > > > >>> > > > >> > > > me.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> -John
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > On
> > Fri,
> > > > > > > >>> 2020-09-11 at
> > > > > > > >>> > > > 10:17
> > > > > > > >>> > > > >> > > -0700,
> > > > > > > >>> > > > >> > > > > > > Justine
> > > > > > > >>> > > > >> > > > > > > > >> > Olshan
> > > > > > > >>> > > > >> > > > > > > > >> > > > > wrote:
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > Hello
> > > all,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > Thanks for
> > > > > > > >>> > continuing
> > > > > > > >>> > > > the
> > > > > > > >>> > > > >> > > > > > discussion! I
> > > > > > > >>> > > > >> > > > > > > > >> have a
> > > > > > > >>> > > > >> > > > > > > > >> > > few
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > responses
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > to
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > your
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > points.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> Tom:
> > > You
> > > > > are
> > > > > > > >>> correct
> > > > > > > >>> > > in
> > > > > > > >>> > > > >> that
> > > > > > > >>> > > > >> > > this
> > > > > > > >>> > > > >> > > > > KIP
> > > > > > > >>> > > > >> > > > > > > has
> > > > > > > >>> > > > >> > > > > > > > >> not
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > mentioned
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > >>> > DeleteTopicsRequest. I
> > > > > > > >>> > > > >> think
> > > > > > > >>> > > > >> > > that
> > > > > > > >>> > > > >> > > > > > this
> > > > > > > >>> > > > >> > > > > > > > >> would
> > > > > > > >>> > > > >> > > > > > > > >> > be
> > > > > > > >>> > > > >> > > > > > > > >> > > > out
> > > > > > > >>> > > > >> > > > > > > > >> > > > > of
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > scope
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > for
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> now,
> but
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> may
> > be
> > > > > > > something
> > > > > > > >>> > worth
> > > > > > > >>> > > > >> adding
> > > > > > > >>> > > > >> > > in
> > > > > > > >>> > > > >> > > > > the
> > > > > > > >>> > > > >> > > > > > > > >> future.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > John:
> > > We
> > > > > did
> > > > > > > >>> > consider
> > > > > > > >>> > > > >> > sequence
> > > > > > > >>> > > > >> > > > ids,
> > > > > > > >>> > > > >> > > > > > but
> > > > > > > >>> > > > >> > > > > > > > >> there
> > > > > > > >>> > > > >> > > > > > > > >> > > are
> > > > > > > >>> > > > >> > > > > > > > >> > > > a
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > few
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > reasons
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> favor
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > UUIDs.
> > > > > There
> > > > > > > are
> > > > > > > >>> > > several
> > > > > > > >>> > > > >> > cases
> > > > > > > >>> > > > >> > > > > where
> > > > > > > >>> > > > >> > > > > > > > topics
> > > > > > > >>> > > > >> > > > > > > > >> > from
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> different
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > clusters
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> may
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > interact
> > > > > now
> > > > > > > >>> and in
> > > > > > > >>> > > the
> > > > > > > >>> > > > >> > future.
> > > > > > > >>> > > > >> > > > For
> > > > > > > >>> > > > >> > > > > > > > >> example,
> > > > > > > >>> > > > >> > > > > > > > >> > > > Mirror
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Maker 2
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > may
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> benefit
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> from
> > > being
> > > > > > > able
> > > > > > > >>> to
> > > > > > > >>> > > > detect
> > > > > > > >>> > > > >> > when
> > > > > > > >>> > > > >> > > a
> > > > > > > >>> > > > >> > > > > > > cluster
> > > > > > > >>> > > > >> > > > > > > > >> being
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > mirrored
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> is
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > deleted
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> and
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > recreated
> > > > > > and
> > > > > > > >>> > globally
> > > > > > > >>> > > > >> unique
> > > > > > > >>> > > > >> > > > > > > identifiers
> > > > > > > >>> > > > >> > > > > > > > >> > would
> > > > > > > >>> > > > >> > > > > > > > >> > > > make
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > resolving
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > issues
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > easier
> > > > > than
> > > > > > > >>> sequence
> > > > > > > >>> > > IDs
> > > > > > > >>> > > > >> > which
> > > > > > > >>> > > > >> > > > may
> > > > > > > >>> > > > >> > > > > > > > collide
> > > > > > > >>> > > > >> > > > > > > > >> > > between
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > clusters.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > KIP-405
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > (tiered
> > > > > > > storage)
> > > > > > > >>> > will
> > > > > > > >>> > > > also
> > > > > > > >>> > > > >> > > > benefit
> > > > > > > >>> > > > >> > > > > > from
> > > > > > > >>> > > > >> > > > > > > > >> > globally
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > unique
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> IDs
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > as
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > shared
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > buckets
> > > > > may
> > > > > > be
> > > > > > > >>> used
> > > > > > > >>> > > > >> between
> > > > > > > >>> > > > >> > > > > clusters.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > Globally
> > > > > > > unique
> > > > > > > >>> IDs
> > > > > > > >>> > > > would
> > > > > > > >>> > > > >> > also
> > > > > > > >>> > > > >> > > > make
> > > > > > > >>> > > > >> > > > > > > > >> > > functionality
> > > > > > > >>> > > > >> > > > > > > > >> > > > > like
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > moving
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > topics
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > between
> > > > > > > >>> disparate
> > > > > > > >>> > > > clusters
> > > > > > > >>> > > > >> > > easier
> > > > > > > >>> > > > >> > > > > in
> > > > > > > >>> > > > >> > > > > > > the
> > > > > > > >>> > > > >> > > > > > > > >> > future,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> simplify
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > any
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > future
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > implementations
> > > > > > > >>> of
> > > > > > > >>> > > > backups
> > > > > > > >>> > > > >> > and
> > > > > > > >>> > > > >> > > > > > > restores,
> > > > > > > >>> > > > >> > > > > > > > >> and
> > > > > > > >>> > > > >> > > > > > > > >> > > more.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > In
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > general,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> unique
> > IDs
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > would
> > > > > ensure
> > > > > > > >>> that
> > > > > > > >>> > the
> > > > > > > >>> > > > >> source
> > > > > > > >>> > > > >> > > > > cluster
> > > > > > > >>> > > > >> > > > > > > > >> topics do
> > > > > > > >>> > > > >> > > > > > > > >> > > not
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> conflict
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > with
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > destination
> > > > > > > >>> cluster
> > > > > > > >>> > > > >> topics.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > If
> > we
> > > were
> > > > > > to
> > > > > > > >>> use
> > > > > > > >>> > > > sequence
> > > > > > > >>> > > > >> > ids,
> > > > > > > >>> > > > >> > > > we
> > > > > > > >>> > > > >> > > > > > > would
> > > > > > > >>> > > > >> > > > > > > > >> need
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> sufficiently
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > large
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> cluster
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> ids
> > > to be
> > > > > > > stored
> > > > > > > >>> > with
> > > > > > > >>> > > > the
> > > > > > > >>> > > > >> > topic
> > > > > > > >>> > > > >> > > > > > > > >> identifiers or
> > > > > > > >>> > > > >> > > > > > > > >> > > we
> > > > > > > >>> > > > >> > > > > > > > >> > > > > run
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > risk
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > collisions.
> > > > > > > This
> > > > > > > >>> > will
> > > > > > > >>> > > > >> give up
> > > > > > > >>> > > > >> > > any
> > > > > > > >>> > > > >> > > > > > > > >> advantage in
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> compactness
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> sequence
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > numbers
> > > > > may
> > > > > > > >>> bring.
> > > > > > > >>> > > Given
> > > > > > > >>> > > > >> > these
> > > > > > > >>> > > > >> > > > > > > > advantages I
> > > > > > > >>> > > > >> > > > > > > > >> > > think
> > > > > > > >>> > > > >> > > > > > > > >> > > > it
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> makes
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > sense
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> use
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > UUIDs.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > Gokul:
> > > > > This
> > > > > > is
> > > > > > > >>> an
> > > > > > > >>> > > > >> interesting
> > > > > > > >>> > > > >> > > > idea,
> > > > > > > >>> > > > >> > > > > > but
> > > > > > > >>> > > > >> > > > > > > > >> this
> > > > > > > >>> > > > >> > > > > > > > >> > is
> > > > > > > >>> > > > >> > > > > > > > >> > > a
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> breaking
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > change.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Out of
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > scope
> > > for
> > > > > > now,
> > > > > > > >>> but
> > > > > > > >>> > > maybe
> > > > > > > >>> > > > >> > worth
> > > > > > > >>> > > > >> > > > > > > discussing
> > > > > > > >>> > > > >> > > > > > > > >> in
> > > > > > > >>> > > > >> > > > > > > > >> > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > future.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> Hope
> > > this
> > > > > > > >>> explains
> > > > > > > >>> > > some
> > > > > > > >>> > > > of
> > > > > > > >>> > > > >> > the
> > > > > > > >>> > > > >> > > > > > > decisions,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > Justine
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > On
> > > Fri,
> > > > > Sep
> > > > > > > 11,
> > > > > > > >>> 2020
> > > > > > > >>> > > at
> > > > > > > >>> > > > >> 8:27
> > > > > > > >>> > > > >> > AM
> > > > > > > >>> > > > >> > > > > Gokul
> > > > > > > >>> > > > >> > > > > > > > >> Ramanan
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Subramanian <
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > >>> > gokul2411s@gmail.com>
> > > > > > > >>> > > > >> wrote:
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > Hi.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > Thanks
> > > > > for
> > > > > > > the
> > > > > > > >>> > KIP.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > Have you
> > > > > > > >>> thought
> > > > > > > >>> > > about
> > > > > > > >>> > > > >> > > whether
> > > > > > > >>> > > > >> > > > it
> > > > > > > >>> > > > >> > > > > > > makes
> > > > > > > >>> > > > >> > > > > > > > >> > sense
> > > > > > > >>> > > > >> > > > > > > > >> > > to
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> support
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > authorizing a
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > principal
> > > > > > > for
> > > > > > > >>> a
> > > > > > > >>> > > topic
> > > > > > > >>> > > > ID
> > > > > > > >>> > > > >> > > rather
> > > > > > > >>> > > > >> > > > > > than
> > > > > > > >>> > > > >> > > > > > > a
> > > > > > > >>> > > > >> > > > > > > > >> topic
> > > > > > > >>> > > > >> > > > > > > > >> > > > name
> > > > > > > >>> > > > >> > > > > > > > >> > > > > to
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > achieve
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> tighter
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > security?
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> Or
> > > is
> > > > > the
> > > > > > > >>> topic ID
> > > > > > > >>> > > > >> > > > fundamentally
> > > > > > > >>> > > > >> > > > > an
> > > > > > > >>> > > > >> > > > > > > > >> internal
> > > > > > > >>> > > > >> > > > > > > > >> > > > > detail
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > similar
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > to
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> epochs
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > used
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > in a
> > > > > bunch
> > > > > > > of
> > > > > > > >>> > other
> > > > > > > >>> > > > >> places
> > > > > > > >>> > > > >> > in
> > > > > > > >>> > > > >> > > > > > Kafka?
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > Thanks.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> On
> > > Fri,
> > > > > > Sep
> > > > > > > >>> 11,
> > > > > > > >>> > 2020
> > > > > > > >>> > > > at
> > > > > > > >>> > > > >> > 4:06
> > > > > > > >>> > > > >> > > PM
> > > > > > > >>> > > > >> > > > > > John
> > > > > > > >>> > > > >> > > > > > > > >> > Roesler <
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > vvcephei@apache.org
> > > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> wrote:
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > Hello
> > > > > > > >>> Justine,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > Thanks
> > > > > > for
> > > > > > > >>> the
> > > > > > > >>> > > KIP!
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > I
> > > > > happen
> > > > > > > to
> > > > > > > >>> have
> > > > > > > >>> > > > been
> > > > > > > >>> > > > >> > > > > confronted
> > > > > > > >>> > > > >> > > > > > > > >> recently
> > > > > > > >>> > > > >> > > > > > > > >> > > with
> > > > > > > >>> > > > >> > > > > > > > >> > > > > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> need
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > to
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > keep
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> track
> > of
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> a
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > large
> > > > > > > >>> number of
> > > > > > > >>> > > > >> topics as
> > > > > > > >>> > > > >> > > > > > compactly
> > > > > > > >>> > > > >> > > > > > > > as
> > > > > > > >>> > > > >> > > > > > > > >> > > > > possible. I
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> was
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > going
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> come
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > up
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > with
> > > > > > some
> > > > > > > >>> way to
> > > > > > > >>> > > > >> > dictionary
> > > > > > > >>> > > > >> > > > > > encode
> > > > > > > >>> > > > >> > > > > > > > the
> > > > > > > >>> > > > >> > > > > > > > >> > topic
> > > > > > > >>> > > > >> > > > > > > > >> > > > > names
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> as
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > integers,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> but
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > this
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > seems
> > > > > > much
> > > > > > > >>> > better!
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > > > > Apologies
> > > > > > > if
> > > > > > > >>> > this
> > > > > > > >>> > > > has
> > > > > > > >>> > > > >> > been
> > > > > > > >>> > > > >> > > > > raised
> > > > > > > >>> > > > >> > > > > > > > >> before,
> > > > > > > >>> > > > >> > > > > > > > >> > > but
> > > > > > > >>> > > > >> > > > > > > > >> > > > > I’m
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > wondering
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> about
> > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > choice
> > > > > > of
> > > > > > > >>> UUID
> > > > > > > >>> > vs
> > > > > > > >>> > > > >> > sequence
> > > > > > > >>> > > > >> > > > > number
> > > > > > > >>> > > > >> > > > > > > for
> > > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > > >>> > > > >> > > > > > > > >> > > ids.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Typically,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > I’ve
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> seen
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> UUIDs
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > in two
> > > > > > > >>> > situations:
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > 1.
> > > > > When
> > > > > > > >>> > processes
> > > > > > > >>> > > > >> need to
> > > > > > > >>> > > > >> > > > > > generate
> > > > > > > >>> > > > >> > > > > > > > >> > > > non-colliding
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > identifiers
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> without
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > > > > > >>> coordination.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > 2.
> > > > > When
> > > > > > > the
> > > > > > > >>> > > > identifier
> > > > > > > >>> > > > >> > > needs
> > > > > > > >>> > > > >> > > > to
> > > > > > > >>> > > > >> > > > > > be
> > > > > > > >>> > > > >> > > > > > > > >> > > > “universally
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > unique”;
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > I.e.,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > > > > identifier
> > > > > > > >>> needs
> > > > > > > >>> > > to
> > > > > > > >>> > > > >> > > > distinguish
> > > > > > > >>> > > > >> > > > > > the
> > > > > > > >>> > > > >> > > > > > > > >> entity
> > > > > > > >>> > > > >> > > > > > > > >> > > > from
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > all
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > other
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> entities
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > that
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > could
> > > > > > ever
> > > > > > > >>> > exist.
> > > > > > > >>> > > > >> This is
> > > > > > > >>> > > > >> > > > > useful
> > > > > > > >>> > > > >> > > > > > in
> > > > > > > >>> > > > >> > > > > > > > >> cases
> > > > > > > >>> > > > >> > > > > > > > >> > > > where
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > entities
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > from
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > all
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> kinds
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> of
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > > > systems
> > > > > > > get
> > > > > > > >>> > mixed
> > > > > > > >>> > > > >> > together,
> > > > > > > >>> > > > >> > > > > such
> > > > > > > >>> > > > >> > > > > > as
> > > > > > > >>> > > > >> > > > > > > > >> when
> > > > > > > >>> > > > >> > > > > > > > >> > > > dumping
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> logs
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > from
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > all
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > processes
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> in
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > a
> > > > > > company
> > > > > > > >>> into a
> > > > > > > >>> > > > >> common
> > > > > > > >>> > > > >> > > > system.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > Maybe
> > > > > > I’m
> > > > > > > >>> being
> > > > > > > >>> > > > >> > > > short-sighted,
> > > > > > > >>> > > > >> > > > > > but
> > > > > > > >>> > > > >> > > > > > > it
> > > > > > > >>> > > > >> > > > > > > > >> > > doesn’t
> > > > > > > >>> > > > >> > > > > > > > >> > > > > seem
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> like
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > either
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> really
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > > > applies
> > > > > > > >>> here. It
> > > > > > > >>> > > > seems
> > > > > > > >>> > > > >> > like
> > > > > > > >>> > > > >> > > > the
> > > > > > > >>> > > > >> > > > > > > > brokers
> > > > > > > >>> > > > >> > > > > > > > >> > > could
> > > > > > > >>> > > > >> > > > > > > > >> > > > > and
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> would
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > achieve
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > consensus
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > when
> > > > > > > >>> creating a
> > > > > > > >>> > > > topic
> > > > > > > >>> > > > >> > > anyway,
> > > > > > > >>> > > > >> > > > > > which
> > > > > > > >>> > > > >> > > > > > > > is
> > > > > > > >>> > > > >> > > > > > > > >> all
> > > > > > > >>> > > > >> > > > > > > > >> > > > > that’s
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > required
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > to
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > generate
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > > > > > >>> non-colliding
> > > > > > > >>> > > > sequence
> > > > > > > >>> > > > >> > ids.
> > > > > > > >>> > > > >> > > > For
> > > > > > > >>> > > > >> > > > > > the
> > > > > > > >>> > > > >> > > > > > > > >> > second,
> > > > > > > >>> > > > >> > > > > > > > >> > > as
> > > > > > > >>> > > > >> > > > > > > > >> > > > > you
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > mention,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > we
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> could
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > assign
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > a
> > > UUID
> > > > > > to
> > > > > > > >>> the
> > > > > > > >>> > > > cluster
> > > > > > > >>> > > > >> as
> > > > > > > >>> > > > >> > a
> > > > > > > >>> > > > >> > > > > whole,
> > > > > > > >>> > > > >> > > > > > > > which
> > > > > > > >>> > > > >> > > > > > > > >> > > would
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > render
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > any
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > resource
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> scoped
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> to
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > the
> > > > > > broker
> > > > > > > >>> > > > universally
> > > > > > > >>> > > > >> > > unique
> > > > > > > >>> > > > >> > > > > as
> > > > > > > >>> > > > >> > > > > > > > well.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > The
> > > > > > > reason I
> > > > > > > >>> > > mention
> > > > > > > >>> > > > >> this
> > > > > > > >>> > > > >> > > is
> > > > > > > >>> > > > >> > > > > > that,
> > > > > > > >>> > > > >> > > > > > > > >> > although
> > > > > > > >>> > > > >> > > > > > > > >> > > a
> > > > > > > >>> > > > >> > > > > > > > >> > > > > UUID
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> is
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > way
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > more
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > compact
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > than
> > > > > > topic
> > > > > > > >>> > names,
> > > > > > > >>> > > > it’s
> > > > > > > >>> > > > >> > > still
> > > > > > > >>> > > > >> > > > 16
> > > > > > > >>> > > > >> > > > > > > > bytes.
> > > > > > > >>> > > > >> > > > > > > > >> In
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > contrast,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> a
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > 4-byte
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> integer
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > > > sequence
> > > > > > > id
> > > > > > > >>> > would
> > > > > > > >>> > > > give
> > > > > > > >>> > > > >> > us 4
> > > > > > > >>> > > > >> > > > > > billion
> > > > > > > >>> > > > >> > > > > > > > >> unique
> > > > > > > >>> > > > >> > > > > > > > >> > > > > topics
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> per
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > cluster,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> which
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > seems
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > like
> > > > > > > enough
> > > > > > > >>> ;)
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > > > > > Considering
> > > > > > > >>> the
> > > > > > > >>> > > > >> number of
> > > > > > > >>> > > > >> > > > > > different
> > > > > > > >>> > > > >> > > > > > > > >> times
> > > > > > > >>> > > > >> > > > > > > > >> > > > these
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> topic
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > identifiers
> > > > > are
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > sent
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > over
> > > > > the
> > > > > > > >>> wire or
> > > > > > > >>> > > > >> stored
> > > > > > > >>> > > > >> > in
> > > > > > > >>> > > > >> > > > > > memory,
> > > > > > > >>> > > > >> > > > > > > it
> > > > > > > >>> > > > >> > > > > > > > >> > seems
> > > > > > > >>> > > > >> > > > > > > > >> > > > like
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > it
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > might
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > be
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> worth
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > > > > additional
> > > > > > > >>> 4x
> > > > > > > >>> > > space
> > > > > > > >>> > > > >> > > savings.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > What
> > > > > do
> > > > > > > you
> > > > > > > >>> > think
> > > > > > > >>> > > > >> about
> > > > > > > >>> > > > >> > > this?
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > > > Thanks,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > John
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > On
> > > > > Fri,
> > > > > > > Sep
> > > > > > > >>> 11,
> > > > > > > >>> > > > 2020,
> > > > > > > >>> > > > >> at
> > > > > > > >>> > > > >> > > > 03:20,
> > > > > > > >>> > > > >> > > > > > Tom
> > > > > > > >>> > > > >> > > > > > > > >> > Bentley
> > > > > > > >>> > > > >> > > > > > > > >> > > > > wrote:
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > Hi
> > > > > > > >>> Justine,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > This
> > > > > > > looks
> > > > > > > >>> > like
> > > > > > > >>> > > a
> > > > > > > >>> > > > >> very
> > > > > > > >>> > > > >> > > > > welcome
> > > > > > > >>> > > > >> > > > > > > > >> > > improvement.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Thanks!
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > > Maybe
> > > > > > I
> > > > > > > >>> missed
> > > > > > > >>> > > it,
> > > > > > > >>> > > > >> but
> > > > > > > >>> > > > >> > > the
> > > > > > > >>> > > > >> > > > > KIP
> > > > > > > >>> > > > >> > > > > > > > >> doesn't
> > > > > > > >>> > > > >> > > > > > > > >> > > seem
> > > > > > > >>> > > > >> > > > > > > > >> > > > to
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > mention
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > changing
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > > > > >>> > > > DeleteTopicsRequest
> > > > > > > >>> > > > >> to
> > > > > > > >>> > > > >> > > > > identify
> > > > > > > >>> > > > >> > > > > > > the
> > > > > > > >>> > > > >> > > > > > > > >> > topic
> > > > > > > >>> > > > >> > > > > > > > >> > > > > using
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > an
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > id.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Maybe
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> that's
> > > out
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > of
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > > scope,
> > > > > > > but
> > > > > > > >>> > > > >> > > > > DeleteTopicsRequest
> > > > > > > >>> > > > >> > > > > > is
> > > > > > > >>> > > > >> > > > > > > > not
> > > > > > > >>> > > > >> > > > > > > > >> > > listed
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > among
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Future
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Work
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > APIs
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > > > either.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > Kind
> > > > > > > >>> regards,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > Tom
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > On
> > > > > > Thu,
> > > > > > > >>> Sep
> > > > > > > >>> > 10,
> > > > > > > >>> > > > >> 2020 at
> > > > > > > >>> > > > >> > > > 3:59
> > > > > > > >>> > > > >> > > > > PM
> > > > > > > >>> > > > >> > > > > > > > >> Satish
> > > > > > > >>> > > > >> > > > > > > > >> > > > > Duggana <
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > > > >>> > > > >> satish.duggana@gmail.com
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > > wrote:
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > Thanks
> > > > > > > >>> > > > >> Lucas/Justine
> > > > > > > >>> > > > >> > > for
> > > > > > > >>> > > > >> > > > > the
> > > > > > > >>> > > > >> > > > > > > nice
> > > > > > > >>> > > > >> > > > > > > > >> KIP.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > It
> > > > > > has
> > > > > > > >>> > several
> > > > > > > >>> > > > >> > benefits
> > > > > > > >>> > > > >> > > > > which
> > > > > > > >>> > > > >> > > > > > > > also
> > > > > > > >>> > > > >> > > > > > > > >> > > include
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > simplifying
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> topic
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > > deletion
> > > > > > > >>> > > process
> > > > > > > >>> > > > >> by
> > > > > > > >>> > > > >> > > > > > controller
> > > > > > > >>> > > > >> > > > > > > > and
> > > > > > > >>> > > > >> > > > > > > > >> > logs
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > cleanup
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> by
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > brokers
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > in
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> corner
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > cases.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > Best,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > > Satish.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > On
> > > > > > > Wed,
> > > > > > > >>> Sep
> > > > > > > >>> > 9,
> > > > > > > >>> > > > >> 2020
> > > > > > > >>> > > > >> > at
> > > > > > > >>> > > > >> > > > > 10:07
> > > > > > > >>> > > > >> > > > > > PM
> > > > > > > >>> > > > >> > > > > > > > >> > Justine
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > Olshan <
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > >>> jolshan@confluent.io
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > wrote:
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > > > Hello
> > > > > > > >>> all,
> > > > > > > >>> > > > it's
> > > > > > > >>> > > > >> > been
> > > > > > > >>> > > > >> > > > > > almost a
> > > > > > > >>> > > > >> > > > > > > > >> year!
> > > > > > > >>> > > > >> > > > > > > > >> > > I've
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > made
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > some
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > changes
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> to
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > this
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > KIP
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > and
> > > > > > > >>> hope to
> > > > > > > >>> > > > >> continue
> > > > > > > >>> > > > >> > > the
> > > > > > > >>> > > > >> > > > > > > > >> discussion.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > > One
> > > > > > > >>> of the
> > > > > > > >>> > > > main
> > > > > > > >>> > > > >> > > changes
> > > > > > > >>> > > > >> > > > > > I've
> > > > > > > >>> > > > >> > > > > > > > >> added
> > > > > > > >>> > > > >> > > > > > > > >> > is
> > > > > > > >>> > > > >> > > > > > > > >> > > > now
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > metadata
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> response
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > will
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > > include
> > > > > > > >>> the
> > > > > > > >>> > > > topic
> > > > > > > >>> > > > >> ID
> > > > > > > >>> > > > >> > > (as
> > > > > > > >>> > > > >> > > > > > Colin
> > > > > > > >>> > > > >> > > > > > > > >> > > suggested).
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Clients
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > can
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> obtain
> > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > > > topicID
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > of a
> > > > > > > >>> given
> > > > > > > >>> > > topic
> > > > > > > >>> > > > >> > > through
> > > > > > > >>> > > > >> > > > a
> > > > > > > >>> > > > >> > > > > > > > >> > > > TopicDescription.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > The
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > topicId
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > will
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > also
> > be
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > > included
> > > > > > > >>> > with
> > > > > > > >>> > > > the
> > > > > > > >>> > > > >> > > > > > > UpdateMetadata
> > > > > > > >>> > > > >> > > > > > > > >> > > request.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > > Let
> > > > > > > me
> > > > > > > >>> > know
> > > > > > > >>> > > > what
> > > > > > > >>> > > > >> > you
> > > > > > > >>> > > > >> > > > all
> > > > > > > >>> > > > >> > > > > > > think.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > > > Thank
> > > > > > > >>> you,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > > > >>> Justine
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > On
> > > > > > > >>> > > 2019/09/13
> > > > > > > >>> > > > >> > > 16:38:26,
> > > > > > > >>> > > > >> > > > > > > "Colin
> > > > > > > >>> > > > >> > > > > > > > >> > > McCabe" <
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > cmccabe@apache.org
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> wrote:
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > Hi
> > > > > > > >>> > Lucas,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > > >>> Thanks
> > > > > > > >>> > for
> > > > > > > >>> > > > >> > tackling
> > > > > > > >>> > > > >> > > > > this.
> > > > > > > >>> > > > >> > > > > > > > >> Topic
> > > > > > > >>> > > > >> > > > > > > > >> > IDs
> > > > > > > >>> > > > >> > > > > > > > >> > > > > are a
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > great
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > idea,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> and
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > this
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> is
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > a
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > really
> > > > > > > >>> good
> > > > > > > >>> > > > >> writeup.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > > For
> > > > > > > >>> > > > >> > > > > > > /brokers/topics/[topic],
> > > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > > >>> > > > >> > > > > > > > >> > > > schema
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> version
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > should
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > be
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> bumped
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> to
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > > version
> > > > > > > >>> 3,
> > > > > > > >>> > > > rather
> > > > > > > >>> > > > >> > than
> > > > > > > >>> > > > >> > > 2.
> > > > > > > >>> > > > >> > > > > > > > KIP-455
> > > > > > > >>> > > > >> > > > > > > > >> > > bumped
> > > > > > > >>> > > > >> > > > > > > > >> > > > > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > version
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > this
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> znode
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> to
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > 2
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > > already
> > > > > > > >>> :)
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > > >>> Given
> > > > > > > >>> > that
> > > > > > > >>> > > > >> we're
> > > > > > > >>> > > > >> > > > going
> > > > > > > >>> > > > >> > > > > to
> > > > > > > >>> > > > >> > > > > > > be
> > > > > > > >>> > > > >> > > > > > > > >> > seeing
> > > > > > > >>> > > > >> > > > > > > > >> > > > > these
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > things
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > as
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> strings
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > as
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > lot
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > (in
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > logs,
> > > > > > > in
> > > > > > > >>> > > > >> ZooKeeper,
> > > > > > > >>> > > > >> > on
> > > > > > > >>> > > > >> > > > the
> > > > > > > >>> > > > >> > > > > > > > >> > command-line,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > etc.),
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > does
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > it
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > make
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> sense
> > to
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > use
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > base64
> > > > > > > >>> when
> > > > > > > >>> > > > >> > converting
> > > > > > > >>> > > > >> > > > them
> > > > > > > >>> > > > >> > > > > > to
> > > > > > > >>> > > > >> > > > > > > > >> > strings?
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > > >>> Here is
> > > > > > > >>> > an
> > > > > > > >>> > > > >> > example
> > > > > > > >>> > > > >> > > of
> > > > > > > >>> > > > >> > > > > the
> > > > > > > >>> > > > >> > > > > > > hex
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> representation:
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> >
> > > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > > And
> > > > > > > >>> here
> > > > > > > >>> > > is
> > > > > > > >>> > > > an
> > > > > > > >>> > > > >> > > > example
> > > > > > > >>> > > > >> > > > > in
> > > > > > > >>> > > > >> > > > > > > > >> base64.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > > >>> > > > >> > > > b8tRS7h4TJ2Vt43Dp85v2A
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > > The
> > > > > > > >>> > base64
> > > > > > > >>> > > > >> > version
> > > > > > > >>> > > > >> > > > > saves
> > > > > > > >>> > > > >> > > > > > 15
> > > > > > > >>> > > > >> > > > > > > > >> > letters
> > > > > > > >>> > > > >> > > > > > > > >> > > > (to
> > > > > > > >>> > > > >> > > > > > > > >> > > > > be
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > fair,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > 4
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> those
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > were
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > dashes
> > > > > > > >>> that
> > > > > > > >>> > we
> > > > > > > >>> > > > >> could
> > > > > > > >>> > > > >> > > have
> > > > > > > >>> > > > >> > > > > > > elided
> > > > > > > >>> > > > >> > > > > > > > in
> > > > > > > >>> > > > >> > > > > > > > >> > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > hex
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > representation.)
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > > >>> Another
> > > > > > > >>> > > > thing
> > > > > > > >>> > > > >> to
> > > > > > > >>> > > > >> > > > > consider
> > > > > > > >>> > > > >> > > > > > > is
> > > > > > > >>> > > > >> > > > > > > > >> that
> > > > > > > >>> > > > >> > > > > > > > >> > we
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > should
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > specify
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > that
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > > >>> all-zeroes
> > > > > > > >>> > > UUID
> > > > > > > >>> > > > is
> > > > > > > >>> > > > >> > not
> > > > > > > >>> > > > >> > > a
> > > > > > > >>> > > > >> > > > > > valid
> > > > > > > >>> > > > >> > > > > > > > >> topic
> > > > > > > >>> > > > >> > > > > > > > >> > > UUID.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > >  We
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > can't
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > use
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> null
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > for
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > this
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > > because
> > > > > > > >>> we
> > > > > > > >>> > > can't
> > > > > > > >>> > > > >> > pass a
> > > > > > > >>> > > > >> > > > > null
> > > > > > > >>> > > > >> > > > > > > UUID
> > > > > > > >>> > > > >> > > > > > > > >> over
> > > > > > > >>> > > > >> > > > > > > > >> > > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > RPC
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > protocol
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> (there
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > is
> no
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > > special
> > > > > > > >>> > > pattern
> > > > > > > >>> > > > >> for
> > > > > > > >>> > > > >> > > null,
> > > > > > > >>> > > > >> > > > > nor
> > > > > > > >>> > > > >> > > > > > > do
> > > > > > > >>> > > > >> > > > > > > > we
> > > > > > > >>> > > > >> > > > > > > > >> > want
> > > > > > > >>> > > > >> > > > > > > > >> > > > to
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> waste
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > space
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > reserving
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > such
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > a
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > > >>> pattern.)
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > > >>> Maybe I
> > > > > > > >>> > > > missed
> > > > > > > >>> > > > >> > it,
> > > > > > > >>> > > > >> > > > but
> > > > > > > >>> > > > >> > > > > > did
> > > > > > > >>> > > > >> > > > > > > > you
> > > > > > > >>> > > > >> > > > > > > > >> > > > describe
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > "migration
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > of...
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > existing
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > > topic[s]
> > > > > > > >>> > > without
> > > > > > > >>> > > > >> > topic
> > > > > > > >>> > > > >> > > > IDs"
> > > > > > > >>> > > > >> > > > > > in
> > > > > > > >>> > > > >> > > > > > > > >> detail
> > > > > > > >>> > > > >> > > > > > > > >> > in
> > > > > > > >>> > > > >> > > > > > > > >> > > > any
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > section?
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > It
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> seems
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > like
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > when
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > the
> > > > > > > new
> > > > > > > >>> > > > controller
> > > > > > > >>> > > > >> > > > becomes
> > > > > > > >>> > > > >> > > > > > > > active,
> > > > > > > >>> > > > >> > > > > > > > >> it
> > > > > > > >>> > > > >> > > > > > > > >> > > > should
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> just
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > generate
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> random
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > UUIDs
> > > > > > for
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > these,
> > > > > > > >>> and
> > > > > > > >>> > > write
> > > > > > > >>> > > > >> the
> > > > > > > >>> > > > >> > > > random
> > > > > > > >>> > > > >> > > > > > > UUIDs
> > > > > > > >>> > > > >> > > > > > > > >> back
> > > > > > > >>> > > > >> > > > > > > > >> > > to
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > ZooKeeper.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > It
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> would
> be
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > good
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > to
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > spell
> > > > > > > >>> that
> > > > > > > >>> > > out.
> > > > > > > >>> > > > >> We
> > > > > > > >>> > > > >> > > > should
> > > > > > > >>> > > > >> > > > > > make
> > > > > > > >>> > > > >> > > > > > > > it
> > > > > > > >>> > > > >> > > > > > > > >> > clear
> > > > > > > >>> > > > >> > > > > > > > >> > > > > that
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> this
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > happens
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > regardless
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > of
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > the
> > > > > > > >>> > > inter-broker
> > > > > > > >>> > > > >> > > protocol
> > > > > > > >>> > > > >> > > > > > > version
> > > > > > > >>> > > > >> > > > > > > > >> > (it's
> > > > > > > >>> > > > >> > > > > > > > >> > > a
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > compatible
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > change).
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > > >>> > > > >> > > "LeaderAndIsrRequests
> > > > > > > >>> > > > >> > > > > > > > >> including an
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > is_every_partition
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> flag"
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > seems a
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > bit
> > > > > > > >>> wordy.
> > > > > > > >>> > > Can
> > > > > > > >>> > > > we
> > > > > > > >>> > > > >> > just
> > > > > > > >>> > > > >> > > > > call
> > > > > > > >>> > > > >> > > > > > > > these
> > > > > > > >>> > > > >> > > > > > > > >> > "full
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > >>> LeaderAndIsrRequests"?
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > Then
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > RPC
> > > > > > > >>> field
> > > > > > > >>> > > could
> > > > > > > >>> > > > be
> > > > > > > >>> > > > >> > > named
> > > > > > > >>> > > > >> > > > > > > "full".
> > > > > > > >>> > > > >> > > > > > > > >> > Also,
> > > > > > > >>> > > > >> > > > > > > > >> > > it
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > would
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > probably
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > be
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> better
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > for
> > > > > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > RPC
> > > > > > > >>> field to
> > > > > > > >>> > > be
> > > > > > > >>> > > > an
> > > > > > > >>> > > > >> > enum
> > > > > > > >>> > > > >> > > > of
> > > > > > > >>> > > > >> > > > > {
> > > > > > > >>> > > > >> > > > > > > > >> > > UNSPECIFIED,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > INCREMENTAL,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > FULL
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> }, so
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > that
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > we
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > can
> > > > > > > >>> cleanly
> > > > > > > >>> > > > handle
> > > > > > > >>> > > > >> > old
> > > > > > > >>> > > > >> > > > > > versions
> > > > > > > >>> > > > >> > > > > > > > (by
> > > > > > > >>> > > > >> > > > > > > > >> > > > treating
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> them
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > as
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > UNSPECIFIED)
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > In
> > > > > > > >>> the
> > > > > > > >>> > > > >> > > > > > LeaderAndIsrRequest
> > > > > > > >>> > > > >> > > > > > > > >> > section,
> > > > > > > >>> > > > >> > > > > > > > >> > > > you
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> write
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > "A
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > final
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > deletion
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > event
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > will
> > > > > > > be
> > > > > > > >>> > > > secheduled
> > > > > > > >>> > > > >> > for
> > > > > > > >>> > > > >> > > X
> > > > > > > >>> > > > >> > > > ms
> > > > > > > >>> > > > >> > > > > > > after
> > > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > LeaderAndIsrRequest
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > was
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> first
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > > >>> received..."
> > > > > > > >>> > > I
> > > > > > > >>> > > > >> guess
> > > > > > > >>> > > > >> > > > the X
> > > > > > > >>> > > > >> > > > > > > was a
> > > > > > > >>> > > > >> > > > > > > > >> > > > > placeholder
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> that
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > you
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> intended
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > > > replace
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > before
> > > > > > > >>> > > posting?
> > > > > > > >>> > > > :)
> > > > > > > >>> > > > >> > In
> > > > > > > >>> > > > >> > > > any
> > > > > > > >>> > > > >> > > > > > > case,
> > > > > > > >>> > > > >> > > > > > > > >> this
> > > > > > > >>> > > > >> > > > > > > > >> > > > seems
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > like
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > kind
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > of
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> thing
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > we'd
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > want a
> > > > > > > >>> > > > >> configuration
> > > > > > > >>> > > > >> > > for.
> > > > > > > >>> > > > >> > > > > > > Let's
> > > > > > > >>> > > > >> > > > > > > > >> > > describe
> > > > > > > >>> > > > >> > > > > > > > >> > > > > that
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > configuration
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> key
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > > > > somewhere
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > in
> > > > > > > this
> > > > > > > >>> KIP,
> > > > > > > >>> > > > >> > including
> > > > > > > >>> > > > >> > > > what
> > > > > > > >>> > > > >> > > > > > its
> > > > > > > >>> > > > >> > > > > > > > >> > default
> > > > > > > >>> > > > >> > > > > > > > >> > > > > value
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> is.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > We
> > > > > > > >>> > should
> > > > > > > >>> > > > >> > probably
> > > > > > > >>> > > > >> > > > also
> > > > > > > >>> > > > >> > > > > > > log a
> > > > > > > >>> > > > >> > > > > > > > >> > bunch
> > > > > > > >>> > > > >> > > > > > > > >> > > of
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> messages
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > at
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > WARN
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> level
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > when
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > > >>> something is
> > > > > > > >>> > > > >> > scheduled
> > > > > > > >>> > > > >> > > > for
> > > > > > > >>> > > > >> > > > > > > > >> deletion,
> > > > > > > >>> > > > >> > > > > > > > >> > as
> > > > > > > >>> > > > >> > > > > > > > >> > > > > well.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > (Maybe
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > this
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> was
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > > > assumed,
> > > > > > > but
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > it
> > > > > > > >>> would be
> > > > > > > >>> > > good
> > > > > > > >>> > > > >> to
> > > > > > > >>> > > > >> > > > mention
> > > > > > > >>> > > > >> > > > > > > it).
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > I
> > > > > > > >>> feel
> > > > > > > >>> > > like
> > > > > > > >>> > > > >> there
> > > > > > > >>> > > > >> > > > are a
> > > > > > > >>> > > > >> > > > > > few
> > > > > > > >>> > > > >> > > > > > > > >> > sections
> > > > > > > >>> > > > >> > > > > > > > >> > > > > that
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > should
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > be
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> moved
> to
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > > > > "rejected
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > > >>> > alternatives."
> > > > > > > >>> > > > >> For
> > > > > > > >>> > > > >> > > > > example,
> > > > > > > >>> > > > >> > > > > > in
> > > > > > > >>> > > > >> > > > > > > > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > DeleteTopics
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > section,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> since
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> we're
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > not
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > going
> > > > > > > >>> to do
> > > > > > > >>> > > > >> option 1
> > > > > > > >>> > > > >> > or
> > > > > > > >>> > > > >> > > > 2,
> > > > > > > >>> > > > >> > > > > > > these
> > > > > > > >>> > > > >> > > > > > > > >> > should
> > > > > > > >>> > > > >> > > > > > > > >> > > be
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > moved
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > into
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > "rejected
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > > >>> > alternatives,"
> > > > > > > >>> > > > >> > rather
> > > > > > > >>> > > > >> > > > than
> > > > > > > >>> > > > >> > > > > > > > >> appearing
> > > > > > > >>> > > > >> > > > > > > > >> > > > > inline.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Another
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > case
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> is
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > > > "Should
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > we
> > > > > > > >>> remove
> > > > > > > >>> > > topic
> > > > > > > >>> > > > >> name
> > > > > > > >>> > > > >> > > from
> > > > > > > >>> > > > >> > > > > the
> > > > > > > >>> > > > >> > > > > > > > >> protocol
> > > > > > > >>> > > > >> > > > > > > > >> > > > where
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > possible"
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> section.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > This
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > is
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > > clearly
> > > > > > > >>> > > > >> discussing a
> > > > > > > >>> > > > >> > > > design
> > > > > > > >>> > > > >> > > > > > > > >> > alternative
> > > > > > > >>> > > > >> > > > > > > > >> > > > that
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> we're
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > not
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > proposing
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > > >>> implement:
> > > > > > > >>> > > > >> removing
> > > > > > > >>> > > > >> > the
> > > > > > > >>> > > > >> > > > > topic
> > > > > > > >>> > > > >> > > > > > > > name
> > > > > > > >>> > > > >> > > > > > > > >> > from
> > > > > > > >>> > > > >> > > > > > > > >> > > > > those
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > protocols.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > Is
> > > > > > > >>> it
> > > > > > > >>> > > really
> > > > > > > >>> > > > >> > > > necessary
> > > > > > > >>> > > > >> > > > > to
> > > > > > > >>> > > > >> > > > > > > > have
> > > > > > > >>> > > > >> > > > > > > > >> a
> > > > > > > >>> > > > >> > > > > > > > >> > new
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > >>> > > /admin/delete_topics_by_id
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > path
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > in
> > > > > > > >>> > ZooKeeper?
> > > > > > > >>> > > > It
> > > > > > > >>> > > > >> > seems
> > > > > > > >>> > > > >> > > > > like
> > > > > > > >>> > > > >> > > > > > we
> > > > > > > >>> > > > >> > > > > > > > >> don't
> > > > > > > >>> > > > >> > > > > > > > >> > > > really
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> need
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > this.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> Whenever
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > there
> > > > > is
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > a
> > > > > > new
> > > > > > > >>> > > > controller,
> > > > > > > >>> > > > >> > we'll
> > > > > > > >>> > > > >> > > > > send
> > > > > > > >>> > > > >> > > > > > > out
> > > > > > > >>> > > > >> > > > > > > > >> full
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > LeaderAndIsrRequests
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> which
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > will
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > > trigger
> > > > > > > >>> the
> > > > > > > >>> > > > stale
> > > > > > > >>> > > > >> > > topics
> > > > > > > >>> > > > >> > > > to
> > > > > > > >>> > > > >> > > > > > be
> > > > > > > >>> > > > >> > > > > > > > >> cleaned
> > > > > > > >>> > > > >> > > > > > > > >> > > up.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > >  The
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > active
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > controller
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > will
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > also
> > > > > > > >>> send
> > > > > > > >>> > the
> > > > > > > >>> > > > full
> > > > > > > >>> > > > >> > > > > > > > >> LeaderAndIsrRequest
> > > > > > > >>> > > > >> > > > > > > > >> > > to
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> brokers
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > are
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> just
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > > > starting
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > up.
> > > > > > > >>> So we
> > > > > > > >>> > > > don't
> > > > > > > >>> > > > >> > > really
> > > > > > > >>> > > > >> > > > > > need
> > > > > > > >>> > > > >> > > > > > > > this
> > > > > > > >>> > > > >> > > > > > > > >> > kind
> > > > > > > >>> > > > >> > > > > > > > >> > > > of
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > two-phase
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > commit
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> (send
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > out
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > > >>> > > > >> StopReplicasRequest,
> > > > > > > >>> > > > >> > > get
> > > > > > > >>> > > > >> > > > > ACKs
> > > > > > > >>> > > > >> > > > > > > > from
> > > > > > > >>> > > > >> > > > > > > > >> all
> > > > > > > >>> > > > >> > > > > > > > >> > > > > nodes,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > commit
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > by
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> removing
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > > >>> > > > >> /admin/delete_topics
> > > > > > > >>> > > > >> > > > node)
> > > > > > > >>> > > > >> > > > > > any
> > > > > > > >>> > > > >> > > > > > > > >> more.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > > You
> > > > > > > >>> > > mention
> > > > > > > >>> > > > >> that
> > > > > > > >>> > > > >> > > > > > > FetchRequest
> > > > > > > >>> > > > >> > > > > > > > >> will
> > > > > > > >>> > > > >> > > > > > > > >> > > now
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> include
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > UUID
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> avoid
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > issues
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > where
> > > > > > > >>> > requests
> > > > > > > >>> > > > are
> > > > > > > >>> > > > >> > made
> > > > > > > >>> > > > >> > > > to
> > > > > > > >>> > > > >> > > > > > > stale
> > > > > > > >>> > > > >> > > > > > > > >> > > > partitions.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > However,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> adding a
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > UUID
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> to
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > > >>> > > MetadataRequest
> > > > > > > >>> > > > is
> > > > > > > >>> > > > >> > > listed
> > > > > > > >>> > > > >> > > > > as
> > > > > > > >>> > > > >> > > > > > > > future
> > > > > > > >>> > > > >> > > > > > > > >> > > work,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > out
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > of
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > scope
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > for
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> this
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > KIP.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > How
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > will
> > > > > > > the
> > > > > > > >>> > > client
> > > > > > > >>> > > > >> learn
> > > > > > > >>> > > > >> > > > what
> > > > > > > >>> > > > >> > > > > > the
> > > > > > > >>> > > > >> > > > > > > > >> topic
> > > > > > > >>> > > > >> > > > > > > > >> > > UUID
> > > > > > > >>> > > > >> > > > > > > > >> > > > > is,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > if
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > metadata
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > response
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > > doesn't
> > > > > > > >>> > > include
> > > > > > > >>> > > > >> that
> > > > > > > >>> > > > >> > > > > > > information?
> > > > > > > >>> > > > >> > > > > > > > >> It
> > > > > > > >>> > > > >> > > > > > > > >> > > > seems
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > like
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > adding
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> UUID
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > > >>> > > MetadataResponse
> > > > > > > >>> > > > >> > would
> > > > > > > >>> > > > >> > > be
> > > > > > > >>> > > > >> > > > > an
> > > > > > > >>> > > > >> > > > > > > > >> > improvement
> > > > > > > >>> > > > >> > > > > > > > >> > > > > here
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> that
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > might
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > not
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> be
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > too
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > hard
> > > > > to
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > make.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > > >>> best,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > > >>> Colin
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > On
> > > > > > > >>> Mon,
> > > > > > > >>> > > Sep
> > > > > > > >>> > > > 9,
> > > > > > > >>> > > > >> > > 2019,
> > > > > > > >>> > > > >> > > > at
> > > > > > > >>> > > > >> > > > > > > > 17:48,
> > > > > > > >>> > > > >> > > > > > > > >> > > Ryanne
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > Dolan
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > wrote:
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >>> Lucas,
> > > > > > > >>> > > > this
> > > > > > > >>> > > > >> > would
> > > > > > > >>> > > > >> > > > be
> > > > > > > >>> > > > >> > > > > > > great.
> > > > > > > >>> > > > >> > > > > > > > >> I've
> > > > > > > >>> > > > >> > > > > > > > >> > > run
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > into
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > issues
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > with
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> topics
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > being
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >>> > > > resurrected
> > > > > > > >>> > > > >> > > > > > accidentally,
> > > > > > > >>> > > > >> > > > > > > > >> since
> > > > > > > >>> > > > >> > > > > > > > >> > a
> > > > > > > >>> > > > >> > > > > > > > >> > > > > client
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > cannot
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > easily
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > > > > > distinguish
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > > between
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > a
> > > > > > > >>> > > deleted
> > > > > > > >>> > > > >> topic
> > > > > > > >>> > > > >> > > > and a
> > > > > > > >>> > > > >> > > > > > new
> > > > > > > >>> > > > >> > > > > > > > >> topic
> > > > > > > >>> > > > >> > > > > > > > >> > > with
> > > > > > > >>> > > > >> > > > > > > > >> > > > > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> same
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > name.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I'd
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > need
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > ID
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >>> > > accessible
> > > > > > > >>> > > > >> from
> > > > > > > >>> > > > >> > > the
> > > > > > > >>> > > > >> > > > > > > client
> > > > > > > >>> > > > >> > > > > > > > to
> > > > > > > >>> > > > >> > > > > > > > >> > > solve
> > > > > > > >>> > > > >> > > > > > > > >> > > > > that
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > issue,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > but
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> this
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > is a
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > good
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > first
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >>> step.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >>> Ryanne
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > On
> > > > > > > >>> > Wed,
> > > > > > > >>> > > > Sep
> > > > > > > >>> > > > >> 4,
> > > > > > > >>> > > > >> > > 2019
> > > > > > > >>> > > > >> > > > > at
> > > > > > > >>> > > > >> > > > > > > 1:41
> > > > > > > >>> > > > >> > > > > > > > >> PM
> > > > > > > >>> > > > >> > > > > > > > >> > > Lucas
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Bradstreet <
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > > > > > >>> > > lucas@confluent.io>
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > wrote:
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > >>> Hi
> > > > > > > >>> > > all,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > >>> I
> > > > > > > >>> > > would
> > > > > > > >>> > > > >> like
> > > > > > > >>> > > > >> > to
> > > > > > > >>> > > > >> > > > > kick
> > > > > > > >>> > > > >> > > > > > > off
> > > > > > > >>> > > > >> > > > > > > > >> > > > discussion
> > > > > > > >>> > > > >> > > > > > > > >> > > > > of
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > KIP-516,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > an
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > > > > > >>> implementation
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > of
> > > > > > > topic
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > >>> IDs
> > > > > > > >>> > > for
> > > > > > > >>> > > > >> > Kafka.
> > > > > > > >>> > > > >> > > > > Topic
> > > > > > > >>> > > > >> > > > > > > IDs
> > > > > > > >>> > > > >> > > > > > > > >> aim
> > > > > > > >>> > > > >> > > > > > > > >> > to
> > > > > > > >>> > > > >> > > > > > > > >> > > > > solve
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> topic
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > uniqueness
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > > > problems
> > > > > > > in
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > Kafka,
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > >>> > where
> > > > > > > >>> > > > >> > referring
> > > > > > > >>> > > > >> > > > to
> > > > > > > >>> > > > >> > > > > a
> > > > > > > >>> > > > >> > > > > > > > topic
> > > > > > > >>> > > > >> > > > > > > > >> by
> > > > > > > >>> > > > >> > > > > > > > >> > > name
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> alone is
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > insufficient.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > Such
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > cases
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > >>> > > include
> > > > > > > >>> > > > >> when
> > > > > > > >>> > > > >> > a
> > > > > > > >>> > > > >> > > > > topic
> > > > > > > >>> > > > >> > > > > > > has
> > > > > > > >>> > > > >> > > > > > > > >> been
> > > > > > > >>> > > > >> > > > > > > > >> > > > > deleted
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > recreated
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> with
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > the
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > > same
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > name.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > >>> > Unique
> > > > > > > >>> > > > >> > > > identifiers
> > > > > > > >>> > > > >> > > > > > will
> > > > > > > >>> > > > >> > > > > > > > >> help
> > > > > > > >>> > > > >> > > > > > > > >> > > > > simplify
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > improve
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> Kafka's
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > topic
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > > deletion
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > >>> > > process,
> > > > > > > >>> > > > >> as
> > > > > > > >>> > > > >> > > well
> > > > > > > >>> > > > >> > > > as
> > > > > > > >>> > > > >> > > > > > > > prevent
> > > > > > > >>> > > > >> > > > > > > > >> > > cases
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > where
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > brokers
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > may
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > > incorrectly
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > >
> > > > > > > interact
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > >>> with
> > > > > > > >>> > > > stale
> > > > > > > >>> > > > >> > > > versions
> > > > > > > >>> > > > >> > > > > > of
> > > > > > > >>> > > > >> > > > > > > > >> topics.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > >>> > > > >> > > > > > > >
> > > > > > > >>> > > > >> > > > > > >
> > > > > > > >>> > > > >> > > > > >
> > > > > > > >>> > > > >> > > > >
> > > > > > > >>> > > > >> > > >
> > > > > > > >>> > > > >> > >
> > > > > > > >>> > > > >> >
> > > > > > > >>> > > > >>
> > > > > > > >>> > > >
> > > > > > > >>> > >
> > > > > > > >>> >
> > > > > > > >>>
> > > > > > >
> > > > > >
> > > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > >>> > > Looking
> > > > > > > >>> > > > >> > forward
> > > > > > > >>> > > > >> > > > to
> > > > > > > >>> > > > >> > > > > > your
> > > > > > > >>> > > > >> > > > > > > > >> > > thoughts.
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > >>> > Lucas
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >
> > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > > >>> > > > >> > > > > > > > >> > > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > > >>> > > > >> > > > > > > > >>
> > > > > > > >>> > > > >> > > > > > > > >
> > > > > > > >>> > > > >> > > > > > > >
> > > > > > > >>> > > > >> > > > > > >
> > > > > > > >>> > > > >> > > > > >
> > > > > > > >>> > > > >> > > > >
> > > > > > > >>> > > > >> > > >
> > > > > > > >>> > > > >> > >
> > > > > > > >>> > > > >> >
> > > > > > > >>> > > > >>
> > > > > > > >>> > > > >
> > > > > > > >>> > > >
> > > > > > > >>> > >
> > > > > > > >>> >
> > > > > > > >>>
> > > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > >
> > >
> >
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Jun Rao <ju...@confluent.io>]

Hi, Justine,

I was just wondering if we expose topicId publicly, how would people use
it. One possibility is to use topicId in all those public interfaces where
we use topic name. I am not sure if the need is very strong though. So,
maybe we can list this as future considerations.

Thanks,

Jun

On Wed, Oct 7, 2020 at 8:57 AM Justine Olshan <jo...@confluent.io> wrote:

> Hi all,
>
> After some careful consideration, I've decided that the TopicIdPartition
> will be out of scope for this KIP. After investigating what it will look
> like using it in only the Fetch Request, I found that I would need to
> modify several public APIs in the consumer and I would need to take special
> care in many situations to handle the request correctly based on what
> version of the request was sent. For these reasons, I've added
> TopicIdPartition to potential future work along with an alternative to
> modify TopicPartition to include a topic ID field.
>
> Jun- I was also wondering about some of the details for supporting topic
> IDs for delete and describe topics in the admin client. Is the idea that
> instead of supplying topic names to describe or delete, we would supply a
> collection of ids? If that is the case, I'm wondering if it makes sense to
> add topic Ids to the metadata and delete topics requests.
>
> Thanks,
> Justine
>
> On Mon, Oct 5, 2020 at 9:25 PM Satish Duggana <sa...@gmail.com>
> wrote:
>
> > Hi Jun, Justine,
> > +1 to have TopicIdPartition in public packages. This is useful for
> > KIP-405 tiered storage too. We currently use TopicPartition for
> > pluggable log storage and metadata storage apis. We will be using
> > TopicIdPartition once the proposed changes in this KIP are merged.
> >
> > Thanks,
> > Satish.
> >
> > On Tue, Oct 6, 2020 at 5:21 AM Jun Rao <ju...@confluent.io> wrote:
> > >
> > > Hi, Justine,
> > >
> > > I guess we have to maintain the mapping between topic name and topicId
> in
> > > both the client and the server. If TopicIdPartition is meant for
> internal
> > > usage only, we can create it under a package
> > > like org/apache/kafka/common/internal and we don't need to document it
> in
> > > the KIP. If it's meant for external public usage (and potentially
> > internal
> > > usage too), we can create it in org/apache/kafka/common and need to
> > > document it in the KIP.
> > >
> > > Since topicId is now part of the topic metadata, perhaps it's useful to
> > > expose it in TopicMetadataAndConfig too.
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Mon, Oct 5, 2020 at 3:35 PM Justine Olshan <jo...@confluent.io>
> > wrote:
> > >
> > > > Hi Jun,
> > > >
> > > > This KIP removes the topic name from many requests. A lot of requests
> > keep
> > > > track of the partitions internally with a TopicPartition. So when we
> > remove
> > > > the topic name, we can not differentiate unless we can also use topic
> > ID.
> > > > The Fetch request relies heavily on this, but ListOffset,
> > > > OffsetsForLeaderEpoch, etc all use TopicPartition as a key value for
> > > > internal maps. Digging deeper into Fetch, I've noticed that the code
> > path
> > > > runs into some public APIs that use the TopicPartition. Some examples
> > are
> > > > Cluster.java or RecordTooLargeException.
> > > >
> > > > A related issue is that in some places like
> > TopicAuthorizationException,
> > > > the topic name is required and without the name in the request, we
> > won't
> > > > know it.
> > > >
> > > > I guess an alternative would be to have some mapping from topic name
> to
> > > > topic ID, but I would have to think about that more.
> > > >
> > > > The KIP mentions exposing TopicId in TopicDescription. (The font was
> > > > weirdly colored, so I've made it easier to read.)
> > > >
> > > > Would the idea for the TopicMetadataAndConfig be that the topic ID
> > would be
> > > > returned as a result?
> > > > As for the other methods describe/deletetopics, it makes sense to
> > include
> > > > them at some point, but I'm not sure how far we should go in terms of
> > > > supporting topic IDs everywhere.
> > > >
> > > > Thank you for continuing to look at the KIP!
> > > > Justine
> > > >
> > > >
> > > > On Mon, Oct 5, 2020 at 2:53 PM Jun Rao <ju...@confluent.io> wrote:
> > > >
> > > > > Hi, Justine,
> > > > >
> > > > > Adding a TopicIdPartition in the public interface may be useful.
> > Where
> > > > > would you expect it to be used?
> > > > >
> > > > > Related to this, since we are exposing topicId in the tool, it
> seems
> > that
> > > > > we may want to expose topicId in at least TopicDescription used by
> > > > > AdminClient.describeTopics() and TopicMetadataAndConfig used in
> > > > > AdminClient.createTopics(Collection<NewTopic> newTopics). I am also
> > not
> > > > > sure if we plan to expose topicId in other AdminClient methods in
> > this
> > > > KIP
> > > > > or in some future KIP. For example, it might be useful to support
> > topicId
> > > > > in the following methods.
> > > > >
> > > > >     DescribeTopicsResult describeTopics(Collection<String>
> > topicNames,
> > > > > DescribeTopicsOptions options);
> > > > >     DeleteTopicsResult deleteTopics(Collection<String> topics,
> > > > > DeleteTopicsOptions options);
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Jun
> > > > >
> > > > > On Mon, Oct 5, 2020 at 11:03 AM Justine Olshan <
> jolshan@confluent.io
> > >
> > > > > wrote:
> > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > I was thinking about this KIP some more. We are removing the
> topic
> > name
> > > > > > from many of these request types, and some of them (eg. Fetch)
> use
> > > > > > TopicPartition as keys in a map when handling the request. I was
> > > > thinking
> > > > > > it would make sense to make some sort of TopicIdPartition with
> > topic
> > > > > name,
> > > > > > topic id, and partition fields (name and id would potentially be
> > > > optional
> > > > > > types since different request versions will contain only one of
> the
> > > > two)
> > > > > I
> > > > > > was wondering if it made sense to include this as a public facing
> > API
> > > > > > similar to TopicPartition. If so, I will add this class to the
> KIP
> > as
> > > > > well.
> > > > > >
> > > > > > Thanks,
> > > > > > Justine
> > > > > >
> > > > > > On Mon, Oct 5, 2020 at 8:52 AM Justine Olshan <
> > jolshan@confluent.io>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi all,
> > > > > > > I've made a few minor changes to the UUID class. I've added a
> few
> > > > more
> > > > > > > methods to support UUIDs more generally, by including a
> > constructor
> > > > > and a
> > > > > > > method to generate random UUIDs. Both of these will be useful
> for
> > > > > > testing.
> > > > > > >
> > > > > > > Let me know if there are any other areas for discussion.
> > > > > > >
> > > > > > > Thank you,
> > > > > > > Justine
> > > > > > >
> > > > > > > On Thu, Oct 1, 2020 at 5:29 PM Justine Olshan <
> > jolshan@confluent.io>
> > > > > > > wrote:
> > > > > > >
> > > > > > >> Hi Jun,
> > > > > > >> Thanks for looking at it again. Here's the new spec. (I fixed
> > the
> > > > typo
> > > > > > in
> > > > > > >> it too.)
> > > > > > >> {"name": "id", "type": "string", "doc": option[UUID]}
> > > > > > >>
> > > > > > >> Justine
> > > > > > >>
> > > > > > >>
> > > > > > >> On Thu, Oct 1, 2020 at 5:03 PM Jun Rao <ju...@confluent.io>
> > wrote:
> > > > > > >>
> > > > > > >>> Hi, Justine,
> > > > > > >>>
> > > > > > >>> Thanks for the update. The KIP looks good to me now. Just a
> > minor
> > > > > > comment
> > > > > > >>> below.
> > > > > > >>>
> > > > > > >>> 30. Perhaps "option[UUID]" can be put in the doc.
> > > > > > >>>
> > > > > > >>> Jun
> > > > > > >>>
> > > > > > >>> On Thu, Oct 1, 2020 at 3:28 PM Justine Olshan <
> > > > jolshan@confluent.io>
> > > > > > >>> wrote:
> > > > > > >>>
> > > > > > >>> > Hi Jun,
> > > > > > >>> > Thanks for the response!
> > > > > > >>> >
> > > > > > >>> > 30. I think I might have changed this in between. The
> current
> > > > > version
> > > > > > >>> > says:  {"name":
> > > > > > >>> > "id", "type": "option[UUID]"}, "doc": topic id}
> > > > > > >>> > I have switched to the option type to cover the migration
> > case
> > > > > where
> > > > > > a
> > > > > > >>> > TopicZNode does not yet have a topic ID.
> > > > > > >>> > I understand that due to json, this field is written as a
> > string,
> > > > > so
> > > > > > >>> if I
> > > > > > >>> > should move the "option[uuid]" to the doc field and the
> type
> > > > should
> > > > > > be
> > > > > > >>> > "string" please let me know.
> > > > > > >>> >
> > > > > > >>> > 40. I've added a definition for UUID.
> > > > > > >>> > 41,42. Fixed
> > > > > > >>> >
> > > > > > >>> > Thank you,
> > > > > > >>> > Justine
> > > > > > >>> >
> > > > > > >>> > On Wed, Sep 30, 2020 at 1:15 PM Jun Rao <ju...@confluent.io>
> > > > wrote:
> > > > > > >>> >
> > > > > > >>> > > Hi, Justine,
> > > > > > >>> > >
> > > > > > >>> > > Thanks for the summary. Just a few minor comments blow.
> > > > > > >>> > >
> > > > > > >>> > > 30.  {"name": "id", "type": "string", "doc": "version
> > id"}}:
> > > > The
> > > > > > doc
> > > > > > >>> > should
> > > > > > >>> > > say UUID. The issue seems unfixed.
> > > > > > >>> > >
> > > > > > >>> > > 40. Since UUID is public facing, could you include its
> > > > > definition?
> > > > > > >>> > >
> > > > > > >>> > > 41. StopReplicaResponse still includes the topic field.
> > > > > > >>> > >
> > > > > > >>> > > 42. "It is unnecessary to include the name of the topic
> in
> > the
> > > > > > >>> following
> > > > > > >>> > > Request/Response calls" It would be useful to include all
> > > > > modified
> > > > > > >>> > requests
> > > > > > >>> > > (e.g. produce) in the list below.
> > > > > > >>> > >
> > > > > > >>> > > Thanks,
> > > > > > >>> > >
> > > > > > >>> > > Jun
> > > > > > >>> > >
> > > > > > >>> > >
> > > > > > >>> > > On Wed, Sep 30, 2020 at 10:17 AM Justine Olshan <
> > > > > > >>> jolshan@confluent.io>
> > > > > > >>> > > wrote:
> > > > > > >>> > >
> > > > > > >>> > > > Hello again,
> > > > > > >>> > > >
> > > > > > >>> > > > I've taken some time to discuss some of the remaining
> > points
> > > > > > >>> brought up
> > > > > > >>> > > by
> > > > > > >>> > > > the previous emails offline. Here are some of the
> > > > conclusions.
> > > > > > >>> > > >
> > > > > > >>> > > > 1. Directory Structure:
> > > > > > >>> > > > There was some talk about whether the directory
> structure
> > > > > should
> > > > > > be
> > > > > > >>> > > changed
> > > > > > >>> > > > to replace all topic names with topic IDs.
> > > > > > >>> > > > This will be a useful change, but will prevent
> > downgrades. It
> > > > > > will
> > > > > > >>> be
> > > > > > >>> > > best
> > > > > > >>> > > > to wait until a major release, likely alongside KIP-500
> > > > changes
> > > > > > >>> that
> > > > > > >>> > will
> > > > > > >>> > > > prevent downgrades. I've updated the KIP to include
> this
> > > > change
> > > > > > >>> with
> > > > > > >>> > the
> > > > > > >>> > > > note about migration and deprecation.
> > > > > > >>> > > >
> > > > > > >>> > > >
> > > > > > >>> > >
> > > > > > >>> >
> > > > > > >>>
> > > > > >
> > > > >
> > > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-log.dirlayout
> > > > > > >>> > > >
> > > > > > >>> > > > 2. Partition Metadata file
> > > > > > >>> > > > There was some disagreement about the inclusion of the
> > > > > partition
> > > > > > >>> > metadata
> > > > > > >>> > > > file.
> > > > > > >>> > > > This file will be needed to persist the topic ID,
> > especially
> > > > > > while
> > > > > > >>> we
> > > > > > >>> > > still
> > > > > > >>> > > > have the old directory structure. Even after the
> changes,
> > > > this
> > > > > > >>> file can
> > > > > > >>> > > be
> > > > > > >>> > > > useful for debugging and recovery.
> > > > > > >>> > > > Creating a single mapping file from topic names to
> topic
> > IDs
> > > > > was
> > > > > > >>> > > > considered, but ultimately scrapped as it would not be
> as
> > > > easy
> > > > > to
> > > > > > >>> > > maintain.
> > > > > > >>> > > >
> > > > > > >>> > > >
> > > > > > >>> > >
> > > > > > >>> >
> > > > > > >>>
> > > > > >
> > > > >
> > > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PartitionMetadatafile
> > > > > > >>> > > >
> > > > > > >>> > > >
> > > > > > >>> > > >
> > > > > > >>> > >
> > > > > > >>> >
> > > > > > >>>
> > > > > >
> > > > >
> > > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PersistingTopicIDs
> > > > > > >>> > > >
> > > > > > >>> > > > 3. Produce Protocols
> > > > > > >>> > > > After some further discussion, this replacing the topic
> > name
> > > > > with
> > > > > > >>> topic
> > > > > > >>> > > ID
> > > > > > >>> > > > in these protocols has been added to the KIP.
> > > > > > >>> > > > This will cut down on the size of the protocol. Since
> > changes
> > > > > to
> > > > > > >>> fetch
> > > > > > >>> > > are
> > > > > > >>> > > > included, it does make sense to update these protocols.
> > > > > > >>> > > >
> > > > > > >>> > > >
> > > > > > >>> > >
> > > > > > >>> >
> > > > > > >>>
> > > > > >
> > > > >
> > > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-Produce
> > > > > > >>> > > >
> > > > > > >>> > > > 4. KIP-500 Compatibility
> > > > > > >>> > > > After some discussion with those involved with KIP-500,
> > it
> > > > > seems
> > > > > > >>> best
> > > > > > >>> > to
> > > > > > >>> > > > use a sentinel topic ID for the metadata topic that is
> > used
> > > > > > before
> > > > > > >>> the
> > > > > > >>> > > > first controller election.
> > > > > > >>> > > > We had an issue where this metadata topic may not be
> > assigned
> > > > > an
> > > > > > ID
> > > > > > >>> > > before
> > > > > > >>> > > > utilizing the Vote and Fetch protocols. It was decided
> to
> > > > > > reserve a
> > > > > > >>> > > unique
> > > > > > >>> > > > ID for this topic to be used until the controller could
> > give
> > > > > the
> > > > > > >>> topic
> > > > > > >>> > a
> > > > > > >>> > > > unique ID.
> > > > > > >>> > > > Having the topic keep the sentinel ID (not be
> reassigned
> > to a
> > > > > > >>> unique
> > > > > > >>> > ID)
> > > > > > >>> > > > was considered, but it seemed like a good idea to leave
> > the
> > > > > > option
> > > > > > >>> open
> > > > > > >>> > > for
> > > > > > >>> > > > the metadata topic to have a unique ID in cases where
> it
> > > > would
> > > > > > >>> need to
> > > > > > >>> > be
> > > > > > >>> > > > differentiated from other clusters' metadata topics.
> (ex.
> > > > > tiered
> > > > > > >>> > storage)
> > > > > > >>> > > >
> > > > > > >>> > > >
> > > > > > >>> > >
> > > > > > >>> >
> > > > > > >>>
> > > > > >
> > > > >
> > > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-CompatibilitywithKIP-500
> > > > > > >>> > > >
> > > > > > >>> > > > I've also split up the KIP into sub-tasks on the JIRA.
> > > > > Hopefully
> > > > > > >>> this
> > > > > > >>> > > will
> > > > > > >>> > > > provide a better idea about what tasks we have, and
> > > > eventually
> > > > > > >>> provide
> > > > > > >>> > a
> > > > > > >>> > > > place to see what's done and what is left.
> > > > > > >>> > > > If there is a task I am missing, please let me know!
> > > > > > >>> > > > https://issues.apache.org/jira/browse/KAFKA-8872
> > > > > > >>> > > >
> > > > > > >>> > > > Of course, these decisions are not set in stone, and I
> > would
> > > > > love
> > > > > > >>> to
> > > > > > >>> > hear
> > > > > > >>> > > > any feedback.
> > > > > > >>> > > >
> > > > > > >>> > > > Thanks,
> > > > > > >>> > > > Justine
> > > > > > >>> > > >
> > > > > > >>> > > > On Mon, Sep 28, 2020 at 11:38 AM Justine Olshan <
> > > > > > >>> jolshan@confluent.io>
> > > > > > >>> > > > wrote:
> > > > > > >>> > > >
> > > > > > >>> > > > > Hello all,
> > > > > > >>> > > > >
> > > > > > >>> > > > > I just wanted to follow up on this discussion. Did we
> > come
> > > > to
> > > > > > an
> > > > > > >>> > > > > understanding about the directory structure?
> > > > > > >>> > > > >
> > > > > > >>> > > > > I think the biggest question here is what is
> > acceptable to
> > > > > > leave
> > > > > > >>> out
> > > > > > >>> > > due
> > > > > > >>> > > > > to scope vs. what is considered to be too much tech
> > debt.
> > > > > > >>> > > > > This KIP is already pretty large with the number of
> > > > changes,
> > > > > > but
> > > > > > >>> it
> > > > > > >>> > > also
> > > > > > >>> > > > > makes sense to do things correctly, so I'd love to
> hear
> > > > > > >>> everyone's
> > > > > > >>> > > > thoughts.
> > > > > > >>> > > > >
> > > > > > >>> > > > > Thanks,
> > > > > > >>> > > > > Justine
> > > > > > >>> > > > >
> > > > > > >>> > > > > On Fri, Sep 25, 2020 at 8:19 AM Lucas Bradstreet <
> > > > > > >>> lucas@confluent.io
> > > > > > >>> > >
> > > > > > >>> > > > > wrote:
> > > > > > >>> > > > >
> > > > > > >>> > > > >> Hi Ismael,
> > > > > > >>> > > > >>
> > > > > > >>> > > > >> If you do not store it in a metadata file or in the
> > > > > directory
> > > > > > >>> > > structure
> > > > > > >>> > > > >> would you then
> > > > > > >>> > > > >> require the LeaderAndIsrRequest following startup to
> > give
> > > > > you
> > > > > > >>> some
> > > > > > >>> > > > notion
> > > > > > >>> > > > >> of
> > > > > > >>> > > > >> topic name in memory? We would still need this
> > information
> > > > > for
> > > > > > >>> the
> > > > > > >>> > > older
> > > > > > >>> > > > >> protocols, but
> > > > > > >>> > > > >> perhaps this is what's meant by tech debt.
> > > > > > >>> > > > >>
> > > > > > >>> > > > >> Once we're free of the old non-topicID requests
> then I
> > > > think
> > > > > > you
> > > > > > >>> > > > wouldn't
> > > > > > >>> > > > >> need to retain the topic name.
> > > > > > >>> > > > >> I think the ability to easily look up topic names
> > > > associated
> > > > > > >>> with
> > > > > > >>> > > > >> partition
> > > > > > >>> > > > >> directories would still be missed
> > > > > > >>> > > > >> when diagnosing issues, though maybe it wouldn't be
> a
> > deal
> > > > > > >>> breaker
> > > > > > >>> > > with
> > > > > > >>> > > > >> the
> > > > > > >>> > > > >> right tooling.
> > > > > > >>> > > > >>
> > > > > > >>> > > > >> Thanks,
> > > > > > >>> > > > >>
> > > > > > >>> > > > >> Lucas
> > > > > > >>> > > > >>
> > > > > > >>> > > > >> On Fri, Sep 25, 2020 at 7:55 AM Ismael Juma <
> > > > > > ismael@juma.me.uk>
> > > > > > >>> > > wrote:
> > > > > > >>> > > > >>
> > > > > > >>> > > > >> > Hi Lucas,
> > > > > > >>> > > > >> >
> > > > > > >>> > > > >> > Why would you include the name and id? I think
> you'd
> > > > want
> > > > > to
> > > > > > >>> > remove
> > > > > > >>> > > > the
> > > > > > >>> > > > >> > name from the directory name right? Jason's
> > suggestion
> > > > was
> > > > > > >>> that if
> > > > > > >>> > > you
> > > > > > >>> > > > >> > remove the name from the directory, then why would
> > you
> > > > > need
> > > > > > >>> the id
> > > > > > >>> > > > name
> > > > > > >>> > > > >> > mapping file?
> > > > > > >>> > > > >> >
> > > > > > >>> > > > >> > Ismael
> > > > > > >>> > > > >> >
> > > > > > >>> > > > >> > On Thu, Sep 24, 2020 at 4:24 PM Lucas Bradstreet <
> > > > > > >>> > > lucas@confluent.io>
> > > > > > >>> > > > >> > wrote:
> > > > > > >>> > > > >> >
> > > > > > >>> > > > >> > > > 2. Part of the usage of the file is to have
> > > > persistent
> > > > > > >>> storage
> > > > > > >>> > > of
> > > > > > >>> > > > >> the
> > > > > > >>> > > > >> > > topic
> > > > > > >>> > > > >> > > ID and use it to compare with the ID supplied in
> > the
> > > > > > >>> > LeaderAndIsr
> > > > > > >>> > > > >> > Request.
> > > > > > >>> > > > >> > > There is some discussion in the KIP about
> changes
> > to
> > > > the
> > > > > > >>> > directory
> > > > > > >>> > > > >> > > structure, but I believe directory changes were
> > > > > considered
> > > > > > >>> to be
> > > > > > >>> > > out
> > > > > > >>> > > > >> of
> > > > > > >>> > > > >> > > scope when the KIP was written.
> > > > > > >>> > > > >> > >
> > > > > > >>> > > > >> > >
> > > > > > >>> > > > >> > > Yeah, I was hoping to get a better understanding
> > of
> > > > why
> > > > > it
> > > > > > >>> was
> > > > > > >>> > > taken
> > > > > > >>> > > > >> out
> > > > > > >>> > > > >> > of
> > > > > > >>> > > > >> > > scope. Perhaps Lucas Bradstreet might have more
> > > > insight
> > > > > > >>> about
> > > > > > >>> > the
> > > > > > >>> > > > >> > decision.
> > > > > > >>> > > > >> > > Basically my point is that we have to create
> > > > additional
> > > > > > >>> > > > infrastructure
> > > > > > >>> > > > >> > here
> > > > > > >>> > > > >> > > to support the name/id mapping, so I wanted to
> > > > > understand
> > > > > > >>> if we
> > > > > > >>> > > just
> > > > > > >>> > > > >> > > consider this a sort of tech debt. It would be
> > useful
> > > > to
> > > > > > >>> cover
> > > > > > >>> > how
> > > > > > >>> > > > we
> > > > > > >>> > > > >> > > handle the case when this file gets corrupted.
> > Seems
> > > > > like
> > > > > > we
> > > > > > >>> > just
> > > > > > >>> > > > >> have to
> > > > > > >>> > > > >> > > trust that it matches whatever the controller
> > tells us
> > > > > and
> > > > > > >>> > rewrite
> > > > > > >>> > > > it?
> > > > > > >>> > > > >> > >
> > > > > > >>> > > > >> > >
> > > > > > >>> > > > >> > > Hi Jason, Justine,
> > > > > > >>> > > > >> > >
> > > > > > >>> > > > >> > > My thought process is that we will likely need
> the
> > > > > > metadata
> > > > > > >>> file
> > > > > > >>> > > > >> whether
> > > > > > >>> > > > >> > we
> > > > > > >>> > > > >> > > rename the directories or not.
> > > > > > >>> > > > >> > > Linux supports filenames of up to 255 bytes and
> > that
> > > > > would
> > > > > > >>> not
> > > > > > >>> > be
> > > > > > >>> > > > >> enough
> > > > > > >>> > > > >> > to
> > > > > > >>> > > > >> > > support a directory name
> > > > > > >>> > > > >> > >  including both the name and topic ID. Given
> that
> > > > fact,
> > > > > it
> > > > > > >>> > seemed
> > > > > > >>> > > > >> > > reasonable to add the metadata file
> > > > > > >>> > > > >> > > and leave the directory rename until some time
> in
> > the
> > > > > > future
> > > > > > >>> > > > (possibly
> > > > > > >>> > > > >> > > never).
> > > > > > >>> > > > >> > >
> > > > > > >>> > > > >> > > If the file does get corrupted I think you're
> > right.
> > > > We
> > > > > > >>> would
> > > > > > >>> > > either
> > > > > > >>> > > > >> have
> > > > > > >>> > > > >> > > to trust it matches what the controller tells us
> > > > > > >>> > > > >> > >  or error out and let an administrator resolve
> it
> > by
> > > > > > >>> checking
> > > > > > >>> > > across
> > > > > > >>> > > > >> > > replicas for consistency.
> > > > > > >>> > > > >> > >
> > > > > > >>> > > > >> > > Lucas
> > > > > > >>> > > > >> > >
> > > > > > >>> > > > >> > >
> > > > > > >>> > > > >> > > On Thu, Sep 24, 2020 at 3:41 PM Jason Gustafson
> <
> > > > > > >>> > > jason@confluent.io
> > > > > > >>> > > > >
> > > > > > >>> > > > >> > > wrote:
> > > > > > >>> > > > >> > >
> > > > > > >>> > > > >> > > > Thanks Justine. Responses below:
> > > > > > >>> > > > >> > > >
> > > > > > >>> > > > >> > > > > 1. Yes, the directory will still be based on
> > the
> > > > > topic
> > > > > > >>> > names.
> > > > > > >>> > > > >> > > > LeaderAndIsrRequest is one of the few requests
> > that
> > > > > will
> > > > > > >>> still
> > > > > > >>> > > > >> contain
> > > > > > >>> > > > >> > > the
> > > > > > >>> > > > >> > > > topic name. So I think we have this covered.
> > Sorry
> > > > for
> > > > > > >>> > > confusion.
> > > > > > >>> > > > >> > > >
> > > > > > >>> > > > >> > > > Ah, you're right. My eyes passed right over
> the
> > > > field.
> > > > > > >>> > > > >> > > >
> > > > > > >>> > > > >> > > > > 2. Part of the usage of the file is to have
> > > > > persistent
> > > > > > >>> > storage
> > > > > > >>> > > > of
> > > > > > >>> > > > >> the
> > > > > > >>> > > > >> > > > topic
> > > > > > >>> > > > >> > > > ID and use it to compare with the ID supplied
> > in the
> > > > > > >>> > > LeaderAndIsr
> > > > > > >>> > > > >> > > Request.
> > > > > > >>> > > > >> > > > There is some discussion in the KIP about
> > changes to
> > > > > the
> > > > > > >>> > > directory
> > > > > > >>> > > > >> > > > structure, but I believe directory changes
> were
> > > > > > >>> considered to
> > > > > > >>> > be
> > > > > > >>> > > > >> out of
> > > > > > >>> > > > >> > > > scope when the KIP was written.
> > > > > > >>> > > > >> > > >
> > > > > > >>> > > > >> > > > Yeah, I was hoping to get a better
> > understanding of
> > > > > why
> > > > > > >>> it was
> > > > > > >>> > > > taken
> > > > > > >>> > > > >> > out
> > > > > > >>> > > > >> > > of
> > > > > > >>> > > > >> > > > scope. Perhaps Lucas Bradstreet might have
> more
> > > > > insight
> > > > > > >>> about
> > > > > > >>> > > the
> > > > > > >>> > > > >> > > decision.
> > > > > > >>> > > > >> > > > Basically my point is that we have to create
> > > > > additional
> > > > > > >>> > > > >> infrastructure
> > > > > > >>> > > > >> > > here
> > > > > > >>> > > > >> > > > to support the name/id mapping, so I wanted to
> > > > > > understand
> > > > > > >>> if
> > > > > > >>> > we
> > > > > > >>> > > > just
> > > > > > >>> > > > >> > > > consider this a sort of tech debt. It would be
> > > > useful
> > > > > to
> > > > > > >>> cover
> > > > > > >>> > > how
> > > > > > >>> > > > >> we
> > > > > > >>> > > > >> > > > handle the case when this file gets corrupted.
> > Seems
> > > > > > like
> > > > > > >>> we
> > > > > > >>> > > just
> > > > > > >>> > > > >> have
> > > > > > >>> > > > >> > to
> > > > > > >>> > > > >> > > > trust that it matches whatever the controller
> > tells
> > > > us
> > > > > > and
> > > > > > >>> > > rewrite
> > > > > > >>> > > > >> it?
> > > > > > >>> > > > >> > > >
> > > > > > >>> > > > >> > > > > 3. I think this is a good point, but I
> again I
> > > > > wonder
> > > > > > >>> about
> > > > > > >>> > > the
> > > > > > >>> > > > >> scope
> > > > > > >>> > > > >> > > of
> > > > > > >>> > > > >> > > > the KIP. Most of the changes mentioned in the
> > KIP
> > > > are
> > > > > > for
> > > > > > >>> > > > supporting
> > > > > > >>> > > > >> > > topic
> > > > > > >>> > > > >> > > > deletion and I believe that is why the produce
> > > > request
> > > > > > was
> > > > > > >>> > > listed
> > > > > > >>> > > > >> under
> > > > > > >>> > > > >> > > > future work.
> > > > > > >>> > > > >> > > >
> > > > > > >>> > > > >> > > > That's fair. I brought it up since `Fetch` is
> > > > already
> > > > > > >>> > included.
> > > > > > >>> > > If
> > > > > > >>> > > > >> > we've
> > > > > > >>> > > > >> > > > got `Metadata` and `Fetch`, seems we may as
> > well do
> > > > > > >>> `Produce`
> > > > > > >>> > > and
> > > > > > >>> > > > >> save
> > > > > > >>> > > > >> > an
> > > > > > >>> > > > >> > > > extra kip. No strong objection though if you
> > want to
> > > > > > >>> leave it
> > > > > > >>> > > out.
> > > > > > >>> > > > >> > > >
> > > > > > >>> > > > >> > > >
> > > > > > >>> > > > >> > > > -Jason
> > > > > > >>> > > > >> > > >
> > > > > > >>> > > > >> > > >
> > > > > > >>> > > > >> > > > On Thu, Sep 24, 2020 at 3:26 PM Justine
> Olshan <
> > > > > > >>> > > > >> jolshan@confluent.io>
> > > > > > >>> > > > >> > > > wrote:
> > > > > > >>> > > > >> > > >
> > > > > > >>> > > > >> > > > > Hi Jason,
> > > > > > >>> > > > >> > > > >
> > > > > > >>> > > > >> > > > > Thanks for your comments.
> > > > > > >>> > > > >> > > > >
> > > > > > >>> > > > >> > > > > 1. Yes, the directory will still be based on
> > the
> > > > > topic
> > > > > > >>> > names.
> > > > > > >>> > > > >> > > > > LeaderAndIsrRequest is one of the few
> requests
> > > > that
> > > > > > will
> > > > > > >>> > still
> > > > > > >>> > > > >> > contain
> > > > > > >>> > > > >> > > > the
> > > > > > >>> > > > >> > > > > topic name. So I think we have this covered.
> > Sorry
> > > > > for
> > > > > > >>> > > > confusion.
> > > > > > >>> > > > >> > > > >
> > > > > > >>> > > > >> > > > > 2. Part of the usage of the file is to have
> > > > > persistent
> > > > > > >>> > storage
> > > > > > >>> > > > of
> > > > > > >>> > > > >> the
> > > > > > >>> > > > >> > > > topic
> > > > > > >>> > > > >> > > > > ID and use it to compare with the ID
> supplied
> > in
> > > > the
> > > > > > >>> > > > LeaderAndIsr
> > > > > > >>> > > > >> > > > Request.
> > > > > > >>> > > > >> > > > > There is some discussion in the KIP about
> > changes
> > > > to
> > > > > > the
> > > > > > >>> > > > directory
> > > > > > >>> > > > >> > > > > structure, but I believe directory changes
> > were
> > > > > > >>> considered
> > > > > > >>> > to
> > > > > > >>> > > be
> > > > > > >>> > > > >> out
> > > > > > >>> > > > >> > of
> > > > > > >>> > > > >> > > > > scope when the KIP was written.
> > > > > > >>> > > > >> > > > >
> > > > > > >>> > > > >> > > > > 3. I think this is a good point, but I
> again I
> > > > > wonder
> > > > > > >>> about
> > > > > > >>> > > the
> > > > > > >>> > > > >> scope
> > > > > > >>> > > > >> > > of
> > > > > > >>> > > > >> > > > > the KIP. Most of the changes mentioned in
> the
> > KIP
> > > > > are
> > > > > > >>> for
> > > > > > >>> > > > >> supporting
> > > > > > >>> > > > >> > > > topic
> > > > > > >>> > > > >> > > > > deletion and I believe that is why the
> produce
> > > > > request
> > > > > > >>> was
> > > > > > >>> > > > listed
> > > > > > >>> > > > >> > under
> > > > > > >>> > > > >> > > > > future work.
> > > > > > >>> > > > >> > > > >
> > > > > > >>> > > > >> > > > > 4. This sounds like it might be a good
> > solution,
> > > > > but I
> > > > > > >>> will
> > > > > > >>> > > need
> > > > > > >>> > > > >> to
> > > > > > >>> > > > >> > > > discuss
> > > > > > >>> > > > >> > > > > more with KIP-500 folks to get the details
> > right.
> > > > > > >>> > > > >> > > > >
> > > > > > >>> > > > >> > > > > Thanks,
> > > > > > >>> > > > >> > > > > Justine
> > > > > > >>> > > > >> > > > >
> > > > > > >>> > > > >> > > > > On Thu, Sep 24, 2020 at 12:30 PM Jason
> > Gustafson <
> > > > > > >>> > > > >> jason@confluent.io
> > > > > > >>> > > > >> > >
> > > > > > >>> > > > >> > > > > wrote:
> > > > > > >>> > > > >> > > > >
> > > > > > >>> > > > >> > > > > > Hi Justine,
> > > > > > >>> > > > >> > > > > >
> > > > > > >>> > > > >> > > > > > Thanks for picking up this work. I have a
> > few
> > > > > > >>> > > > >> questions/comments:
> > > > > > >>> > > > >> > > > > >
> > > > > > >>> > > > >> > > > > > 1. It sounds like the directory structure
> is
> > > > still
> > > > > > >>> going
> > > > > > >>> > to
> > > > > > >>> > > be
> > > > > > >>> > > > >> > based
> > > > > > >>> > > > >> > > on
> > > > > > >>> > > > >> > > > > > topic names. Do I have that right? One
> > > > > complication
> > > > > > is
> > > > > > >>> > that
> > > > > > >>> > > > the
> > > > > > >>> > > > >> > > > > > LeaderAndIsr request does not include the
> > topic
> > > > > name
> > > > > > >>> any
> > > > > > >>> > > > longer.
> > > > > > >>> > > > >> > This
> > > > > > >>> > > > >> > > > > means
> > > > > > >>> > > > >> > > > > > that a replica must wait for the
> > UpdateMetadata
> > > > > > >>> request to
> > > > > > >>> > > > >> arrive
> > > > > > >>> > > > >> > > with
> > > > > > >>> > > > >> > > > > the
> > > > > > >>> > > > >> > > > > > topic name mapping before it can create
> the
> > > > > > >>> directory. I
> > > > > > >>> > > > wonder
> > > > > > >>> > > > >> if
> > > > > > >>> > > > >> > it
> > > > > > >>> > > > >> > > > > would
> > > > > > >>> > > > >> > > > > > be simpler to avoid assumptions on the
> > ordering
> > > > of
> > > > > > >>> > > > >> UpdateMetadata
> > > > > > >>> > > > >> > and
> > > > > > >>> > > > >> > > > let
> > > > > > >>> > > > >> > > > > > LeaderAndIsr include the topic name as
> well.
> > > > Feels
> > > > > > >>> like we
> > > > > > >>> > > are
> > > > > > >>> > > > >> not
> > > > > > >>> > > > >> > > > saving
> > > > > > >>> > > > >> > > > > > that much by excluding it.
> > > > > > >>> > > > >> > > > > >
> > > > > > >>> > > > >> > > > > > 2. On a related note, it seems that the
> > reason
> > > > we
> > > > > > >>> have the
> > > > > > >>> > > > >> > partition
> > > > > > >>> > > > >> > > > > > metadata file is because we are not
> > changing the
> > > > > > >>> directory
> > > > > > >>> > > > >> > structure.
> > > > > > >>> > > > >> > > > We
> > > > > > >>> > > > >> > > > > > need it so that we remember which
> > directories
> > > > map
> > > > > to
> > > > > > >>> which
> > > > > > >>> > > > topic
> > > > > > >>> > > > >> > id.
> > > > > > >>> > > > >> > > I
> > > > > > >>> > > > >> > > > > > think it would be useful to add some
> detail
> > > > about
> > > > > > why
> > > > > > >>> > > changing
> > > > > > >>> > > > >> the
> > > > > > >>> > > > >> > > > > > directory layout was rejected. Basically
> > trying
> > > > to
> > > > > > >>> > > understand
> > > > > > >>> > > > >> how
> > > > > > >>> > > > >> > > much
> > > > > > >>> > > > >> > > > > > effort we are saving by skipping this step
> > if we
> > > > > > have
> > > > > > >>> to
> > > > > > >>> > > > >> implement
> > > > > > >>> > > > >> > > this
> > > > > > >>> > > > >> > > > > > additional bookkeeping. One concern I have
> > for
> > > > > > >>> example is
> > > > > > >>> > > that
> > > > > > >>> > > > >> the
> > > > > > >>> > > > >> > > > > > partition metadata file gets corrupt and
> > then
> > > > all
> > > > > > >>> bets are
> > > > > > >>> > > off
> > > > > > >>> > > > >> as
> > > > > > >>> > > > >> > far
> > > > > > >>> > > > >> > > > as
> > > > > > >>> > > > >> > > > > > topic consistency.
> > > > > > >>> > > > >> > > > > >
> > > > > > >>> > > > >> > > > > > 3. Is it worth adding support now for
> topic
> > ids
> > > > in
> > > > > > the
> > > > > > >>> > > > `Produce`
> > > > > > >>> > > > >> > > > request
> > > > > > >>> > > > >> > > > > > now? Overhead is mentioned as one of the
> > > > > motivations
> > > > > > >>> and
> > > > > > >>> > the
> > > > > > >>> > > > >> best
> > > > > > >>> > > > >> > > APIs
> > > > > > >>> > > > >> > > > to
> > > > > > >>> > > > >> > > > > > get savings from are `Produce` and
> `Fetch`.
> > > > > > >>> > > > >> > > > > >
> > > > > > >>> > > > >> > > > > > 4. When it comes to bootstrapping the
> > metadata
> > > > > topic
> > > > > > >>> with
> > > > > > >>> > > > >> respect
> > > > > > >>> > > > >> > to
> > > > > > >>> > > > >> > > > > > KIP-500, one suggestion would be to
> reserve
> > a
> > > > > > sentinel
> > > > > > >>> > topic
> > > > > > >>> > > > ID
> > > > > > >>> > > > >> > which
> > > > > > >>> > > > >> > > > can
> > > > > > >>> > > > >> > > > > > be used initially by new brokers. When the
> > first
> > > > > > >>> > controller
> > > > > > >>> > > is
> > > > > > >>> > > > >> > > elected,
> > > > > > >>> > > > >> > > > > it
> > > > > > >>> > > > >> > > > > > can assign a topicId to the metadata
> topic.
> > This
> > > > > is
> > > > > > a
> > > > > > >>> bit
> > > > > > >>> > > > >> similar
> > > > > > >>> > > > >> > to
> > > > > > >>> > > > >> > > > how
> > > > > > >>> > > > >> > > > > we
> > > > > > >>> > > > >> > > > > > were planning to generate the clusterId.
> > > > > > >>> > > > >> > > > > >
> > > > > > >>> > > > >> > > > > > Thanks,
> > > > > > >>> > > > >> > > > > > Jason
> > > > > > >>> > > > >> > > > > >
> > > > > > >>> > > > >> > > > > >
> > > > > > >>> > > > >> > > > > > On Thu, Sep 24, 2020 at 11:10 AM Jun Rao <
> > > > > > >>> > jun@confluent.io>
> > > > > > >>> > > > >> wrote:
> > > > > > >>> > > > >> > > > > >
> > > > > > >>> > > > >> > > > > > > Hi, Justine,
> > > > > > >>> > > > >> > > > > > >
> > > > > > >>> > > > >> > > > > > > Thanks for the updated KIP. A few more
> > > > comments
> > > > > > >>> below.
> > > > > > >>> > > > >> > > > > > >
> > > > > > >>> > > > >> > > > > > > 30.  {"name": "id", "type": "string",
> > "doc":
> > > > > > >>> "version
> > > > > > >>> > > id"}}:
> > > > > > >>> > > > >> The
> > > > > > >>> > > > >> > > doc
> > > > > > >>> > > > >> > > > > > should
> > > > > > >>> > > > >> > > > > > > say UUID.
> > > > > > >>> > > > >> > > > > > >
> > > > > > >>> > > > >> > > > > > > 31. LeaderAndIsrResponse v5 and
> > > > > > StopReplicaResponse
> > > > > > >>> v4 :
> > > > > > >>> > > It
> > > > > > >>> > > > >> seems
> > > > > > >>> > > > >> > > > there
> > > > > > >>> > > > >> > > > > > is
> > > > > > >>> > > > >> > > > > > > no need to add topic_id at partitions
> > level.
> > > > > > >>> > > > >> > > > > > >
> > > > > > >>> > > > >> > > > > > > 32. Regarding partition metadata file.
> > Perhaps
> > > > > the
> > > > > > >>> key
> > > > > > >>> > can
> > > > > > >>> > > > be
> > > > > > >>> > > > >> a
> > > > > > >>> > > > >> > > > single
> > > > > > >>> > > > >> > > > > > > word, sth like the following.
> > > > > > >>> > > > >> > > > > > > version: 0
> > > > > > >>> > > > >> > > > > > > topic_id:
> > 46bdb63f-9e8d-4a38-bf7b-ee4eb2a794e4
> > > > > > >>> > > > >> > > > > > >
> > > > > > >>> > > > >> > > > > > > 33. Another tricky thing that I realized
> > is
> > > > how
> > > > > to
> > > > > > >>> > support
> > > > > > >>> > > > the
> > > > > > >>> > > > >> > > > metadata
> > > > > > >>> > > > >> > > > > > > topic introduced in KIP-595. It's a bit
> > tricky
> > > > > to
> > > > > > >>> > assign a
> > > > > > >>> > > > >> UUID
> > > > > > >>> > > > >> > to
> > > > > > >>> > > > >> > > > the
> > > > > > >>> > > > >> > > > > > > metadata topic since we have a
> > chicken-and-egg
> > > > > > >>> problem.
> > > > > > >>> > > The
> > > > > > >>> > > > >> > > > controller
> > > > > > >>> > > > >> > > > > > > needs to persist the UUID in the
> metadata
> > > > topic
> > > > > in
> > > > > > >>> order
> > > > > > >>> > > to
> > > > > > >>> > > > >> > assign
> > > > > > >>> > > > >> > > > one
> > > > > > >>> > > > >> > > > > > > successfully, but the metadata topic is
> > needed
> > > > > to
> > > > > > >>> elect
> > > > > > >>> > a
> > > > > > >>> > > > >> > > controller
> > > > > > >>> > > > >> > > > > > > (KIP-631). So, this probably needs a bit
> > more
> > > > > > >>> thought.
> > > > > > >>> > > > >> > > > > > >
> > > > > > >>> > > > >> > > > > > > Jun
> > > > > > >>> > > > >> > > > > > >
> > > > > > >>> > > > >> > > > > > > On Thu, Sep 24, 2020 at 3:04 AM Ismael
> > Juma <
> > > > > > >>> > > > >> ismael@juma.me.uk>
> > > > > > >>> > > > >> > > > wrote:
> > > > > > >>> > > > >> > > > > > >
> > > > > > >>> > > > >> > > > > > > > Also, can we provide more details on
> > how the
> > > > > > >>> Partition
> > > > > > >>> > > > >> Metadata
> > > > > > >>> > > > >> > > > file
> > > > > > >>> > > > >> > > > > > will
> > > > > > >>> > > > >> > > > > > > > be used?
> > > > > > >>> > > > >> > > > > > > >
> > > > > > >>> > > > >> > > > > > > > Ismael
> > > > > > >>> > > > >> > > > > > > >
> > > > > > >>> > > > >> > > > > > > > On Thu, Sep 24, 2020 at 3:01 AM Ismael
> > Juma
> > > > <
> > > > > > >>> > > > >> ismael@juma.me.uk
> > > > > > >>> > > > >> > >
> > > > > > >>> > > > >> > > > > wrote:
> > > > > > >>> > > > >> > > > > > > >
> > > > > > >>> > > > >> > > > > > > > > Hi Justine,
> > > > > > >>> > > > >> > > > > > > > >
> > > > > > >>> > > > >> > > > > > > > > I think we need to update the
> > "Rejected
> > > > > > >>> > Alternatives"
> > > > > > >>> > > > >> section
> > > > > > >>> > > > >> > > to
> > > > > > >>> > > > >> > > > > take
> > > > > > >>> > > > >> > > > > > > > into
> > > > > > >>> > > > >> > > > > > > > > account that the proposal now
> removes
> > the
> > > > > > topic
> > > > > > >>> name
> > > > > > >>> > > > from
> > > > > > >>> > > > >> the
> > > > > > >>> > > > >> > > > fetch
> > > > > > >>> > > > >> > > > > > > > > request. Also, if we are removing it
> > from
> > > > > the
> > > > > > >>> Fetch
> > > > > > >>> > > > >> request,
> > > > > > >>> > > > >> > > does
> > > > > > >>> > > > >> > > > > it
> > > > > > >>> > > > >> > > > > > > make
> > > > > > >>> > > > >> > > > > > > > > sense not to remove it from similar
> > > > requests
> > > > > > >>> like
> > > > > > >>> > > > >> > > > > ListOffsetRequest?
> > > > > > >>> > > > >> > > > > > > > >
> > > > > > >>> > > > >> > > > > > > > > Ismael
> > > > > > >>> > > > >> > > > > > > > >
> > > > > > >>> > > > >> > > > > > > > > On Thu, Sep 24, 2020 at 2:46 AM
> David
> > > > Jacot
> > > > > <
> > > > > > >>> > > > >> > > djacot@confluent.io
> > > > > > >>> > > > >> > > > >
> > > > > > >>> > > > >> > > > > > > wrote:
> > > > > > >>> > > > >> > > > > > > > >
> > > > > > >>> > > > >> > > > > > > > >> Hi Justine,
> > > > > > >>> > > > >> > > > > > > > >>
> > > > > > >>> > > > >> > > > > > > > >> Thanks for the KIP. I finally had
> > time to
> > > > > > read
> > > > > > >>> it
> > > > > > >>> > :).
> > > > > > >>> > > > It
> > > > > > >>> > > > >> is
> > > > > > >>> > > > >> > a
> > > > > > >>> > > > >> > > > > great
> > > > > > >>> > > > >> > > > > > > > >> improvement.
> > > > > > >>> > > > >> > > > > > > > >>
> > > > > > >>> > > > >> > > > > > > > >> I have a few comments/questions:
> > > > > > >>> > > > >> > > > > > > > >>
> > > > > > >>> > > > >> > > > > > > > >> 1. It seems that the schema of the
> > > > > > >>> > StopReplicaRequest
> > > > > > >>> > > > is
> > > > > > >>> > > > >> > > > slightly
> > > > > > >>> > > > >> > > > > > > > >> outdated.
> > > > > > >>> > > > >> > > > > > > > >> We
> > > > > > >>> > > > >> > > > > > > > >> did some changes as part of
> KIP-570.
> > V3
> > > > is
> > > > > > >>> already
> > > > > > >>> > > > >> organized
> > > > > > >>> > > > >> > > by
> > > > > > >>> > > > >> > > > > > > topics.
> > > > > > >>> > > > >> > > > > > > > >>
> > > > > > >>> > > > >> > > > > > > > >> 2. I just want to make sure that I
> > > > > understand
> > > > > > >>> the
> > > > > > >>> > > > >> > > reconciliation
> > > > > > >>> > > > >> > > > > > > > >> logic correctly. When
> > > > > > >>> > > > >> > > > > > > > >> an "INCREMENTAL" LeaderAndIsr
> > Request is
> > > > > > >>> received,
> > > > > > >>> > > the
> > > > > > >>> > > > >> > broker
> > > > > > >>> > > > >> > > > will
> > > > > > >>> > > > >> > > > > > > also
> > > > > > >>> > > > >> > > > > > > > >> reconcile
> > > > > > >>> > > > >> > > > > > > > >> when the local uuid does not match
> > the
> > > > uuid
> > > > > > in
> > > > > > >>> the
> > > > > > >>> > > > >> request,
> > > > > > >>> > > > >> > > > right?
> > > > > > >>> > > > >> > > > > > In
> > > > > > >>> > > > >> > > > > > > > this
> > > > > > >>> > > > >> > > > > > > > >> case, the
> > > > > > >>> > > > >> > > > > > > > >> local log is staged for deletion.
> > > > > > >>> > > > >> > > > > > > > >>
> > > > > > >>> > > > >> > > > > > > > >> 3. In the documentation of the `
> > > > > > >>> > > > >> delete.stale.topic.delay.ms
> > > > > > >>> > > > >> > `
> > > > > > >>> > > > >> > > > > > config,
> > > > > > >>> > > > >> > > > > > > it
> > > > > > >>> > > > >> > > > > > > > >> says "When a
> > > > > > >>> > > > >> > > > > > > > >> FULL LeaderAndIsrRequest is
> > received..."
> > > > > but
> > > > > > I
> > > > > > >>> > > suppose
> > > > > > >>> > > > it
> > > > > > >>> > > > >> > > > applies
> > > > > > >>> > > > >> > > > > to
> > > > > > >>> > > > >> > > > > > > > both
> > > > > > >>> > > > >> > > > > > > > >> types.
> > > > > > >>> > > > >> > > > > > > > >>
> > > > > > >>> > > > >> > > > > > > > >> Best,
> > > > > > >>> > > > >> > > > > > > > >> David
> > > > > > >>> > > > >> > > > > > > > >>
> > > > > > >>> > > > >> > > > > > > > >> On Thu, Sep 24, 2020 at 1:40 AM
> > Justine
> > > > > > Olshan
> > > > > > >>> <
> > > > > > >>> > > > >> > > > > > jolshan@confluent.io>
> > > > > > >>> > > > >> > > > > > > > >> wrote:
> > > > > > >>> > > > >> > > > > > > > >>
> > > > > > >>> > > > >> > > > > > > > >> > Hi Jun,
> > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > >>> > > > >> > > > > > > > >> > Thanks for the comments. I
> > apologize
> > > > for
> > > > > > >>> some of
> > > > > > >>> > > the
> > > > > > >>> > > > >> typos
> > > > > > >>> > > > >> > > and
> > > > > > >>> > > > >> > > > > > > > >> confusion.
> > > > > > >>> > > > >> > > > > > > > >> > I’ve updated the KIP to fix some
> > of the
> > > > > > >>> issues
> > > > > > >>> > you
> > > > > > >>> > > > >> > > mentioned.
> > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > >>> > > > >> > > > > > > > >> > 20.2 I’ve changed the type to
> > String.
> > > > > > >>> > > > >> > > > > > > > >> > 20.1/3 I’ve updated the
> TopicZNode
> > to
> > > > fix
> > > > > > >>> > > formatting
> > > > > > >>> > > > >> and
> > > > > > >>> > > > >> > > > reflect
> > > > > > >>> > > > >> > > > > > the
> > > > > > >>> > > > >> > > > > > > > >> latest
> > > > > > >>> > > > >> > > > > > > > >> > version before this change.
> > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > >>> > > > >> > > > > > > > >> > 21. You are correct and I’ve
> > removed
> > > > this
> > > > > > >>> line.
> > > > > > >>> > > I’ve
> > > > > > >>> > > > >> also
> > > > > > >>> > > > >> > > > added
> > > > > > >>> > > > >> > > > > a
> > > > > > >>> > > > >> > > > > > > line
> > > > > > >>> > > > >> > > > > > > > >> > mentioning an IBP bump is
> > necessary for
> > > > > > >>> migration
> > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > >>> > > > >> > > > > > > > >> > 22. I think the wording was
> > unclear but
> > > > > > your
> > > > > > >>> > > summary
> > > > > > >>> > > > is
> > > > > > >>> > > > >> > what
> > > > > > >>> > > > >> > > > was
> > > > > > >>> > > > >> > > > > > > > >> intended
> > > > > > >>> > > > >> > > > > > > > >> > by this line. I’ve updated to
> make
> > this
> > > > > > point
> > > > > > >>> > more
> > > > > > >>> > > > >> clear.
> > > > > > >>> > > > >> > > > > “Remove
> > > > > > >>> > > > >> > > > > > > > >> deleted
> > > > > > >>> > > > >> > > > > > > > >> > topics from replicas by sending
> > > > > > >>> > StopReplicaRequest
> > > > > > >>> > > V3
> > > > > > >>> > > > >> > before
> > > > > > >>> > > > >> > > > the
> > > > > > >>> > > > >> > > > > > IBP
> > > > > > >>> > > > >> > > > > > > > >> bump
> > > > > > >>> > > > >> > > > > > > > >> > using the old logic, and using V4
> > and
> > > > the
> > > > > > new
> > > > > > >>> > logic
> > > > > > >>> > > > >> with
> > > > > > >>> > > > >> > > topic
> > > > > > >>> > > > >> > > > > IDs
> > > > > > >>> > > > >> > > > > > > > after
> > > > > > >>> > > > >> > > > > > > > >> > the IBP bump.”
> > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > >>> > > > >> > > > > > > > >> > 23. I’ve removed the unspecified
> > type
> > > > > from
> > > > > > >>> the
> > > > > > >>> > KIP
> > > > > > >>> > > > and
> > > > > > >>> > > > >> > > mention
> > > > > > >>> > > > >> > > > > > that
> > > > > > >>> > > > >> > > > > > > > IBP
> > > > > > >>> > > > >> > > > > > > > >> > will be used to handle this
> > request.
> > > > “IBP
> > > > > > >>> will be
> > > > > > >>> > > > used
> > > > > > >>> > > > >> to
> > > > > > >>> > > > >> > > > > > determine
> > > > > > >>> > > > >> > > > > > > > >> whether
> > > > > > >>> > > > >> > > > > > > > >> > this new form of the request will
> > be
> > > > > used.
> > > > > > >>> For
> > > > > > >>> > > older
> > > > > > >>> > > > >> > > requests,
> > > > > > >>> > > > >> > > > > we
> > > > > > >>> > > > >> > > > > > > will
> > > > > > >>> > > > >> > > > > > > > >> > ignore this field and default to
> > > > previous
> > > > > > >>> > > behavior.”
> > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > >>> > > > >> > > > > > > > >> > 24. I’ve fixed this typo.
> > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > >>> > > > >> > > > > > > > >> > 25. I've added a topics at a
> higher
> > > > level
> > > > > > for
> > > > > > >>> > > > >> > > > > LeaderAndIsrResponse
> > > > > > >>> > > > >> > > > > > > v5,
> > > > > > >>> > > > >> > > > > > > > >> > StopReplicaResponse v4. I've also
> > > > changed
> > > > > > >>> > > > >> > StopReplicaRequest
> > > > > > >>> > > > >> > > > v4
> > > > > > >>> > > > >> > > > > to
> > > > > > >>> > > > >> > > > > > > > have
> > > > > > >>> > > > >> > > > > > > > >> > topics at a higher level.
> > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > >>> > > > >> > > > > > > > >> > 26. I’ve updated
> > > > > > >>> forgotten_topics_data--added the
> > > > > > >>> > > > >> topic ID
> > > > > > >>> > > > >> > > and
> > > > > > >>> > > > >> > > > > > > removed
> > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > >>> > > > >> > > > > > > > >> > topic name
> > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > >>> > > > >> > > > > > > > >> > 27. I’ve decided on plain text,
> and
> > > > I’ve
> > > > > > >>> added an
> > > > > > >>> > > > >> example.
> > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > >>> > > > >> > > > > > > > >> > 28. I’ve added this idea to
> future
> > > > work.
> > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > >>> > > > >> > > > > > > > >> > Thanks again for taking a look,
> > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > >>> > > > >> > > > > > > > >> > Justine
> > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > >>> > > > >> > > > > > > > >> > On Wed, Sep 23, 2020 at 10:28 AM
> > Jun
> > > > Rao
> > > > > <
> > > > > > >>> > > > >> > jun@confluent.io>
> > > > > > >>> > > > >> > > > > > wrote:
> > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > >>> > > > >> > > > > > > > >> > > Hi, Justine,
> > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > >>> > > > >> > > > > > > > >> > > Thanks for the response. Made
> > another
> > > > > > >>> pass. A
> > > > > > >>> > few
> > > > > > >>> > > > >> more
> > > > > > >>> > > > >> > > > > comments
> > > > > > >>> > > > >> > > > > > > > below.
> > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > >>> > > > >> > > > > > > > >> > > 20. znode schema:
> > > > > > >>> > > > >> > > > > > > > >> > > 20.1 It seems that {"name":
> > > > "version",
> > > > > > >>> "type":
> > > > > > >>> > > > "int",
> > > > > > >>> > > > >> > > "id":
> > > > > > >>> > > > >> > > > > > > "UUID",
> > > > > > >>> > > > >> > > > > > > > >> > "doc":
> > > > > > >>> > > > >> > > > > > > > >> > > "version id"} should be
> {"name":
> > > > > > "version",
> > > > > > >>> > > "type":
> > > > > > >>> > > > >> > > "int"},
> > > > > > >>> > > > >> > > > > > > {"name":
> > > > > > >>> > > > >> > > > > > > > >> > "id",
> > > > > > >>> > > > >> > > > > > > > >> > > "type": "UUID", "doc": "version
> > id"}.
> > > > > > >>> > > > >> > > > > > > > >> > > 20.2 The znode format is JSON
> > which
> > > > > > doesn't
> > > > > > >>> > have
> > > > > > >>> > > > UUID
> > > > > > >>> > > > >> > > type.
> > > > > > >>> > > > >> > > > So
> > > > > > >>> > > > >> > > > > > the
> > > > > > >>> > > > >> > > > > > > > >> type
> > > > > > >>> > > > >> > > > > > > > >> > > probably should be string?
> > > > > > >>> > > > >> > > > > > > > >> > > 20.3 Also, the existing format
> > used
> > > > > seems
> > > > > > >>> > > outdated.
> > > > > > >>> > > > >> It
> > > > > > >>> > > > >> > > > should
> > > > > > >>> > > > >> > > > > > have
> > > > > > >>> > > > >> > > > > > > > the
> > > > > > >>> > > > >> > > > > > > > >> > > following format.
> > > > > > >>> > > > >> > > > > > > > >> > >     Json.encodeAsBytes(Map(
> > > > > > >>> > > > >> > > > > > > > >> > >       "version" -> 2,
> > > > > > >>> > > > >> > > > > > > > >> > >       "partitions" ->
> > > > > > >>> > > replicaAssignmentJson.asJava,
> > > > > > >>> > > > >> > > > > > > > >> > >       "adding_replicas" ->
> > > > > > >>> > > > >> > > > > addingReplicasAssignmentJson.asJava,
> > > > > > >>> > > > >> > > > > > > > >> > >       "removing_replicas" ->
> > > > > > >>> > > > >> > > > > > removingReplicasAssignmentJson.asJava
> > > > > > >>> > > > >> > > > > > > > >> > >     ).asJava)
> > > > > > >>> > > > >> > > > > > > > >> > >   }
> > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > >>> > > > >> > > > > > > > >> > > 21. Migration: The KIP says
> "The
> > > > > > migration
> > > > > > >>> > > process
> > > > > > >>> > > > >> can
> > > > > > >>> > > > >> > > take
> > > > > > >>> > > > >> > > > > > place
> > > > > > >>> > > > >> > > > > > > > >> without
> > > > > > >>> > > > >> > > > > > > > >> > > an inter-broker protocol bump,
> > as the
> > > > > > >>> format
> > > > > > >>> > > stored
> > > > > > >>> > > > >> in
> > > > > > >>> > > > >> > > > > > > > >> > > /brokers/topics/[topic] will be
> > > > > > compatible
> > > > > > >>> with
> > > > > > >>> > > > older
> > > > > > >>> > > > >> > > broker
> > > > > > >>> > > > >> > > > > > > > >> versions."
> > > > > > >>> > > > >> > > > > > > > >> > > However, since we are bumping
> up
> > the
> > > > > > >>> version of
> > > > > > >>> > > > >> > > inter-broker
> > > > > > >>> > > > >> > > > > > > > >> requests, it
> > > > > > >>> > > > >> > > > > > > > >> > > seems that we need to use IBP
> for
> > > > > > >>> migration.
> > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > >>> > > > >> > > > > > > > >> > > 22. The KIP says "Remove
> deleted
> > > > topics
> > > > > > >>> from
> > > > > > >>> > > > >> replicas by
> > > > > > >>> > > > >> > > > > sending
> > > > > > >>> > > > >> > > > > > > > >> > > StopReplicaRequest V3 for any
> > topics
> > > > > > which
> > > > > > >>> do
> > > > > > >>> > not
> > > > > > >>> > > > >> > contain
> > > > > > >>> > > > >> > > a
> > > > > > >>> > > > >> > > > > > topic
> > > > > > >>> > > > >> > > > > > > > ID,
> > > > > > >>> > > > >> > > > > > > > >> and
> > > > > > >>> > > > >> > > > > > > > >> > > V4 for any topics which do
> > contain a
> > > > > > topic
> > > > > > >>> > ID.".
> > > > > > >>> > > > >> > However,
> > > > > > >>> > > > >> > > if
> > > > > > >>> > > > >> > > > > we
> > > > > > >>> > > > >> > > > > > > use
> > > > > > >>> > > > >> > > > > > > > >> IBP,
> > > > > > >>> > > > >> > > > > > > > >> > it
> > > > > > >>> > > > >> > > > > > > > >> > > seems that the controller will
> > either
> > > > > > send
> > > > > > >>> > > > >> > > > StopReplicaRequest
> > > > > > >>> > > > >> > > > > V3
> > > > > > >>> > > > >> > > > > > > > >> > > or StopReplicaRequest V4, but
> > never
> > > > > mixed
> > > > > > >>> V3
> > > > > > >>> > and
> > > > > > >>> > > V4
> > > > > > >>> > > > >> for
> > > > > > >>> > > > >> > > > > > different
> > > > > > >>> > > > >> > > > > > > > >> topics.
> > > > > > >>> > > > >> > > > > > > > >> > > Basically, before the IBP bump,
> > V3
> > > > will
> > > > > > be
> > > > > > >>> > used.
> > > > > > >>> > > > >> After
> > > > > > >>> > > > >> > the
> > > > > > >>> > > > >> > > > IBP
> > > > > > >>> > > > >> > > > > > > bump,
> > > > > > >>> > > > >> > > > > > > > >> > > topicId will be created and V4
> > will
> > > > be
> > > > > > >>> used.
> > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > >>> > > > >> > > > > > > > >> > > 23. Given that we depend on
> IBP,
> > do
> > > > we
> > > > > > >>> still
> > > > > > >>> > need
> > > > > > >>> > > > "0
> > > > > > >>> > > > >> > > > > > UNSPECIFIED"
> > > > > > >>> > > > >> > > > > > > > >> > > in LeaderAndIsr?
> > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > >>> > > > >> > > > > > > > >> > > 24. LeaderAndIsrResponse v5 :
> It
> > > > still
> > > > > > has
> > > > > > >>> the
> > > > > > >>> > > > topic
> > > > > > >>> > > > >> > > field.
> > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > >>> > > > >> > > > > > > > >> > > 25. LeaderAndIsrResponse v5,
> > > > > > >>> > StopReplicaResponse
> > > > > > >>> > > > v4:
> > > > > > >>> > > > >> > Could
> > > > > > >>> > > > >> > > > we
> > > > > > >>> > > > >> > > > > > use
> > > > > > >>> > > > >> > > > > > > > this
> > > > > > >>> > > > >> > > > > > > > >> > > opportunity to organize the
> > response
> > > > > in 2
> > > > > > >>> > levels,
> > > > > > >>> > > > >> first
> > > > > > >>> > > > >> > by
> > > > > > >>> > > > >> > > > > > topic,
> > > > > > >>> > > > >> > > > > > > > >> then by
> > > > > > >>> > > > >> > > > > > > > >> > > partition, as most other
> > > > > > >>> requests/responses?
> > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > >>> > > > >> > > > > > > > >> > > 26. FetchRequest v13 : Should
> > > > > > >>> > > forgotten_topics_data
> > > > > > >>> > > > >> use
> > > > > > >>> > > > >> > > > > topicId
> > > > > > >>> > > > >> > > > > > > too?
> > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > >>> > > > >> > > > > > > > >> > > 27. "This file can either be
> > plain
> > > > text
> > > > > > >>> > > (key/value
> > > > > > >>> > > > >> > pairs)
> > > > > > >>> > > > >> > > or
> > > > > > >>> > > > >> > > > > > > JSON."
> > > > > > >>> > > > >> > > > > > > > >> Have
> > > > > > >>> > > > >> > > > > > > > >> > we
> > > > > > >>> > > > >> > > > > > > > >> > > decided which one to use? Also,
> > it
> > > > > would
> > > > > > be
> > > > > > >>> > > helpful
> > > > > > >>> > > > >> to
> > > > > > >>> > > > >> > > > provide
> > > > > > >>> > > > >> > > > > > an
> > > > > > >>> > > > >> > > > > > > > >> > example.
> > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > >>> > > > >> > > > > > > > >> > > 28. Future improvement: Another
> > > > future
> > > > > > >>> > > improvement
> > > > > > >>> > > > >> > > > opportunity
> > > > > > >>> > > > >> > > > > > is
> > > > > > >>> > > > >> > > > > > > to
> > > > > > >>> > > > >> > > > > > > > >> use
> > > > > > >>> > > > >> > > > > > > > >> > > topicId in
> > > > > > >>> GroupMetadataManager.offsetCommitKey
> > > > > > >>> > > in
> > > > > > >>> > > > >> the
> > > > > > >>> > > > >> > > > > > > offset_commit
> > > > > > >>> > > > >> > > > > > > > >> > topic.
> > > > > > >>> > > > >> > > > > > > > >> > > This may save some space.
> > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > >>> > > > >> > > > > > > > >> > > Thanks,
> > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > >>> > > > >> > > > > > > > >> > > Jun
> > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > >>> > > > >> > > > > > > > >> > > On Wed, Sep 23, 2020 at 8:50 AM
> > > > Justine
> > > > > > >>> Olshan
> > > > > > >>> > <
> > > > > > >>> > > > >> > > > > > > > jolshan@confluent.io>
> > > > > > >>> > > > >> > > > > > > > >> > > wrote:
> > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > >>> > > > >> > > > > > > > >> > > > Hi Tom,
> > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > Thanks for the comment. I
> think
> > > > this
> > > > > > is a
> > > > > > >>> > > really
> > > > > > >>> > > > >> good
> > > > > > >>> > > > >> > > idea
> > > > > > >>> > > > >> > > > > and
> > > > > > >>> > > > >> > > > > > > it
> > > > > > >>> > > > >> > > > > > > > >> has
> > > > > > >>> > > > >> > > > > > > > >> > > been
> > > > > > >>> > > > >> > > > > > > > >> > > > added to the KIP under the
> > newly
> > > > > added
> > > > > > >>> > tooling
> > > > > > >>> > > > >> > section.
> > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > Thanks again,
> > > > > > >>> > > > >> > > > > > > > >> > > > Justine
> > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > On Wed, Sep 23, 2020 at 3:17
> > AM Tom
> > > > > > >>> Bentley <
> > > > > > >>> > > > >> > > > > > > tbentley@redhat.com>
> > > > > > >>> > > > >> > > > > > > > >> > wrote:
> > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > Hi Justine,
> > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > I know you started the vote
> > > > thread,
> > > > > > >>> but on
> > > > > > >>> > > > >> > re-reading
> > > > > > >>> > > > >> > > > the
> > > > > > >>> > > > >> > > > > > KIP
> > > > > > >>> > > > >> > > > > > > I
> > > > > > >>> > > > >> > > > > > > > >> > noticed
> > > > > > >>> > > > >> > > > > > > > >> > > > > that although the topic id
> is
> > > > > > included
> > > > > > >>> in
> > > > > > >>> > the
> > > > > > >>> > > > >> > > > > > MetadataResponse
> > > > > > >>> > > > >> > > > > > > > >> it's
> > > > > > >>> > > > >> > > > > > > > >> > not
> > > > > > >>> > > > >> > > > > > > > >> > > > > surfaced in the output from
> > > > > > >>> > `kafka-topics.sh
> > > > > > >>> > > > >> > > > --describe`.
> > > > > > >>> > > > >> > > > > > > Maybe
> > > > > > >>> > > > >> > > > > > > > >> that
> > > > > > >>> > > > >> > > > > > > > >> > > was
> > > > > > >>> > > > >> > > > > > > > >> > > > > intentional because ids are
> > > > > > >>> intentionally
> > > > > > >>> > not
> > > > > > >>> > > > >> really
> > > > > > >>> > > > >> > > > > > something
> > > > > > >>> > > > >> > > > > > > > the
> > > > > > >>> > > > >> > > > > > > > >> > user
> > > > > > >>> > > > >> > > > > > > > >> > > > > should care deeply about,
> > but it
> > > > > > would
> > > > > > >>> also
> > > > > > >>> > > > make
> > > > > > >>> > > > >> > life
> > > > > > >>> > > > >> > > > > harder
> > > > > > >>> > > > >> > > > > > > for
> > > > > > >>> > > > >> > > > > > > > >> > anyone
> > > > > > >>> > > > >> > > > > > > > >> > > > > debugging Kafka and this
> > would
> > > > > likely
> > > > > > >>> get
> > > > > > >>> > > worse
> > > > > > >>> > > > >> the
> > > > > > >>> > > > >> > > more
> > > > > > >>> > > > >> > > > > > topic
> > > > > > >>> > > > >> > > > > > > > ids
> > > > > > >>> > > > >> > > > > > > > >> > got
> > > > > > >>> > > > >> > > > > > > > >> > > > > rolled out across the
> > protocols,
> > > > > > >>> clients
> > > > > > >>> > etc.
> > > > > > >>> > > > It
> > > > > > >>> > > > >> > seems
> > > > > > >>> > > > >> > > > > > likely
> > > > > > >>> > > > >> > > > > > > > that
> > > > > > >>> > > > >> > > > > > > > >> > > > > `kafka-topics.sh` will
> > eventually
> > > > > > need
> > > > > > >>> the
> > > > > > >>> > > > >> ability
> > > > > > >>> > > > >> > to
> > > > > > >>> > > > >> > > > show
> > > > > > >>> > > > >> > > > > > the
> > > > > > >>> > > > >> > > > > > > > id
> > > > > > >>> > > > >> > > > > > > > >> of
> > > > > > >>> > > > >> > > > > > > > >> > a
> > > > > > >>> > > > >> > > > > > > > >> > > > > topic and perhaps find a
> > topic
> > > > name
> > > > > > >>> given
> > > > > > >>> > an
> > > > > > >>> > > > id.
> > > > > > >>> > > > >> Is
> > > > > > >>> > > > >> > > > there
> > > > > > >>> > > > >> > > > > > any
> > > > > > >>> > > > >> > > > > > > > >> reason
> > > > > > >>> > > > >> > > > > > > > >> > > not
> > > > > > >>> > > > >> > > > > > > > >> > > > to
> > > > > > >>> > > > >> > > > > > > > >> > > > > implement that in this KIP?
> > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > Many thanks,
> > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > Tom
> > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > On Mon, Sep 21, 2020 at
> 9:54
> > PM
> > > > > > Justine
> > > > > > >>> > > Olshan
> > > > > > >>> > > > <
> > > > > > >>> > > > >> > > > > > > > >> jolshan@confluent.io
> > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > wrote:
> > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > Hi all,
> > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > After thinking about it,
> > I've
> > > > > > >>> decided to
> > > > > > >>> > > > remove
> > > > > > >>> > > > >> > the
> > > > > > >>> > > > >> > > > > topic
> > > > > > >>> > > > >> > > > > > > name
> > > > > > >>> > > > >> > > > > > > > >> from
> > > > > > >>> > > > >> > > > > > > > >> > > the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > Fetch Request and
> Response
> > > > after
> > > > > > all.
> > > > > > >>> > Since
> > > > > > >>> > > > >> there
> > > > > > >>> > > > >> > > are
> > > > > > >>> > > > >> > > > so
> > > > > > >>> > > > >> > > > > > > many
> > > > > > >>> > > > >> > > > > > > > of
> > > > > > >>> > > > >> > > > > > > > >> > > these
> > > > > > >>> > > > >> > > > > > > > >> > > > > > requests per second, it
> is
> > > > worth
> > > > > > >>> removing
> > > > > > >>> > > the
> > > > > > >>> > > > >> > extra
> > > > > > >>> > > > >> > > > > > > > information.
> > > > > > >>> > > > >> > > > > > > > >> > I've
> > > > > > >>> > > > >> > > > > > > > >> > > > > > updated the KIP to
> reflect
> > this
> > > > > > >>> change.
> > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > Please let me know if
> > there is
> > > > > > >>> anything
> > > > > > >>> > > else
> > > > > > >>> > > > we
> > > > > > >>> > > > >> > > should
> > > > > > >>> > > > >> > > > > > > discuss
> > > > > > >>> > > > >> > > > > > > > >> > before
> > > > > > >>> > > > >> > > > > > > > >> > > > > > voting.
> > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > Thank you,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > Justine
> > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > On Fri, Sep 18, 2020 at
> > 9:46 AM
> > > > > > >>> Justine
> > > > > > >>> > > > Olshan
> > > > > > >>> > > > >> <
> > > > > > >>> > > > >> > > > > > > > >> > jolshan@confluent.io
> > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > wrote:
> > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > > Hi Jun,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > > I see what you are
> > saying.
> > > > For
> > > > > > now
> > > > > > >>> we
> > > > > > >>> > can
> > > > > > >>> > > > >> remove
> > > > > > >>> > > > >> > > the
> > > > > > >>> > > > >> > > > > > extra
> > > > > > >>> > > > >> > > > > > > > >> > > > information.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > > I'll leave the option
> to
> > add
> > > > > more
> > > > > > >>> > fields
> > > > > > >>> > > to
> > > > > > >>> > > > >> the
> > > > > > >>> > > > >> > > file
> > > > > > >>> > > > >> > > > > in
> > > > > > >>> > > > >> > > > > > > the
> > > > > > >>> > > > >> > > > > > > > >> > future.
> > > > > > >>> > > > >> > > > > > > > >> > > > The
> > > > > > >>> > > > >> > > > > > > > >> > > > > > KIP
> > > > > > >>> > > > >> > > > > > > > >> > > > > > > has been updated to
> > reflect
> > > > > this
> > > > > > >>> > change.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > > Thanks,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > > Justine
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > > On Fri, Sep 18, 2020 at
> > 8:46
> > > > AM
> > > > > > Jun
> > > > > > >>> > Rao <
> > > > > > >>> > > > >> > > > > > jun@confluent.io
> > > > > > >>> > > > >> > > > > > > >
> > > > > > >>> > > > >> > > > > > > > >> > wrote:
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Hi, Justine,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Thanks for the reply.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> 13. If the log
> > directory is
> > > > > the
> > > > > > >>> source
> > > > > > >>> > > of
> > > > > > >>> > > > >> > truth,
> > > > > > >>> > > > >> > > it
> > > > > > >>> > > > >> > > > > > means
> > > > > > >>> > > > >> > > > > > > > >> that
> > > > > > >>> > > > >> > > > > > > > >> > the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> redundant info in the
> > > > metadata
> > > > > > >>> file
> > > > > > >>> > will
> > > > > > >>> > > > be
> > > > > > >>> > > > >> > > > ignored.
> > > > > > >>> > > > >> > > > > > Then
> > > > > > >>> > > > >> > > > > > > > the
> > > > > > >>> > > > >> > > > > > > > >> > > > question
> > > > > > >>> > > > >> > > > > > > > >> > > > > > is
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> why do we need to put
> > the
> > > > > > >>> redundant
> > > > > > >>> > info
> > > > > > >>> > > > in
> > > > > > >>> > > > >> the
> > > > > > >>> > > > >> > > > > > metadata
> > > > > > >>> > > > >> > > > > > > > file
> > > > > > >>> > > > >> > > > > > > > >> > now?
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Thanks,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Jun
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> On Thu, Sep 17, 2020
> at
> > 5:07
> > > > > PM
> > > > > > >>> > Justine
> > > > > > >>> > > > >> Olshan
> > > > > > >>> > > > >> > <
> > > > > > >>> > > > >> > > > > > > > >> > > > jolshan@confluent.io>
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> wrote:
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Hi Jun,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Thanks for the quick
> > > > > response!
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > 12. I've decided to
> > bump
> > > > up
> > > > > > the
> > > > > > >>> > > versions
> > > > > > >>> > > > >> on
> > > > > > >>> > > > >> > the
> > > > > > >>> > > > >> > > > > > > requests
> > > > > > >>> > > > >> > > > > > > > >> and
> > > > > > >>> > > > >> > > > > > > > >> > > > updated
> > > > > > >>> > > > >> > > > > > > > >> > > > > > the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > KIP. I think it's
> > good we
> > > > > > >>> thoroughly
> > > > > > >>> > > > >> > discussed
> > > > > > >>> > > > >> > > > the
> > > > > > >>> > > > >> > > > > > > > options
> > > > > > >>> > > > >> > > > > > > > >> > here,
> > > > > > >>> > > > >> > > > > > > > >> > > > so
> > > > > > >>> > > > >> > > > > > > > >> > > > > we
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> know
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > we made a good
> > choice. :)
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > 13. This is an
> > interesting
> > > > > > >>> > situation.
> > > > > > >>> > > I
> > > > > > >>> > > > >> think
> > > > > > >>> > > > >> > > if
> > > > > > >>> > > > >> > > > > this
> > > > > > >>> > > > >> > > > > > > > does
> > > > > > >>> > > > >> > > > > > > > >> > occur
> > > > > > >>> > > > >> > > > > > > > >> > > > we
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> should
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > give a warning. I
> > agree
> > > > that
> > > > > > >>> it's
> > > > > > >>> > hard
> > > > > > >>> > > > to
> > > > > > >>> > > > >> > know
> > > > > > >>> > > > >> > > > the
> > > > > > >>> > > > >> > > > > > > source
> > > > > > >>> > > > >> > > > > > > > >> of
> > > > > > >>> > > > >> > > > > > > > >> > > truth
> > > > > > >>> > > > >> > > > > > > > >> > > > > for
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> sure
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > since the directory
> > or the
> > > > > > file
> > > > > > >>> > could
> > > > > > >>> > > be
> > > > > > >>> > > > >> > > manually
> > > > > > >>> > > > >> > > > > > > > >> modified. I
> > > > > > >>> > > > >> > > > > > > > >> > > > guess
> > > > > > >>> > > > >> > > > > > > > >> > > > > > the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > directory could be
> > used as
> > > > > the
> > > > > > >>> > source
> > > > > > >>> > > of
> > > > > > >>> > > > >> > truth.
> > > > > > >>> > > > >> > > > To
> > > > > > >>> > > > >> > > > > be
> > > > > > >>> > > > >> > > > > > > > >> honest,
> > > > > > >>> > > > >> > > > > > > > >> > > I'm
> > > > > > >>> > > > >> > > > > > > > >> > > > > not
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > really sure what
> > happens
> > > > in
> > > > > > >>> kafka
> > > > > > >>> > when
> > > > > > >>> > > > the
> > > > > > >>> > > > >> > log
> > > > > > >>> > > > >> > > > > > > directory
> > > > > > >>> > > > >> > > > > > > > is
> > > > > > >>> > > > >> > > > > > > > >> > > > renamed
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > manually in such a
> > way.
> > > > I'm
> > > > > > also
> > > > > > >>> > > > >> wondering if
> > > > > > >>> > > > >> > > the
> > > > > > >>> > > > >> > > > > > > > >> situation is
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> recoverable
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > in this scenario.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Thanks,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Justine
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > On Thu, Sep 17, 2020
> > at
> > > > 4:28
> > > > > > PM
> > > > > > >>> Jun
> > > > > > >>> > > Rao
> > > > > > >>> > > > <
> > > > > > >>> > > > >> > > > > > > > jun@confluent.io>
> > > > > > >>> > > > >> > > > > > > > >> > > wrote:
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Hi, Justine,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Thanks for the
> > reply.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > 12. I don't have a
> > > > strong
> > > > > > >>> > preference
> > > > > > >>> > > > >> > either.
> > > > > > >>> > > > >> > > > > > However,
> > > > > > >>> > > > >> > > > > > > > if
> > > > > > >>> > > > >> > > > > > > > >> we
> > > > > > >>> > > > >> > > > > > > > >> > > need
> > > > > > >>> > > > >> > > > > > > > >> > > > > IBP
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > anyway, maybe it's
> > > > easier
> > > > > to
> > > > > > >>> just
> > > > > > >>> > > bump
> > > > > > >>> > > > >> up
> > > > > > >>> > > > >> > the
> > > > > > >>> > > > >> > > > > > version
> > > > > > >>> > > > >> > > > > > > > for
> > > > > > >>> > > > >> > > > > > > > >> > all
> > > > > > >>> > > > >> > > > > > > > >> > > > > inter
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > broker
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > requests and add
> the
> > > > topic
> > > > > > id
> > > > > > >>> > field
> > > > > > >>> > > > as a
> > > > > > >>> > > > >> > > > regular
> > > > > > >>> > > > >> > > > > > > > field. A
> > > > > > >>> > > > >> > > > > > > > >> > > > regular
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> field
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > is
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > a bit more concise
> > in
> > > > wire
> > > > > > >>> > transfer
> > > > > > >>> > > > >> than a
> > > > > > >>> > > > >> > > > > flexible
> > > > > > >>> > > > >> > > > > > > > >> field.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > 13. The confusion
> > that I
> > > > > was
> > > > > > >>> > > referring
> > > > > > >>> > > > >> to
> > > > > > >>> > > > >> > is
> > > > > > >>> > > > >> > > > > > between
> > > > > > >>> > > > >> > > > > > > > the
> > > > > > >>> > > > >> > > > > > > > >> > topic
> > > > > > >>> > > > >> > > > > > > > >> > > > > name
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > partition number
> > between
> > > > > the
> > > > > > >>> log
> > > > > > >>> > dir
> > > > > > >>> > > > and
> > > > > > >>> > > > >> > the
> > > > > > >>> > > > >> > > > > > metadata
> > > > > > >>> > > > >> > > > > > > > >> file.
> > > > > > >>> > > > >> > > > > > > > >> > > For
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> example,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > if
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > the log dir is
> > topicA-1
> > > > > and
> > > > > > >>> the
> > > > > > >>> > > > metadata
> > > > > > >>> > > > >> > file
> > > > > > >>> > > > >> > > > in
> > > > > > >>> > > > >> > > > > it
> > > > > > >>> > > > >> > > > > > > has
> > > > > > >>> > > > >> > > > > > > > >> > topicB
> > > > > > >>> > > > >> > > > > > > > >> > > > and
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > partition 0 (say
> > due to
> > > > a
> > > > > > bug
> > > > > > >>> or
> > > > > > >>> > > > manual
> > > > > > >>> > > > >> > > > > > > modification),
> > > > > > >>> > > > >> > > > > > > > >> which
> > > > > > >>> > > > >> > > > > > > > >> > > one
> > > > > > >>> > > > >> > > > > > > > >> > > > > do
> > > > > > >>> > > > >> > > > > > > > >> > > > > > we
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > use
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > as the source of
> > truth?
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Jun
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > On Thu, Sep 17,
> > 2020 at
> > > > > 3:43
> > > > > > >>> PM
> > > > > > >>> > > > Justine
> > > > > > >>> > > > >> > > Olshan
> > > > > > >>> > > > >> > > > <
> > > > > > >>> > > > >> > > > > > > > >> > > > > > jolshan@confluent.io>
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > wrote:
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > Hi Jun,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > Thanks for the
> > > > comments.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > 12. I bumped the
> > > > > > >>> > > LeaderAndIsrRequest
> > > > > > >>> > > > >> > > because
> > > > > > >>> > > > >> > > > I
> > > > > > >>> > > > >> > > > > > > > removed
> > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > >>> > > > >> > > > > > > > >> > > > topic
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> name
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > field
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > in the response.
> > It
> > > > may
> > > > > be
> > > > > > >>> > > possible
> > > > > > >>> > > > to
> > > > > > >>> > > > >> > > avoid
> > > > > > >>> > > > >> > > > > > > bumping
> > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > >>> > > > >> > > > > > > > >> > > > version
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > without
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > that change, but
> > I may
> > > > > be
> > > > > > >>> > missing
> > > > > > >>> > > > >> > > something.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > I believe
> > StopReplica
> > > > is
> > > > > > >>> > actually
> > > > > > >>> > > on
> > > > > > >>> > > > >> > > version
> > > > > > >>> > > > >> > > > 3
> > > > > > >>> > > > >> > > > > > now,
> > > > > > >>> > > > >> > > > > > > > but
> > > > > > >>> > > > >> > > > > > > > >> > > > because
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > version 2
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > is flexible, I
> > kept
> > > > that
> > > > > > >>> listed
> > > > > > >>> > as
> > > > > > >>> > > > >> > version
> > > > > > >>> > > > >> > > 2
> > > > > > >>> > > > >> > > > on
> > > > > > >>> > > > >> > > > > > the
> > > > > > >>> > > > >> > > > > > > > KIP
> > > > > > >>> > > > >> > > > > > > > >> > > page.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> However,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > you
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > may be right in
> > that
> > > > we
> > > > > > may
> > > > > > >>> need
> > > > > > >>> > > to
> > > > > > >>> > > > >> bump
> > > > > > >>> > > > >> > > the
> > > > > > >>> > > > >> > > > > > > version
> > > > > > >>> > > > >> > > > > > > > on
> > > > > > >>> > > > >> > > > > > > > >> > > > > > StopReplica
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> to
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > deal
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > with deletion
> > > > > differently
> > > > > > as
> > > > > > >>> > > > mentioned
> > > > > > >>> > > > >> > > > above. I
> > > > > > >>> > > > >> > > > > > > don't
> > > > > > >>> > > > >> > > > > > > > >> know
> > > > > > >>> > > > >> > > > > > > > >> > > if
> > > > > > >>> > > > >> > > > > > > > >> > > > I
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> have a
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > big
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > preference over
> > used
> > > > > > tagged
> > > > > > >>> > fields
> > > > > > >>> > > > or
> > > > > > >>> > > > >> > not.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > 13. I was
> thinking
> > > > that
> > > > > in
> > > > > > >>> the
> > > > > > >>> > > case
> > > > > > >>> > > > >> where
> > > > > > >>> > > > >> > > the
> > > > > > >>> > > > >> > > > > > file
> > > > > > >>> > > > >> > > > > > > > and
> > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > >>> > > > >> > > > > > > > >> > > > > request
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > topic
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > ids don't match,
> > it
> > > > > means
> > > > > > >>> that
> > > > > > >>> > the
> > > > > > >>> > > > >> > broker's
> > > > > > >>> > > > >> > > > > > > topic/the
> > > > > > >>> > > > >> > > > > > > > >> one
> > > > > > >>> > > > >> > > > > > > > >> > in
> > > > > > >>> > > > >> > > > > > > > >> > > > the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> file
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > has
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > been deleted. In
> > that
> > > > > > case,
> > > > > > >>> we
> > > > > > >>> > > would
> > > > > > >>> > > > >> need
> > > > > > >>> > > > >> > > to
> > > > > > >>> > > > >> > > > > > delete
> > > > > > >>> > > > >> > > > > > > > the
> > > > > > >>> > > > >> > > > > > > > >> > old
> > > > > > >>> > > > >> > > > > > > > >> > > > > topic
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > start
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > receiving the
> new
> > > > > version.
> > > > > > >>> If
> > > > > > >>> > the
> > > > > > >>> > > > >> topic
> > > > > > >>> > > > >> > > name
> > > > > > >>> > > > >> > > > > were
> > > > > > >>> > > > >> > > > > > > to
> > > > > > >>> > > > >> > > > > > > > >> > change,
> > > > > > >>> > > > >> > > > > > > > >> > > > but
> > > > > > >>> > > > >> > > > > > > > >> > > > > > the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > ids
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > still matched,
> the
> > > > file
> > > > > > >>> would
> > > > > > >>> > also
> > > > > > >>> > > > >> need
> > > > > > >>> > > > >> > to
> > > > > > >>> > > > >> > > > > > update.
> > > > > > >>> > > > >> > > > > > > > Am I
> > > > > > >>> > > > >> > > > > > > > >> > > > missing
> > > > > > >>> > > > >> > > > > > > > >> > > > > a
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> case
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > where the file
> > would
> > > > be
> > > > > > >>> correct
> > > > > > >>> > > and
> > > > > > >>> > > > >> not
> > > > > > >>> > > > >> > the
> > > > > > >>> > > > >> > > > > > > request?
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > Thanks,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > Justine
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > On Thu, Sep 17,
> > 2020
> > > > at
> > > > > > >>> 3:18 PM
> > > > > > >>> > > Jun
> > > > > > >>> > > > >> Rao <
> > > > > > >>> > > > >> > > > > > > > >> jun@confluent.io
> > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > wrote:
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Hi, Justine,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Thanks for the
> > > > reply.
> > > > > A
> > > > > > >>> couple
> > > > > > >>> > > of
> > > > > > >>> > > > >> more
> > > > > > >>> > > > >> > > > > comments
> > > > > > >>> > > > >> > > > > > > > >> below.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > 12. ListOffset
> > and
> > > > > > >>> > > OffsetForLeader
> > > > > > >>> > > > >> > > > currently
> > > > > > >>> > > > >> > > > > > > don't
> > > > > > >>> > > > >> > > > > > > > >> > support
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> flexible
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > fields.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > So, we have to
> > bump
> > > > up
> > > > > > the
> > > > > > >>> > > version
> > > > > > >>> > > > >> > number
> > > > > > >>> > > > >> > > > and
> > > > > > >>> > > > >> > > > > > use
> > > > > > >>> > > > >> > > > > > > > >> IBP at
> > > > > > >>> > > > >> > > > > > > > >> > > > least
> > > > > > >>> > > > >> > > > > > > > >> > > > > > for
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > these
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > two requests.
> > Note
> > > > > that
> > > > > > it
> > > > > > >>> > seems
> > > > > > >>> > > > >> 2.7.0
> > > > > > >>> > > > >> > > will
> > > > > > >>> > > > >> > > > > > > require
> > > > > > >>> > > > >> > > > > > > > >> IBP
> > > > > > >>> > > > >> > > > > > > > >> > > > anyway
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > because
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > of
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > changes in
> > > > > KAFKA-10435.
> > > > > > >>> Also,
> > > > > > >>> > it
> > > > > > >>> > > > >> seems
> > > > > > >>> > > > >> > > that
> > > > > > >>> > > > >> > > > > the
> > > > > > >>> > > > >> > > > > > > > >> version
> > > > > > >>> > > > >> > > > > > > > >> > > for
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > LeaderAndIsrRequest
> > > > > and
> > > > > > >>> > > > StopReplica
> > > > > > >>> > > > >> are
> > > > > > >>> > > > >> > > > > bumped
> > > > > > >>> > > > >> > > > > > > even
> > > > > > >>> > > > >> > > > > > > > >> > though
> > > > > > >>> > > > >> > > > > > > > >> > > > we
> > > > > > >>> > > > >> > > > > > > > >> > > > > > only
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > added
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > a
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > tagged field.
> > But
> > > > > since
> > > > > > >>> IBP is
> > > > > > >>> > > > >> needed
> > > > > > >>> > > > >> > > > anyway,
> > > > > > >>> > > > >> > > > > > we
> > > > > > >>> > > > >> > > > > > > > may
> > > > > > >>> > > > >> > > > > > > > >> > want
> > > > > > >>> > > > >> > > > > > > > >> > > to
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> revisit
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > overall tagged
> > field
> > > > > > >>> choice.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > 13. The only
> > > > downside
> > > > > is
> > > > > > >>> the
> > > > > > >>> > > > >> potential
> > > > > > >>> > > > >> > > > > > confusion
> > > > > > >>> > > > >> > > > > > > on
> > > > > > >>> > > > >> > > > > > > > >> > which
> > > > > > >>> > > > >> > > > > > > > >> > > > one
> > > > > > >>> > > > >> > > > > > > > >> > > > > is
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > source
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > of truth if
> they
> > > > don't
> > > > > > >>> match.
> > > > > > >>> > > > >> Another
> > > > > > >>> > > > >> > > > option
> > > > > > >>> > > > >> > > > > is
> > > > > > >>> > > > >> > > > > > > to
> > > > > > >>> > > > >> > > > > > > > >> > include
> > > > > > >>> > > > >> > > > > > > > >> > > > > those
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > fields
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > in
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the metadata
> > file
> > > > when
> > > > > > we
> > > > > > >>> > > actually
> > > > > > >>> > > > >> > change
> > > > > > >>> > > > >> > > > the
> > > > > > >>> > > > >> > > > > > > > >> directory
> > > > > > >>> > > > >> > > > > > > > >> > > > > > structure.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Thanks,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Jun
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > On Thu, Sep
> 17,
> > 2020
> > > > > at
> > > > > > >>> 2:01
> > > > > > >>> > PM
> > > > > > >>> > > > >> Justine
> > > > > > >>> > > > >> > > > > Olshan
> > > > > > >>> > > > >> > > > > > <
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> jolshan@confluent.io
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > wrote:
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > Hello all,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I've thought
> > some
> > > > > more
> > > > > > >>> about
> > > > > > >>> > > > >> removing
> > > > > > >>> > > > >> > > the
> > > > > > >>> > > > >> > > > > > topic
> > > > > > >>> > > > >> > > > > > > > >> name
> > > > > > >>> > > > >> > > > > > > > >> > > field
> > > > > > >>> > > > >> > > > > > > > >> > > > > > from
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > some
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > of
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > requests. On
> > > > closer
> > > > > > >>> > inspection
> > > > > > >>> > > > of
> > > > > > >>> > > > >> the
> > > > > > >>> > > > >> > > > > > > > >> > > requests/responses,
> > > > > > >>> > > > >> > > > > > > > >> > > > it
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> seems
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > internal
> > changes
> > > > > would
> > > > > > >>> be
> > > > > > >>> > much
> > > > > > >>> > > > >> larger
> > > > > > >>> > > > >> > > > than
> > > > > > >>> > > > >> > > > > I
> > > > > > >>> > > > >> > > > > > > > >> expected.
> > > > > > >>> > > > >> > > > > > > > >> > > > Some
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > protocols
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > involve
> > clients,
> > > > so
> > > > > > they
> > > > > > >>> > would
> > > > > > >>> > > > >> > require
> > > > > > >>> > > > >> > > > > > changes
> > > > > > >>> > > > >> > > > > > > > too.
> > > > > > >>> > > > >> > > > > > > > >> > I'm
> > > > > > >>> > > > >> > > > > > > > >> > > > > > thinking
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > that
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > for
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > now,
> removing
> > the
> > > > > > topic
> > > > > > >>> name
> > > > > > >>> > > > from
> > > > > > >>> > > > >> > these
> > > > > > >>> > > > >> > > > > > > requests
> > > > > > >>> > > > >> > > > > > > > >> and
> > > > > > >>> > > > >> > > > > > > > >> > > > > responses
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> are
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > out
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > scope.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I have
> > decided to
> > > > > just
> > > > > > >>> keep
> > > > > > >>> > > the
> > > > > > >>> > > > >> > change
> > > > > > >>> > > > >> > > > > > > > >> > > > LeaderAndIsrResponse
> > > > > > >>> > > > >> > > > > > > > >> > > > > to
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > remove
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > topic name,
> > and
> > > > have
> > > > > > >>> updated
> > > > > > >>> > > the
> > > > > > >>> > > > >> KIP
> > > > > > >>> > > > >> > to
> > > > > > >>> > > > >> > > > > > reflect
> > > > > > >>> > > > >> > > > > > > > >> this
> > > > > > >>> > > > >> > > > > > > > >> > > > > change. I
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> have
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > also
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > mentioned
> the
> > > > other
> > > > > > >>> requests
> > > > > > >>> > > and
> > > > > > >>> > > > >> > > > responses
> > > > > > >>> > > > >> > > > > in
> > > > > > >>> > > > >> > > > > > > > >> future
> > > > > > >>> > > > >> > > > > > > > >> > > work.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I'm hoping
> to
> > > > start
> > > > > > the
> > > > > > >>> > voting
> > > > > > >>> > > > >> > process
> > > > > > >>> > > > >> > > > > soon,
> > > > > > >>> > > > >> > > > > > so
> > > > > > >>> > > > >> > > > > > > > >> let me
> > > > > > >>> > > > >> > > > > > > > >> > > > know
> > > > > > >>> > > > >> > > > > > > > >> > > > > if
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > there
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > is
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > anything
> else
> > we
> > > > > > should
> > > > > > >>> > > discuss.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > Thank you,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > Justine
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > On Tue, Sep
> > 15,
> > > > 2020
> > > > > > at
> > > > > > >>> 3:57
> > > > > > >>> > > PM
> > > > > > >>> > > > >> > Justine
> > > > > > >>> > > > >> > > > > > Olshan
> > > > > > >>> > > > >> > > > > > > <
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> jolshan@confluent.io
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > wrote:
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > Hello
> again,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > To follow
> > up on
> > > > > some
> > > > > > >>> of
> > > > > > >>> > the
> > > > > > >>> > > > >> other
> > > > > > >>> > > > >> > > > > comments:
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 10/11) We
> > can
> > > > > remove
> > > > > > >>> the
> > > > > > >>> > > topic
> > > > > > >>> > > > >> name
> > > > > > >>> > > > >> > > > from
> > > > > > >>> > > > >> > > > > > > these
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > requests/responses,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > and
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > that means
> > we
> > > > will
> > > > > > >>> just
> > > > > > >>> > have
> > > > > > >>> > > > to
> > > > > > >>> > > > >> > make
> > > > > > >>> > > > >> > > a
> > > > > > >>> > > > >> > > > > few
> > > > > > >>> > > > >> > > > > > > > >> internal
> > > > > > >>> > > > >> > > > > > > > >> > > > > changes
> > > > > > >>> > > > >> > > > > > > > >> > > > > > to
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > make
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > partitions
> > > > > > accessible
> > > > > > >>> by
> > > > > > >>> > > topic
> > > > > > >>> > > > >> id
> > > > > > >>> > > > >> > and
> > > > > > >>> > > > >> > > > > > > > partition.
> > > > > > >>> > > > >> > > > > > > > >> I
> > > > > > >>> > > > >> > > > > > > > >> > can
> > > > > > >>> > > > >> > > > > > > > >> > > > > > update
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > KIP
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > remove
> them
> > > > unless
> > > > > > >>> anyone
> > > > > > >>> > > > thinks
> > > > > > >>> > > > >> > they
> > > > > > >>> > > > >> > > > > > should
> > > > > > >>> > > > >> > > > > > > > >> stay.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 12)
> > Addressed in
> > > > > the
> > > > > > >>> > > previous
> > > > > > >>> > > > >> > email.
> > > > > > >>> > > > >> > > > I've
> > > > > > >>> > > > >> > > > > > > > updated
> > > > > > >>> > > > >> > > > > > > > >> > the
> > > > > > >>> > > > >> > > > > > > > >> > > > KIP
> > > > > > >>> > > > >> > > > > > > > >> > > > > to
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > include
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > tagged
> > fields
> > > > for
> > > > > > the
> > > > > > >>> > > requests
> > > > > > >>> > > > >> and
> > > > > > >>> > > > >> > > > > > responses.
> > > > > > >>> > > > >> > > > > > > > >> (More
> > > > > > >>> > > > >> > > > > > > > >> > on
> > > > > > >>> > > > >> > > > > > > > >> > > > > that
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > below)
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 13) I
> think
> > part
> > > > > of
> > > > > > >>> the
> > > > > > >>> > idea
> > > > > > >>> > > > for
> > > > > > >>> > > > >> > > > > including
> > > > > > >>> > > > >> > > > > > > this
> > > > > > >>> > > > >> > > > > > > > >> > > > > information
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> is to
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > prepare
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > for future
> > > > > changes.
> > > > > > >>> > Perhaps
> > > > > > >>> > > > the
> > > > > > >>> > > > >> > > > directory
> > > > > > >>> > > > >> > > > > > > > >> structure
> > > > > > >>> > > > >> > > > > > > > >> > > > might
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> change
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > from
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > >>> topicName_partitionNumber
> > > > > > >>> > to
> > > > > > >>> > > > >> > > something
> > > > > > >>> > > > >> > > > > like
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > topicID_partitionNumber.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Then
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > it would
> be
> > > > useful
> > > > > > to
> > > > > > >>> have
> > > > > > >>> > > the
> > > > > > >>> > > > >> > topic
> > > > > > >>> > > > >> > > > name
> > > > > > >>> > > > >> > > > > > in
> > > > > > >>> > > > >> > > > > > > > the
> > > > > > >>> > > > >> > > > > > > > >> > file
> > > > > > >>> > > > >> > > > > > > > >> > > > > since
> > > > > > >>> > > > >> > > > > > > > >> > > > > > it
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > would
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > not
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > be
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > in the
> > directory
> > > > > > >>> > structure.
> > > > > > >>> > > > >> > > Supporting
> > > > > > >>> > > > >> > > > > > topic
> > > > > > >>> > > > >> > > > > > > > >> renames
> > > > > > >>> > > > >> > > > > > > > >> > > > might
> > > > > > >>> > > > >> > > > > > > > >> > > > > > be
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > easier
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > if
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > other
> > fields are
> > > > > > >>> included.
> > > > > > >>> > > > Would
> > > > > > >>> > > > >> > > there
> > > > > > >>> > > > >> > > > be
> > > > > > >>> > > > >> > > > > > any
> > > > > > >>> > > > >> > > > > > > > >> > > downsides
> > > > > > >>> > > > >> > > > > > > > >> > > > to
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > including
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > this
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> information?
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 14)  Yes,
> we
> > > > would
> > > > > > >>> need to
> > > > > > >>> > > > copy
> > > > > > >>> > > > >> the
> > > > > > >>> > > > >> > > > > > partition
> > > > > > >>> > > > >> > > > > > > > >> > metadata
> > > > > > >>> > > > >> > > > > > > > >> > > > > file
> > > > > > >>> > > > >> > > > > > > > >> > > > > > in
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > this
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > process.
> > I've
> > > > > > updated
> > > > > > >>> the
> > > > > > >>> > > KIP
> > > > > > >>> > > > to
> > > > > > >>> > > > >> > > > include
> > > > > > >>> > > > >> > > > > > > this.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 15) I
> > believe
> > > > > Lucas
> > > > > > >>> meant
> > > > > > >>> > v1
> > > > > > >>> > > > >> and v2
> > > > > > >>> > > > >> > > > here.
> > > > > > >>> > > > >> > > > > > He
> > > > > > >>> > > > >> > > > > > > > was
> > > > > > >>> > > > >> > > > > > > > >> > > > referring
> > > > > > >>> > > > >> > > > > > > > >> > > > > > to
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> how
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > requests
> > would
> > > > > fall
> > > > > > >>> under
> > > > > > >>> > > > >> different
> > > > > > >>> > > > >> > > IBP
> > > > > > >>> > > > >> > > > > and
> > > > > > >>> > > > >> > > > > > > > meant
> > > > > > >>> > > > >> > > > > > > > >> > that
> > > > > > >>> > > > >> > > > > > > > >> > > > > older
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > brokers
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > would
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > have to
> use
> > the
> > > > > > older
> > > > > > >>> > > version
> > > > > > >>> > > > of
> > > > > > >>> > > > >> > the
> > > > > > >>> > > > >> > > > > > request
> > > > > > >>> > > > >> > > > > > > > and
> > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > existing
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > topic
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > deletion
> > > > process.
> > > > > At
> > > > > > >>> > first,
> > > > > > >>> > > it
> > > > > > >>> > > > >> > seemed
> > > > > > >>> > > > >> > > > > like
> > > > > > >>> > > > >> > > > > > > > tagged
> > > > > > >>> > > > >> > > > > > > > >> > > fields
> > > > > > >>> > > > >> > > > > > > > >> > > > > > would
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > resolve
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > the IBP
> > issue.
> > > > > > >>> However, we
> > > > > > >>> > > may
> > > > > > >>> > > > >> need
> > > > > > >>> > > > >> > > IBP
> > > > > > >>> > > > >> > > > > for
> > > > > > >>> > > > >> > > > > > > > this
> > > > > > >>> > > > >> > > > > > > > >> > > request
> > > > > > >>> > > > >> > > > > > > > >> > > > > > after
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > all
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > since
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > the
> > controller
> > > > > > >>> handles the
> > > > > > >>> > > > topic
> > > > > > >>> > > > >> > > > deletion
> > > > > > >>> > > > >> > > > > > > > >> > differently
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> depending
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > on
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > IBP
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > version.
> In
> > an
> > > > > older
> > > > > > >>> > > version,
> > > > > > >>> > > > we
> > > > > > >>> > > > >> > > can't
> > > > > > >>> > > > >> > > > > just
> > > > > > >>> > > > >> > > > > > > > send
> > > > > > >>> > > > >> > > > > > > > >> a
> > > > > > >>> > > > >> > > > > > > > >> > > > > > StopReplica
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > delete
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > topic
> > > > immediately
> > > > > > like
> > > > > > >>> > we'd
> > > > > > >>> > > > >> want to
> > > > > > >>> > > > >> > > for
> > > > > > >>> > > > >> > > > > > this
> > > > > > >>> > > > >> > > > > > > > KIP.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > This makes
> > me
> > > > > wonder
> > > > > > >>> if we
> > > > > > >>> > > > want
> > > > > > >>> > > > >> > > tagged
> > > > > > >>> > > > >> > > > > > fields
> > > > > > >>> > > > >> > > > > > > > on
> > > > > > >>> > > > >> > > > > > > > >> all
> > > > > > >>> > > > >> > > > > > > > >> > > the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> requests
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > after
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > all. Let
> me
> > know
> > > > > > your
> > > > > > >>> > > > thoughts!
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > Justine
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > On Tue,
> Sep
> > 15,
> > > > > 2020
> > > > > > >>> at
> > > > > > >>> > 1:03
> > > > > > >>> > > > PM
> > > > > > >>> > > > >> > > Justine
> > > > > > >>> > > > >> > > > > > > Olshan
> > > > > > >>> > > > >> > > > > > > > <
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > jolshan@confluent.io
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > wrote:
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Hi all,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Jun
> > brought up
> > > > a
> > > > > > good
> > > > > > >>> > point
> > > > > > >>> > > > in
> > > > > > >>> > > > >> his
> > > > > > >>> > > > >> > > > last
> > > > > > >>> > > > >> > > > > > > email
> > > > > > >>> > > > >> > > > > > > > >> about
> > > > > > >>> > > > >> > > > > > > > >> > > > > tagged
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > fields,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > and
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> I've
> > updated
> > > > the
> > > > > > KIP
> > > > > > >>> to
> > > > > > >>> > > > reflect
> > > > > > >>> > > > >> > that
> > > > > > >>> > > > >> > > > the
> > > > > > >>> > > > >> > > > > > > > >> changes to
> > > > > > >>> > > > >> > > > > > > > >> > > > > > requests
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > responses
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> will be
> in
> > the
> > > > > form
> > > > > > >>> of
> > > > > > >>> > > tagged
> > > > > > >>> > > > >> > fields
> > > > > > >>> > > > >> > > > to
> > > > > > >>> > > > >> > > > > > > avoid
> > > > > > >>> > > > >> > > > > > > > >> > > changing
> > > > > > >>> > > > >> > > > > > > > >> > > > > IBP.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Jun: I
> > plan on
> > > > > > >>> sending a
> > > > > > >>> > > > >> followup
> > > > > > >>> > > > >> > > > email
> > > > > > >>> > > > >> > > > > to
> > > > > > >>> > > > >> > > > > > > > >> address
> > > > > > >>> > > > >> > > > > > > > >> > > some
> > > > > > >>> > > > >> > > > > > > > >> > > > > of
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > other
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> points.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Thanks,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Justine
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> On Mon,
> > Sep 14,
> > > > > > 2020
> > > > > > >>> at
> > > > > > >>> > > 4:25
> > > > > > >>> > > > PM
> > > > > > >>> > > > >> > Jun
> > > > > > >>> > > > >> > > > Rao
> > > > > > >>> > > > >> > > > > <
> > > > > > >>> > > > >> > > > > > > > >> > > > > jun@confluent.io>
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > wrote:
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Hi,
> > Justine,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Thanks
> > for the
> > > > > > >>> updated
> > > > > > >>> > > KIP.
> > > > > > >>> > > > A
> > > > > > >>> > > > >> few
> > > > > > >>> > > > >> > > > > > comments
> > > > > > >>> > > > >> > > > > > > > >> below.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 10.
> > > > LeaderAndIsr
> > > > > > >>> > Response:
> > > > > > >>> > > > Do
> > > > > > >>> > > > >> we
> > > > > > >>> > > > >> > > need
> > > > > > >>> > > > >> > > > > the
> > > > > > >>> > > > >> > > > > > > > topic
> > > > > > >>> > > > >> > > > > > > > >> > > name?
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 11. For
> > the
> > > > > > changed
> > > > > > >>> > > > >> > > request/response,
> > > > > > >>> > > > >> > > > > > other
> > > > > > >>> > > > >> > > > > > > > >> than
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> LeaderAndIsr,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > UpdateMetadata,
> > > > > > >>> > Metadata,
> > > > > > >>> > > do
> > > > > > >>> > > > >> we
> > > > > > >>> > > > >> > > need
> > > > > > >>> > > > >> > > > to
> > > > > > >>> > > > >> > > > > > > > include
> > > > > > >>> > > > >> > > > > > > > >> > the
> > > > > > >>> > > > >> > > > > > > > >> > > > > topic
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> name?
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 12. It
> > seems
> > > > > that
> > > > > > >>> > upgrades
> > > > > > >>> > > > >> don't
> > > > > > >>> > > > >> > > > > require
> > > > > > >>> > > > >> > > > > > > IBP.
> > > > > > >>> > > > >> > > > > > > > >> Does
> > > > > > >>> > > > >> > > > > > > > >> > > > that
> > > > > > >>> > > > >> > > > > > > > >> > > > > > mean
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > new
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> fields
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> in all
> the
> > > > > > >>> > > request/response
> > > > > > >>> > > > >> are
> > > > > > >>> > > > >> > > added
> > > > > > >>> > > > >> > > > > as
> > > > > > >>> > > > >> > > > > > > > tagged
> > > > > > >>> > > > >> > > > > > > > >> > > fields
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> without
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > bumping
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > up
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> the
> > request
> > > > > > >>> version? It
> > > > > > >>> > > > would
> > > > > > >>> > > > >> be
> > > > > > >>> > > > >> > > > useful
> > > > > > >>> > > > >> > > > > > to
> > > > > > >>> > > > >> > > > > > > > make
> > > > > > >>> > > > >> > > > > > > > >> > that
> > > > > > >>> > > > >> > > > > > > > >> > > > > > clear.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 13.
> > Partition
> > > > > > >>> Metadata
> > > > > > >>> > > file:
> > > > > > >>> > > > >> Do
> > > > > > >>> > > > >> > we
> > > > > > >>> > > > >> > > > need
> > > > > > >>> > > > >> > > > > > to
> > > > > > >>> > > > >> > > > > > > > >> include
> > > > > > >>> > > > >> > > > > > > > >> > > the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > topic
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > name
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > and
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> partition
> > id
> > > > > since
> > > > > > >>> they
> > > > > > >>> > > are
> > > > > > >>> > > > >> > implied
> > > > > > >>> > > > >> > > > in
> > > > > > >>> > > > >> > > > > > the
> > > > > > >>> > > > >> > > > > > > > >> > directory
> > > > > > >>> > > > >> > > > > > > > >> > > > > name?
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 14. In
> the
> > > > JBOD
> > > > > > >>> mode, we
> > > > > > >>> > > > >> support
> > > > > > >>> > > > >> > > > > moving a
> > > > > > >>> > > > >> > > > > > > > >> > > partition's
> > > > > > >>> > > > >> > > > > > > > >> > > > > data
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> from
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > one
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > disk
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> to
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> another.
> > Will
> > > > > the
> > > > > > >>> new
> > > > > > >>> > > > >> partition
> > > > > > >>> > > > >> > > > > metadata
> > > > > > >>> > > > >> > > > > > > file
> > > > > > >>> > > > >> > > > > > > > >> be
> > > > > > >>> > > > >> > > > > > > > >> > > > copied
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> during
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> process?
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 15. The
> > KIP
> > > > says
> > > > > > >>> "Remove
> > > > > > >>> > > > >> deleted
> > > > > > >>> > > > >> > > > topics
> > > > > > >>> > > > >> > > > > > > from
> > > > > > >>> > > > >> > > > > > > > >> > > replicas
> > > > > > >>> > > > >> > > > > > > > >> > > > by
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > sending
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > StopReplicaRequest
> > > > > > >>> V2
> > > > > > >>> > for
> > > > > > >>> > > > any
> > > > > > >>> > > > >> > > topics
> > > > > > >>> > > > >> > > > > > which
> > > > > > >>> > > > >> > > > > > > do
> > > > > > >>> > > > >> > > > > > > > >> not
> > > > > > >>> > > > >> > > > > > > > >> > > > > contain
> > > > > > >>> > > > >> > > > > > > > >> > > > > > a
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > topic
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > ID,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > and
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> V3 for
> any
> > > > > topics
> > > > > > >>> which
> > > > > > >>> > do
> > > > > > >>> > > > >> > contain
> > > > > > >>> > > > >> > > a
> > > > > > >>> > > > >> > > > > > topic
> > > > > > >>> > > > >> > > > > > > > >> ID.".
> > > > > > >>> > > > >> > > > > > > > >> > > > > However,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > it
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > seems
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> updated
> > > > > controller
> > > > > > >>> will
> > > > > > >>> > > > create
> > > > > > >>> > > > >> > all
> > > > > > >>> > > > >> > > > > > missing
> > > > > > >>> > > > >> > > > > > > > >> topic
> > > > > > >>> > > > >> > > > > > > > >> > IDs
> > > > > > >>> > > > >> > > > > > > > >> > > > > first
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > before
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > doing
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> other
> > actions.
> > > > > So,
> > > > > > >>> is
> > > > > > >>> > > > >> > > > > StopReplicaRequest
> > > > > > >>> > > > >> > > > > > V2
> > > > > > >>> > > > >> > > > > > > > >> > needed?
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Jun
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> On Fri,
> > Sep
> > > > 11,
> > > > > > >>> 2020 at
> > > > > > >>> > > > 10:31
> > > > > > >>> > > > >> AM
> > > > > > >>> > > > >> > > John
> > > > > > >>> > > > >> > > > > > > > Roesler <
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> vvcephei@apache.org
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> wrote:
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> Thanks,
> > > > > Justine!
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > Your
> > > > response
> > > > > > >>> seems
> > > > > > >>> > > > >> compelling
> > > > > > >>> > > > >> > to
> > > > > > >>> > > > >> > > > me.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > -John
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > On
> Fri,
> > > > > > >>> 2020-09-11 at
> > > > > > >>> > > > 10:17
> > > > > > >>> > > > >> > > -0700,
> > > > > > >>> > > > >> > > > > > > Justine
> > > > > > >>> > > > >> > > > > > > > >> > Olshan
> > > > > > >>> > > > >> > > > > > > > >> > > > > wrote:
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> Hello
> > all,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > Thanks for
> > > > > > >>> > continuing
> > > > > > >>> > > > the
> > > > > > >>> > > > >> > > > > > discussion! I
> > > > > > >>> > > > >> > > > > > > > >> have a
> > > > > > >>> > > > >> > > > > > > > >> > > few
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > responses
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > to
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > your
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> points.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Tom:
> > You
> > > > are
> > > > > > >>> correct
> > > > > > >>> > > in
> > > > > > >>> > > > >> that
> > > > > > >>> > > > >> > > this
> > > > > > >>> > > > >> > > > > KIP
> > > > > > >>> > > > >> > > > > > > has
> > > > > > >>> > > > >> > > > > > > > >> not
> > > > > > >>> > > > >> > > > > > > > >> > > > > > mentioned
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > >>> > DeleteTopicsRequest. I
> > > > > > >>> > > > >> think
> > > > > > >>> > > > >> > > that
> > > > > > >>> > > > >> > > > > > this
> > > > > > >>> > > > >> > > > > > > > >> would
> > > > > > >>> > > > >> > > > > > > > >> > be
> > > > > > >>> > > > >> > > > > > > > >> > > > out
> > > > > > >>> > > > >> > > > > > > > >> > > > > of
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > scope
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > for
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> now, but
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > may
> be
> > > > > > something
> > > > > > >>> > worth
> > > > > > >>> > > > >> adding
> > > > > > >>> > > > >> > > in
> > > > > > >>> > > > >> > > > > the
> > > > > > >>> > > > >> > > > > > > > >> future.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> John:
> > We
> > > > did
> > > > > > >>> > consider
> > > > > > >>> > > > >> > sequence
> > > > > > >>> > > > >> > > > ids,
> > > > > > >>> > > > >> > > > > > but
> > > > > > >>> > > > >> > > > > > > > >> there
> > > > > > >>> > > > >> > > > > > > > >> > > are
> > > > > > >>> > > > >> > > > > > > > >> > > > a
> > > > > > >>> > > > >> > > > > > > > >> > > > > > few
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > reasons
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> favor
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> UUIDs.
> > > > There
> > > > > > are
> > > > > > >>> > > several
> > > > > > >>> > > > >> > cases
> > > > > > >>> > > > >> > > > > where
> > > > > > >>> > > > >> > > > > > > > topics
> > > > > > >>> > > > >> > > > > > > > >> > from
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> different
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > clusters
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> may
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > interact
> > > > now
> > > > > > >>> and in
> > > > > > >>> > > the
> > > > > > >>> > > > >> > future.
> > > > > > >>> > > > >> > > > For
> > > > > > >>> > > > >> > > > > > > > >> example,
> > > > > > >>> > > > >> > > > > > > > >> > > > Mirror
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Maker 2
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > may
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> benefit
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > from
> > being
> > > > > > able
> > > > > > >>> to
> > > > > > >>> > > > detect
> > > > > > >>> > > > >> > when
> > > > > > >>> > > > >> > > a
> > > > > > >>> > > > >> > > > > > > cluster
> > > > > > >>> > > > >> > > > > > > > >> being
> > > > > > >>> > > > >> > > > > > > > >> > > > > > mirrored
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> is
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > deleted
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> and
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > recreated
> > > > > and
> > > > > > >>> > globally
> > > > > > >>> > > > >> unique
> > > > > > >>> > > > >> > > > > > > identifiers
> > > > > > >>> > > > >> > > > > > > > >> > would
> > > > > > >>> > > > >> > > > > > > > >> > > > make
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > resolving
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > issues
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> easier
> > > > than
> > > > > > >>> sequence
> > > > > > >>> > > IDs
> > > > > > >>> > > > >> > which
> > > > > > >>> > > > >> > > > may
> > > > > > >>> > > > >> > > > > > > > collide
> > > > > > >>> > > > >> > > > > > > > >> > > between
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > clusters.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > KIP-405
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > (tiered
> > > > > > storage)
> > > > > > >>> > will
> > > > > > >>> > > > also
> > > > > > >>> > > > >> > > > benefit
> > > > > > >>> > > > >> > > > > > from
> > > > > > >>> > > > >> > > > > > > > >> > globally
> > > > > > >>> > > > >> > > > > > > > >> > > > > > unique
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> IDs
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > as
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > shared
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > buckets
> > > > may
> > > > > be
> > > > > > >>> used
> > > > > > >>> > > > >> between
> > > > > > >>> > > > >> > > > > clusters.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > Globally
> > > > > > unique
> > > > > > >>> IDs
> > > > > > >>> > > > would
> > > > > > >>> > > > >> > also
> > > > > > >>> > > > >> > > > make
> > > > > > >>> > > > >> > > > > > > > >> > > functionality
> > > > > > >>> > > > >> > > > > > > > >> > > > > like
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > moving
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > topics
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > between
> > > > > > >>> disparate
> > > > > > >>> > > > clusters
> > > > > > >>> > > > >> > > easier
> > > > > > >>> > > > >> > > > > in
> > > > > > >>> > > > >> > > > > > > the
> > > > > > >>> > > > >> > > > > > > > >> > future,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> simplify
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > any
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > future
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > implementations
> > > > > > >>> of
> > > > > > >>> > > > backups
> > > > > > >>> > > > >> > and
> > > > > > >>> > > > >> > > > > > > restores,
> > > > > > >>> > > > >> > > > > > > > >> and
> > > > > > >>> > > > >> > > > > > > > >> > > more.
> > > > > > >>> > > > >> > > > > > > > >> > > > > In
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > general,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> unique
> IDs
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> would
> > > > ensure
> > > > > > >>> that
> > > > > > >>> > the
> > > > > > >>> > > > >> source
> > > > > > >>> > > > >> > > > > cluster
> > > > > > >>> > > > >> > > > > > > > >> topics do
> > > > > > >>> > > > >> > > > > > > > >> > > not
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> conflict
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > with
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > destination
> > > > > > >>> cluster
> > > > > > >>> > > > >> topics.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > If
> we
> > were
> > > > > to
> > > > > > >>> use
> > > > > > >>> > > > sequence
> > > > > > >>> > > > >> > ids,
> > > > > > >>> > > > >> > > > we
> > > > > > >>> > > > >> > > > > > > would
> > > > > > >>> > > > >> > > > > > > > >> need
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> sufficiently
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > large
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> cluster
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > ids
> > to be
> > > > > > stored
> > > > > > >>> > with
> > > > > > >>> > > > the
> > > > > > >>> > > > >> > topic
> > > > > > >>> > > > >> > > > > > > > >> identifiers or
> > > > > > >>> > > > >> > > > > > > > >> > > we
> > > > > > >>> > > > >> > > > > > > > >> > > > > run
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > risk
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > collisions.
> > > > > > This
> > > > > > >>> > will
> > > > > > >>> > > > >> give up
> > > > > > >>> > > > >> > > any
> > > > > > >>> > > > >> > > > > > > > >> advantage in
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> compactness
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> sequence
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > numbers
> > > > may
> > > > > > >>> bring.
> > > > > > >>> > > Given
> > > > > > >>> > > > >> > these
> > > > > > >>> > > > >> > > > > > > > advantages I
> > > > > > >>> > > > >> > > > > > > > >> > > think
> > > > > > >>> > > > >> > > > > > > > >> > > > it
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> makes
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > sense
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> use
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> UUIDs.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> Gokul:
> > > > This
> > > > > is
> > > > > > >>> an
> > > > > > >>> > > > >> interesting
> > > > > > >>> > > > >> > > > idea,
> > > > > > >>> > > > >> > > > > > but
> > > > > > >>> > > > >> > > > > > > > >> this
> > > > > > >>> > > > >> > > > > > > > >> > is
> > > > > > >>> > > > >> > > > > > > > >> > > a
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> breaking
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > change.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Out of
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> scope
> > for
> > > > > now,
> > > > > > >>> but
> > > > > > >>> > > maybe
> > > > > > >>> > > > >> > worth
> > > > > > >>> > > > >> > > > > > > discussing
> > > > > > >>> > > > >> > > > > > > > >> in
> > > > > > >>> > > > >> > > > > > > > >> > the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > future.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Hope
> > this
> > > > > > >>> explains
> > > > > > >>> > > some
> > > > > > >>> > > > of
> > > > > > >>> > > > >> > the
> > > > > > >>> > > > >> > > > > > > decisions,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > Justine
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > On
> > Fri,
> > > > Sep
> > > > > > 11,
> > > > > > >>> 2020
> > > > > > >>> > > at
> > > > > > >>> > > > >> 8:27
> > > > > > >>> > > > >> > AM
> > > > > > >>> > > > >> > > > > Gokul
> > > > > > >>> > > > >> > > > > > > > >> Ramanan
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Subramanian <
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > >>> > gokul2411s@gmail.com>
> > > > > > >>> > > > >> wrote:
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> Hi.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > Thanks
> > > > for
> > > > > > the
> > > > > > >>> > KIP.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > Have you
> > > > > > >>> thought
> > > > > > >>> > > about
> > > > > > >>> > > > >> > > whether
> > > > > > >>> > > > >> > > > it
> > > > > > >>> > > > >> > > > > > > makes
> > > > > > >>> > > > >> > > > > > > > >> > sense
> > > > > > >>> > > > >> > > > > > > > >> > > to
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> support
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > authorizing a
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > principal
> > > > > > for
> > > > > > >>> a
> > > > > > >>> > > topic
> > > > > > >>> > > > ID
> > > > > > >>> > > > >> > > rather
> > > > > > >>> > > > >> > > > > > than
> > > > > > >>> > > > >> > > > > > > a
> > > > > > >>> > > > >> > > > > > > > >> topic
> > > > > > >>> > > > >> > > > > > > > >> > > > name
> > > > > > >>> > > > >> > > > > > > > >> > > > > to
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > achieve
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> tighter
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > security?
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Or
> > is
> > > > the
> > > > > > >>> topic ID
> > > > > > >>> > > > >> > > > fundamentally
> > > > > > >>> > > > >> > > > > an
> > > > > > >>> > > > >> > > > > > > > >> internal
> > > > > > >>> > > > >> > > > > > > > >> > > > > detail
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > similar
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > to
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> epochs
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > used
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> in a
> > > > bunch
> > > > > > of
> > > > > > >>> > other
> > > > > > >>> > > > >> places
> > > > > > >>> > > > >> > in
> > > > > > >>> > > > >> > > > > > Kafka?
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > Thanks.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > On
> > Fri,
> > > > > Sep
> > > > > > >>> 11,
> > > > > > >>> > 2020
> > > > > > >>> > > > at
> > > > > > >>> > > > >> > 4:06
> > > > > > >>> > > > >> > > PM
> > > > > > >>> > > > >> > > > > > John
> > > > > > >>> > > > >> > > > > > > > >> > Roesler <
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > vvcephei@apache.org
> > > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > wrote:
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > Hello
> > > > > > >>> Justine,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > Thanks
> > > > > for
> > > > > > >>> the
> > > > > > >>> > > KIP!
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> I
> > > > happen
> > > > > > to
> > > > > > >>> have
> > > > > > >>> > > > been
> > > > > > >>> > > > >> > > > > confronted
> > > > > > >>> > > > >> > > > > > > > >> recently
> > > > > > >>> > > > >> > > > > > > > >> > > with
> > > > > > >>> > > > >> > > > > > > > >> > > > > the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> need
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > to
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > keep
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > track
> of
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > a
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > large
> > > > > > >>> number of
> > > > > > >>> > > > >> topics as
> > > > > > >>> > > > >> > > > > > compactly
> > > > > > >>> > > > >> > > > > > > > as
> > > > > > >>> > > > >> > > > > > > > >> > > > > possible. I
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> was
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > going
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> come
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > up
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > with
> > > > > some
> > > > > > >>> way to
> > > > > > >>> > > > >> > dictionary
> > > > > > >>> > > > >> > > > > > encode
> > > > > > >>> > > > >> > > > > > > > the
> > > > > > >>> > > > >> > > > > > > > >> > topic
> > > > > > >>> > > > >> > > > > > > > >> > > > > names
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> as
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > integers,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> but
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > this
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > seems
> > > > > much
> > > > > > >>> > better!
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > Apologies
> > > > > > if
> > > > > > >>> > this
> > > > > > >>> > > > has
> > > > > > >>> > > > >> > been
> > > > > > >>> > > > >> > > > > raised
> > > > > > >>> > > > >> > > > > > > > >> before,
> > > > > > >>> > > > >> > > > > > > > >> > > but
> > > > > > >>> > > > >> > > > > > > > >> > > > > I’m
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > wondering
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> about
> the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > choice
> > > > > of
> > > > > > >>> UUID
> > > > > > >>> > vs
> > > > > > >>> > > > >> > sequence
> > > > > > >>> > > > >> > > > > number
> > > > > > >>> > > > >> > > > > > > for
> > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > >>> > > > >> > > > > > > > >> > > ids.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Typically,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > I’ve
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> seen
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > UUIDs
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > in two
> > > > > > >>> > situations:
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> 1.
> > > > When
> > > > > > >>> > processes
> > > > > > >>> > > > >> need to
> > > > > > >>> > > > >> > > > > > generate
> > > > > > >>> > > > >> > > > > > > > >> > > > non-colliding
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > identifiers
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> without
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > >>> coordination.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> 2.
> > > > When
> > > > > > the
> > > > > > >>> > > > identifier
> > > > > > >>> > > > >> > > needs
> > > > > > >>> > > > >> > > > to
> > > > > > >>> > > > >> > > > > > be
> > > > > > >>> > > > >> > > > > > > > >> > > > “universally
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > unique”;
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > I.e.,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > identifier
> > > > > > >>> needs
> > > > > > >>> > > to
> > > > > > >>> > > > >> > > > distinguish
> > > > > > >>> > > > >> > > > > > the
> > > > > > >>> > > > >> > > > > > > > >> entity
> > > > > > >>> > > > >> > > > > > > > >> > > > from
> > > > > > >>> > > > >> > > > > > > > >> > > > > > all
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > other
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> entities
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > that
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > could
> > > > > ever
> > > > > > >>> > exist.
> > > > > > >>> > > > >> This is
> > > > > > >>> > > > >> > > > > useful
> > > > > > >>> > > > >> > > > > > in
> > > > > > >>> > > > >> > > > > > > > >> cases
> > > > > > >>> > > > >> > > > > > > > >> > > > where
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > entities
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > from
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > all
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > kinds
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > of
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > systems
> > > > > > get
> > > > > > >>> > mixed
> > > > > > >>> > > > >> > together,
> > > > > > >>> > > > >> > > > > such
> > > > > > >>> > > > >> > > > > > as
> > > > > > >>> > > > >> > > > > > > > >> when
> > > > > > >>> > > > >> > > > > > > > >> > > > dumping
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> logs
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > from
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > all
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > processes
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > in
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> a
> > > > > company
> > > > > > >>> into a
> > > > > > >>> > > > >> common
> > > > > > >>> > > > >> > > > system.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > Maybe
> > > > > I’m
> > > > > > >>> being
> > > > > > >>> > > > >> > > > short-sighted,
> > > > > > >>> > > > >> > > > > > but
> > > > > > >>> > > > >> > > > > > > it
> > > > > > >>> > > > >> > > > > > > > >> > > doesn’t
> > > > > > >>> > > > >> > > > > > > > >> > > > > seem
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> like
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > either
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> really
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > applies
> > > > > > >>> here. It
> > > > > > >>> > > > seems
> > > > > > >>> > > > >> > like
> > > > > > >>> > > > >> > > > the
> > > > > > >>> > > > >> > > > > > > > brokers
> > > > > > >>> > > > >> > > > > > > > >> > > could
> > > > > > >>> > > > >> > > > > > > > >> > > > > and
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> would
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > achieve
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > consensus
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > when
> > > > > > >>> creating a
> > > > > > >>> > > > topic
> > > > > > >>> > > > >> > > anyway,
> > > > > > >>> > > > >> > > > > > which
> > > > > > >>> > > > >> > > > > > > > is
> > > > > > >>> > > > >> > > > > > > > >> all
> > > > > > >>> > > > >> > > > > > > > >> > > > > that’s
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > required
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > to
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> generate
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > >>> non-colliding
> > > > > > >>> > > > sequence
> > > > > > >>> > > > >> > ids.
> > > > > > >>> > > > >> > > > For
> > > > > > >>> > > > >> > > > > > the
> > > > > > >>> > > > >> > > > > > > > >> > second,
> > > > > > >>> > > > >> > > > > > > > >> > > as
> > > > > > >>> > > > >> > > > > > > > >> > > > > you
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > mention,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > we
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> could
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > assign
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> a
> > UUID
> > > > > to
> > > > > > >>> the
> > > > > > >>> > > > cluster
> > > > > > >>> > > > >> as
> > > > > > >>> > > > >> > a
> > > > > > >>> > > > >> > > > > whole,
> > > > > > >>> > > > >> > > > > > > > which
> > > > > > >>> > > > >> > > > > > > > >> > > would
> > > > > > >>> > > > >> > > > > > > > >> > > > > > render
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > any
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > resource
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > scoped
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > the
> > > > > broker
> > > > > > >>> > > > universally
> > > > > > >>> > > > >> > > unique
> > > > > > >>> > > > >> > > > > as
> > > > > > >>> > > > >> > > > > > > > well.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > The
> > > > > > reason I
> > > > > > >>> > > mention
> > > > > > >>> > > > >> this
> > > > > > >>> > > > >> > > is
> > > > > > >>> > > > >> > > > > > that,
> > > > > > >>> > > > >> > > > > > > > >> > although
> > > > > > >>> > > > >> > > > > > > > >> > > a
> > > > > > >>> > > > >> > > > > > > > >> > > > > UUID
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> is
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > way
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > more
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> compact
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > than
> > > > > topic
> > > > > > >>> > names,
> > > > > > >>> > > > it’s
> > > > > > >>> > > > >> > > still
> > > > > > >>> > > > >> > > > 16
> > > > > > >>> > > > >> > > > > > > > bytes.
> > > > > > >>> > > > >> > > > > > > > >> In
> > > > > > >>> > > > >> > > > > > > > >> > > > > > contrast,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> a
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > 4-byte
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> integer
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > sequence
> > > > > > id
> > > > > > >>> > would
> > > > > > >>> > > > give
> > > > > > >>> > > > >> > us 4
> > > > > > >>> > > > >> > > > > > billion
> > > > > > >>> > > > >> > > > > > > > >> unique
> > > > > > >>> > > > >> > > > > > > > >> > > > > topics
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> per
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > cluster,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> which
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > seems
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > like
> > > > > > enough
> > > > > > >>> ;)
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > Considering
> > > > > > >>> the
> > > > > > >>> > > > >> number of
> > > > > > >>> > > > >> > > > > > different
> > > > > > >>> > > > >> > > > > > > > >> times
> > > > > > >>> > > > >> > > > > > > > >> > > > these
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> topic
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > identifiers
> > > > are
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> sent
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > over
> > > > the
> > > > > > >>> wire or
> > > > > > >>> > > > >> stored
> > > > > > >>> > > > >> > in
> > > > > > >>> > > > >> > > > > > memory,
> > > > > > >>> > > > >> > > > > > > it
> > > > > > >>> > > > >> > > > > > > > >> > seems
> > > > > > >>> > > > >> > > > > > > > >> > > > like
> > > > > > >>> > > > >> > > > > > > > >> > > > > > it
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > might
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > be
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> worth
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > additional
> > > > > > >>> 4x
> > > > > > >>> > > space
> > > > > > >>> > > > >> > > savings.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > What
> > > > do
> > > > > > you
> > > > > > >>> > think
> > > > > > >>> > > > >> about
> > > > > > >>> > > > >> > > this?
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > Thanks,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > John
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> On
> > > > Fri,
> > > > > > Sep
> > > > > > >>> 11,
> > > > > > >>> > > > 2020,
> > > > > > >>> > > > >> at
> > > > > > >>> > > > >> > > > 03:20,
> > > > > > >>> > > > >> > > > > > Tom
> > > > > > >>> > > > >> > > > > > > > >> > Bentley
> > > > > > >>> > > > >> > > > > > > > >> > > > > wrote:
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > Hi
> > > > > > >>> Justine,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > This
> > > > > > looks
> > > > > > >>> > like
> > > > > > >>> > > a
> > > > > > >>> > > > >> very
> > > > > > >>> > > > >> > > > > welcome
> > > > > > >>> > > > >> > > > > > > > >> > > improvement.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Thanks!
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > > Maybe
> > > > > I
> > > > > > >>> missed
> > > > > > >>> > > it,
> > > > > > >>> > > > >> but
> > > > > > >>> > > > >> > > the
> > > > > > >>> > > > >> > > > > KIP
> > > > > > >>> > > > >> > > > > > > > >> doesn't
> > > > > > >>> > > > >> > > > > > > > >> > > seem
> > > > > > >>> > > > >> > > > > > > > >> > > > to
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > mention
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > changing
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > > > > >>> > > > DeleteTopicsRequest
> > > > > > >>> > > > >> to
> > > > > > >>> > > > >> > > > > identify
> > > > > > >>> > > > >> > > > > > > the
> > > > > > >>> > > > >> > > > > > > > >> > topic
> > > > > > >>> > > > >> > > > > > > > >> > > > > using
> > > > > > >>> > > > >> > > > > > > > >> > > > > > an
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > id.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Maybe
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > that's
> > out
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> of
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > > scope,
> > > > > > but
> > > > > > >>> > > > >> > > > > DeleteTopicsRequest
> > > > > > >>> > > > >> > > > > > is
> > > > > > >>> > > > >> > > > > > > > not
> > > > > > >>> > > > >> > > > > > > > >> > > listed
> > > > > > >>> > > > >> > > > > > > > >> > > > > > among
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Future
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Work
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > APIs
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > > > either.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > Kind
> > > > > > >>> regards,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > Tom
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > On
> > > > > Thu,
> > > > > > >>> Sep
> > > > > > >>> > 10,
> > > > > > >>> > > > >> 2020 at
> > > > > > >>> > > > >> > > > 3:59
> > > > > > >>> > > > >> > > > > PM
> > > > > > >>> > > > >> > > > > > > > >> Satish
> > > > > > >>> > > > >> > > > > > > > >> > > > > Duggana <
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > > >>> > > > >> satish.duggana@gmail.com
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > > wrote:
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > Thanks
> > > > > > >>> > > > >> Lucas/Justine
> > > > > > >>> > > > >> > > for
> > > > > > >>> > > > >> > > > > the
> > > > > > >>> > > > >> > > > > > > nice
> > > > > > >>> > > > >> > > > > > > > >> KIP.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > It
> > > > > has
> > > > > > >>> > several
> > > > > > >>> > > > >> > benefits
> > > > > > >>> > > > >> > > > > which
> > > > > > >>> > > > >> > > > > > > > also
> > > > > > >>> > > > >> > > > > > > > >> > > include
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > simplifying
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> topic
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > > deletion
> > > > > > >>> > > process
> > > > > > >>> > > > >> by
> > > > > > >>> > > > >> > > > > > controller
> > > > > > >>> > > > >> > > > > > > > and
> > > > > > >>> > > > >> > > > > > > > >> > logs
> > > > > > >>> > > > >> > > > > > > > >> > > > > > cleanup
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> by
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > brokers
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > in
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > corner
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > cases.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > Best,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > > Satish.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > On
> > > > > > Wed,
> > > > > > >>> Sep
> > > > > > >>> > 9,
> > > > > > >>> > > > >> 2020
> > > > > > >>> > > > >> > at
> > > > > > >>> > > > >> > > > > 10:07
> > > > > > >>> > > > >> > > > > > PM
> > > > > > >>> > > > >> > > > > > > > >> > Justine
> > > > > > >>> > > > >> > > > > > > > >> > > > > > Olshan <
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > >>> jolshan@confluent.io
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > wrote:
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > > > Hello
> > > > > > >>> all,
> > > > > > >>> > > > it's
> > > > > > >>> > > > >> > been
> > > > > > >>> > > > >> > > > > > almost a
> > > > > > >>> > > > >> > > > > > > > >> year!
> > > > > > >>> > > > >> > > > > > > > >> > > I've
> > > > > > >>> > > > >> > > > > > > > >> > > > > > made
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > some
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > changes
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> to
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > this
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > KIP
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > and
> > > > > > >>> hope to
> > > > > > >>> > > > >> continue
> > > > > > >>> > > > >> > > the
> > > > > > >>> > > > >> > > > > > > > >> discussion.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > > One
> > > > > > >>> of the
> > > > > > >>> > > > main
> > > > > > >>> > > > >> > > changes
> > > > > > >>> > > > >> > > > > > I've
> > > > > > >>> > > > >> > > > > > > > >> added
> > > > > > >>> > > > >> > > > > > > > >> > is
> > > > > > >>> > > > >> > > > > > > > >> > > > now
> > > > > > >>> > > > >> > > > > > > > >> > > > > > the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > metadata
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> response
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> will
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > > include
> > > > > > >>> the
> > > > > > >>> > > > topic
> > > > > > >>> > > > >> ID
> > > > > > >>> > > > >> > > (as
> > > > > > >>> > > > >> > > > > > Colin
> > > > > > >>> > > > >> > > > > > > > >> > > suggested).
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> Clients
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > can
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> obtain
> the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > topicID
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > of a
> > > > > > >>> given
> > > > > > >>> > > topic
> > > > > > >>> > > > >> > > through
> > > > > > >>> > > > >> > > > a
> > > > > > >>> > > > >> > > > > > > > >> > > > TopicDescription.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > The
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > topicId
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > will
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > also
> be
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > > included
> > > > > > >>> > with
> > > > > > >>> > > > the
> > > > > > >>> > > > >> > > > > > > UpdateMetadata
> > > > > > >>> > > > >> > > > > > > > >> > > request.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > > Let
> > > > > > me
> > > > > > >>> > know
> > > > > > >>> > > > what
> > > > > > >>> > > > >> > you
> > > > > > >>> > > > >> > > > all
> > > > > > >>> > > > >> > > > > > > think.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > > > Thank
> > > > > > >>> you,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > > > >>> Justine
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > On
> > > > > > >>> > > 2019/09/13
> > > > > > >>> > > > >> > > 16:38:26,
> > > > > > >>> > > > >> > > > > > > "Colin
> > > > > > >>> > > > >> > > > > > > > >> > > McCabe" <
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > cmccabe@apache.org
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > wrote:
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > Hi
> > > > > > >>> > Lucas,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > > >>> Thanks
> > > > > > >>> > for
> > > > > > >>> > > > >> > tackling
> > > > > > >>> > > > >> > > > > this.
> > > > > > >>> > > > >> > > > > > > > >> Topic
> > > > > > >>> > > > >> > > > > > > > >> > IDs
> > > > > > >>> > > > >> > > > > > > > >> > > > > are a
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > great
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > idea,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> and
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > this
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > is
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> a
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > really
> > > > > > >>> good
> > > > > > >>> > > > >> writeup.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > > For
> > > > > > >>> > > > >> > > > > > > /brokers/topics/[topic],
> > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > >>> > > > >> > > > > > > > >> > > > schema
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> version
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > should
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > be
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > bumped
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > > version
> > > > > > >>> 3,
> > > > > > >>> > > > rather
> > > > > > >>> > > > >> > than
> > > > > > >>> > > > >> > > 2.
> > > > > > >>> > > > >> > > > > > > > KIP-455
> > > > > > >>> > > > >> > > > > > > > >> > > bumped
> > > > > > >>> > > > >> > > > > > > > >> > > > > the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > version
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > this
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > znode
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> 2
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > > already
> > > > > > >>> :)
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > > >>> Given
> > > > > > >>> > that
> > > > > > >>> > > > >> we're
> > > > > > >>> > > > >> > > > going
> > > > > > >>> > > > >> > > > > to
> > > > > > >>> > > > >> > > > > > > be
> > > > > > >>> > > > >> > > > > > > > >> > seeing
> > > > > > >>> > > > >> > > > > > > > >> > > > > these
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > things
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > as
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> strings
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > as
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> lot
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > (in
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > logs,
> > > > > > in
> > > > > > >>> > > > >> ZooKeeper,
> > > > > > >>> > > > >> > on
> > > > > > >>> > > > >> > > > the
> > > > > > >>> > > > >> > > > > > > > >> > command-line,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > etc.),
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > does
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > it
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > make
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > sense
> to
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > use
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > base64
> > > > > > >>> when
> > > > > > >>> > > > >> > converting
> > > > > > >>> > > > >> > > > them
> > > > > > >>> > > > >> > > > > > to
> > > > > > >>> > > > >> > > > > > > > >> > strings?
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > > >>> Here is
> > > > > > >>> > an
> > > > > > >>> > > > >> > example
> > > > > > >>> > > > >> > > of
> > > > > > >>> > > > >> > > > > the
> > > > > > >>> > > > >> > > > > > > hex
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> representation:
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > > >>> > > > >> > > > > > > > >> >
> > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > > And
> > > > > > >>> here
> > > > > > >>> > > is
> > > > > > >>> > > > an
> > > > > > >>> > > > >> > > > example
> > > > > > >>> > > > >> > > > > in
> > > > > > >>> > > > >> > > > > > > > >> base64.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > > >>> > > > >> > > > b8tRS7h4TJ2Vt43Dp85v2A
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > > The
> > > > > > >>> > base64
> > > > > > >>> > > > >> > version
> > > > > > >>> > > > >> > > > > saves
> > > > > > >>> > > > >> > > > > > 15
> > > > > > >>> > > > >> > > > > > > > >> > letters
> > > > > > >>> > > > >> > > > > > > > >> > > > (to
> > > > > > >>> > > > >> > > > > > > > >> > > > > be
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > fair,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > 4
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> those
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > were
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > dashes
> > > > > > >>> that
> > > > > > >>> > we
> > > > > > >>> > > > >> could
> > > > > > >>> > > > >> > > have
> > > > > > >>> > > > >> > > > > > > elided
> > > > > > >>> > > > >> > > > > > > > in
> > > > > > >>> > > > >> > > > > > > > >> > the
> > > > > > >>> > > > >> > > > > > > > >> > > > hex
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > representation.)
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > > >>> Another
> > > > > > >>> > > > thing
> > > > > > >>> > > > >> to
> > > > > > >>> > > > >> > > > > consider
> > > > > > >>> > > > >> > > > > > > is
> > > > > > >>> > > > >> > > > > > > > >> that
> > > > > > >>> > > > >> > > > > > > > >> > we
> > > > > > >>> > > > >> > > > > > > > >> > > > > > should
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > specify
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > that
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > > >>> all-zeroes
> > > > > > >>> > > UUID
> > > > > > >>> > > > is
> > > > > > >>> > > > >> > not
> > > > > > >>> > > > >> > > a
> > > > > > >>> > > > >> > > > > > valid
> > > > > > >>> > > > >> > > > > > > > >> topic
> > > > > > >>> > > > >> > > > > > > > >> > > UUID.
> > > > > > >>> > > > >> > > > > > > > >> > > > > >  We
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > can't
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > use
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> null
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > for
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> this
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > > because
> > > > > > >>> we
> > > > > > >>> > > can't
> > > > > > >>> > > > >> > pass a
> > > > > > >>> > > > >> > > > > null
> > > > > > >>> > > > >> > > > > > > UUID
> > > > > > >>> > > > >> > > > > > > > >> over
> > > > > > >>> > > > >> > > > > > > > >> > > the
> > > > > > >>> > > > >> > > > > > > > >> > > > > RPC
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > protocol
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> (there
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > is no
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > > special
> > > > > > >>> > > pattern
> > > > > > >>> > > > >> for
> > > > > > >>> > > > >> > > null,
> > > > > > >>> > > > >> > > > > nor
> > > > > > >>> > > > >> > > > > > > do
> > > > > > >>> > > > >> > > > > > > > we
> > > > > > >>> > > > >> > > > > > > > >> > want
> > > > > > >>> > > > >> > > > > > > > >> > > > to
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> waste
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > space
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> reserving
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> such
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> a
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > > >>> pattern.)
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > > >>> Maybe I
> > > > > > >>> > > > missed
> > > > > > >>> > > > >> > it,
> > > > > > >>> > > > >> > > > but
> > > > > > >>> > > > >> > > > > > did
> > > > > > >>> > > > >> > > > > > > > you
> > > > > > >>> > > > >> > > > > > > > >> > > > describe
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > "migration
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > of...
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> existing
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > > topic[s]
> > > > > > >>> > > without
> > > > > > >>> > > > >> > topic
> > > > > > >>> > > > >> > > > IDs"
> > > > > > >>> > > > >> > > > > > in
> > > > > > >>> > > > >> > > > > > > > >> detail
> > > > > > >>> > > > >> > > > > > > > >> > in
> > > > > > >>> > > > >> > > > > > > > >> > > > any
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > section?
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > It
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> seems
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > like
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > when
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > the
> > > > > > new
> > > > > > >>> > > > controller
> > > > > > >>> > > > >> > > > becomes
> > > > > > >>> > > > >> > > > > > > > active,
> > > > > > >>> > > > >> > > > > > > > >> it
> > > > > > >>> > > > >> > > > > > > > >> > > > should
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> just
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > generate
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> random
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > UUIDs
> > > > > for
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > these,
> > > > > > >>> and
> > > > > > >>> > > write
> > > > > > >>> > > > >> the
> > > > > > >>> > > > >> > > > random
> > > > > > >>> > > > >> > > > > > > UUIDs
> > > > > > >>> > > > >> > > > > > > > >> back
> > > > > > >>> > > > >> > > > > > > > >> > > to
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > ZooKeeper.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > It
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> would be
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> good
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> to
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > spell
> > > > > > >>> that
> > > > > > >>> > > out.
> > > > > > >>> > > > >> We
> > > > > > >>> > > > >> > > > should
> > > > > > >>> > > > >> > > > > > make
> > > > > > >>> > > > >> > > > > > > > it
> > > > > > >>> > > > >> > > > > > > > >> > clear
> > > > > > >>> > > > >> > > > > > > > >> > > > > that
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> this
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > happens
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > regardless
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> of
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > the
> > > > > > >>> > > inter-broker
> > > > > > >>> > > > >> > > protocol
> > > > > > >>> > > > >> > > > > > > version
> > > > > > >>> > > > >> > > > > > > > >> > (it's
> > > > > > >>> > > > >> > > > > > > > >> > > a
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > compatible
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > change).
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > > >>> > > > >> > > "LeaderAndIsrRequests
> > > > > > >>> > > > >> > > > > > > > >> including an
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> is_every_partition
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> flag"
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > seems a
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > bit
> > > > > > >>> wordy.
> > > > > > >>> > > Can
> > > > > > >>> > > > we
> > > > > > >>> > > > >> > just
> > > > > > >>> > > > >> > > > > call
> > > > > > >>> > > > >> > > > > > > > these
> > > > > > >>> > > > >> > > > > > > > >> > "full
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > >>> LeaderAndIsrRequests"?
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > Then
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > RPC
> > > > > > >>> field
> > > > > > >>> > > could
> > > > > > >>> > > > be
> > > > > > >>> > > > >> > > named
> > > > > > >>> > > > >> > > > > > > "full".
> > > > > > >>> > > > >> > > > > > > > >> > Also,
> > > > > > >>> > > > >> > > > > > > > >> > > it
> > > > > > >>> > > > >> > > > > > > > >> > > > > > would
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > probably
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > be
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > better
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > for
> > > > the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > RPC
> > > > > > >>> field to
> > > > > > >>> > > be
> > > > > > >>> > > > an
> > > > > > >>> > > > >> > enum
> > > > > > >>> > > > >> > > > of
> > > > > > >>> > > > >> > > > > {
> > > > > > >>> > > > >> > > > > > > > >> > > UNSPECIFIED,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > INCREMENTAL,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > FULL
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> }, so
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> that
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> we
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > can
> > > > > > >>> cleanly
> > > > > > >>> > > > handle
> > > > > > >>> > > > >> > old
> > > > > > >>> > > > >> > > > > > versions
> > > > > > >>> > > > >> > > > > > > > (by
> > > > > > >>> > > > >> > > > > > > > >> > > > treating
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> them
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > as
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > UNSPECIFIED)
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > In
> > > > > > >>> the
> > > > > > >>> > > > >> > > > > > LeaderAndIsrRequest
> > > > > > >>> > > > >> > > > > > > > >> > section,
> > > > > > >>> > > > >> > > > > > > > >> > > > you
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> write
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > "A
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > final
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> deletion
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > event
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > will
> > > > > > be
> > > > > > >>> > > > secheduled
> > > > > > >>> > > > >> > for
> > > > > > >>> > > > >> > > X
> > > > > > >>> > > > >> > > > ms
> > > > > > >>> > > > >> > > > > > > after
> > > > > > >>> > > > >> > > > > > > > >> the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > LeaderAndIsrRequest
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > was
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > first
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > > >>> received..."
> > > > > > >>> > > I
> > > > > > >>> > > > >> guess
> > > > > > >>> > > > >> > > > the X
> > > > > > >>> > > > >> > > > > > > was a
> > > > > > >>> > > > >> > > > > > > > >> > > > > placeholder
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> that
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > you
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> intended
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > replace
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > before
> > > > > > >>> > > posting?
> > > > > > >>> > > > :)
> > > > > > >>> > > > >> > In
> > > > > > >>> > > > >> > > > any
> > > > > > >>> > > > >> > > > > > > case,
> > > > > > >>> > > > >> > > > > > > > >> this
> > > > > > >>> > > > >> > > > > > > > >> > > > seems
> > > > > > >>> > > > >> > > > > > > > >> > > > > > like
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > kind
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > of
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > thing
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> we'd
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > want a
> > > > > > >>> > > > >> configuration
> > > > > > >>> > > > >> > > for.
> > > > > > >>> > > > >> > > > > > > Let's
> > > > > > >>> > > > >> > > > > > > > >> > > describe
> > > > > > >>> > > > >> > > > > > > > >> > > > > that
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> configuration
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> key
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > somewhere
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > in
> > > > > > this
> > > > > > >>> KIP,
> > > > > > >>> > > > >> > including
> > > > > > >>> > > > >> > > > what
> > > > > > >>> > > > >> > > > > > its
> > > > > > >>> > > > >> > > > > > > > >> > default
> > > > > > >>> > > > >> > > > > > > > >> > > > > value
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> is.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > We
> > > > > > >>> > should
> > > > > > >>> > > > >> > probably
> > > > > > >>> > > > >> > > > also
> > > > > > >>> > > > >> > > > > > > log a
> > > > > > >>> > > > >> > > > > > > > >> > bunch
> > > > > > >>> > > > >> > > > > > > > >> > > of
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> messages
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > at
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > WARN
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> level
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> when
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > > >>> something is
> > > > > > >>> > > > >> > scheduled
> > > > > > >>> > > > >> > > > for
> > > > > > >>> > > > >> > > > > > > > >> deletion,
> > > > > > >>> > > > >> > > > > > > > >> > as
> > > > > > >>> > > > >> > > > > > > > >> > > > > well.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > (Maybe
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > this
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> was
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > assumed,
> > > > > > but
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > it
> > > > > > >>> would be
> > > > > > >>> > > good
> > > > > > >>> > > > >> to
> > > > > > >>> > > > >> > > > mention
> > > > > > >>> > > > >> > > > > > > it).
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > I
> > > > > > >>> feel
> > > > > > >>> > > like
> > > > > > >>> > > > >> there
> > > > > > >>> > > > >> > > > are a
> > > > > > >>> > > > >> > > > > > few
> > > > > > >>> > > > >> > > > > > > > >> > sections
> > > > > > >>> > > > >> > > > > > > > >> > > > > that
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > should
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > be
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> moved to
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > "rejected
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > > >>> > alternatives."
> > > > > > >>> > > > >> For
> > > > > > >>> > > > >> > > > > example,
> > > > > > >>> > > > >> > > > > > in
> > > > > > >>> > > > >> > > > > > > > the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > DeleteTopics
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > section,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> since
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > we're
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > not
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > going
> > > > > > >>> to do
> > > > > > >>> > > > >> option 1
> > > > > > >>> > > > >> > or
> > > > > > >>> > > > >> > > > 2,
> > > > > > >>> > > > >> > > > > > > these
> > > > > > >>> > > > >> > > > > > > > >> > should
> > > > > > >>> > > > >> > > > > > > > >> > > be
> > > > > > >>> > > > >> > > > > > > > >> > > > > > moved
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > into
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> "rejected
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > > >>> > alternatives,"
> > > > > > >>> > > > >> > rather
> > > > > > >>> > > > >> > > > than
> > > > > > >>> > > > >> > > > > > > > >> appearing
> > > > > > >>> > > > >> > > > > > > > >> > > > > inline.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Another
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > case
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> is
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > "Should
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > we
> > > > > > >>> remove
> > > > > > >>> > > topic
> > > > > > >>> > > > >> name
> > > > > > >>> > > > >> > > from
> > > > > > >>> > > > >> > > > > the
> > > > > > >>> > > > >> > > > > > > > >> protocol
> > > > > > >>> > > > >> > > > > > > > >> > > > where
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > possible"
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> section.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > This
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> is
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > > clearly
> > > > > > >>> > > > >> discussing a
> > > > > > >>> > > > >> > > > design
> > > > > > >>> > > > >> > > > > > > > >> > alternative
> > > > > > >>> > > > >> > > > > > > > >> > > > that
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> we're
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > not
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> proposing
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > > >>> implement:
> > > > > > >>> > > > >> removing
> > > > > > >>> > > > >> > the
> > > > > > >>> > > > >> > > > > topic
> > > > > > >>> > > > >> > > > > > > > name
> > > > > > >>> > > > >> > > > > > > > >> > from
> > > > > > >>> > > > >> > > > > > > > >> > > > > those
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > protocols.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > Is
> > > > > > >>> it
> > > > > > >>> > > really
> > > > > > >>> > > > >> > > > necessary
> > > > > > >>> > > > >> > > > > to
> > > > > > >>> > > > >> > > > > > > > have
> > > > > > >>> > > > >> > > > > > > > >> a
> > > > > > >>> > > > >> > > > > > > > >> > new
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > >>> > > /admin/delete_topics_by_id
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > path
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > in
> > > > > > >>> > ZooKeeper?
> > > > > > >>> > > > It
> > > > > > >>> > > > >> > seems
> > > > > > >>> > > > >> > > > > like
> > > > > > >>> > > > >> > > > > > we
> > > > > > >>> > > > >> > > > > > > > >> don't
> > > > > > >>> > > > >> > > > > > > > >> > > > really
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> need
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > this.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Whenever
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > there
> > > > is
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > a
> > > > > new
> > > > > > >>> > > > controller,
> > > > > > >>> > > > >> > we'll
> > > > > > >>> > > > >> > > > > send
> > > > > > >>> > > > >> > > > > > > out
> > > > > > >>> > > > >> > > > > > > > >> full
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > LeaderAndIsrRequests
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> which
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > will
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > > trigger
> > > > > > >>> the
> > > > > > >>> > > > stale
> > > > > > >>> > > > >> > > topics
> > > > > > >>> > > > >> > > > to
> > > > > > >>> > > > >> > > > > > be
> > > > > > >>> > > > >> > > > > > > > >> cleaned
> > > > > > >>> > > > >> > > > > > > > >> > > up.
> > > > > > >>> > > > >> > > > > > > > >> > > > > >  The
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > active
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > controller
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> will
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > also
> > > > > > >>> send
> > > > > > >>> > the
> > > > > > >>> > > > full
> > > > > > >>> > > > >> > > > > > > > >> LeaderAndIsrRequest
> > > > > > >>> > > > >> > > > > > > > >> > > to
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> brokers
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > are
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> just
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > starting
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > up.
> > > > > > >>> So we
> > > > > > >>> > > > don't
> > > > > > >>> > > > >> > > really
> > > > > > >>> > > > >> > > > > > need
> > > > > > >>> > > > >> > > > > > > > this
> > > > > > >>> > > > >> > > > > > > > >> > kind
> > > > > > >>> > > > >> > > > > > > > >> > > > of
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > two-phase
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > commit
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > (send
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> out
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > > >>> > > > >> StopReplicasRequest,
> > > > > > >>> > > > >> > > get
> > > > > > >>> > > > >> > > > > ACKs
> > > > > > >>> > > > >> > > > > > > > from
> > > > > > >>> > > > >> > > > > > > > >> all
> > > > > > >>> > > > >> > > > > > > > >> > > > > nodes,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > commit
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > by
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> removing
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > > >>> > > > >> /admin/delete_topics
> > > > > > >>> > > > >> > > > node)
> > > > > > >>> > > > >> > > > > > any
> > > > > > >>> > > > >> > > > > > > > >> more.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > > You
> > > > > > >>> > > mention
> > > > > > >>> > > > >> that
> > > > > > >>> > > > >> > > > > > > FetchRequest
> > > > > > >>> > > > >> > > > > > > > >> will
> > > > > > >>> > > > >> > > > > > > > >> > > now
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> include
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > UUID
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> avoid
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > issues
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > where
> > > > > > >>> > requests
> > > > > > >>> > > > are
> > > > > > >>> > > > >> > made
> > > > > > >>> > > > >> > > > to
> > > > > > >>> > > > >> > > > > > > stale
> > > > > > >>> > > > >> > > > > > > > >> > > > partitions.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > However,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> adding a
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > UUID
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > > >>> > > MetadataRequest
> > > > > > >>> > > > is
> > > > > > >>> > > > >> > > listed
> > > > > > >>> > > > >> > > > > as
> > > > > > >>> > > > >> > > > > > > > future
> > > > > > >>> > > > >> > > > > > > > >> > > work,
> > > > > > >>> > > > >> > > > > > > > >> > > > > out
> > > > > > >>> > > > >> > > > > > > > >> > > > > > of
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > scope
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > for
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> this
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > KIP.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > How
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > will
> > > > > > the
> > > > > > >>> > > client
> > > > > > >>> > > > >> learn
> > > > > > >>> > > > >> > > > what
> > > > > > >>> > > > >> > > > > > the
> > > > > > >>> > > > >> > > > > > > > >> topic
> > > > > > >>> > > > >> > > > > > > > >> > > UUID
> > > > > > >>> > > > >> > > > > > > > >> > > > > is,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > if
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > metadata
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > response
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > > doesn't
> > > > > > >>> > > include
> > > > > > >>> > > > >> that
> > > > > > >>> > > > >> > > > > > > information?
> > > > > > >>> > > > >> > > > > > > > >> It
> > > > > > >>> > > > >> > > > > > > > >> > > > seems
> > > > > > >>> > > > >> > > > > > > > >> > > > > > like
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > adding
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> UUID
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > > >>> > > MetadataResponse
> > > > > > >>> > > > >> > would
> > > > > > >>> > > > >> > > be
> > > > > > >>> > > > >> > > > > an
> > > > > > >>> > > > >> > > > > > > > >> > improvement
> > > > > > >>> > > > >> > > > > > > > >> > > > > here
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> that
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > might
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > not
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> be
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > too
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > hard
> > > > to
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > make.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > > >>> best,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > > >>> Colin
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > On
> > > > > > >>> Mon,
> > > > > > >>> > > Sep
> > > > > > >>> > > > 9,
> > > > > > >>> > > > >> > > 2019,
> > > > > > >>> > > > >> > > > at
> > > > > > >>> > > > >> > > > > > > > 17:48,
> > > > > > >>> > > > >> > > > > > > > >> > > Ryanne
> > > > > > >>> > > > >> > > > > > > > >> > > > > > Dolan
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > wrote:
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > >
> > > > > > >>> Lucas,
> > > > > > >>> > > > this
> > > > > > >>> > > > >> > would
> > > > > > >>> > > > >> > > > be
> > > > > > >>> > > > >> > > > > > > great.
> > > > > > >>> > > > >> > > > > > > > >> I've
> > > > > > >>> > > > >> > > > > > > > >> > > run
> > > > > > >>> > > > >> > > > > > > > >> > > > > > into
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > issues
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > with
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > topics
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > being
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > >
> > > > > > >>> > > > resurrected
> > > > > > >>> > > > >> > > > > > accidentally,
> > > > > > >>> > > > >> > > > > > > > >> since
> > > > > > >>> > > > >> > > > > > > > >> > a
> > > > > > >>> > > > >> > > > > > > > >> > > > > client
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > cannot
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > easily
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > distinguish
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > > between
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > >
> > > > > > a
> > > > > > >>> > > deleted
> > > > > > >>> > > > >> topic
> > > > > > >>> > > > >> > > > and a
> > > > > > >>> > > > >> > > > > > new
> > > > > > >>> > > > >> > > > > > > > >> topic
> > > > > > >>> > > > >> > > > > > > > >> > > with
> > > > > > >>> > > > >> > > > > > > > >> > > > > the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> same
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > name.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I'd
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > need
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> ID
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > >
> > > > > > >>> > > accessible
> > > > > > >>> > > > >> from
> > > > > > >>> > > > >> > > the
> > > > > > >>> > > > >> > > > > > > client
> > > > > > >>> > > > >> > > > > > > > to
> > > > > > >>> > > > >> > > > > > > > >> > > solve
> > > > > > >>> > > > >> > > > > > > > >> > > > > that
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > issue,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > but
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> this
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > is a
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > good
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > first
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > >
> > > > > > >>> step.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > >
> > > > > > >>> Ryanne
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > >
> > > > > > On
> > > > > > >>> > Wed,
> > > > > > >>> > > > Sep
> > > > > > >>> > > > >> 4,
> > > > > > >>> > > > >> > > 2019
> > > > > > >>> > > > >> > > > > at
> > > > > > >>> > > > >> > > > > > > 1:41
> > > > > > >>> > > > >> > > > > > > > >> PM
> > > > > > >>> > > > >> > > > > > > > >> > > Lucas
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Bradstreet <
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > >>> > > lucas@confluent.io>
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > wrote:
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > >>> Hi
> > > > > > >>> > > all,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > >>> I
> > > > > > >>> > > would
> > > > > > >>> > > > >> like
> > > > > > >>> > > > >> > to
> > > > > > >>> > > > >> > > > > kick
> > > > > > >>> > > > >> > > > > > > off
> > > > > > >>> > > > >> > > > > > > > >> > > > discussion
> > > > > > >>> > > > >> > > > > > > > >> > > > > of
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > KIP-516,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > an
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > >>> implementation
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > of
> > > > > > topic
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > >>> IDs
> > > > > > >>> > > for
> > > > > > >>> > > > >> > Kafka.
> > > > > > >>> > > > >> > > > > Topic
> > > > > > >>> > > > >> > > > > > > IDs
> > > > > > >>> > > > >> > > > > > > > >> aim
> > > > > > >>> > > > >> > > > > > > > >> > to
> > > > > > >>> > > > >> > > > > > > > >> > > > > solve
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> topic
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> uniqueness
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > problems
> > > > > > in
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > Kafka,
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > >>> > where
> > > > > > >>> > > > >> > referring
> > > > > > >>> > > > >> > > > to
> > > > > > >>> > > > >> > > > > a
> > > > > > >>> > > > >> > > > > > > > topic
> > > > > > >>> > > > >> > > > > > > > >> by
> > > > > > >>> > > > >> > > > > > > > >> > > name
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> alone is
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > insufficient.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> Such
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > cases
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > >>> > > include
> > > > > > >>> > > > >> when
> > > > > > >>> > > > >> > a
> > > > > > >>> > > > >> > > > > topic
> > > > > > >>> > > > >> > > > > > > has
> > > > > > >>> > > > >> > > > > > > > >> been
> > > > > > >>> > > > >> > > > > > > > >> > > > > deleted
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > recreated
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> with
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > the
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > same
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > name.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > >>> > Unique
> > > > > > >>> > > > >> > > > identifiers
> > > > > > >>> > > > >> > > > > > will
> > > > > > >>> > > > >> > > > > > > > >> help
> > > > > > >>> > > > >> > > > > > > > >> > > > > simplify
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > improve
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Kafka's
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > topic
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > > deletion
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > >>> > > process,
> > > > > > >>> > > > >> as
> > > > > > >>> > > > >> > > well
> > > > > > >>> > > > >> > > > as
> > > > > > >>> > > > >> > > > > > > > prevent
> > > > > > >>> > > > >> > > > > > > > >> > > cases
> > > > > > >>> > > > >> > > > > > > > >> > > > > > where
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > brokers
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > may
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > incorrectly
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > >
> > > > > > interact
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > >>> with
> > > > > > >>> > > > stale
> > > > > > >>> > > > >> > > > versions
> > > > > > >>> > > > >> > > > > > of
> > > > > > >>> > > > >> > > > > > > > >> topics.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > >>> > > > >> > > > > > > > >>
> > > > > > >>> > > > >> > > > > > > >
> > > > > > >>> > > > >> > > > > > >
> > > > > > >>> > > > >> > > > > >
> > > > > > >>> > > > >> > > > >
> > > > > > >>> > > > >> > > >
> > > > > > >>> > > > >> > >
> > > > > > >>> > > > >> >
> > > > > > >>> > > > >>
> > > > > > >>> > > >
> > > > > > >>> > >
> > > > > > >>> >
> > > > > > >>>
> > > > > >
> > > > >
> > > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > >>> > > Looking
> > > > > > >>> > > > >> > forward
> > > > > > >>> > > > >> > > > to
> > > > > > >>> > > > >> > > > > > your
> > > > > > >>> > > > >> > > > > > > > >> > > thoughts.
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > >>> > Lucas
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > > >>> > > > >> > > > > > > > >> > > > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > > >>> > > > >> > > > > > > > >> > >
> > > > > > >>> > > > >> > > > > > > > >> >
> > > > > > >>> > > > >> > > > > > > > >>
> > > > > > >>> > > > >> > > > > > > > >
> > > > > > >>> > > > >> > > > > > > >
> > > > > > >>> > > > >> > > > > > >
> > > > > > >>> > > > >> > > > > >
> > > > > > >>> > > > >> > > > >
> > > > > > >>> > > > >> > > >
> > > > > > >>> > > > >> > >
> > > > > > >>> > > > >> >
> > > > > > >>> > > > >>
> > > > > > >>> > > > >
> > > > > > >>> > > >
> > > > > > >>> > >
> > > > > > >>> >
> > > > > > >>>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> >
> >
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Justine Olshan <jo...@confluent.io>]

Hi all,

After some careful consideration, I've decided that the TopicIdPartition
will be out of scope for this KIP. After investigating what it will look
like using it in only the Fetch Request, I found that I would need to
modify several public APIs in the consumer and I would need to take special
care in many situations to handle the request correctly based on what
version of the request was sent. For these reasons, I've added
TopicIdPartition to potential future work along with an alternative to
modify TopicPartition to include a topic ID field.

Jun- I was also wondering about some of the details for supporting topic
IDs for delete and describe topics in the admin client. Is the idea that
instead of supplying topic names to describe or delete, we would supply a
collection of ids? If that is the case, I'm wondering if it makes sense to
add topic Ids to the metadata and delete topics requests.

Thanks,
Justine

On Mon, Oct 5, 2020 at 9:25 PM Satish Duggana <sa...@gmail.com>
wrote:

> Hi Jun, Justine,
> +1 to have TopicIdPartition in public packages. This is useful for
> KIP-405 tiered storage too. We currently use TopicPartition for
> pluggable log storage and metadata storage apis. We will be using
> TopicIdPartition once the proposed changes in this KIP are merged.
>
> Thanks,
> Satish.
>
> On Tue, Oct 6, 2020 at 5:21 AM Jun Rao <ju...@confluent.io> wrote:
> >
> > Hi, Justine,
> >
> > I guess we have to maintain the mapping between topic name and topicId in
> > both the client and the server. If TopicIdPartition is meant for internal
> > usage only, we can create it under a package
> > like org/apache/kafka/common/internal and we don't need to document it in
> > the KIP. If it's meant for external public usage (and potentially
> internal
> > usage too), we can create it in org/apache/kafka/common and need to
> > document it in the KIP.
> >
> > Since topicId is now part of the topic metadata, perhaps it's useful to
> > expose it in TopicMetadataAndConfig too.
> >
> > Thanks,
> >
> > Jun
> >
> > On Mon, Oct 5, 2020 at 3:35 PM Justine Olshan <jo...@confluent.io>
> wrote:
> >
> > > Hi Jun,
> > >
> > > This KIP removes the topic name from many requests. A lot of requests
> keep
> > > track of the partitions internally with a TopicPartition. So when we
> remove
> > > the topic name, we can not differentiate unless we can also use topic
> ID.
> > > The Fetch request relies heavily on this, but ListOffset,
> > > OffsetsForLeaderEpoch, etc all use TopicPartition as a key value for
> > > internal maps. Digging deeper into Fetch, I've noticed that the code
> path
> > > runs into some public APIs that use the TopicPartition. Some examples
> are
> > > Cluster.java or RecordTooLargeException.
> > >
> > > A related issue is that in some places like
> TopicAuthorizationException,
> > > the topic name is required and without the name in the request, we
> won't
> > > know it.
> > >
> > > I guess an alternative would be to have some mapping from topic name to
> > > topic ID, but I would have to think about that more.
> > >
> > > The KIP mentions exposing TopicId in TopicDescription. (The font was
> > > weirdly colored, so I've made it easier to read.)
> > >
> > > Would the idea for the TopicMetadataAndConfig be that the topic ID
> would be
> > > returned as a result?
> > > As for the other methods describe/deletetopics, it makes sense to
> include
> > > them at some point, but I'm not sure how far we should go in terms of
> > > supporting topic IDs everywhere.
> > >
> > > Thank you for continuing to look at the KIP!
> > > Justine
> > >
> > >
> > > On Mon, Oct 5, 2020 at 2:53 PM Jun Rao <ju...@confluent.io> wrote:
> > >
> > > > Hi, Justine,
> > > >
> > > > Adding a TopicIdPartition in the public interface may be useful.
> Where
> > > > would you expect it to be used?
> > > >
> > > > Related to this, since we are exposing topicId in the tool, it seems
> that
> > > > we may want to expose topicId in at least TopicDescription used by
> > > > AdminClient.describeTopics() and TopicMetadataAndConfig used in
> > > > AdminClient.createTopics(Collection<NewTopic> newTopics). I am also
> not
> > > > sure if we plan to expose topicId in other AdminClient methods in
> this
> > > KIP
> > > > or in some future KIP. For example, it might be useful to support
> topicId
> > > > in the following methods.
> > > >
> > > >     DescribeTopicsResult describeTopics(Collection<String>
> topicNames,
> > > > DescribeTopicsOptions options);
> > > >     DeleteTopicsResult deleteTopics(Collection<String> topics,
> > > > DeleteTopicsOptions options);
> > > >
> > > > Thanks,
> > > >
> > > > Jun
> > > >
> > > > On Mon, Oct 5, 2020 at 11:03 AM Justine Olshan <jolshan@confluent.io
> >
> > > > wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > I was thinking about this KIP some more. We are removing the topic
> name
> > > > > from many of these request types, and some of them (eg. Fetch) use
> > > > > TopicPartition as keys in a map when handling the request. I was
> > > thinking
> > > > > it would make sense to make some sort of TopicIdPartition with
> topic
> > > > name,
> > > > > topic id, and partition fields (name and id would potentially be
> > > optional
> > > > > types since different request versions will contain only one of the
> > > two)
> > > > I
> > > > > was wondering if it made sense to include this as a public facing
> API
> > > > > similar to TopicPartition. If so, I will add this class to the KIP
> as
> > > > well.
> > > > >
> > > > > Thanks,
> > > > > Justine
> > > > >
> > > > > On Mon, Oct 5, 2020 at 8:52 AM Justine Olshan <
> jolshan@confluent.io>
> > > > > wrote:
> > > > >
> > > > > > Hi all,
> > > > > > I've made a few minor changes to the UUID class. I've added a few
> > > more
> > > > > > methods to support UUIDs more generally, by including a
> constructor
> > > > and a
> > > > > > method to generate random UUIDs. Both of these will be useful for
> > > > > testing.
> > > > > >
> > > > > > Let me know if there are any other areas for discussion.
> > > > > >
> > > > > > Thank you,
> > > > > > Justine
> > > > > >
> > > > > > On Thu, Oct 1, 2020 at 5:29 PM Justine Olshan <
> jolshan@confluent.io>
> > > > > > wrote:
> > > > > >
> > > > > >> Hi Jun,
> > > > > >> Thanks for looking at it again. Here's the new spec. (I fixed
> the
> > > typo
> > > > > in
> > > > > >> it too.)
> > > > > >> {"name": "id", "type": "string", "doc": option[UUID]}
> > > > > >>
> > > > > >> Justine
> > > > > >>
> > > > > >>
> > > > > >> On Thu, Oct 1, 2020 at 5:03 PM Jun Rao <ju...@confluent.io>
> wrote:
> > > > > >>
> > > > > >>> Hi, Justine,
> > > > > >>>
> > > > > >>> Thanks for the update. The KIP looks good to me now. Just a
> minor
> > > > > comment
> > > > > >>> below.
> > > > > >>>
> > > > > >>> 30. Perhaps "option[UUID]" can be put in the doc.
> > > > > >>>
> > > > > >>> Jun
> > > > > >>>
> > > > > >>> On Thu, Oct 1, 2020 at 3:28 PM Justine Olshan <
> > > jolshan@confluent.io>
> > > > > >>> wrote:
> > > > > >>>
> > > > > >>> > Hi Jun,
> > > > > >>> > Thanks for the response!
> > > > > >>> >
> > > > > >>> > 30. I think I might have changed this in between. The current
> > > > version
> > > > > >>> > says:  {"name":
> > > > > >>> > "id", "type": "option[UUID]"}, "doc": topic id}
> > > > > >>> > I have switched to the option type to cover the migration
> case
> > > > where
> > > > > a
> > > > > >>> > TopicZNode does not yet have a topic ID.
> > > > > >>> > I understand that due to json, this field is written as a
> string,
> > > > so
> > > > > >>> if I
> > > > > >>> > should move the "option[uuid]" to the doc field and the type
> > > should
> > > > > be
> > > > > >>> > "string" please let me know.
> > > > > >>> >
> > > > > >>> > 40. I've added a definition for UUID.
> > > > > >>> > 41,42. Fixed
> > > > > >>> >
> > > > > >>> > Thank you,
> > > > > >>> > Justine
> > > > > >>> >
> > > > > >>> > On Wed, Sep 30, 2020 at 1:15 PM Jun Rao <ju...@confluent.io>
> > > wrote:
> > > > > >>> >
> > > > > >>> > > Hi, Justine,
> > > > > >>> > >
> > > > > >>> > > Thanks for the summary. Just a few minor comments blow.
> > > > > >>> > >
> > > > > >>> > > 30.  {"name": "id", "type": "string", "doc": "version
> id"}}:
> > > The
> > > > > doc
> > > > > >>> > should
> > > > > >>> > > say UUID. The issue seems unfixed.
> > > > > >>> > >
> > > > > >>> > > 40. Since UUID is public facing, could you include its
> > > > definition?
> > > > > >>> > >
> > > > > >>> > > 41. StopReplicaResponse still includes the topic field.
> > > > > >>> > >
> > > > > >>> > > 42. "It is unnecessary to include the name of the topic in
> the
> > > > > >>> following
> > > > > >>> > > Request/Response calls" It would be useful to include all
> > > > modified
> > > > > >>> > requests
> > > > > >>> > > (e.g. produce) in the list below.
> > > > > >>> > >
> > > > > >>> > > Thanks,
> > > > > >>> > >
> > > > > >>> > > Jun
> > > > > >>> > >
> > > > > >>> > >
> > > > > >>> > > On Wed, Sep 30, 2020 at 10:17 AM Justine Olshan <
> > > > > >>> jolshan@confluent.io>
> > > > > >>> > > wrote:
> > > > > >>> > >
> > > > > >>> > > > Hello again,
> > > > > >>> > > >
> > > > > >>> > > > I've taken some time to discuss some of the remaining
> points
> > > > > >>> brought up
> > > > > >>> > > by
> > > > > >>> > > > the previous emails offline. Here are some of the
> > > conclusions.
> > > > > >>> > > >
> > > > > >>> > > > 1. Directory Structure:
> > > > > >>> > > > There was some talk about whether the directory structure
> > > > should
> > > > > be
> > > > > >>> > > changed
> > > > > >>> > > > to replace all topic names with topic IDs.
> > > > > >>> > > > This will be a useful change, but will prevent
> downgrades. It
> > > > > will
> > > > > >>> be
> > > > > >>> > > best
> > > > > >>> > > > to wait until a major release, likely alongside KIP-500
> > > changes
> > > > > >>> that
> > > > > >>> > will
> > > > > >>> > > > prevent downgrades. I've updated the KIP to include this
> > > change
> > > > > >>> with
> > > > > >>> > the
> > > > > >>> > > > note about migration and deprecation.
> > > > > >>> > > >
> > > > > >>> > > >
> > > > > >>> > >
> > > > > >>> >
> > > > > >>>
> > > > >
> > > >
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-log.dirlayout
> > > > > >>> > > >
> > > > > >>> > > > 2. Partition Metadata file
> > > > > >>> > > > There was some disagreement about the inclusion of the
> > > > partition
> > > > > >>> > metadata
> > > > > >>> > > > file.
> > > > > >>> > > > This file will be needed to persist the topic ID,
> especially
> > > > > while
> > > > > >>> we
> > > > > >>> > > still
> > > > > >>> > > > have the old directory structure. Even after the changes,
> > > this
> > > > > >>> file can
> > > > > >>> > > be
> > > > > >>> > > > useful for debugging and recovery.
> > > > > >>> > > > Creating a single mapping file from topic names to topic
> IDs
> > > > was
> > > > > >>> > > > considered, but ultimately scrapped as it would not be as
> > > easy
> > > > to
> > > > > >>> > > maintain.
> > > > > >>> > > >
> > > > > >>> > > >
> > > > > >>> > >
> > > > > >>> >
> > > > > >>>
> > > > >
> > > >
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PartitionMetadatafile
> > > > > >>> > > >
> > > > > >>> > > >
> > > > > >>> > > >
> > > > > >>> > >
> > > > > >>> >
> > > > > >>>
> > > > >
> > > >
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PersistingTopicIDs
> > > > > >>> > > >
> > > > > >>> > > > 3. Produce Protocols
> > > > > >>> > > > After some further discussion, this replacing the topic
> name
> > > > with
> > > > > >>> topic
> > > > > >>> > > ID
> > > > > >>> > > > in these protocols has been added to the KIP.
> > > > > >>> > > > This will cut down on the size of the protocol. Since
> changes
> > > > to
> > > > > >>> fetch
> > > > > >>> > > are
> > > > > >>> > > > included, it does make sense to update these protocols.
> > > > > >>> > > >
> > > > > >>> > > >
> > > > > >>> > >
> > > > > >>> >
> > > > > >>>
> > > > >
> > > >
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-Produce
> > > > > >>> > > >
> > > > > >>> > > > 4. KIP-500 Compatibility
> > > > > >>> > > > After some discussion with those involved with KIP-500,
> it
> > > > seems
> > > > > >>> best
> > > > > >>> > to
> > > > > >>> > > > use a sentinel topic ID for the metadata topic that is
> used
> > > > > before
> > > > > >>> the
> > > > > >>> > > > first controller election.
> > > > > >>> > > > We had an issue where this metadata topic may not be
> assigned
> > > > an
> > > > > ID
> > > > > >>> > > before
> > > > > >>> > > > utilizing the Vote and Fetch protocols. It was decided to
> > > > > reserve a
> > > > > >>> > > unique
> > > > > >>> > > > ID for this topic to be used until the controller could
> give
> > > > the
> > > > > >>> topic
> > > > > >>> > a
> > > > > >>> > > > unique ID.
> > > > > >>> > > > Having the topic keep the sentinel ID (not be reassigned
> to a
> > > > > >>> unique
> > > > > >>> > ID)
> > > > > >>> > > > was considered, but it seemed like a good idea to leave
> the
> > > > > option
> > > > > >>> open
> > > > > >>> > > for
> > > > > >>> > > > the metadata topic to have a unique ID in cases where it
> > > would
> > > > > >>> need to
> > > > > >>> > be
> > > > > >>> > > > differentiated from other clusters' metadata topics. (ex.
> > > > tiered
> > > > > >>> > storage)
> > > > > >>> > > >
> > > > > >>> > > >
> > > > > >>> > >
> > > > > >>> >
> > > > > >>>
> > > > >
> > > >
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-CompatibilitywithKIP-500
> > > > > >>> > > >
> > > > > >>> > > > I've also split up the KIP into sub-tasks on the JIRA.
> > > > Hopefully
> > > > > >>> this
> > > > > >>> > > will
> > > > > >>> > > > provide a better idea about what tasks we have, and
> > > eventually
> > > > > >>> provide
> > > > > >>> > a
> > > > > >>> > > > place to see what's done and what is left.
> > > > > >>> > > > If there is a task I am missing, please let me know!
> > > > > >>> > > > https://issues.apache.org/jira/browse/KAFKA-8872
> > > > > >>> > > >
> > > > > >>> > > > Of course, these decisions are not set in stone, and I
> would
> > > > love
> > > > > >>> to
> > > > > >>> > hear
> > > > > >>> > > > any feedback.
> > > > > >>> > > >
> > > > > >>> > > > Thanks,
> > > > > >>> > > > Justine
> > > > > >>> > > >
> > > > > >>> > > > On Mon, Sep 28, 2020 at 11:38 AM Justine Olshan <
> > > > > >>> jolshan@confluent.io>
> > > > > >>> > > > wrote:
> > > > > >>> > > >
> > > > > >>> > > > > Hello all,
> > > > > >>> > > > >
> > > > > >>> > > > > I just wanted to follow up on this discussion. Did we
> come
> > > to
> > > > > an
> > > > > >>> > > > > understanding about the directory structure?
> > > > > >>> > > > >
> > > > > >>> > > > > I think the biggest question here is what is
> acceptable to
> > > > > leave
> > > > > >>> out
> > > > > >>> > > due
> > > > > >>> > > > > to scope vs. what is considered to be too much tech
> debt.
> > > > > >>> > > > > This KIP is already pretty large with the number of
> > > changes,
> > > > > but
> > > > > >>> it
> > > > > >>> > > also
> > > > > >>> > > > > makes sense to do things correctly, so I'd love to hear
> > > > > >>> everyone's
> > > > > >>> > > > thoughts.
> > > > > >>> > > > >
> > > > > >>> > > > > Thanks,
> > > > > >>> > > > > Justine
> > > > > >>> > > > >
> > > > > >>> > > > > On Fri, Sep 25, 2020 at 8:19 AM Lucas Bradstreet <
> > > > > >>> lucas@confluent.io
> > > > > >>> > >
> > > > > >>> > > > > wrote:
> > > > > >>> > > > >
> > > > > >>> > > > >> Hi Ismael,
> > > > > >>> > > > >>
> > > > > >>> > > > >> If you do not store it in a metadata file or in the
> > > > directory
> > > > > >>> > > structure
> > > > > >>> > > > >> would you then
> > > > > >>> > > > >> require the LeaderAndIsrRequest following startup to
> give
> > > > you
> > > > > >>> some
> > > > > >>> > > > notion
> > > > > >>> > > > >> of
> > > > > >>> > > > >> topic name in memory? We would still need this
> information
> > > > for
> > > > > >>> the
> > > > > >>> > > older
> > > > > >>> > > > >> protocols, but
> > > > > >>> > > > >> perhaps this is what's meant by tech debt.
> > > > > >>> > > > >>
> > > > > >>> > > > >> Once we're free of the old non-topicID requests then I
> > > think
> > > > > you
> > > > > >>> > > > wouldn't
> > > > > >>> > > > >> need to retain the topic name.
> > > > > >>> > > > >> I think the ability to easily look up topic names
> > > associated
> > > > > >>> with
> > > > > >>> > > > >> partition
> > > > > >>> > > > >> directories would still be missed
> > > > > >>> > > > >> when diagnosing issues, though maybe it wouldn't be a
> deal
> > > > > >>> breaker
> > > > > >>> > > with
> > > > > >>> > > > >> the
> > > > > >>> > > > >> right tooling.
> > > > > >>> > > > >>
> > > > > >>> > > > >> Thanks,
> > > > > >>> > > > >>
> > > > > >>> > > > >> Lucas
> > > > > >>> > > > >>
> > > > > >>> > > > >> On Fri, Sep 25, 2020 at 7:55 AM Ismael Juma <
> > > > > ismael@juma.me.uk>
> > > > > >>> > > wrote:
> > > > > >>> > > > >>
> > > > > >>> > > > >> > Hi Lucas,
> > > > > >>> > > > >> >
> > > > > >>> > > > >> > Why would you include the name and id? I think you'd
> > > want
> > > > to
> > > > > >>> > remove
> > > > > >>> > > > the
> > > > > >>> > > > >> > name from the directory name right? Jason's
> suggestion
> > > was
> > > > > >>> that if
> > > > > >>> > > you
> > > > > >>> > > > >> > remove the name from the directory, then why would
> you
> > > > need
> > > > > >>> the id
> > > > > >>> > > > name
> > > > > >>> > > > >> > mapping file?
> > > > > >>> > > > >> >
> > > > > >>> > > > >> > Ismael
> > > > > >>> > > > >> >
> > > > > >>> > > > >> > On Thu, Sep 24, 2020 at 4:24 PM Lucas Bradstreet <
> > > > > >>> > > lucas@confluent.io>
> > > > > >>> > > > >> > wrote:
> > > > > >>> > > > >> >
> > > > > >>> > > > >> > > > 2. Part of the usage of the file is to have
> > > persistent
> > > > > >>> storage
> > > > > >>> > > of
> > > > > >>> > > > >> the
> > > > > >>> > > > >> > > topic
> > > > > >>> > > > >> > > ID and use it to compare with the ID supplied in
> the
> > > > > >>> > LeaderAndIsr
> > > > > >>> > > > >> > Request.
> > > > > >>> > > > >> > > There is some discussion in the KIP about changes
> to
> > > the
> > > > > >>> > directory
> > > > > >>> > > > >> > > structure, but I believe directory changes were
> > > > considered
> > > > > >>> to be
> > > > > >>> > > out
> > > > > >>> > > > >> of
> > > > > >>> > > > >> > > scope when the KIP was written.
> > > > > >>> > > > >> > >
> > > > > >>> > > > >> > >
> > > > > >>> > > > >> > > Yeah, I was hoping to get a better understanding
> of
> > > why
> > > > it
> > > > > >>> was
> > > > > >>> > > taken
> > > > > >>> > > > >> out
> > > > > >>> > > > >> > of
> > > > > >>> > > > >> > > scope. Perhaps Lucas Bradstreet might have more
> > > insight
> > > > > >>> about
> > > > > >>> > the
> > > > > >>> > > > >> > decision.
> > > > > >>> > > > >> > > Basically my point is that we have to create
> > > additional
> > > > > >>> > > > infrastructure
> > > > > >>> > > > >> > here
> > > > > >>> > > > >> > > to support the name/id mapping, so I wanted to
> > > > understand
> > > > > >>> if we
> > > > > >>> > > just
> > > > > >>> > > > >> > > consider this a sort of tech debt. It would be
> useful
> > > to
> > > > > >>> cover
> > > > > >>> > how
> > > > > >>> > > > we
> > > > > >>> > > > >> > > handle the case when this file gets corrupted.
> Seems
> > > > like
> > > > > we
> > > > > >>> > just
> > > > > >>> > > > >> have to
> > > > > >>> > > > >> > > trust that it matches whatever the controller
> tells us
> > > > and
> > > > > >>> > rewrite
> > > > > >>> > > > it?
> > > > > >>> > > > >> > >
> > > > > >>> > > > >> > >
> > > > > >>> > > > >> > > Hi Jason, Justine,
> > > > > >>> > > > >> > >
> > > > > >>> > > > >> > > My thought process is that we will likely need the
> > > > > metadata
> > > > > >>> file
> > > > > >>> > > > >> whether
> > > > > >>> > > > >> > we
> > > > > >>> > > > >> > > rename the directories or not.
> > > > > >>> > > > >> > > Linux supports filenames of up to 255 bytes and
> that
> > > > would
> > > > > >>> not
> > > > > >>> > be
> > > > > >>> > > > >> enough
> > > > > >>> > > > >> > to
> > > > > >>> > > > >> > > support a directory name
> > > > > >>> > > > >> > >  including both the name and topic ID. Given that
> > > fact,
> > > > it
> > > > > >>> > seemed
> > > > > >>> > > > >> > > reasonable to add the metadata file
> > > > > >>> > > > >> > > and leave the directory rename until some time in
> the
> > > > > future
> > > > > >>> > > > (possibly
> > > > > >>> > > > >> > > never).
> > > > > >>> > > > >> > >
> > > > > >>> > > > >> > > If the file does get corrupted I think you're
> right.
> > > We
> > > > > >>> would
> > > > > >>> > > either
> > > > > >>> > > > >> have
> > > > > >>> > > > >> > > to trust it matches what the controller tells us
> > > > > >>> > > > >> > >  or error out and let an administrator resolve it
> by
> > > > > >>> checking
> > > > > >>> > > across
> > > > > >>> > > > >> > > replicas for consistency.
> > > > > >>> > > > >> > >
> > > > > >>> > > > >> > > Lucas
> > > > > >>> > > > >> > >
> > > > > >>> > > > >> > >
> > > > > >>> > > > >> > > On Thu, Sep 24, 2020 at 3:41 PM Jason Gustafson <
> > > > > >>> > > jason@confluent.io
> > > > > >>> > > > >
> > > > > >>> > > > >> > > wrote:
> > > > > >>> > > > >> > >
> > > > > >>> > > > >> > > > Thanks Justine. Responses below:
> > > > > >>> > > > >> > > >
> > > > > >>> > > > >> > > > > 1. Yes, the directory will still be based on
> the
> > > > topic
> > > > > >>> > names.
> > > > > >>> > > > >> > > > LeaderAndIsrRequest is one of the few requests
> that
> > > > will
> > > > > >>> still
> > > > > >>> > > > >> contain
> > > > > >>> > > > >> > > the
> > > > > >>> > > > >> > > > topic name. So I think we have this covered.
> Sorry
> > > for
> > > > > >>> > > confusion.
> > > > > >>> > > > >> > > >
> > > > > >>> > > > >> > > > Ah, you're right. My eyes passed right over the
> > > field.
> > > > > >>> > > > >> > > >
> > > > > >>> > > > >> > > > > 2. Part of the usage of the file is to have
> > > > persistent
> > > > > >>> > storage
> > > > > >>> > > > of
> > > > > >>> > > > >> the
> > > > > >>> > > > >> > > > topic
> > > > > >>> > > > >> > > > ID and use it to compare with the ID supplied
> in the
> > > > > >>> > > LeaderAndIsr
> > > > > >>> > > > >> > > Request.
> > > > > >>> > > > >> > > > There is some discussion in the KIP about
> changes to
> > > > the
> > > > > >>> > > directory
> > > > > >>> > > > >> > > > structure, but I believe directory changes were
> > > > > >>> considered to
> > > > > >>> > be
> > > > > >>> > > > >> out of
> > > > > >>> > > > >> > > > scope when the KIP was written.
> > > > > >>> > > > >> > > >
> > > > > >>> > > > >> > > > Yeah, I was hoping to get a better
> understanding of
> > > > why
> > > > > >>> it was
> > > > > >>> > > > taken
> > > > > >>> > > > >> > out
> > > > > >>> > > > >> > > of
> > > > > >>> > > > >> > > > scope. Perhaps Lucas Bradstreet might have more
> > > > insight
> > > > > >>> about
> > > > > >>> > > the
> > > > > >>> > > > >> > > decision.
> > > > > >>> > > > >> > > > Basically my point is that we have to create
> > > > additional
> > > > > >>> > > > >> infrastructure
> > > > > >>> > > > >> > > here
> > > > > >>> > > > >> > > > to support the name/id mapping, so I wanted to
> > > > > understand
> > > > > >>> if
> > > > > >>> > we
> > > > > >>> > > > just
> > > > > >>> > > > >> > > > consider this a sort of tech debt. It would be
> > > useful
> > > > to
> > > > > >>> cover
> > > > > >>> > > how
> > > > > >>> > > > >> we
> > > > > >>> > > > >> > > > handle the case when this file gets corrupted.
> Seems
> > > > > like
> > > > > >>> we
> > > > > >>> > > just
> > > > > >>> > > > >> have
> > > > > >>> > > > >> > to
> > > > > >>> > > > >> > > > trust that it matches whatever the controller
> tells
> > > us
> > > > > and
> > > > > >>> > > rewrite
> > > > > >>> > > > >> it?
> > > > > >>> > > > >> > > >
> > > > > >>> > > > >> > > > > 3. I think this is a good point, but I again I
> > > > wonder
> > > > > >>> about
> > > > > >>> > > the
> > > > > >>> > > > >> scope
> > > > > >>> > > > >> > > of
> > > > > >>> > > > >> > > > the KIP. Most of the changes mentioned in the
> KIP
> > > are
> > > > > for
> > > > > >>> > > > supporting
> > > > > >>> > > > >> > > topic
> > > > > >>> > > > >> > > > deletion and I believe that is why the produce
> > > request
> > > > > was
> > > > > >>> > > listed
> > > > > >>> > > > >> under
> > > > > >>> > > > >> > > > future work.
> > > > > >>> > > > >> > > >
> > > > > >>> > > > >> > > > That's fair. I brought it up since `Fetch` is
> > > already
> > > > > >>> > included.
> > > > > >>> > > If
> > > > > >>> > > > >> > we've
> > > > > >>> > > > >> > > > got `Metadata` and `Fetch`, seems we may as
> well do
> > > > > >>> `Produce`
> > > > > >>> > > and
> > > > > >>> > > > >> save
> > > > > >>> > > > >> > an
> > > > > >>> > > > >> > > > extra kip. No strong objection though if you
> want to
> > > > > >>> leave it
> > > > > >>> > > out.
> > > > > >>> > > > >> > > >
> > > > > >>> > > > >> > > >
> > > > > >>> > > > >> > > > -Jason
> > > > > >>> > > > >> > > >
> > > > > >>> > > > >> > > >
> > > > > >>> > > > >> > > > On Thu, Sep 24, 2020 at 3:26 PM Justine Olshan <
> > > > > >>> > > > >> jolshan@confluent.io>
> > > > > >>> > > > >> > > > wrote:
> > > > > >>> > > > >> > > >
> > > > > >>> > > > >> > > > > Hi Jason,
> > > > > >>> > > > >> > > > >
> > > > > >>> > > > >> > > > > Thanks for your comments.
> > > > > >>> > > > >> > > > >
> > > > > >>> > > > >> > > > > 1. Yes, the directory will still be based on
> the
> > > > topic
> > > > > >>> > names.
> > > > > >>> > > > >> > > > > LeaderAndIsrRequest is one of the few requests
> > > that
> > > > > will
> > > > > >>> > still
> > > > > >>> > > > >> > contain
> > > > > >>> > > > >> > > > the
> > > > > >>> > > > >> > > > > topic name. So I think we have this covered.
> Sorry
> > > > for
> > > > > >>> > > > confusion.
> > > > > >>> > > > >> > > > >
> > > > > >>> > > > >> > > > > 2. Part of the usage of the file is to have
> > > > persistent
> > > > > >>> > storage
> > > > > >>> > > > of
> > > > > >>> > > > >> the
> > > > > >>> > > > >> > > > topic
> > > > > >>> > > > >> > > > > ID and use it to compare with the ID supplied
> in
> > > the
> > > > > >>> > > > LeaderAndIsr
> > > > > >>> > > > >> > > > Request.
> > > > > >>> > > > >> > > > > There is some discussion in the KIP about
> changes
> > > to
> > > > > the
> > > > > >>> > > > directory
> > > > > >>> > > > >> > > > > structure, but I believe directory changes
> were
> > > > > >>> considered
> > > > > >>> > to
> > > > > >>> > > be
> > > > > >>> > > > >> out
> > > > > >>> > > > >> > of
> > > > > >>> > > > >> > > > > scope when the KIP was written.
> > > > > >>> > > > >> > > > >
> > > > > >>> > > > >> > > > > 3. I think this is a good point, but I again I
> > > > wonder
> > > > > >>> about
> > > > > >>> > > the
> > > > > >>> > > > >> scope
> > > > > >>> > > > >> > > of
> > > > > >>> > > > >> > > > > the KIP. Most of the changes mentioned in the
> KIP
> > > > are
> > > > > >>> for
> > > > > >>> > > > >> supporting
> > > > > >>> > > > >> > > > topic
> > > > > >>> > > > >> > > > > deletion and I believe that is why the produce
> > > > request
> > > > > >>> was
> > > > > >>> > > > listed
> > > > > >>> > > > >> > under
> > > > > >>> > > > >> > > > > future work.
> > > > > >>> > > > >> > > > >
> > > > > >>> > > > >> > > > > 4. This sounds like it might be a good
> solution,
> > > > but I
> > > > > >>> will
> > > > > >>> > > need
> > > > > >>> > > > >> to
> > > > > >>> > > > >> > > > discuss
> > > > > >>> > > > >> > > > > more with KIP-500 folks to get the details
> right.
> > > > > >>> > > > >> > > > >
> > > > > >>> > > > >> > > > > Thanks,
> > > > > >>> > > > >> > > > > Justine
> > > > > >>> > > > >> > > > >
> > > > > >>> > > > >> > > > > On Thu, Sep 24, 2020 at 12:30 PM Jason
> Gustafson <
> > > > > >>> > > > >> jason@confluent.io
> > > > > >>> > > > >> > >
> > > > > >>> > > > >> > > > > wrote:
> > > > > >>> > > > >> > > > >
> > > > > >>> > > > >> > > > > > Hi Justine,
> > > > > >>> > > > >> > > > > >
> > > > > >>> > > > >> > > > > > Thanks for picking up this work. I have a
> few
> > > > > >>> > > > >> questions/comments:
> > > > > >>> > > > >> > > > > >
> > > > > >>> > > > >> > > > > > 1. It sounds like the directory structure is
> > > still
> > > > > >>> going
> > > > > >>> > to
> > > > > >>> > > be
> > > > > >>> > > > >> > based
> > > > > >>> > > > >> > > on
> > > > > >>> > > > >> > > > > > topic names. Do I have that right? One
> > > > complication
> > > > > is
> > > > > >>> > that
> > > > > >>> > > > the
> > > > > >>> > > > >> > > > > > LeaderAndIsr request does not include the
> topic
> > > > name
> > > > > >>> any
> > > > > >>> > > > longer.
> > > > > >>> > > > >> > This
> > > > > >>> > > > >> > > > > means
> > > > > >>> > > > >> > > > > > that a replica must wait for the
> UpdateMetadata
> > > > > >>> request to
> > > > > >>> > > > >> arrive
> > > > > >>> > > > >> > > with
> > > > > >>> > > > >> > > > > the
> > > > > >>> > > > >> > > > > > topic name mapping before it can create the
> > > > > >>> directory. I
> > > > > >>> > > > wonder
> > > > > >>> > > > >> if
> > > > > >>> > > > >> > it
> > > > > >>> > > > >> > > > > would
> > > > > >>> > > > >> > > > > > be simpler to avoid assumptions on the
> ordering
> > > of
> > > > > >>> > > > >> UpdateMetadata
> > > > > >>> > > > >> > and
> > > > > >>> > > > >> > > > let
> > > > > >>> > > > >> > > > > > LeaderAndIsr include the topic name as well.
> > > Feels
> > > > > >>> like we
> > > > > >>> > > are
> > > > > >>> > > > >> not
> > > > > >>> > > > >> > > > saving
> > > > > >>> > > > >> > > > > > that much by excluding it.
> > > > > >>> > > > >> > > > > >
> > > > > >>> > > > >> > > > > > 2. On a related note, it seems that the
> reason
> > > we
> > > > > >>> have the
> > > > > >>> > > > >> > partition
> > > > > >>> > > > >> > > > > > metadata file is because we are not
> changing the
> > > > > >>> directory
> > > > > >>> > > > >> > structure.
> > > > > >>> > > > >> > > > We
> > > > > >>> > > > >> > > > > > need it so that we remember which
> directories
> > > map
> > > > to
> > > > > >>> which
> > > > > >>> > > > topic
> > > > > >>> > > > >> > id.
> > > > > >>> > > > >> > > I
> > > > > >>> > > > >> > > > > > think it would be useful to add some detail
> > > about
> > > > > why
> > > > > >>> > > changing
> > > > > >>> > > > >> the
> > > > > >>> > > > >> > > > > > directory layout was rejected. Basically
> trying
> > > to
> > > > > >>> > > understand
> > > > > >>> > > > >> how
> > > > > >>> > > > >> > > much
> > > > > >>> > > > >> > > > > > effort we are saving by skipping this step
> if we
> > > > > have
> > > > > >>> to
> > > > > >>> > > > >> implement
> > > > > >>> > > > >> > > this
> > > > > >>> > > > >> > > > > > additional bookkeeping. One concern I have
> for
> > > > > >>> example is
> > > > > >>> > > that
> > > > > >>> > > > >> the
> > > > > >>> > > > >> > > > > > partition metadata file gets corrupt and
> then
> > > all
> > > > > >>> bets are
> > > > > >>> > > off
> > > > > >>> > > > >> as
> > > > > >>> > > > >> > far
> > > > > >>> > > > >> > > > as
> > > > > >>> > > > >> > > > > > topic consistency.
> > > > > >>> > > > >> > > > > >
> > > > > >>> > > > >> > > > > > 3. Is it worth adding support now for topic
> ids
> > > in
> > > > > the
> > > > > >>> > > > `Produce`
> > > > > >>> > > > >> > > > request
> > > > > >>> > > > >> > > > > > now? Overhead is mentioned as one of the
> > > > motivations
> > > > > >>> and
> > > > > >>> > the
> > > > > >>> > > > >> best
> > > > > >>> > > > >> > > APIs
> > > > > >>> > > > >> > > > to
> > > > > >>> > > > >> > > > > > get savings from are `Produce` and `Fetch`.
> > > > > >>> > > > >> > > > > >
> > > > > >>> > > > >> > > > > > 4. When it comes to bootstrapping the
> metadata
> > > > topic
> > > > > >>> with
> > > > > >>> > > > >> respect
> > > > > >>> > > > >> > to
> > > > > >>> > > > >> > > > > > KIP-500, one suggestion would be to reserve
> a
> > > > > sentinel
> > > > > >>> > topic
> > > > > >>> > > > ID
> > > > > >>> > > > >> > which
> > > > > >>> > > > >> > > > can
> > > > > >>> > > > >> > > > > > be used initially by new brokers. When the
> first
> > > > > >>> > controller
> > > > > >>> > > is
> > > > > >>> > > > >> > > elected,
> > > > > >>> > > > >> > > > > it
> > > > > >>> > > > >> > > > > > can assign a topicId to the metadata topic.
> This
> > > > is
> > > > > a
> > > > > >>> bit
> > > > > >>> > > > >> similar
> > > > > >>> > > > >> > to
> > > > > >>> > > > >> > > > how
> > > > > >>> > > > >> > > > > we
> > > > > >>> > > > >> > > > > > were planning to generate the clusterId.
> > > > > >>> > > > >> > > > > >
> > > > > >>> > > > >> > > > > > Thanks,
> > > > > >>> > > > >> > > > > > Jason
> > > > > >>> > > > >> > > > > >
> > > > > >>> > > > >> > > > > >
> > > > > >>> > > > >> > > > > > On Thu, Sep 24, 2020 at 11:10 AM Jun Rao <
> > > > > >>> > jun@confluent.io>
> > > > > >>> > > > >> wrote:
> > > > > >>> > > > >> > > > > >
> > > > > >>> > > > >> > > > > > > Hi, Justine,
> > > > > >>> > > > >> > > > > > >
> > > > > >>> > > > >> > > > > > > Thanks for the updated KIP. A few more
> > > comments
> > > > > >>> below.
> > > > > >>> > > > >> > > > > > >
> > > > > >>> > > > >> > > > > > > 30.  {"name": "id", "type": "string",
> "doc":
> > > > > >>> "version
> > > > > >>> > > id"}}:
> > > > > >>> > > > >> The
> > > > > >>> > > > >> > > doc
> > > > > >>> > > > >> > > > > > should
> > > > > >>> > > > >> > > > > > > say UUID.
> > > > > >>> > > > >> > > > > > >
> > > > > >>> > > > >> > > > > > > 31. LeaderAndIsrResponse v5 and
> > > > > StopReplicaResponse
> > > > > >>> v4 :
> > > > > >>> > > It
> > > > > >>> > > > >> seems
> > > > > >>> > > > >> > > > there
> > > > > >>> > > > >> > > > > > is
> > > > > >>> > > > >> > > > > > > no need to add topic_id at partitions
> level.
> > > > > >>> > > > >> > > > > > >
> > > > > >>> > > > >> > > > > > > 32. Regarding partition metadata file.
> Perhaps
> > > > the
> > > > > >>> key
> > > > > >>> > can
> > > > > >>> > > > be
> > > > > >>> > > > >> a
> > > > > >>> > > > >> > > > single
> > > > > >>> > > > >> > > > > > > word, sth like the following.
> > > > > >>> > > > >> > > > > > > version: 0
> > > > > >>> > > > >> > > > > > > topic_id:
> 46bdb63f-9e8d-4a38-bf7b-ee4eb2a794e4
> > > > > >>> > > > >> > > > > > >
> > > > > >>> > > > >> > > > > > > 33. Another tricky thing that I realized
> is
> > > how
> > > > to
> > > > > >>> > support
> > > > > >>> > > > the
> > > > > >>> > > > >> > > > metadata
> > > > > >>> > > > >> > > > > > > topic introduced in KIP-595. It's a bit
> tricky
> > > > to
> > > > > >>> > assign a
> > > > > >>> > > > >> UUID
> > > > > >>> > > > >> > to
> > > > > >>> > > > >> > > > the
> > > > > >>> > > > >> > > > > > > metadata topic since we have a
> chicken-and-egg
> > > > > >>> problem.
> > > > > >>> > > The
> > > > > >>> > > > >> > > > controller
> > > > > >>> > > > >> > > > > > > needs to persist the UUID in the metadata
> > > topic
> > > > in
> > > > > >>> order
> > > > > >>> > > to
> > > > > >>> > > > >> > assign
> > > > > >>> > > > >> > > > one
> > > > > >>> > > > >> > > > > > > successfully, but the metadata topic is
> needed
> > > > to
> > > > > >>> elect
> > > > > >>> > a
> > > > > >>> > > > >> > > controller
> > > > > >>> > > > >> > > > > > > (KIP-631). So, this probably needs a bit
> more
> > > > > >>> thought.
> > > > > >>> > > > >> > > > > > >
> > > > > >>> > > > >> > > > > > > Jun
> > > > > >>> > > > >> > > > > > >
> > > > > >>> > > > >> > > > > > > On Thu, Sep 24, 2020 at 3:04 AM Ismael
> Juma <
> > > > > >>> > > > >> ismael@juma.me.uk>
> > > > > >>> > > > >> > > > wrote:
> > > > > >>> > > > >> > > > > > >
> > > > > >>> > > > >> > > > > > > > Also, can we provide more details on
> how the
> > > > > >>> Partition
> > > > > >>> > > > >> Metadata
> > > > > >>> > > > >> > > > file
> > > > > >>> > > > >> > > > > > will
> > > > > >>> > > > >> > > > > > > > be used?
> > > > > >>> > > > >> > > > > > > >
> > > > > >>> > > > >> > > > > > > > Ismael
> > > > > >>> > > > >> > > > > > > >
> > > > > >>> > > > >> > > > > > > > On Thu, Sep 24, 2020 at 3:01 AM Ismael
> Juma
> > > <
> > > > > >>> > > > >> ismael@juma.me.uk
> > > > > >>> > > > >> > >
> > > > > >>> > > > >> > > > > wrote:
> > > > > >>> > > > >> > > > > > > >
> > > > > >>> > > > >> > > > > > > > > Hi Justine,
> > > > > >>> > > > >> > > > > > > > >
> > > > > >>> > > > >> > > > > > > > > I think we need to update the
> "Rejected
> > > > > >>> > Alternatives"
> > > > > >>> > > > >> section
> > > > > >>> > > > >> > > to
> > > > > >>> > > > >> > > > > take
> > > > > >>> > > > >> > > > > > > > into
> > > > > >>> > > > >> > > > > > > > > account that the proposal now removes
> the
> > > > > topic
> > > > > >>> name
> > > > > >>> > > > from
> > > > > >>> > > > >> the
> > > > > >>> > > > >> > > > fetch
> > > > > >>> > > > >> > > > > > > > > request. Also, if we are removing it
> from
> > > > the
> > > > > >>> Fetch
> > > > > >>> > > > >> request,
> > > > > >>> > > > >> > > does
> > > > > >>> > > > >> > > > > it
> > > > > >>> > > > >> > > > > > > make
> > > > > >>> > > > >> > > > > > > > > sense not to remove it from similar
> > > requests
> > > > > >>> like
> > > > > >>> > > > >> > > > > ListOffsetRequest?
> > > > > >>> > > > >> > > > > > > > >
> > > > > >>> > > > >> > > > > > > > > Ismael
> > > > > >>> > > > >> > > > > > > > >
> > > > > >>> > > > >> > > > > > > > > On Thu, Sep 24, 2020 at 2:46 AM David
> > > Jacot
> > > > <
> > > > > >>> > > > >> > > djacot@confluent.io
> > > > > >>> > > > >> > > > >
> > > > > >>> > > > >> > > > > > > wrote:
> > > > > >>> > > > >> > > > > > > > >
> > > > > >>> > > > >> > > > > > > > >> Hi Justine,
> > > > > >>> > > > >> > > > > > > > >>
> > > > > >>> > > > >> > > > > > > > >> Thanks for the KIP. I finally had
> time to
> > > > > read
> > > > > >>> it
> > > > > >>> > :).
> > > > > >>> > > > It
> > > > > >>> > > > >> is
> > > > > >>> > > > >> > a
> > > > > >>> > > > >> > > > > great
> > > > > >>> > > > >> > > > > > > > >> improvement.
> > > > > >>> > > > >> > > > > > > > >>
> > > > > >>> > > > >> > > > > > > > >> I have a few comments/questions:
> > > > > >>> > > > >> > > > > > > > >>
> > > > > >>> > > > >> > > > > > > > >> 1. It seems that the schema of the
> > > > > >>> > StopReplicaRequest
> > > > > >>> > > > is
> > > > > >>> > > > >> > > > slightly
> > > > > >>> > > > >> > > > > > > > >> outdated.
> > > > > >>> > > > >> > > > > > > > >> We
> > > > > >>> > > > >> > > > > > > > >> did some changes as part of KIP-570.
> V3
> > > is
> > > > > >>> already
> > > > > >>> > > > >> organized
> > > > > >>> > > > >> > > by
> > > > > >>> > > > >> > > > > > > topics.
> > > > > >>> > > > >> > > > > > > > >>
> > > > > >>> > > > >> > > > > > > > >> 2. I just want to make sure that I
> > > > understand
> > > > > >>> the
> > > > > >>> > > > >> > > reconciliation
> > > > > >>> > > > >> > > > > > > > >> logic correctly. When
> > > > > >>> > > > >> > > > > > > > >> an "INCREMENTAL" LeaderAndIsr
> Request is
> > > > > >>> received,
> > > > > >>> > > the
> > > > > >>> > > > >> > broker
> > > > > >>> > > > >> > > > will
> > > > > >>> > > > >> > > > > > > also
> > > > > >>> > > > >> > > > > > > > >> reconcile
> > > > > >>> > > > >> > > > > > > > >> when the local uuid does not match
> the
> > > uuid
> > > > > in
> > > > > >>> the
> > > > > >>> > > > >> request,
> > > > > >>> > > > >> > > > right?
> > > > > >>> > > > >> > > > > > In
> > > > > >>> > > > >> > > > > > > > this
> > > > > >>> > > > >> > > > > > > > >> case, the
> > > > > >>> > > > >> > > > > > > > >> local log is staged for deletion.
> > > > > >>> > > > >> > > > > > > > >>
> > > > > >>> > > > >> > > > > > > > >> 3. In the documentation of the `
> > > > > >>> > > > >> delete.stale.topic.delay.ms
> > > > > >>> > > > >> > `
> > > > > >>> > > > >> > > > > > config,
> > > > > >>> > > > >> > > > > > > it
> > > > > >>> > > > >> > > > > > > > >> says "When a
> > > > > >>> > > > >> > > > > > > > >> FULL LeaderAndIsrRequest is
> received..."
> > > > but
> > > > > I
> > > > > >>> > > suppose
> > > > > >>> > > > it
> > > > > >>> > > > >> > > > applies
> > > > > >>> > > > >> > > > > to
> > > > > >>> > > > >> > > > > > > > both
> > > > > >>> > > > >> > > > > > > > >> types.
> > > > > >>> > > > >> > > > > > > > >>
> > > > > >>> > > > >> > > > > > > > >> Best,
> > > > > >>> > > > >> > > > > > > > >> David
> > > > > >>> > > > >> > > > > > > > >>
> > > > > >>> > > > >> > > > > > > > >> On Thu, Sep 24, 2020 at 1:40 AM
> Justine
> > > > > Olshan
> > > > > >>> <
> > > > > >>> > > > >> > > > > > jolshan@confluent.io>
> > > > > >>> > > > >> > > > > > > > >> wrote:
> > > > > >>> > > > >> > > > > > > > >>
> > > > > >>> > > > >> > > > > > > > >> > Hi Jun,
> > > > > >>> > > > >> > > > > > > > >> >
> > > > > >>> > > > >> > > > > > > > >> > Thanks for the comments. I
> apologize
> > > for
> > > > > >>> some of
> > > > > >>> > > the
> > > > > >>> > > > >> typos
> > > > > >>> > > > >> > > and
> > > > > >>> > > > >> > > > > > > > >> confusion.
> > > > > >>> > > > >> > > > > > > > >> > I’ve updated the KIP to fix some
> of the
> > > > > >>> issues
> > > > > >>> > you
> > > > > >>> > > > >> > > mentioned.
> > > > > >>> > > > >> > > > > > > > >> >
> > > > > >>> > > > >> > > > > > > > >> > 20.2 I’ve changed the type to
> String.
> > > > > >>> > > > >> > > > > > > > >> > 20.1/3 I’ve updated the TopicZNode
> to
> > > fix
> > > > > >>> > > formatting
> > > > > >>> > > > >> and
> > > > > >>> > > > >> > > > reflect
> > > > > >>> > > > >> > > > > > the
> > > > > >>> > > > >> > > > > > > > >> latest
> > > > > >>> > > > >> > > > > > > > >> > version before this change.
> > > > > >>> > > > >> > > > > > > > >> >
> > > > > >>> > > > >> > > > > > > > >> > 21. You are correct and I’ve
> removed
> > > this
> > > > > >>> line.
> > > > > >>> > > I’ve
> > > > > >>> > > > >> also
> > > > > >>> > > > >> > > > added
> > > > > >>> > > > >> > > > > a
> > > > > >>> > > > >> > > > > > > line
> > > > > >>> > > > >> > > > > > > > >> > mentioning an IBP bump is
> necessary for
> > > > > >>> migration
> > > > > >>> > > > >> > > > > > > > >> >
> > > > > >>> > > > >> > > > > > > > >> > 22. I think the wording was
> unclear but
> > > > > your
> > > > > >>> > > summary
> > > > > >>> > > > is
> > > > > >>> > > > >> > what
> > > > > >>> > > > >> > > > was
> > > > > >>> > > > >> > > > > > > > >> intended
> > > > > >>> > > > >> > > > > > > > >> > by this line. I’ve updated to make
> this
> > > > > point
> > > > > >>> > more
> > > > > >>> > > > >> clear.
> > > > > >>> > > > >> > > > > “Remove
> > > > > >>> > > > >> > > > > > > > >> deleted
> > > > > >>> > > > >> > > > > > > > >> > topics from replicas by sending
> > > > > >>> > StopReplicaRequest
> > > > > >>> > > V3
> > > > > >>> > > > >> > before
> > > > > >>> > > > >> > > > the
> > > > > >>> > > > >> > > > > > IBP
> > > > > >>> > > > >> > > > > > > > >> bump
> > > > > >>> > > > >> > > > > > > > >> > using the old logic, and using V4
> and
> > > the
> > > > > new
> > > > > >>> > logic
> > > > > >>> > > > >> with
> > > > > >>> > > > >> > > topic
> > > > > >>> > > > >> > > > > IDs
> > > > > >>> > > > >> > > > > > > > after
> > > > > >>> > > > >> > > > > > > > >> > the IBP bump.”
> > > > > >>> > > > >> > > > > > > > >> >
> > > > > >>> > > > >> > > > > > > > >> > 23. I’ve removed the unspecified
> type
> > > > from
> > > > > >>> the
> > > > > >>> > KIP
> > > > > >>> > > > and
> > > > > >>> > > > >> > > mention
> > > > > >>> > > > >> > > > > > that
> > > > > >>> > > > >> > > > > > > > IBP
> > > > > >>> > > > >> > > > > > > > >> > will be used to handle this
> request.
> > > “IBP
> > > > > >>> will be
> > > > > >>> > > > used
> > > > > >>> > > > >> to
> > > > > >>> > > > >> > > > > > determine
> > > > > >>> > > > >> > > > > > > > >> whether
> > > > > >>> > > > >> > > > > > > > >> > this new form of the request will
> be
> > > > used.
> > > > > >>> For
> > > > > >>> > > older
> > > > > >>> > > > >> > > requests,
> > > > > >>> > > > >> > > > > we
> > > > > >>> > > > >> > > > > > > will
> > > > > >>> > > > >> > > > > > > > >> > ignore this field and default to
> > > previous
> > > > > >>> > > behavior.”
> > > > > >>> > > > >> > > > > > > > >> >
> > > > > >>> > > > >> > > > > > > > >> > 24. I’ve fixed this typo.
> > > > > >>> > > > >> > > > > > > > >> >
> > > > > >>> > > > >> > > > > > > > >> > 25. I've added a topics at a higher
> > > level
> > > > > for
> > > > > >>> > > > >> > > > > LeaderAndIsrResponse
> > > > > >>> > > > >> > > > > > > v5,
> > > > > >>> > > > >> > > > > > > > >> > StopReplicaResponse v4. I've also
> > > changed
> > > > > >>> > > > >> > StopReplicaRequest
> > > > > >>> > > > >> > > > v4
> > > > > >>> > > > >> > > > > to
> > > > > >>> > > > >> > > > > > > > have
> > > > > >>> > > > >> > > > > > > > >> > topics at a higher level.
> > > > > >>> > > > >> > > > > > > > >> >
> > > > > >>> > > > >> > > > > > > > >> > 26. I’ve updated
> > > > > >>> forgotten_topics_data--added the
> > > > > >>> > > > >> topic ID
> > > > > >>> > > > >> > > and
> > > > > >>> > > > >> > > > > > > removed
> > > > > >>> > > > >> > > > > > > > >> the
> > > > > >>> > > > >> > > > > > > > >> > topic name
> > > > > >>> > > > >> > > > > > > > >> >
> > > > > >>> > > > >> > > > > > > > >> > 27. I’ve decided on plain text, and
> > > I’ve
> > > > > >>> added an
> > > > > >>> > > > >> example.
> > > > > >>> > > > >> > > > > > > > >> >
> > > > > >>> > > > >> > > > > > > > >> > 28. I’ve added this idea to future
> > > work.
> > > > > >>> > > > >> > > > > > > > >> >
> > > > > >>> > > > >> > > > > > > > >> > Thanks again for taking a look,
> > > > > >>> > > > >> > > > > > > > >> >
> > > > > >>> > > > >> > > > > > > > >> > Justine
> > > > > >>> > > > >> > > > > > > > >> >
> > > > > >>> > > > >> > > > > > > > >> > On Wed, Sep 23, 2020 at 10:28 AM
> Jun
> > > Rao
> > > > <
> > > > > >>> > > > >> > jun@confluent.io>
> > > > > >>> > > > >> > > > > > wrote:
> > > > > >>> > > > >> > > > > > > > >> >
> > > > > >>> > > > >> > > > > > > > >> > > Hi, Justine,
> > > > > >>> > > > >> > > > > > > > >> > >
> > > > > >>> > > > >> > > > > > > > >> > > Thanks for the response. Made
> another
> > > > > >>> pass. A
> > > > > >>> > few
> > > > > >>> > > > >> more
> > > > > >>> > > > >> > > > > comments
> > > > > >>> > > > >> > > > > > > > below.
> > > > > >>> > > > >> > > > > > > > >> > >
> > > > > >>> > > > >> > > > > > > > >> > > 20. znode schema:
> > > > > >>> > > > >> > > > > > > > >> > > 20.1 It seems that {"name":
> > > "version",
> > > > > >>> "type":
> > > > > >>> > > > "int",
> > > > > >>> > > > >> > > "id":
> > > > > >>> > > > >> > > > > > > "UUID",
> > > > > >>> > > > >> > > > > > > > >> > "doc":
> > > > > >>> > > > >> > > > > > > > >> > > "version id"} should be {"name":
> > > > > "version",
> > > > > >>> > > "type":
> > > > > >>> > > > >> > > "int"},
> > > > > >>> > > > >> > > > > > > {"name":
> > > > > >>> > > > >> > > > > > > > >> > "id",
> > > > > >>> > > > >> > > > > > > > >> > > "type": "UUID", "doc": "version
> id"}.
> > > > > >>> > > > >> > > > > > > > >> > > 20.2 The znode format is JSON
> which
> > > > > doesn't
> > > > > >>> > have
> > > > > >>> > > > UUID
> > > > > >>> > > > >> > > type.
> > > > > >>> > > > >> > > > So
> > > > > >>> > > > >> > > > > > the
> > > > > >>> > > > >> > > > > > > > >> type
> > > > > >>> > > > >> > > > > > > > >> > > probably should be string?
> > > > > >>> > > > >> > > > > > > > >> > > 20.3 Also, the existing format
> used
> > > > seems
> > > > > >>> > > outdated.
> > > > > >>> > > > >> It
> > > > > >>> > > > >> > > > should
> > > > > >>> > > > >> > > > > > have
> > > > > >>> > > > >> > > > > > > > the
> > > > > >>> > > > >> > > > > > > > >> > > following format.
> > > > > >>> > > > >> > > > > > > > >> > >     Json.encodeAsBytes(Map(
> > > > > >>> > > > >> > > > > > > > >> > >       "version" -> 2,
> > > > > >>> > > > >> > > > > > > > >> > >       "partitions" ->
> > > > > >>> > > replicaAssignmentJson.asJava,
> > > > > >>> > > > >> > > > > > > > >> > >       "adding_replicas" ->
> > > > > >>> > > > >> > > > > addingReplicasAssignmentJson.asJava,
> > > > > >>> > > > >> > > > > > > > >> > >       "removing_replicas" ->
> > > > > >>> > > > >> > > > > > removingReplicasAssignmentJson.asJava
> > > > > >>> > > > >> > > > > > > > >> > >     ).asJava)
> > > > > >>> > > > >> > > > > > > > >> > >   }
> > > > > >>> > > > >> > > > > > > > >> > >
> > > > > >>> > > > >> > > > > > > > >> > > 21. Migration: The KIP says "The
> > > > > migration
> > > > > >>> > > process
> > > > > >>> > > > >> can
> > > > > >>> > > > >> > > take
> > > > > >>> > > > >> > > > > > place
> > > > > >>> > > > >> > > > > > > > >> without
> > > > > >>> > > > >> > > > > > > > >> > > an inter-broker protocol bump,
> as the
> > > > > >>> format
> > > > > >>> > > stored
> > > > > >>> > > > >> in
> > > > > >>> > > > >> > > > > > > > >> > > /brokers/topics/[topic] will be
> > > > > compatible
> > > > > >>> with
> > > > > >>> > > > older
> > > > > >>> > > > >> > > broker
> > > > > >>> > > > >> > > > > > > > >> versions."
> > > > > >>> > > > >> > > > > > > > >> > > However, since we are bumping up
> the
> > > > > >>> version of
> > > > > >>> > > > >> > > inter-broker
> > > > > >>> > > > >> > > > > > > > >> requests, it
> > > > > >>> > > > >> > > > > > > > >> > > seems that we need to use IBP for
> > > > > >>> migration.
> > > > > >>> > > > >> > > > > > > > >> > >
> > > > > >>> > > > >> > > > > > > > >> > > 22. The KIP says "Remove deleted
> > > topics
> > > > > >>> from
> > > > > >>> > > > >> replicas by
> > > > > >>> > > > >> > > > > sending
> > > > > >>> > > > >> > > > > > > > >> > > StopReplicaRequest V3 for any
> topics
> > > > > which
> > > > > >>> do
> > > > > >>> > not
> > > > > >>> > > > >> > contain
> > > > > >>> > > > >> > > a
> > > > > >>> > > > >> > > > > > topic
> > > > > >>> > > > >> > > > > > > > ID,
> > > > > >>> > > > >> > > > > > > > >> and
> > > > > >>> > > > >> > > > > > > > >> > > V4 for any topics which do
> contain a
> > > > > topic
> > > > > >>> > ID.".
> > > > > >>> > > > >> > However,
> > > > > >>> > > > >> > > if
> > > > > >>> > > > >> > > > > we
> > > > > >>> > > > >> > > > > > > use
> > > > > >>> > > > >> > > > > > > > >> IBP,
> > > > > >>> > > > >> > > > > > > > >> > it
> > > > > >>> > > > >> > > > > > > > >> > > seems that the controller will
> either
> > > > > send
> > > > > >>> > > > >> > > > StopReplicaRequest
> > > > > >>> > > > >> > > > > V3
> > > > > >>> > > > >> > > > > > > > >> > > or StopReplicaRequest V4, but
> never
> > > > mixed
> > > > > >>> V3
> > > > > >>> > and
> > > > > >>> > > V4
> > > > > >>> > > > >> for
> > > > > >>> > > > >> > > > > > different
> > > > > >>> > > > >> > > > > > > > >> topics.
> > > > > >>> > > > >> > > > > > > > >> > > Basically, before the IBP bump,
> V3
> > > will
> > > > > be
> > > > > >>> > used.
> > > > > >>> > > > >> After
> > > > > >>> > > > >> > the
> > > > > >>> > > > >> > > > IBP
> > > > > >>> > > > >> > > > > > > bump,
> > > > > >>> > > > >> > > > > > > > >> > > topicId will be created and V4
> will
> > > be
> > > > > >>> used.
> > > > > >>> > > > >> > > > > > > > >> > >
> > > > > >>> > > > >> > > > > > > > >> > > 23. Given that we depend on IBP,
> do
> > > we
> > > > > >>> still
> > > > > >>> > need
> > > > > >>> > > > "0
> > > > > >>> > > > >> > > > > > UNSPECIFIED"
> > > > > >>> > > > >> > > > > > > > >> > > in LeaderAndIsr?
> > > > > >>> > > > >> > > > > > > > >> > >
> > > > > >>> > > > >> > > > > > > > >> > > 24. LeaderAndIsrResponse v5 : It
> > > still
> > > > > has
> > > > > >>> the
> > > > > >>> > > > topic
> > > > > >>> > > > >> > > field.
> > > > > >>> > > > >> > > > > > > > >> > >
> > > > > >>> > > > >> > > > > > > > >> > > 25. LeaderAndIsrResponse v5,
> > > > > >>> > StopReplicaResponse
> > > > > >>> > > > v4:
> > > > > >>> > > > >> > Could
> > > > > >>> > > > >> > > > we
> > > > > >>> > > > >> > > > > > use
> > > > > >>> > > > >> > > > > > > > this
> > > > > >>> > > > >> > > > > > > > >> > > opportunity to organize the
> response
> > > > in 2
> > > > > >>> > levels,
> > > > > >>> > > > >> first
> > > > > >>> > > > >> > by
> > > > > >>> > > > >> > > > > > topic,
> > > > > >>> > > > >> > > > > > > > >> then by
> > > > > >>> > > > >> > > > > > > > >> > > partition, as most other
> > > > > >>> requests/responses?
> > > > > >>> > > > >> > > > > > > > >> > >
> > > > > >>> > > > >> > > > > > > > >> > > 26. FetchRequest v13 : Should
> > > > > >>> > > forgotten_topics_data
> > > > > >>> > > > >> use
> > > > > >>> > > > >> > > > > topicId
> > > > > >>> > > > >> > > > > > > too?
> > > > > >>> > > > >> > > > > > > > >> > >
> > > > > >>> > > > >> > > > > > > > >> > > 27. "This file can either be
> plain
> > > text
> > > > > >>> > > (key/value
> > > > > >>> > > > >> > pairs)
> > > > > >>> > > > >> > > or
> > > > > >>> > > > >> > > > > > > JSON."
> > > > > >>> > > > >> > > > > > > > >> Have
> > > > > >>> > > > >> > > > > > > > >> > we
> > > > > >>> > > > >> > > > > > > > >> > > decided which one to use? Also,
> it
> > > > would
> > > > > be
> > > > > >>> > > helpful
> > > > > >>> > > > >> to
> > > > > >>> > > > >> > > > provide
> > > > > >>> > > > >> > > > > > an
> > > > > >>> > > > >> > > > > > > > >> > example.
> > > > > >>> > > > >> > > > > > > > >> > >
> > > > > >>> > > > >> > > > > > > > >> > > 28. Future improvement: Another
> > > future
> > > > > >>> > > improvement
> > > > > >>> > > > >> > > > opportunity
> > > > > >>> > > > >> > > > > > is
> > > > > >>> > > > >> > > > > > > to
> > > > > >>> > > > >> > > > > > > > >> use
> > > > > >>> > > > >> > > > > > > > >> > > topicId in
> > > > > >>> GroupMetadataManager.offsetCommitKey
> > > > > >>> > > in
> > > > > >>> > > > >> the
> > > > > >>> > > > >> > > > > > > offset_commit
> > > > > >>> > > > >> > > > > > > > >> > topic.
> > > > > >>> > > > >> > > > > > > > >> > > This may save some space.
> > > > > >>> > > > >> > > > > > > > >> > >
> > > > > >>> > > > >> > > > > > > > >> > > Thanks,
> > > > > >>> > > > >> > > > > > > > >> > >
> > > > > >>> > > > >> > > > > > > > >> > > Jun
> > > > > >>> > > > >> > > > > > > > >> > >
> > > > > >>> > > > >> > > > > > > > >> > > On Wed, Sep 23, 2020 at 8:50 AM
> > > Justine
> > > > > >>> Olshan
> > > > > >>> > <
> > > > > >>> > > > >> > > > > > > > jolshan@confluent.io>
> > > > > >>> > > > >> > > > > > > > >> > > wrote:
> > > > > >>> > > > >> > > > > > > > >> > >
> > > > > >>> > > > >> > > > > > > > >> > > > Hi Tom,
> > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > >>> > > > >> > > > > > > > >> > > > Thanks for the comment. I think
> > > this
> > > > > is a
> > > > > >>> > > really
> > > > > >>> > > > >> good
> > > > > >>> > > > >> > > idea
> > > > > >>> > > > >> > > > > and
> > > > > >>> > > > >> > > > > > > it
> > > > > >>> > > > >> > > > > > > > >> has
> > > > > >>> > > > >> > > > > > > > >> > > been
> > > > > >>> > > > >> > > > > > > > >> > > > added to the KIP under the
> newly
> > > > added
> > > > > >>> > tooling
> > > > > >>> > > > >> > section.
> > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > >>> > > > >> > > > > > > > >> > > > Thanks again,
> > > > > >>> > > > >> > > > > > > > >> > > > Justine
> > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > >>> > > > >> > > > > > > > >> > > > On Wed, Sep 23, 2020 at 3:17
> AM Tom
> > > > > >>> Bentley <
> > > > > >>> > > > >> > > > > > > tbentley@redhat.com>
> > > > > >>> > > > >> > > > > > > > >> > wrote:
> > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > Hi Justine,
> > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > I know you started the vote
> > > thread,
> > > > > >>> but on
> > > > > >>> > > > >> > re-reading
> > > > > >>> > > > >> > > > the
> > > > > >>> > > > >> > > > > > KIP
> > > > > >>> > > > >> > > > > > > I
> > > > > >>> > > > >> > > > > > > > >> > noticed
> > > > > >>> > > > >> > > > > > > > >> > > > > that although the topic id is
> > > > > included
> > > > > >>> in
> > > > > >>> > the
> > > > > >>> > > > >> > > > > > MetadataResponse
> > > > > >>> > > > >> > > > > > > > >> it's
> > > > > >>> > > > >> > > > > > > > >> > not
> > > > > >>> > > > >> > > > > > > > >> > > > > surfaced in the output from
> > > > > >>> > `kafka-topics.sh
> > > > > >>> > > > >> > > > --describe`.
> > > > > >>> > > > >> > > > > > > Maybe
> > > > > >>> > > > >> > > > > > > > >> that
> > > > > >>> > > > >> > > > > > > > >> > > was
> > > > > >>> > > > >> > > > > > > > >> > > > > intentional because ids are
> > > > > >>> intentionally
> > > > > >>> > not
> > > > > >>> > > > >> really
> > > > > >>> > > > >> > > > > > something
> > > > > >>> > > > >> > > > > > > > the
> > > > > >>> > > > >> > > > > > > > >> > user
> > > > > >>> > > > >> > > > > > > > >> > > > > should care deeply about,
> but it
> > > > > would
> > > > > >>> also
> > > > > >>> > > > make
> > > > > >>> > > > >> > life
> > > > > >>> > > > >> > > > > harder
> > > > > >>> > > > >> > > > > > > for
> > > > > >>> > > > >> > > > > > > > >> > anyone
> > > > > >>> > > > >> > > > > > > > >> > > > > debugging Kafka and this
> would
> > > > likely
> > > > > >>> get
> > > > > >>> > > worse
> > > > > >>> > > > >> the
> > > > > >>> > > > >> > > more
> > > > > >>> > > > >> > > > > > topic
> > > > > >>> > > > >> > > > > > > > ids
> > > > > >>> > > > >> > > > > > > > >> > got
> > > > > >>> > > > >> > > > > > > > >> > > > > rolled out across the
> protocols,
> > > > > >>> clients
> > > > > >>> > etc.
> > > > > >>> > > > It
> > > > > >>> > > > >> > seems
> > > > > >>> > > > >> > > > > > likely
> > > > > >>> > > > >> > > > > > > > that
> > > > > >>> > > > >> > > > > > > > >> > > > > `kafka-topics.sh` will
> eventually
> > > > > need
> > > > > >>> the
> > > > > >>> > > > >> ability
> > > > > >>> > > > >> > to
> > > > > >>> > > > >> > > > show
> > > > > >>> > > > >> > > > > > the
> > > > > >>> > > > >> > > > > > > > id
> > > > > >>> > > > >> > > > > > > > >> of
> > > > > >>> > > > >> > > > > > > > >> > a
> > > > > >>> > > > >> > > > > > > > >> > > > > topic and perhaps find a
> topic
> > > name
> > > > > >>> given
> > > > > >>> > an
> > > > > >>> > > > id.
> > > > > >>> > > > >> Is
> > > > > >>> > > > >> > > > there
> > > > > >>> > > > >> > > > > > any
> > > > > >>> > > > >> > > > > > > > >> reason
> > > > > >>> > > > >> > > > > > > > >> > > not
> > > > > >>> > > > >> > > > > > > > >> > > > to
> > > > > >>> > > > >> > > > > > > > >> > > > > implement that in this KIP?
> > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > Many thanks,
> > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > Tom
> > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > On Mon, Sep 21, 2020 at 9:54
> PM
> > > > > Justine
> > > > > >>> > > Olshan
> > > > > >>> > > > <
> > > > > >>> > > > >> > > > > > > > >> jolshan@confluent.io
> > > > > >>> > > > >> > > > > > > > >> > >
> > > > > >>> > > > >> > > > > > > > >> > > > > wrote:
> > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > Hi all,
> > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > After thinking about it,
> I've
> > > > > >>> decided to
> > > > > >>> > > > remove
> > > > > >>> > > > >> > the
> > > > > >>> > > > >> > > > > topic
> > > > > >>> > > > >> > > > > > > name
> > > > > >>> > > > >> > > > > > > > >> from
> > > > > >>> > > > >> > > > > > > > >> > > the
> > > > > >>> > > > >> > > > > > > > >> > > > > > Fetch Request and Response
> > > after
> > > > > all.
> > > > > >>> > Since
> > > > > >>> > > > >> there
> > > > > >>> > > > >> > > are
> > > > > >>> > > > >> > > > so
> > > > > >>> > > > >> > > > > > > many
> > > > > >>> > > > >> > > > > > > > of
> > > > > >>> > > > >> > > > > > > > >> > > these
> > > > > >>> > > > >> > > > > > > > >> > > > > > requests per second, it is
> > > worth
> > > > > >>> removing
> > > > > >>> > > the
> > > > > >>> > > > >> > extra
> > > > > >>> > > > >> > > > > > > > information.
> > > > > >>> > > > >> > > > > > > > >> > I've
> > > > > >>> > > > >> > > > > > > > >> > > > > > updated the KIP to reflect
> this
> > > > > >>> change.
> > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > Please let me know if
> there is
> > > > > >>> anything
> > > > > >>> > > else
> > > > > >>> > > > we
> > > > > >>> > > > >> > > should
> > > > > >>> > > > >> > > > > > > discuss
> > > > > >>> > > > >> > > > > > > > >> > before
> > > > > >>> > > > >> > > > > > > > >> > > > > > voting.
> > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > Thank you,
> > > > > >>> > > > >> > > > > > > > >> > > > > > Justine
> > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > On Fri, Sep 18, 2020 at
> 9:46 AM
> > > > > >>> Justine
> > > > > >>> > > > Olshan
> > > > > >>> > > > >> <
> > > > > >>> > > > >> > > > > > > > >> > jolshan@confluent.io
> > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > wrote:
> > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > > Hi Jun,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > > I see what you are
> saying.
> > > For
> > > > > now
> > > > > >>> we
> > > > > >>> > can
> > > > > >>> > > > >> remove
> > > > > >>> > > > >> > > the
> > > > > >>> > > > >> > > > > > extra
> > > > > >>> > > > >> > > > > > > > >> > > > information.
> > > > > >>> > > > >> > > > > > > > >> > > > > > > I'll leave the option to
> add
> > > > more
> > > > > >>> > fields
> > > > > >>> > > to
> > > > > >>> > > > >> the
> > > > > >>> > > > >> > > file
> > > > > >>> > > > >> > > > > in
> > > > > >>> > > > >> > > > > > > the
> > > > > >>> > > > >> > > > > > > > >> > future.
> > > > > >>> > > > >> > > > > > > > >> > > > The
> > > > > >>> > > > >> > > > > > > > >> > > > > > KIP
> > > > > >>> > > > >> > > > > > > > >> > > > > > > has been updated to
> reflect
> > > > this
> > > > > >>> > change.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > > Thanks,
> > > > > >>> > > > >> > > > > > > > >> > > > > > > Justine
> > > > > >>> > > > >> > > > > > > > >> > > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > > On Fri, Sep 18, 2020 at
> 8:46
> > > AM
> > > > > Jun
> > > > > >>> > Rao <
> > > > > >>> > > > >> > > > > > jun@confluent.io
> > > > > >>> > > > >> > > > > > > >
> > > > > >>> > > > >> > > > > > > > >> > wrote:
> > > > > >>> > > > >> > > > > > > > >> > > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> Hi, Justine,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> Thanks for the reply.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> 13. If the log
> directory is
> > > > the
> > > > > >>> source
> > > > > >>> > > of
> > > > > >>> > > > >> > truth,
> > > > > >>> > > > >> > > it
> > > > > >>> > > > >> > > > > > means
> > > > > >>> > > > >> > > > > > > > >> that
> > > > > >>> > > > >> > > > > > > > >> > the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> redundant info in the
> > > metadata
> > > > > >>> file
> > > > > >>> > will
> > > > > >>> > > > be
> > > > > >>> > > > >> > > > ignored.
> > > > > >>> > > > >> > > > > > Then
> > > > > >>> > > > >> > > > > > > > the
> > > > > >>> > > > >> > > > > > > > >> > > > question
> > > > > >>> > > > >> > > > > > > > >> > > > > > is
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> why do we need to put
> the
> > > > > >>> redundant
> > > > > >>> > info
> > > > > >>> > > > in
> > > > > >>> > > > >> the
> > > > > >>> > > > >> > > > > > metadata
> > > > > >>> > > > >> > > > > > > > file
> > > > > >>> > > > >> > > > > > > > >> > now?
> > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> Thanks,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> Jun
> > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> On Thu, Sep 17, 2020 at
> 5:07
> > > > PM
> > > > > >>> > Justine
> > > > > >>> > > > >> Olshan
> > > > > >>> > > > >> > <
> > > > > >>> > > > >> > > > > > > > >> > > > jolshan@confluent.io>
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> wrote:
> > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Hi Jun,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Thanks for the quick
> > > > response!
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > 12. I've decided to
> bump
> > > up
> > > > > the
> > > > > >>> > > versions
> > > > > >>> > > > >> on
> > > > > >>> > > > >> > the
> > > > > >>> > > > >> > > > > > > requests
> > > > > >>> > > > >> > > > > > > > >> and
> > > > > >>> > > > >> > > > > > > > >> > > > updated
> > > > > >>> > > > >> > > > > > > > >> > > > > > the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > KIP. I think it's
> good we
> > > > > >>> thoroughly
> > > > > >>> > > > >> > discussed
> > > > > >>> > > > >> > > > the
> > > > > >>> > > > >> > > > > > > > options
> > > > > >>> > > > >> > > > > > > > >> > here,
> > > > > >>> > > > >> > > > > > > > >> > > > so
> > > > > >>> > > > >> > > > > > > > >> > > > > we
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> know
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > we made a good
> choice. :)
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > 13. This is an
> interesting
> > > > > >>> > situation.
> > > > > >>> > > I
> > > > > >>> > > > >> think
> > > > > >>> > > > >> > > if
> > > > > >>> > > > >> > > > > this
> > > > > >>> > > > >> > > > > > > > does
> > > > > >>> > > > >> > > > > > > > >> > occur
> > > > > >>> > > > >> > > > > > > > >> > > > we
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> should
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > give a warning. I
> agree
> > > that
> > > > > >>> it's
> > > > > >>> > hard
> > > > > >>> > > > to
> > > > > >>> > > > >> > know
> > > > > >>> > > > >> > > > the
> > > > > >>> > > > >> > > > > > > source
> > > > > >>> > > > >> > > > > > > > >> of
> > > > > >>> > > > >> > > > > > > > >> > > truth
> > > > > >>> > > > >> > > > > > > > >> > > > > for
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> sure
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > since the directory
> or the
> > > > > file
> > > > > >>> > could
> > > > > >>> > > be
> > > > > >>> > > > >> > > manually
> > > > > >>> > > > >> > > > > > > > >> modified. I
> > > > > >>> > > > >> > > > > > > > >> > > > guess
> > > > > >>> > > > >> > > > > > > > >> > > > > > the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > directory could be
> used as
> > > > the
> > > > > >>> > source
> > > > > >>> > > of
> > > > > >>> > > > >> > truth.
> > > > > >>> > > > >> > > > To
> > > > > >>> > > > >> > > > > be
> > > > > >>> > > > >> > > > > > > > >> honest,
> > > > > >>> > > > >> > > > > > > > >> > > I'm
> > > > > >>> > > > >> > > > > > > > >> > > > > not
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > really sure what
> happens
> > > in
> > > > > >>> kafka
> > > > > >>> > when
> > > > > >>> > > > the
> > > > > >>> > > > >> > log
> > > > > >>> > > > >> > > > > > > directory
> > > > > >>> > > > >> > > > > > > > is
> > > > > >>> > > > >> > > > > > > > >> > > > renamed
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > manually in such a
> way.
> > > I'm
> > > > > also
> > > > > >>> > > > >> wondering if
> > > > > >>> > > > >> > > the
> > > > > >>> > > > >> > > > > > > > >> situation is
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> recoverable
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > in this scenario.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Thanks,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Justine
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > On Thu, Sep 17, 2020
> at
> > > 4:28
> > > > > PM
> > > > > >>> Jun
> > > > > >>> > > Rao
> > > > > >>> > > > <
> > > > > >>> > > > >> > > > > > > > jun@confluent.io>
> > > > > >>> > > > >> > > > > > > > >> > > wrote:
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Hi, Justine,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Thanks for the
> reply.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > 12. I don't have a
> > > strong
> > > > > >>> > preference
> > > > > >>> > > > >> > either.
> > > > > >>> > > > >> > > > > > However,
> > > > > >>> > > > >> > > > > > > > if
> > > > > >>> > > > >> > > > > > > > >> we
> > > > > >>> > > > >> > > > > > > > >> > > need
> > > > > >>> > > > >> > > > > > > > >> > > > > IBP
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > anyway, maybe it's
> > > easier
> > > > to
> > > > > >>> just
> > > > > >>> > > bump
> > > > > >>> > > > >> up
> > > > > >>> > > > >> > the
> > > > > >>> > > > >> > > > > > version
> > > > > >>> > > > >> > > > > > > > for
> > > > > >>> > > > >> > > > > > > > >> > all
> > > > > >>> > > > >> > > > > > > > >> > > > > inter
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > broker
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > requests and add the
> > > topic
> > > > > id
> > > > > >>> > field
> > > > > >>> > > > as a
> > > > > >>> > > > >> > > > regular
> > > > > >>> > > > >> > > > > > > > field. A
> > > > > >>> > > > >> > > > > > > > >> > > > regular
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> field
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > is
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > a bit more concise
> in
> > > wire
> > > > > >>> > transfer
> > > > > >>> > > > >> than a
> > > > > >>> > > > >> > > > > flexible
> > > > > >>> > > > >> > > > > > > > >> field.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > 13. The confusion
> that I
> > > > was
> > > > > >>> > > referring
> > > > > >>> > > > >> to
> > > > > >>> > > > >> > is
> > > > > >>> > > > >> > > > > > between
> > > > > >>> > > > >> > > > > > > > the
> > > > > >>> > > > >> > > > > > > > >> > topic
> > > > > >>> > > > >> > > > > > > > >> > > > > name
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > partition number
> between
> > > > the
> > > > > >>> log
> > > > > >>> > dir
> > > > > >>> > > > and
> > > > > >>> > > > >> > the
> > > > > >>> > > > >> > > > > > metadata
> > > > > >>> > > > >> > > > > > > > >> file.
> > > > > >>> > > > >> > > > > > > > >> > > For
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> example,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > if
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > the log dir is
> topicA-1
> > > > and
> > > > > >>> the
> > > > > >>> > > > metadata
> > > > > >>> > > > >> > file
> > > > > >>> > > > >> > > > in
> > > > > >>> > > > >> > > > > it
> > > > > >>> > > > >> > > > > > > has
> > > > > >>> > > > >> > > > > > > > >> > topicB
> > > > > >>> > > > >> > > > > > > > >> > > > and
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > partition 0 (say
> due to
> > > a
> > > > > bug
> > > > > >>> or
> > > > > >>> > > > manual
> > > > > >>> > > > >> > > > > > > modification),
> > > > > >>> > > > >> > > > > > > > >> which
> > > > > >>> > > > >> > > > > > > > >> > > one
> > > > > >>> > > > >> > > > > > > > >> > > > > do
> > > > > >>> > > > >> > > > > > > > >> > > > > > we
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > use
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > as the source of
> truth?
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Jun
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > On Thu, Sep 17,
> 2020 at
> > > > 3:43
> > > > > >>> PM
> > > > > >>> > > > Justine
> > > > > >>> > > > >> > > Olshan
> > > > > >>> > > > >> > > > <
> > > > > >>> > > > >> > > > > > > > >> > > > > > jolshan@confluent.io>
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > wrote:
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > Hi Jun,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > Thanks for the
> > > comments.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > 12. I bumped the
> > > > > >>> > > LeaderAndIsrRequest
> > > > > >>> > > > >> > > because
> > > > > >>> > > > >> > > > I
> > > > > >>> > > > >> > > > > > > > removed
> > > > > >>> > > > >> > > > > > > > >> the
> > > > > >>> > > > >> > > > > > > > >> > > > topic
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> name
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > field
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > in the response.
> It
> > > may
> > > > be
> > > > > >>> > > possible
> > > > > >>> > > > to
> > > > > >>> > > > >> > > avoid
> > > > > >>> > > > >> > > > > > > bumping
> > > > > >>> > > > >> > > > > > > > >> the
> > > > > >>> > > > >> > > > > > > > >> > > > version
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > without
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > that change, but
> I may
> > > > be
> > > > > >>> > missing
> > > > > >>> > > > >> > > something.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > I believe
> StopReplica
> > > is
> > > > > >>> > actually
> > > > > >>> > > on
> > > > > >>> > > > >> > > version
> > > > > >>> > > > >> > > > 3
> > > > > >>> > > > >> > > > > > now,
> > > > > >>> > > > >> > > > > > > > but
> > > > > >>> > > > >> > > > > > > > >> > > > because
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > version 2
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > is flexible, I
> kept
> > > that
> > > > > >>> listed
> > > > > >>> > as
> > > > > >>> > > > >> > version
> > > > > >>> > > > >> > > 2
> > > > > >>> > > > >> > > > on
> > > > > >>> > > > >> > > > > > the
> > > > > >>> > > > >> > > > > > > > KIP
> > > > > >>> > > > >> > > > > > > > >> > > page.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> However,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > you
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > may be right in
> that
> > > we
> > > > > may
> > > > > >>> need
> > > > > >>> > > to
> > > > > >>> > > > >> bump
> > > > > >>> > > > >> > > the
> > > > > >>> > > > >> > > > > > > version
> > > > > >>> > > > >> > > > > > > > on
> > > > > >>> > > > >> > > > > > > > >> > > > > > StopReplica
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> to
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > deal
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > with deletion
> > > > differently
> > > > > as
> > > > > >>> > > > mentioned
> > > > > >>> > > > >> > > > above. I
> > > > > >>> > > > >> > > > > > > don't
> > > > > >>> > > > >> > > > > > > > >> know
> > > > > >>> > > > >> > > > > > > > >> > > if
> > > > > >>> > > > >> > > > > > > > >> > > > I
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> have a
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > big
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > preference over
> used
> > > > > tagged
> > > > > >>> > fields
> > > > > >>> > > > or
> > > > > >>> > > > >> > not.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > 13. I was thinking
> > > that
> > > > in
> > > > > >>> the
> > > > > >>> > > case
> > > > > >>> > > > >> where
> > > > > >>> > > > >> > > the
> > > > > >>> > > > >> > > > > > file
> > > > > >>> > > > >> > > > > > > > and
> > > > > >>> > > > >> > > > > > > > >> the
> > > > > >>> > > > >> > > > > > > > >> > > > > request
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > topic
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > ids don't match,
> it
> > > > means
> > > > > >>> that
> > > > > >>> > the
> > > > > >>> > > > >> > broker's
> > > > > >>> > > > >> > > > > > > topic/the
> > > > > >>> > > > >> > > > > > > > >> one
> > > > > >>> > > > >> > > > > > > > >> > in
> > > > > >>> > > > >> > > > > > > > >> > > > the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> file
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > has
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > been deleted. In
> that
> > > > > case,
> > > > > >>> we
> > > > > >>> > > would
> > > > > >>> > > > >> need
> > > > > >>> > > > >> > > to
> > > > > >>> > > > >> > > > > > delete
> > > > > >>> > > > >> > > > > > > > the
> > > > > >>> > > > >> > > > > > > > >> > old
> > > > > >>> > > > >> > > > > > > > >> > > > > topic
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > start
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > receiving the new
> > > > version.
> > > > > >>> If
> > > > > >>> > the
> > > > > >>> > > > >> topic
> > > > > >>> > > > >> > > name
> > > > > >>> > > > >> > > > > were
> > > > > >>> > > > >> > > > > > > to
> > > > > >>> > > > >> > > > > > > > >> > change,
> > > > > >>> > > > >> > > > > > > > >> > > > but
> > > > > >>> > > > >> > > > > > > > >> > > > > > the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > ids
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > still matched, the
> > > file
> > > > > >>> would
> > > > > >>> > also
> > > > > >>> > > > >> need
> > > > > >>> > > > >> > to
> > > > > >>> > > > >> > > > > > update.
> > > > > >>> > > > >> > > > > > > > Am I
> > > > > >>> > > > >> > > > > > > > >> > > > missing
> > > > > >>> > > > >> > > > > > > > >> > > > > a
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> case
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > where the file
> would
> > > be
> > > > > >>> correct
> > > > > >>> > > and
> > > > > >>> > > > >> not
> > > > > >>> > > > >> > the
> > > > > >>> > > > >> > > > > > > request?
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > Thanks,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > Justine
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > On Thu, Sep 17,
> 2020
> > > at
> > > > > >>> 3:18 PM
> > > > > >>> > > Jun
> > > > > >>> > > > >> Rao <
> > > > > >>> > > > >> > > > > > > > >> jun@confluent.io
> > > > > >>> > > > >> > > > > > > > >> > >
> > > > > >>> > > > >> > > > > > > > >> > > > > wrote:
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Hi, Justine,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Thanks for the
> > > reply.
> > > > A
> > > > > >>> couple
> > > > > >>> > > of
> > > > > >>> > > > >> more
> > > > > >>> > > > >> > > > > comments
> > > > > >>> > > > >> > > > > > > > >> below.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > 12. ListOffset
> and
> > > > > >>> > > OffsetForLeader
> > > > > >>> > > > >> > > > currently
> > > > > >>> > > > >> > > > > > > don't
> > > > > >>> > > > >> > > > > > > > >> > support
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> flexible
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > fields.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > So, we have to
> bump
> > > up
> > > > > the
> > > > > >>> > > version
> > > > > >>> > > > >> > number
> > > > > >>> > > > >> > > > and
> > > > > >>> > > > >> > > > > > use
> > > > > >>> > > > >> > > > > > > > >> IBP at
> > > > > >>> > > > >> > > > > > > > >> > > > least
> > > > > >>> > > > >> > > > > > > > >> > > > > > for
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > these
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > two requests.
> Note
> > > > that
> > > > > it
> > > > > >>> > seems
> > > > > >>> > > > >> 2.7.0
> > > > > >>> > > > >> > > will
> > > > > >>> > > > >> > > > > > > require
> > > > > >>> > > > >> > > > > > > > >> IBP
> > > > > >>> > > > >> > > > > > > > >> > > > anyway
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > because
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > of
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > changes in
> > > > KAFKA-10435.
> > > > > >>> Also,
> > > > > >>> > it
> > > > > >>> > > > >> seems
> > > > > >>> > > > >> > > that
> > > > > >>> > > > >> > > > > the
> > > > > >>> > > > >> > > > > > > > >> version
> > > > > >>> > > > >> > > > > > > > >> > > for
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> LeaderAndIsrRequest
> > > > and
> > > > > >>> > > > StopReplica
> > > > > >>> > > > >> are
> > > > > >>> > > > >> > > > > bumped
> > > > > >>> > > > >> > > > > > > even
> > > > > >>> > > > >> > > > > > > > >> > though
> > > > > >>> > > > >> > > > > > > > >> > > > we
> > > > > >>> > > > >> > > > > > > > >> > > > > > only
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > added
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > a
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > tagged field.
> But
> > > > since
> > > > > >>> IBP is
> > > > > >>> > > > >> needed
> > > > > >>> > > > >> > > > anyway,
> > > > > >>> > > > >> > > > > > we
> > > > > >>> > > > >> > > > > > > > may
> > > > > >>> > > > >> > > > > > > > >> > want
> > > > > >>> > > > >> > > > > > > > >> > > to
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> revisit
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > overall tagged
> field
> > > > > >>> choice.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > 13. The only
> > > downside
> > > > is
> > > > > >>> the
> > > > > >>> > > > >> potential
> > > > > >>> > > > >> > > > > > confusion
> > > > > >>> > > > >> > > > > > > on
> > > > > >>> > > > >> > > > > > > > >> > which
> > > > > >>> > > > >> > > > > > > > >> > > > one
> > > > > >>> > > > >> > > > > > > > >> > > > > is
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > source
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > of truth if they
> > > don't
> > > > > >>> match.
> > > > > >>> > > > >> Another
> > > > > >>> > > > >> > > > option
> > > > > >>> > > > >> > > > > is
> > > > > >>> > > > >> > > > > > > to
> > > > > >>> > > > >> > > > > > > > >> > include
> > > > > >>> > > > >> > > > > > > > >> > > > > those
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > fields
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > in
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the metadata
> file
> > > when
> > > > > we
> > > > > >>> > > actually
> > > > > >>> > > > >> > change
> > > > > >>> > > > >> > > > the
> > > > > >>> > > > >> > > > > > > > >> directory
> > > > > >>> > > > >> > > > > > > > >> > > > > > structure.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Thanks,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Jun
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > On Thu, Sep 17,
> 2020
> > > > at
> > > > > >>> 2:01
> > > > > >>> > PM
> > > > > >>> > > > >> Justine
> > > > > >>> > > > >> > > > > Olshan
> > > > > >>> > > > >> > > > > > <
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> jolshan@confluent.io
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > wrote:
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > Hello all,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I've thought
> some
> > > > more
> > > > > >>> about
> > > > > >>> > > > >> removing
> > > > > >>> > > > >> > > the
> > > > > >>> > > > >> > > > > > topic
> > > > > >>> > > > >> > > > > > > > >> name
> > > > > >>> > > > >> > > > > > > > >> > > field
> > > > > >>> > > > >> > > > > > > > >> > > > > > from
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > some
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > of
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > requests. On
> > > closer
> > > > > >>> > inspection
> > > > > >>> > > > of
> > > > > >>> > > > >> the
> > > > > >>> > > > >> > > > > > > > >> > > requests/responses,
> > > > > >>> > > > >> > > > > > > > >> > > > it
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> seems
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > internal
> changes
> > > > would
> > > > > >>> be
> > > > > >>> > much
> > > > > >>> > > > >> larger
> > > > > >>> > > > >> > > > than
> > > > > >>> > > > >> > > > > I
> > > > > >>> > > > >> > > > > > > > >> expected.
> > > > > >>> > > > >> > > > > > > > >> > > > Some
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > protocols
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > involve
> clients,
> > > so
> > > > > they
> > > > > >>> > would
> > > > > >>> > > > >> > require
> > > > > >>> > > > >> > > > > > changes
> > > > > >>> > > > >> > > > > > > > too.
> > > > > >>> > > > >> > > > > > > > >> > I'm
> > > > > >>> > > > >> > > > > > > > >> > > > > > thinking
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > that
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > for
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > now, removing
> the
> > > > > topic
> > > > > >>> name
> > > > > >>> > > > from
> > > > > >>> > > > >> > these
> > > > > >>> > > > >> > > > > > > requests
> > > > > >>> > > > >> > > > > > > > >> and
> > > > > >>> > > > >> > > > > > > > >> > > > > responses
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> are
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > out
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > scope.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I have
> decided to
> > > > just
> > > > > >>> keep
> > > > > >>> > > the
> > > > > >>> > > > >> > change
> > > > > >>> > > > >> > > > > > > > >> > > > LeaderAndIsrResponse
> > > > > >>> > > > >> > > > > > > > >> > > > > to
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > remove
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > topic name,
> and
> > > have
> > > > > >>> updated
> > > > > >>> > > the
> > > > > >>> > > > >> KIP
> > > > > >>> > > > >> > to
> > > > > >>> > > > >> > > > > > reflect
> > > > > >>> > > > >> > > > > > > > >> this
> > > > > >>> > > > >> > > > > > > > >> > > > > change. I
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> have
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > also
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > mentioned the
> > > other
> > > > > >>> requests
> > > > > >>> > > and
> > > > > >>> > > > >> > > > responses
> > > > > >>> > > > >> > > > > in
> > > > > >>> > > > >> > > > > > > > >> future
> > > > > >>> > > > >> > > > > > > > >> > > work.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I'm hoping to
> > > start
> > > > > the
> > > > > >>> > voting
> > > > > >>> > > > >> > process
> > > > > >>> > > > >> > > > > soon,
> > > > > >>> > > > >> > > > > > so
> > > > > >>> > > > >> > > > > > > > >> let me
> > > > > >>> > > > >> > > > > > > > >> > > > know
> > > > > >>> > > > >> > > > > > > > >> > > > > if
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > there
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > is
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > anything else
> we
> > > > > should
> > > > > >>> > > discuss.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > Thank you,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > Justine
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > On Tue, Sep
> 15,
> > > 2020
> > > > > at
> > > > > >>> 3:57
> > > > > >>> > > PM
> > > > > >>> > > > >> > Justine
> > > > > >>> > > > >> > > > > > Olshan
> > > > > >>> > > > >> > > > > > > <
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > jolshan@confluent.io
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > wrote:
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > Hello again,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > To follow
> up on
> > > > some
> > > > > >>> of
> > > > > >>> > the
> > > > > >>> > > > >> other
> > > > > >>> > > > >> > > > > comments:
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 10/11) We
> can
> > > > remove
> > > > > >>> the
> > > > > >>> > > topic
> > > > > >>> > > > >> name
> > > > > >>> > > > >> > > > from
> > > > > >>> > > > >> > > > > > > these
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > requests/responses,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > and
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > that means
> we
> > > will
> > > > > >>> just
> > > > > >>> > have
> > > > > >>> > > > to
> > > > > >>> > > > >> > make
> > > > > >>> > > > >> > > a
> > > > > >>> > > > >> > > > > few
> > > > > >>> > > > >> > > > > > > > >> internal
> > > > > >>> > > > >> > > > > > > > >> > > > > changes
> > > > > >>> > > > >> > > > > > > > >> > > > > > to
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > make
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > partitions
> > > > > accessible
> > > > > >>> by
> > > > > >>> > > topic
> > > > > >>> > > > >> id
> > > > > >>> > > > >> > and
> > > > > >>> > > > >> > > > > > > > partition.
> > > > > >>> > > > >> > > > > > > > >> I
> > > > > >>> > > > >> > > > > > > > >> > can
> > > > > >>> > > > >> > > > > > > > >> > > > > > update
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > KIP
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > remove them
> > > unless
> > > > > >>> anyone
> > > > > >>> > > > thinks
> > > > > >>> > > > >> > they
> > > > > >>> > > > >> > > > > > should
> > > > > >>> > > > >> > > > > > > > >> stay.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 12)
> Addressed in
> > > > the
> > > > > >>> > > previous
> > > > > >>> > > > >> > email.
> > > > > >>> > > > >> > > > I've
> > > > > >>> > > > >> > > > > > > > updated
> > > > > >>> > > > >> > > > > > > > >> > the
> > > > > >>> > > > >> > > > > > > > >> > > > KIP
> > > > > >>> > > > >> > > > > > > > >> > > > > to
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > include
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > tagged
> fields
> > > for
> > > > > the
> > > > > >>> > > requests
> > > > > >>> > > > >> and
> > > > > >>> > > > >> > > > > > responses.
> > > > > >>> > > > >> > > > > > > > >> (More
> > > > > >>> > > > >> > > > > > > > >> > on
> > > > > >>> > > > >> > > > > > > > >> > > > > that
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > below)
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 13) I think
> part
> > > > of
> > > > > >>> the
> > > > > >>> > idea
> > > > > >>> > > > for
> > > > > >>> > > > >> > > > > including
> > > > > >>> > > > >> > > > > > > this
> > > > > >>> > > > >> > > > > > > > >> > > > > information
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> is to
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > prepare
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > for future
> > > > changes.
> > > > > >>> > Perhaps
> > > > > >>> > > > the
> > > > > >>> > > > >> > > > directory
> > > > > >>> > > > >> > > > > > > > >> structure
> > > > > >>> > > > >> > > > > > > > >> > > > might
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> change
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > from
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > >>> topicName_partitionNumber
> > > > > >>> > to
> > > > > >>> > > > >> > > something
> > > > > >>> > > > >> > > > > like
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> topicID_partitionNumber.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Then
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > it would be
> > > useful
> > > > > to
> > > > > >>> have
> > > > > >>> > > the
> > > > > >>> > > > >> > topic
> > > > > >>> > > > >> > > > name
> > > > > >>> > > > >> > > > > > in
> > > > > >>> > > > >> > > > > > > > the
> > > > > >>> > > > >> > > > > > > > >> > file
> > > > > >>> > > > >> > > > > > > > >> > > > > since
> > > > > >>> > > > >> > > > > > > > >> > > > > > it
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > would
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > not
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > be
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > in the
> directory
> > > > > >>> > structure.
> > > > > >>> > > > >> > > Supporting
> > > > > >>> > > > >> > > > > > topic
> > > > > >>> > > > >> > > > > > > > >> renames
> > > > > >>> > > > >> > > > > > > > >> > > > might
> > > > > >>> > > > >> > > > > > > > >> > > > > > be
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > easier
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > if
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > other
> fields are
> > > > > >>> included.
> > > > > >>> > > > Would
> > > > > >>> > > > >> > > there
> > > > > >>> > > > >> > > > be
> > > > > >>> > > > >> > > > > > any
> > > > > >>> > > > >> > > > > > > > >> > > downsides
> > > > > >>> > > > >> > > > > > > > >> > > > to
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > including
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > this
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > information?
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 14)  Yes, we
> > > would
> > > > > >>> need to
> > > > > >>> > > > copy
> > > > > >>> > > > >> the
> > > > > >>> > > > >> > > > > > partition
> > > > > >>> > > > >> > > > > > > > >> > metadata
> > > > > >>> > > > >> > > > > > > > >> > > > > file
> > > > > >>> > > > >> > > > > > > > >> > > > > > in
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > this
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > process.
> I've
> > > > > updated
> > > > > >>> the
> > > > > >>> > > KIP
> > > > > >>> > > > to
> > > > > >>> > > > >> > > > include
> > > > > >>> > > > >> > > > > > > this.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 15) I
> believe
> > > > Lucas
> > > > > >>> meant
> > > > > >>> > v1
> > > > > >>> > > > >> and v2
> > > > > >>> > > > >> > > > here.
> > > > > >>> > > > >> > > > > > He
> > > > > >>> > > > >> > > > > > > > was
> > > > > >>> > > > >> > > > > > > > >> > > > referring
> > > > > >>> > > > >> > > > > > > > >> > > > > > to
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> how
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > requests
> would
> > > > fall
> > > > > >>> under
> > > > > >>> > > > >> different
> > > > > >>> > > > >> > > IBP
> > > > > >>> > > > >> > > > > and
> > > > > >>> > > > >> > > > > > > > meant
> > > > > >>> > > > >> > > > > > > > >> > that
> > > > > >>> > > > >> > > > > > > > >> > > > > older
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > brokers
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > would
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > have to use
> the
> > > > > older
> > > > > >>> > > version
> > > > > >>> > > > of
> > > > > >>> > > > >> > the
> > > > > >>> > > > >> > > > > > request
> > > > > >>> > > > >> > > > > > > > and
> > > > > >>> > > > >> > > > > > > > >> the
> > > > > >>> > > > >> > > > > > > > >> > > > > > existing
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > topic
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > deletion
> > > process.
> > > > At
> > > > > >>> > first,
> > > > > >>> > > it
> > > > > >>> > > > >> > seemed
> > > > > >>> > > > >> > > > > like
> > > > > >>> > > > >> > > > > > > > tagged
> > > > > >>> > > > >> > > > > > > > >> > > fields
> > > > > >>> > > > >> > > > > > > > >> > > > > > would
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > resolve
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > the IBP
> issue.
> > > > > >>> However, we
> > > > > >>> > > may
> > > > > >>> > > > >> need
> > > > > >>> > > > >> > > IBP
> > > > > >>> > > > >> > > > > for
> > > > > >>> > > > >> > > > > > > > this
> > > > > >>> > > > >> > > > > > > > >> > > request
> > > > > >>> > > > >> > > > > > > > >> > > > > > after
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > all
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > since
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > the
> controller
> > > > > >>> handles the
> > > > > >>> > > > topic
> > > > > >>> > > > >> > > > deletion
> > > > > >>> > > > >> > > > > > > > >> > differently
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> depending
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > on
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > IBP
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > version. In
> an
> > > > older
> > > > > >>> > > version,
> > > > > >>> > > > we
> > > > > >>> > > > >> > > can't
> > > > > >>> > > > >> > > > > just
> > > > > >>> > > > >> > > > > > > > send
> > > > > >>> > > > >> > > > > > > > >> a
> > > > > >>> > > > >> > > > > > > > >> > > > > > StopReplica
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > delete
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > topic
> > > immediately
> > > > > like
> > > > > >>> > we'd
> > > > > >>> > > > >> want to
> > > > > >>> > > > >> > > for
> > > > > >>> > > > >> > > > > > this
> > > > > >>> > > > >> > > > > > > > KIP.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > This makes
> me
> > > > wonder
> > > > > >>> if we
> > > > > >>> > > > want
> > > > > >>> > > > >> > > tagged
> > > > > >>> > > > >> > > > > > fields
> > > > > >>> > > > >> > > > > > > > on
> > > > > >>> > > > >> > > > > > > > >> all
> > > > > >>> > > > >> > > > > > > > >> > > the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> requests
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > after
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > all. Let me
> know
> > > > > your
> > > > > >>> > > > thoughts!
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > Justine
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > On Tue, Sep
> 15,
> > > > 2020
> > > > > >>> at
> > > > > >>> > 1:03
> > > > > >>> > > > PM
> > > > > >>> > > > >> > > Justine
> > > > > >>> > > > >> > > > > > > Olshan
> > > > > >>> > > > >> > > > > > > > <
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> jolshan@confluent.io
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > wrote:
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Hi all,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Jun
> brought up
> > > a
> > > > > good
> > > > > >>> > point
> > > > > >>> > > > in
> > > > > >>> > > > >> his
> > > > > >>> > > > >> > > > last
> > > > > >>> > > > >> > > > > > > email
> > > > > >>> > > > >> > > > > > > > >> about
> > > > > >>> > > > >> > > > > > > > >> > > > > tagged
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > fields,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > and
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> I've
> updated
> > > the
> > > > > KIP
> > > > > >>> to
> > > > > >>> > > > reflect
> > > > > >>> > > > >> > that
> > > > > >>> > > > >> > > > the
> > > > > >>> > > > >> > > > > > > > >> changes to
> > > > > >>> > > > >> > > > > > > > >> > > > > > requests
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > responses
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> will be in
> the
> > > > form
> > > > > >>> of
> > > > > >>> > > tagged
> > > > > >>> > > > >> > fields
> > > > > >>> > > > >> > > > to
> > > > > >>> > > > >> > > > > > > avoid
> > > > > >>> > > > >> > > > > > > > >> > > changing
> > > > > >>> > > > >> > > > > > > > >> > > > > IBP.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Jun: I
> plan on
> > > > > >>> sending a
> > > > > >>> > > > >> followup
> > > > > >>> > > > >> > > > email
> > > > > >>> > > > >> > > > > to
> > > > > >>> > > > >> > > > > > > > >> address
> > > > > >>> > > > >> > > > > > > > >> > > some
> > > > > >>> > > > >> > > > > > > > >> > > > > of
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > other
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> points.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Thanks,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Justine
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> On Mon,
> Sep 14,
> > > > > 2020
> > > > > >>> at
> > > > > >>> > > 4:25
> > > > > >>> > > > PM
> > > > > >>> > > > >> > Jun
> > > > > >>> > > > >> > > > Rao
> > > > > >>> > > > >> > > > > <
> > > > > >>> > > > >> > > > > > > > >> > > > > jun@confluent.io>
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > wrote:
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Hi,
> Justine,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Thanks
> for the
> > > > > >>> updated
> > > > > >>> > > KIP.
> > > > > >>> > > > A
> > > > > >>> > > > >> few
> > > > > >>> > > > >> > > > > > comments
> > > > > >>> > > > >> > > > > > > > >> below.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 10.
> > > LeaderAndIsr
> > > > > >>> > Response:
> > > > > >>> > > > Do
> > > > > >>> > > > >> we
> > > > > >>> > > > >> > > need
> > > > > >>> > > > >> > > > > the
> > > > > >>> > > > >> > > > > > > > topic
> > > > > >>> > > > >> > > > > > > > >> > > name?
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 11. For
> the
> > > > > changed
> > > > > >>> > > > >> > > request/response,
> > > > > >>> > > > >> > > > > > other
> > > > > >>> > > > >> > > > > > > > >> than
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> LeaderAndIsr,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > UpdateMetadata,
> > > > > >>> > Metadata,
> > > > > >>> > > do
> > > > > >>> > > > >> we
> > > > > >>> > > > >> > > need
> > > > > >>> > > > >> > > > to
> > > > > >>> > > > >> > > > > > > > include
> > > > > >>> > > > >> > > > > > > > >> > the
> > > > > >>> > > > >> > > > > > > > >> > > > > topic
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> name?
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 12. It
> seems
> > > > that
> > > > > >>> > upgrades
> > > > > >>> > > > >> don't
> > > > > >>> > > > >> > > > > require
> > > > > >>> > > > >> > > > > > > IBP.
> > > > > >>> > > > >> > > > > > > > >> Does
> > > > > >>> > > > >> > > > > > > > >> > > > that
> > > > > >>> > > > >> > > > > > > > >> > > > > > mean
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > new
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> fields
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> in all the
> > > > > >>> > > request/response
> > > > > >>> > > > >> are
> > > > > >>> > > > >> > > added
> > > > > >>> > > > >> > > > > as
> > > > > >>> > > > >> > > > > > > > tagged
> > > > > >>> > > > >> > > > > > > > >> > > fields
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> without
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > bumping
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > up
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> the
> request
> > > > > >>> version? It
> > > > > >>> > > > would
> > > > > >>> > > > >> be
> > > > > >>> > > > >> > > > useful
> > > > > >>> > > > >> > > > > > to
> > > > > >>> > > > >> > > > > > > > make
> > > > > >>> > > > >> > > > > > > > >> > that
> > > > > >>> > > > >> > > > > > > > >> > > > > > clear.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 13.
> Partition
> > > > > >>> Metadata
> > > > > >>> > > file:
> > > > > >>> > > > >> Do
> > > > > >>> > > > >> > we
> > > > > >>> > > > >> > > > need
> > > > > >>> > > > >> > > > > > to
> > > > > >>> > > > >> > > > > > > > >> include
> > > > > >>> > > > >> > > > > > > > >> > > the
> > > > > >>> > > > >> > > > > > > > >> > > > > > topic
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > name
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > and
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> partition
> id
> > > > since
> > > > > >>> they
> > > > > >>> > > are
> > > > > >>> > > > >> > implied
> > > > > >>> > > > >> > > > in
> > > > > >>> > > > >> > > > > > the
> > > > > >>> > > > >> > > > > > > > >> > directory
> > > > > >>> > > > >> > > > > > > > >> > > > > name?
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 14. In the
> > > JBOD
> > > > > >>> mode, we
> > > > > >>> > > > >> support
> > > > > >>> > > > >> > > > > moving a
> > > > > >>> > > > >> > > > > > > > >> > > partition's
> > > > > >>> > > > >> > > > > > > > >> > > > > data
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> from
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > one
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > disk
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> to
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> another.
> Will
> > > > the
> > > > > >>> new
> > > > > >>> > > > >> partition
> > > > > >>> > > > >> > > > > metadata
> > > > > >>> > > > >> > > > > > > file
> > > > > >>> > > > >> > > > > > > > >> be
> > > > > >>> > > > >> > > > > > > > >> > > > copied
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> during
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> process?
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 15. The
> KIP
> > > says
> > > > > >>> "Remove
> > > > > >>> > > > >> deleted
> > > > > >>> > > > >> > > > topics
> > > > > >>> > > > >> > > > > > > from
> > > > > >>> > > > >> > > > > > > > >> > > replicas
> > > > > >>> > > > >> > > > > > > > >> > > > by
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > sending
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > StopReplicaRequest
> > > > > >>> V2
> > > > > >>> > for
> > > > > >>> > > > any
> > > > > >>> > > > >> > > topics
> > > > > >>> > > > >> > > > > > which
> > > > > >>> > > > >> > > > > > > do
> > > > > >>> > > > >> > > > > > > > >> not
> > > > > >>> > > > >> > > > > > > > >> > > > > contain
> > > > > >>> > > > >> > > > > > > > >> > > > > > a
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > topic
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > ID,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > and
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> V3 for any
> > > > topics
> > > > > >>> which
> > > > > >>> > do
> > > > > >>> > > > >> > contain
> > > > > >>> > > > >> > > a
> > > > > >>> > > > >> > > > > > topic
> > > > > >>> > > > >> > > > > > > > >> ID.".
> > > > > >>> > > > >> > > > > > > > >> > > > > However,
> > > > > >>> > > > >> > > > > > > > >> > > > > > it
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > seems
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> updated
> > > > controller
> > > > > >>> will
> > > > > >>> > > > create
> > > > > >>> > > > >> > all
> > > > > >>> > > > >> > > > > > missing
> > > > > >>> > > > >> > > > > > > > >> topic
> > > > > >>> > > > >> > > > > > > > >> > IDs
> > > > > >>> > > > >> > > > > > > > >> > > > > first
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > before
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > doing
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> other
> actions.
> > > > So,
> > > > > >>> is
> > > > > >>> > > > >> > > > > StopReplicaRequest
> > > > > >>> > > > >> > > > > > V2
> > > > > >>> > > > >> > > > > > > > >> > needed?
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Jun
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> On Fri,
> Sep
> > > 11,
> > > > > >>> 2020 at
> > > > > >>> > > > 10:31
> > > > > >>> > > > >> AM
> > > > > >>> > > > >> > > John
> > > > > >>> > > > >> > > > > > > > Roesler <
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > vvcephei@apache.org
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> wrote:
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > Thanks,
> > > > Justine!
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > Your
> > > response
> > > > > >>> seems
> > > > > >>> > > > >> compelling
> > > > > >>> > > > >> > to
> > > > > >>> > > > >> > > > me.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > -John
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > On Fri,
> > > > > >>> 2020-09-11 at
> > > > > >>> > > > 10:17
> > > > > >>> > > > >> > > -0700,
> > > > > >>> > > > >> > > > > > > Justine
> > > > > >>> > > > >> > > > > > > > >> > Olshan
> > > > > >>> > > > >> > > > > > > > >> > > > > wrote:
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Hello
> all,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> Thanks for
> > > > > >>> > continuing
> > > > > >>> > > > the
> > > > > >>> > > > >> > > > > > discussion! I
> > > > > >>> > > > >> > > > > > > > >> have a
> > > > > >>> > > > >> > > > > > > > >> > > few
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > responses
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > to
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > your
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > points.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Tom:
> You
> > > are
> > > > > >>> correct
> > > > > >>> > > in
> > > > > >>> > > > >> that
> > > > > >>> > > > >> > > this
> > > > > >>> > > > >> > > > > KIP
> > > > > >>> > > > >> > > > > > > has
> > > > > >>> > > > >> > > > > > > > >> not
> > > > > >>> > > > >> > > > > > > > >> > > > > > mentioned
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > >>> > DeleteTopicsRequest. I
> > > > > >>> > > > >> think
> > > > > >>> > > > >> > > that
> > > > > >>> > > > >> > > > > > this
> > > > > >>> > > > >> > > > > > > > >> would
> > > > > >>> > > > >> > > > > > > > >> > be
> > > > > >>> > > > >> > > > > > > > >> > > > out
> > > > > >>> > > > >> > > > > > > > >> > > > > of
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > scope
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > for
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> now, but
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > may be
> > > > > something
> > > > > >>> > worth
> > > > > >>> > > > >> adding
> > > > > >>> > > > >> > > in
> > > > > >>> > > > >> > > > > the
> > > > > >>> > > > >> > > > > > > > >> future.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > John:
> We
> > > did
> > > > > >>> > consider
> > > > > >>> > > > >> > sequence
> > > > > >>> > > > >> > > > ids,
> > > > > >>> > > > >> > > > > > but
> > > > > >>> > > > >> > > > > > > > >> there
> > > > > >>> > > > >> > > > > > > > >> > > are
> > > > > >>> > > > >> > > > > > > > >> > > > a
> > > > > >>> > > > >> > > > > > > > >> > > > > > few
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > reasons
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> favor
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > UUIDs.
> > > There
> > > > > are
> > > > > >>> > > several
> > > > > >>> > > > >> > cases
> > > > > >>> > > > >> > > > > where
> > > > > >>> > > > >> > > > > > > > topics
> > > > > >>> > > > >> > > > > > > > >> > from
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> different
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > clusters
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> may
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> interact
> > > now
> > > > > >>> and in
> > > > > >>> > > the
> > > > > >>> > > > >> > future.
> > > > > >>> > > > >> > > > For
> > > > > >>> > > > >> > > > > > > > >> example,
> > > > > >>> > > > >> > > > > > > > >> > > > Mirror
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> Maker 2
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > may
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> benefit
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > from
> being
> > > > > able
> > > > > >>> to
> > > > > >>> > > > detect
> > > > > >>> > > > >> > when
> > > > > >>> > > > >> > > a
> > > > > >>> > > > >> > > > > > > cluster
> > > > > >>> > > > >> > > > > > > > >> being
> > > > > >>> > > > >> > > > > > > > >> > > > > > mirrored
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> is
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > deleted
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> and
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> recreated
> > > > and
> > > > > >>> > globally
> > > > > >>> > > > >> unique
> > > > > >>> > > > >> > > > > > > identifiers
> > > > > >>> > > > >> > > > > > > > >> > would
> > > > > >>> > > > >> > > > > > > > >> > > > make
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > resolving
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > issues
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > easier
> > > than
> > > > > >>> sequence
> > > > > >>> > > IDs
> > > > > >>> > > > >> > which
> > > > > >>> > > > >> > > > may
> > > > > >>> > > > >> > > > > > > > collide
> > > > > >>> > > > >> > > > > > > > >> > > between
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > clusters.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > KIP-405
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> (tiered
> > > > > storage)
> > > > > >>> > will
> > > > > >>> > > > also
> > > > > >>> > > > >> > > > benefit
> > > > > >>> > > > >> > > > > > from
> > > > > >>> > > > >> > > > > > > > >> > globally
> > > > > >>> > > > >> > > > > > > > >> > > > > > unique
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> IDs
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > as
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > shared
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> buckets
> > > may
> > > > be
> > > > > >>> used
> > > > > >>> > > > >> between
> > > > > >>> > > > >> > > > > clusters.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> Globally
> > > > > unique
> > > > > >>> IDs
> > > > > >>> > > > would
> > > > > >>> > > > >> > also
> > > > > >>> > > > >> > > > make
> > > > > >>> > > > >> > > > > > > > >> > > functionality
> > > > > >>> > > > >> > > > > > > > >> > > > > like
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > moving
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > topics
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> between
> > > > > >>> disparate
> > > > > >>> > > > clusters
> > > > > >>> > > > >> > > easier
> > > > > >>> > > > >> > > > > in
> > > > > >>> > > > >> > > > > > > the
> > > > > >>> > > > >> > > > > > > > >> > future,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> simplify
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > any
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > future
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > implementations
> > > > > >>> of
> > > > > >>> > > > backups
> > > > > >>> > > > >> > and
> > > > > >>> > > > >> > > > > > > restores,
> > > > > >>> > > > >> > > > > > > > >> and
> > > > > >>> > > > >> > > > > > > > >> > > more.
> > > > > >>> > > > >> > > > > > > > >> > > > > In
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > general,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> unique IDs
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > would
> > > ensure
> > > > > >>> that
> > > > > >>> > the
> > > > > >>> > > > >> source
> > > > > >>> > > > >> > > > > cluster
> > > > > >>> > > > >> > > > > > > > >> topics do
> > > > > >>> > > > >> > > > > > > > >> > > not
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> conflict
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > with
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > destination
> > > > > >>> cluster
> > > > > >>> > > > >> topics.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > If we
> were
> > > > to
> > > > > >>> use
> > > > > >>> > > > sequence
> > > > > >>> > > > >> > ids,
> > > > > >>> > > > >> > > > we
> > > > > >>> > > > >> > > > > > > would
> > > > > >>> > > > >> > > > > > > > >> need
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> sufficiently
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > large
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> cluster
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > ids
> to be
> > > > > stored
> > > > > >>> > with
> > > > > >>> > > > the
> > > > > >>> > > > >> > topic
> > > > > >>> > > > >> > > > > > > > >> identifiers or
> > > > > >>> > > > >> > > > > > > > >> > > we
> > > > > >>> > > > >> > > > > > > > >> > > > > run
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > risk
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > collisions.
> > > > > This
> > > > > >>> > will
> > > > > >>> > > > >> give up
> > > > > >>> > > > >> > > any
> > > > > >>> > > > >> > > > > > > > >> advantage in
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> compactness
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> sequence
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> numbers
> > > may
> > > > > >>> bring.
> > > > > >>> > > Given
> > > > > >>> > > > >> > these
> > > > > >>> > > > >> > > > > > > > advantages I
> > > > > >>> > > > >> > > > > > > > >> > > think
> > > > > >>> > > > >> > > > > > > > >> > > > it
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> makes
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > sense
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> use
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > UUIDs.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Gokul:
> > > This
> > > > is
> > > > > >>> an
> > > > > >>> > > > >> interesting
> > > > > >>> > > > >> > > > idea,
> > > > > >>> > > > >> > > > > > but
> > > > > >>> > > > >> > > > > > > > >> this
> > > > > >>> > > > >> > > > > > > > >> > is
> > > > > >>> > > > >> > > > > > > > >> > > a
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> breaking
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > change.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Out of
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > scope
> for
> > > > now,
> > > > > >>> but
> > > > > >>> > > maybe
> > > > > >>> > > > >> > worth
> > > > > >>> > > > >> > > > > > > discussing
> > > > > >>> > > > >> > > > > > > > >> in
> > > > > >>> > > > >> > > > > > > > >> > the
> > > > > >>> > > > >> > > > > > > > >> > > > > > future.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Hope
> this
> > > > > >>> explains
> > > > > >>> > > some
> > > > > >>> > > > of
> > > > > >>> > > > >> > the
> > > > > >>> > > > >> > > > > > > decisions,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> Justine
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > On
> Fri,
> > > Sep
> > > > > 11,
> > > > > >>> 2020
> > > > > >>> > > at
> > > > > >>> > > > >> 8:27
> > > > > >>> > > > >> > AM
> > > > > >>> > > > >> > > > > Gokul
> > > > > >>> > > > >> > > > > > > > >> Ramanan
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> Subramanian <
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > >>> > gokul2411s@gmail.com>
> > > > > >>> > > > >> wrote:
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Hi.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> Thanks
> > > for
> > > > > the
> > > > > >>> > KIP.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> Have you
> > > > > >>> thought
> > > > > >>> > > about
> > > > > >>> > > > >> > > whether
> > > > > >>> > > > >> > > > it
> > > > > >>> > > > >> > > > > > > makes
> > > > > >>> > > > >> > > > > > > > >> > sense
> > > > > >>> > > > >> > > > > > > > >> > > to
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> support
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> authorizing a
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > principal
> > > > > for
> > > > > >>> a
> > > > > >>> > > topic
> > > > > >>> > > > ID
> > > > > >>> > > > >> > > rather
> > > > > >>> > > > >> > > > > > than
> > > > > >>> > > > >> > > > > > > a
> > > > > >>> > > > >> > > > > > > > >> topic
> > > > > >>> > > > >> > > > > > > > >> > > > name
> > > > > >>> > > > >> > > > > > > > >> > > > > to
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > achieve
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> tighter
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > security?
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Or
> is
> > > the
> > > > > >>> topic ID
> > > > > >>> > > > >> > > > fundamentally
> > > > > >>> > > > >> > > > > an
> > > > > >>> > > > >> > > > > > > > >> internal
> > > > > >>> > > > >> > > > > > > > >> > > > > detail
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > similar
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > to
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> epochs
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > used
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > in a
> > > bunch
> > > > > of
> > > > > >>> > other
> > > > > >>> > > > >> places
> > > > > >>> > > > >> > in
> > > > > >>> > > > >> > > > > > Kafka?
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> Thanks.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > On
> Fri,
> > > > Sep
> > > > > >>> 11,
> > > > > >>> > 2020
> > > > > >>> > > > at
> > > > > >>> > > > >> > 4:06
> > > > > >>> > > > >> > > PM
> > > > > >>> > > > >> > > > > > John
> > > > > >>> > > > >> > > > > > > > >> > Roesler <
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > vvcephei@apache.org
> > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > wrote:
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> Hello
> > > > > >>> Justine,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> Thanks
> > > > for
> > > > > >>> the
> > > > > >>> > > KIP!
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > I
> > > happen
> > > > > to
> > > > > >>> have
> > > > > >>> > > > been
> > > > > >>> > > > >> > > > > confronted
> > > > > >>> > > > >> > > > > > > > >> recently
> > > > > >>> > > > >> > > > > > > > >> > > with
> > > > > >>> > > > >> > > > > > > > >> > > > > the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> need
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > to
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > keep
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > track of
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > a
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> large
> > > > > >>> number of
> > > > > >>> > > > >> topics as
> > > > > >>> > > > >> > > > > > compactly
> > > > > >>> > > > >> > > > > > > > as
> > > > > >>> > > > >> > > > > > > > >> > > > > possible. I
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> was
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > going
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> come
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > up
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> with
> > > > some
> > > > > >>> way to
> > > > > >>> > > > >> > dictionary
> > > > > >>> > > > >> > > > > > encode
> > > > > >>> > > > >> > > > > > > > the
> > > > > >>> > > > >> > > > > > > > >> > topic
> > > > > >>> > > > >> > > > > > > > >> > > > > names
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> as
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > integers,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> but
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > this
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> seems
> > > > much
> > > > > >>> > better!
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > Apologies
> > > > > if
> > > > > >>> > this
> > > > > >>> > > > has
> > > > > >>> > > > >> > been
> > > > > >>> > > > >> > > > > raised
> > > > > >>> > > > >> > > > > > > > >> before,
> > > > > >>> > > > >> > > > > > > > >> > > but
> > > > > >>> > > > >> > > > > > > > >> > > > > I’m
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > wondering
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> about the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> choice
> > > > of
> > > > > >>> UUID
> > > > > >>> > vs
> > > > > >>> > > > >> > sequence
> > > > > >>> > > > >> > > > > number
> > > > > >>> > > > >> > > > > > > for
> > > > > >>> > > > >> > > > > > > > >> the
> > > > > >>> > > > >> > > > > > > > >> > > ids.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > Typically,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > I’ve
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> seen
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > UUIDs
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> in two
> > > > > >>> > situations:
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 1.
> > > When
> > > > > >>> > processes
> > > > > >>> > > > >> need to
> > > > > >>> > > > >> > > > > > generate
> > > > > >>> > > > >> > > > > > > > >> > > > non-colliding
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > identifiers
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> without
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > >>> coordination.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 2.
> > > When
> > > > > the
> > > > > >>> > > > identifier
> > > > > >>> > > > >> > > needs
> > > > > >>> > > > >> > > > to
> > > > > >>> > > > >> > > > > > be
> > > > > >>> > > > >> > > > > > > > >> > > > “universally
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > unique”;
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > I.e.,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > identifier
> > > > > >>> needs
> > > > > >>> > > to
> > > > > >>> > > > >> > > > distinguish
> > > > > >>> > > > >> > > > > > the
> > > > > >>> > > > >> > > > > > > > >> entity
> > > > > >>> > > > >> > > > > > > > >> > > > from
> > > > > >>> > > > >> > > > > > > > >> > > > > > all
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > other
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> entities
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > that
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> could
> > > > ever
> > > > > >>> > exist.
> > > > > >>> > > > >> This is
> > > > > >>> > > > >> > > > > useful
> > > > > >>> > > > >> > > > > > in
> > > > > >>> > > > >> > > > > > > > >> cases
> > > > > >>> > > > >> > > > > > > > >> > > > where
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > entities
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > from
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > all
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > kinds
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > of
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > systems
> > > > > get
> > > > > >>> > mixed
> > > > > >>> > > > >> > together,
> > > > > >>> > > > >> > > > > such
> > > > > >>> > > > >> > > > > > as
> > > > > >>> > > > >> > > > > > > > >> when
> > > > > >>> > > > >> > > > > > > > >> > > > dumping
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> logs
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > from
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > all
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> processes
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > in
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a
> > > > company
> > > > > >>> into a
> > > > > >>> > > > >> common
> > > > > >>> > > > >> > > > system.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> Maybe
> > > > I’m
> > > > > >>> being
> > > > > >>> > > > >> > > > short-sighted,
> > > > > >>> > > > >> > > > > > but
> > > > > >>> > > > >> > > > > > > it
> > > > > >>> > > > >> > > > > > > > >> > > doesn’t
> > > > > >>> > > > >> > > > > > > > >> > > > > seem
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> like
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > either
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> really
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > applies
> > > > > >>> here. It
> > > > > >>> > > > seems
> > > > > >>> > > > >> > like
> > > > > >>> > > > >> > > > the
> > > > > >>> > > > >> > > > > > > > brokers
> > > > > >>> > > > >> > > > > > > > >> > > could
> > > > > >>> > > > >> > > > > > > > >> > > > > and
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> would
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > achieve
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> consensus
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> when
> > > > > >>> creating a
> > > > > >>> > > > topic
> > > > > >>> > > > >> > > anyway,
> > > > > >>> > > > >> > > > > > which
> > > > > >>> > > > >> > > > > > > > is
> > > > > >>> > > > >> > > > > > > > >> all
> > > > > >>> > > > >> > > > > > > > >> > > > > that’s
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > required
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > to
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > generate
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > >>> non-colliding
> > > > > >>> > > > sequence
> > > > > >>> > > > >> > ids.
> > > > > >>> > > > >> > > > For
> > > > > >>> > > > >> > > > > > the
> > > > > >>> > > > >> > > > > > > > >> > second,
> > > > > >>> > > > >> > > > > > > > >> > > as
> > > > > >>> > > > >> > > > > > > > >> > > > > you
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > mention,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > we
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> could
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> assign
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a
> UUID
> > > > to
> > > > > >>> the
> > > > > >>> > > > cluster
> > > > > >>> > > > >> as
> > > > > >>> > > > >> > a
> > > > > >>> > > > >> > > > > whole,
> > > > > >>> > > > >> > > > > > > > which
> > > > > >>> > > > >> > > > > > > > >> > > would
> > > > > >>> > > > >> > > > > > > > >> > > > > > render
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > any
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > resource
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > scoped
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> the
> > > > broker
> > > > > >>> > > > universally
> > > > > >>> > > > >> > > unique
> > > > > >>> > > > >> > > > > as
> > > > > >>> > > > >> > > > > > > > well.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> The
> > > > > reason I
> > > > > >>> > > mention
> > > > > >>> > > > >> this
> > > > > >>> > > > >> > > is
> > > > > >>> > > > >> > > > > > that,
> > > > > >>> > > > >> > > > > > > > >> > although
> > > > > >>> > > > >> > > > > > > > >> > > a
> > > > > >>> > > > >> > > > > > > > >> > > > > UUID
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> is
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > way
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > more
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > compact
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> than
> > > > topic
> > > > > >>> > names,
> > > > > >>> > > > it’s
> > > > > >>> > > > >> > > still
> > > > > >>> > > > >> > > > 16
> > > > > >>> > > > >> > > > > > > > bytes.
> > > > > >>> > > > >> > > > > > > > >> In
> > > > > >>> > > > >> > > > > > > > >> > > > > > contrast,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> a
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > 4-byte
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> integer
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > sequence
> > > > > id
> > > > > >>> > would
> > > > > >>> > > > give
> > > > > >>> > > > >> > us 4
> > > > > >>> > > > >> > > > > > billion
> > > > > >>> > > > >> > > > > > > > >> unique
> > > > > >>> > > > >> > > > > > > > >> > > > > topics
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> per
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > cluster,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> which
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> seems
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> like
> > > > > enough
> > > > > >>> ;)
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > Considering
> > > > > >>> the
> > > > > >>> > > > >> number of
> > > > > >>> > > > >> > > > > > different
> > > > > >>> > > > >> > > > > > > > >> times
> > > > > >>> > > > >> > > > > > > > >> > > > these
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> topic
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> identifiers
> > > are
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > sent
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> over
> > > the
> > > > > >>> wire or
> > > > > >>> > > > >> stored
> > > > > >>> > > > >> > in
> > > > > >>> > > > >> > > > > > memory,
> > > > > >>> > > > >> > > > > > > it
> > > > > >>> > > > >> > > > > > > > >> > seems
> > > > > >>> > > > >> > > > > > > > >> > > > like
> > > > > >>> > > > >> > > > > > > > >> > > > > > it
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > might
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > be
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> worth
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > additional
> > > > > >>> 4x
> > > > > >>> > > space
> > > > > >>> > > > >> > > savings.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> What
> > > do
> > > > > you
> > > > > >>> > think
> > > > > >>> > > > >> about
> > > > > >>> > > > >> > > this?
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > Thanks,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> John
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > On
> > > Fri,
> > > > > Sep
> > > > > >>> 11,
> > > > > >>> > > > 2020,
> > > > > >>> > > > >> at
> > > > > >>> > > > >> > > > 03:20,
> > > > > >>> > > > >> > > > > > Tom
> > > > > >>> > > > >> > > > > > > > >> > Bentley
> > > > > >>> > > > >> > > > > > > > >> > > > > wrote:
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> Hi
> > > > > >>> Justine,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> This
> > > > > looks
> > > > > >>> > like
> > > > > >>> > > a
> > > > > >>> > > > >> very
> > > > > >>> > > > >> > > > > welcome
> > > > > >>> > > > >> > > > > > > > >> > > improvement.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> Thanks!
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > Maybe
> > > > I
> > > > > >>> missed
> > > > > >>> > > it,
> > > > > >>> > > > >> but
> > > > > >>> > > > >> > > the
> > > > > >>> > > > >> > > > > KIP
> > > > > >>> > > > >> > > > > > > > >> doesn't
> > > > > >>> > > > >> > > > > > > > >> > > seem
> > > > > >>> > > > >> > > > > > > > >> > > > to
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > mention
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > changing
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > > >>> > > > DeleteTopicsRequest
> > > > > >>> > > > >> to
> > > > > >>> > > > >> > > > > identify
> > > > > >>> > > > >> > > > > > > the
> > > > > >>> > > > >> > > > > > > > >> > topic
> > > > > >>> > > > >> > > > > > > > >> > > > > using
> > > > > >>> > > > >> > > > > > > > >> > > > > > an
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > id.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Maybe
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > that's
> out
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > of
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > scope,
> > > > > but
> > > > > >>> > > > >> > > > > DeleteTopicsRequest
> > > > > >>> > > > >> > > > > > is
> > > > > >>> > > > >> > > > > > > > not
> > > > > >>> > > > >> > > > > > > > >> > > listed
> > > > > >>> > > > >> > > > > > > > >> > > > > > among
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Future
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Work
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > APIs
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > either.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> Kind
> > > > > >>> regards,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> Tom
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> On
> > > > Thu,
> > > > > >>> Sep
> > > > > >>> > 10,
> > > > > >>> > > > >> 2020 at
> > > > > >>> > > > >> > > > 3:59
> > > > > >>> > > > >> > > > > PM
> > > > > >>> > > > >> > > > > > > > >> Satish
> > > > > >>> > > > >> > > > > > > > >> > > > > Duggana <
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > >>> > > > >> satish.duggana@gmail.com
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > wrote:
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > Thanks
> > > > > >>> > > > >> Lucas/Justine
> > > > > >>> > > > >> > > for
> > > > > >>> > > > >> > > > > the
> > > > > >>> > > > >> > > > > > > nice
> > > > > >>> > > > >> > > > > > > > >> KIP.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > It
> > > > has
> > > > > >>> > several
> > > > > >>> > > > >> > benefits
> > > > > >>> > > > >> > > > > which
> > > > > >>> > > > >> > > > > > > > also
> > > > > >>> > > > >> > > > > > > > >> > > include
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > simplifying
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> topic
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > > deletion
> > > > > >>> > > process
> > > > > >>> > > > >> by
> > > > > >>> > > > >> > > > > > controller
> > > > > >>> > > > >> > > > > > > > and
> > > > > >>> > > > >> > > > > > > > >> > logs
> > > > > >>> > > > >> > > > > > > > >> > > > > > cleanup
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> by
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > brokers
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > in
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > corner
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > cases.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > Best,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > > Satish.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > On
> > > > > Wed,
> > > > > >>> Sep
> > > > > >>> > 9,
> > > > > >>> > > > >> 2020
> > > > > >>> > > > >> > at
> > > > > >>> > > > >> > > > > 10:07
> > > > > >>> > > > >> > > > > > PM
> > > > > >>> > > > >> > > > > > > > >> > Justine
> > > > > >>> > > > >> > > > > > > > >> > > > > > Olshan <
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > >>> jolshan@confluent.io
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > wrote:
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > > > Hello
> > > > > >>> all,
> > > > > >>> > > > it's
> > > > > >>> > > > >> > been
> > > > > >>> > > > >> > > > > > almost a
> > > > > >>> > > > >> > > > > > > > >> year!
> > > > > >>> > > > >> > > > > > > > >> > > I've
> > > > > >>> > > > >> > > > > > > > >> > > > > > made
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > some
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > changes
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> to
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > this
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> KIP
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > and
> > > > > >>> hope to
> > > > > >>> > > > >> continue
> > > > > >>> > > > >> > > the
> > > > > >>> > > > >> > > > > > > > >> discussion.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > > One
> > > > > >>> of the
> > > > > >>> > > > main
> > > > > >>> > > > >> > > changes
> > > > > >>> > > > >> > > > > > I've
> > > > > >>> > > > >> > > > > > > > >> added
> > > > > >>> > > > >> > > > > > > > >> > is
> > > > > >>> > > > >> > > > > > > > >> > > > now
> > > > > >>> > > > >> > > > > > > > >> > > > > > the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > metadata
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> response
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > will
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > > include
> > > > > >>> the
> > > > > >>> > > > topic
> > > > > >>> > > > >> ID
> > > > > >>> > > > >> > > (as
> > > > > >>> > > > >> > > > > > Colin
> > > > > >>> > > > >> > > > > > > > >> > > suggested).
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> Clients
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > can
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> obtain the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > topicID
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > of a
> > > > > >>> given
> > > > > >>> > > topic
> > > > > >>> > > > >> > > through
> > > > > >>> > > > >> > > > a
> > > > > >>> > > > >> > > > > > > > >> > > > TopicDescription.
> > > > > >>> > > > >> > > > > > > > >> > > > > > The
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > topicId
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > will
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > also be
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > > included
> > > > > >>> > with
> > > > > >>> > > > the
> > > > > >>> > > > >> > > > > > > UpdateMetadata
> > > > > >>> > > > >> > > > > > > > >> > > request.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > > Let
> > > > > me
> > > > > >>> > know
> > > > > >>> > > > what
> > > > > >>> > > > >> > you
> > > > > >>> > > > >> > > > all
> > > > > >>> > > > >> > > > > > > think.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > > > Thank
> > > > > >>> you,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > > > >>> Justine
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > On
> > > > > >>> > > 2019/09/13
> > > > > >>> > > > >> > > 16:38:26,
> > > > > >>> > > > >> > > > > > > "Colin
> > > > > >>> > > > >> > > > > > > > >> > > McCabe" <
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > cmccabe@apache.org
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > wrote:
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > Hi
> > > > > >>> > Lucas,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > > >>> Thanks
> > > > > >>> > for
> > > > > >>> > > > >> > tackling
> > > > > >>> > > > >> > > > > this.
> > > > > >>> > > > >> > > > > > > > >> Topic
> > > > > >>> > > > >> > > > > > > > >> > IDs
> > > > > >>> > > > >> > > > > > > > >> > > > > are a
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > great
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > idea,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> and
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > this
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > is
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > really
> > > > > >>> good
> > > > > >>> > > > >> writeup.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > > For
> > > > > >>> > > > >> > > > > > > /brokers/topics/[topic],
> > > > > >>> > > > >> > > > > > > > >> the
> > > > > >>> > > > >> > > > > > > > >> > > > schema
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> version
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > should
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > be
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > bumped
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > > version
> > > > > >>> 3,
> > > > > >>> > > > rather
> > > > > >>> > > > >> > than
> > > > > >>> > > > >> > > 2.
> > > > > >>> > > > >> > > > > > > > KIP-455
> > > > > >>> > > > >> > > > > > > > >> > > bumped
> > > > > >>> > > > >> > > > > > > > >> > > > > the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > version
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > this
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > znode
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 2
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > > already
> > > > > >>> :)
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > > >>> Given
> > > > > >>> > that
> > > > > >>> > > > >> we're
> > > > > >>> > > > >> > > > going
> > > > > >>> > > > >> > > > > to
> > > > > >>> > > > >> > > > > > > be
> > > > > >>> > > > >> > > > > > > > >> > seeing
> > > > > >>> > > > >> > > > > > > > >> > > > > these
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > things
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > as
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> strings
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > as
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > lot
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> (in
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > logs,
> > > > > in
> > > > > >>> > > > >> ZooKeeper,
> > > > > >>> > > > >> > on
> > > > > >>> > > > >> > > > the
> > > > > >>> > > > >> > > > > > > > >> > command-line,
> > > > > >>> > > > >> > > > > > > > >> > > > > > etc.),
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > does
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > it
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > make
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > sense to
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> use
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > base64
> > > > > >>> when
> > > > > >>> > > > >> > converting
> > > > > >>> > > > >> > > > them
> > > > > >>> > > > >> > > > > > to
> > > > > >>> > > > >> > > > > > > > >> > strings?
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > > >>> Here is
> > > > > >>> > an
> > > > > >>> > > > >> > example
> > > > > >>> > > > >> > > of
> > > > > >>> > > > >> > > > > the
> > > > > >>> > > > >> > > > > > > hex
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> representation:
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > > >>> > > > >> > > > > > > > >> >
> 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > > And
> > > > > >>> here
> > > > > >>> > > is
> > > > > >>> > > > an
> > > > > >>> > > > >> > > > example
> > > > > >>> > > > >> > > > > in
> > > > > >>> > > > >> > > > > > > > >> base64.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > > >>> > > > >> > > > b8tRS7h4TJ2Vt43Dp85v2A
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > > The
> > > > > >>> > base64
> > > > > >>> > > > >> > version
> > > > > >>> > > > >> > > > > saves
> > > > > >>> > > > >> > > > > > 15
> > > > > >>> > > > >> > > > > > > > >> > letters
> > > > > >>> > > > >> > > > > > > > >> > > > (to
> > > > > >>> > > > >> > > > > > > > >> > > > > be
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > fair,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > 4
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> those
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > were
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > dashes
> > > > > >>> that
> > > > > >>> > we
> > > > > >>> > > > >> could
> > > > > >>> > > > >> > > have
> > > > > >>> > > > >> > > > > > > elided
> > > > > >>> > > > >> > > > > > > > in
> > > > > >>> > > > >> > > > > > > > >> > the
> > > > > >>> > > > >> > > > > > > > >> > > > hex
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> representation.)
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > > >>> Another
> > > > > >>> > > > thing
> > > > > >>> > > > >> to
> > > > > >>> > > > >> > > > > consider
> > > > > >>> > > > >> > > > > > > is
> > > > > >>> > > > >> > > > > > > > >> that
> > > > > >>> > > > >> > > > > > > > >> > we
> > > > > >>> > > > >> > > > > > > > >> > > > > > should
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > specify
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > that
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > > >>> all-zeroes
> > > > > >>> > > UUID
> > > > > >>> > > > is
> > > > > >>> > > > >> > not
> > > > > >>> > > > >> > > a
> > > > > >>> > > > >> > > > > > valid
> > > > > >>> > > > >> > > > > > > > >> topic
> > > > > >>> > > > >> > > > > > > > >> > > UUID.
> > > > > >>> > > > >> > > > > > > > >> > > > > >  We
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > can't
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > use
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> null
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > for
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > this
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > > because
> > > > > >>> we
> > > > > >>> > > can't
> > > > > >>> > > > >> > pass a
> > > > > >>> > > > >> > > > > null
> > > > > >>> > > > >> > > > > > > UUID
> > > > > >>> > > > >> > > > > > > > >> over
> > > > > >>> > > > >> > > > > > > > >> > > the
> > > > > >>> > > > >> > > > > > > > >> > > > > RPC
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > protocol
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> (there
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > is no
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > > special
> > > > > >>> > > pattern
> > > > > >>> > > > >> for
> > > > > >>> > > > >> > > null,
> > > > > >>> > > > >> > > > > nor
> > > > > >>> > > > >> > > > > > > do
> > > > > >>> > > > >> > > > > > > > we
> > > > > >>> > > > >> > > > > > > > >> > want
> > > > > >>> > > > >> > > > > > > > >> > > > to
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> waste
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > space
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> reserving
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > such
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > > >>> pattern.)
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > > >>> Maybe I
> > > > > >>> > > > missed
> > > > > >>> > > > >> > it,
> > > > > >>> > > > >> > > > but
> > > > > >>> > > > >> > > > > > did
> > > > > >>> > > > >> > > > > > > > you
> > > > > >>> > > > >> > > > > > > > >> > > > describe
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > "migration
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > of...
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > existing
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > > topic[s]
> > > > > >>> > > without
> > > > > >>> > > > >> > topic
> > > > > >>> > > > >> > > > IDs"
> > > > > >>> > > > >> > > > > > in
> > > > > >>> > > > >> > > > > > > > >> detail
> > > > > >>> > > > >> > > > > > > > >> > in
> > > > > >>> > > > >> > > > > > > > >> > > > any
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > section?
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > It
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> seems
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > like
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> when
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > the
> > > > > new
> > > > > >>> > > > controller
> > > > > >>> > > > >> > > > becomes
> > > > > >>> > > > >> > > > > > > > active,
> > > > > >>> > > > >> > > > > > > > >> it
> > > > > >>> > > > >> > > > > > > > >> > > > should
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> just
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > generate
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> random
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> UUIDs
> > > > for
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > these,
> > > > > >>> and
> > > > > >>> > > write
> > > > > >>> > > > >> the
> > > > > >>> > > > >> > > > random
> > > > > >>> > > > >> > > > > > > UUIDs
> > > > > >>> > > > >> > > > > > > > >> back
> > > > > >>> > > > >> > > > > > > > >> > > to
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > ZooKeeper.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > It
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> would be
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > good
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > to
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > spell
> > > > > >>> that
> > > > > >>> > > out.
> > > > > >>> > > > >> We
> > > > > >>> > > > >> > > > should
> > > > > >>> > > > >> > > > > > make
> > > > > >>> > > > >> > > > > > > > it
> > > > > >>> > > > >> > > > > > > > >> > clear
> > > > > >>> > > > >> > > > > > > > >> > > > > that
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> this
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > happens
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> regardless
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > of
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > the
> > > > > >>> > > inter-broker
> > > > > >>> > > > >> > > protocol
> > > > > >>> > > > >> > > > > > > version
> > > > > >>> > > > >> > > > > > > > >> > (it's
> > > > > >>> > > > >> > > > > > > > >> > > a
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > compatible
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > change).
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > > >>> > > > >> > > "LeaderAndIsrRequests
> > > > > >>> > > > >> > > > > > > > >> including an
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > is_every_partition
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> flag"
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> seems a
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > bit
> > > > > >>> wordy.
> > > > > >>> > > Can
> > > > > >>> > > > we
> > > > > >>> > > > >> > just
> > > > > >>> > > > >> > > > > call
> > > > > >>> > > > >> > > > > > > > these
> > > > > >>> > > > >> > > > > > > > >> > "full
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > >>> LeaderAndIsrRequests"?
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > Then
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > RPC
> > > > > >>> field
> > > > > >>> > > could
> > > > > >>> > > > be
> > > > > >>> > > > >> > > named
> > > > > >>> > > > >> > > > > > > "full".
> > > > > >>> > > > >> > > > > > > > >> > Also,
> > > > > >>> > > > >> > > > > > > > >> > > it
> > > > > >>> > > > >> > > > > > > > >> > > > > > would
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > probably
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > be
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > better
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> for
> > > the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > RPC
> > > > > >>> field to
> > > > > >>> > > be
> > > > > >>> > > > an
> > > > > >>> > > > >> > enum
> > > > > >>> > > > >> > > > of
> > > > > >>> > > > >> > > > > {
> > > > > >>> > > > >> > > > > > > > >> > > UNSPECIFIED,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > INCREMENTAL,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > FULL
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> }, so
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > that
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > we
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > can
> > > > > >>> cleanly
> > > > > >>> > > > handle
> > > > > >>> > > > >> > old
> > > > > >>> > > > >> > > > > > versions
> > > > > >>> > > > >> > > > > > > > (by
> > > > > >>> > > > >> > > > > > > > >> > > > treating
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> them
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > as
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> UNSPECIFIED)
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > In
> > > > > >>> the
> > > > > >>> > > > >> > > > > > LeaderAndIsrRequest
> > > > > >>> > > > >> > > > > > > > >> > section,
> > > > > >>> > > > >> > > > > > > > >> > > > you
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> write
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > "A
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > final
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > deletion
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> event
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > will
> > > > > be
> > > > > >>> > > > secheduled
> > > > > >>> > > > >> > for
> > > > > >>> > > > >> > > X
> > > > > >>> > > > >> > > > ms
> > > > > >>> > > > >> > > > > > > after
> > > > > >>> > > > >> > > > > > > > >> the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> LeaderAndIsrRequest
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > was
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > first
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > > >>> received..."
> > > > > >>> > > I
> > > > > >>> > > > >> guess
> > > > > >>> > > > >> > > > the X
> > > > > >>> > > > >> > > > > > > was a
> > > > > >>> > > > >> > > > > > > > >> > > > > placeholder
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> that
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > you
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> intended
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > replace
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > before
> > > > > >>> > > posting?
> > > > > >>> > > > :)
> > > > > >>> > > > >> > In
> > > > > >>> > > > >> > > > any
> > > > > >>> > > > >> > > > > > > case,
> > > > > >>> > > > >> > > > > > > > >> this
> > > > > >>> > > > >> > > > > > > > >> > > > seems
> > > > > >>> > > > >> > > > > > > > >> > > > > > like
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > kind
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > of
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > thing
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > we'd
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > want a
> > > > > >>> > > > >> configuration
> > > > > >>> > > > >> > > for.
> > > > > >>> > > > >> > > > > > > Let's
> > > > > >>> > > > >> > > > > > > > >> > > describe
> > > > > >>> > > > >> > > > > > > > >> > > > > that
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > configuration
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> key
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > somewhere
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > in
> > > > > this
> > > > > >>> KIP,
> > > > > >>> > > > >> > including
> > > > > >>> > > > >> > > > what
> > > > > >>> > > > >> > > > > > its
> > > > > >>> > > > >> > > > > > > > >> > default
> > > > > >>> > > > >> > > > > > > > >> > > > > value
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> is.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > We
> > > > > >>> > should
> > > > > >>> > > > >> > probably
> > > > > >>> > > > >> > > > also
> > > > > >>> > > > >> > > > > > > log a
> > > > > >>> > > > >> > > > > > > > >> > bunch
> > > > > >>> > > > >> > > > > > > > >> > > of
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> messages
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > at
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > WARN
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> level
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > when
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > > >>> something is
> > > > > >>> > > > >> > scheduled
> > > > > >>> > > > >> > > > for
> > > > > >>> > > > >> > > > > > > > >> deletion,
> > > > > >>> > > > >> > > > > > > > >> > as
> > > > > >>> > > > >> > > > > > > > >> > > > > well.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > (Maybe
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > this
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> was
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > assumed,
> > > > > but
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > it
> > > > > >>> would be
> > > > > >>> > > good
> > > > > >>> > > > >> to
> > > > > >>> > > > >> > > > mention
> > > > > >>> > > > >> > > > > > > it).
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > I
> > > > > >>> feel
> > > > > >>> > > like
> > > > > >>> > > > >> there
> > > > > >>> > > > >> > > > are a
> > > > > >>> > > > >> > > > > > few
> > > > > >>> > > > >> > > > > > > > >> > sections
> > > > > >>> > > > >> > > > > > > > >> > > > > that
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > should
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > be
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> moved to
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > "rejected
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > > >>> > alternatives."
> > > > > >>> > > > >> For
> > > > > >>> > > > >> > > > > example,
> > > > > >>> > > > >> > > > > > in
> > > > > >>> > > > >> > > > > > > > the
> > > > > >>> > > > >> > > > > > > > >> > > > > > DeleteTopics
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > section,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> since
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > we're
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> not
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > going
> > > > > >>> to do
> > > > > >>> > > > >> option 1
> > > > > >>> > > > >> > or
> > > > > >>> > > > >> > > > 2,
> > > > > >>> > > > >> > > > > > > these
> > > > > >>> > > > >> > > > > > > > >> > should
> > > > > >>> > > > >> > > > > > > > >> > > be
> > > > > >>> > > > >> > > > > > > > >> > > > > > moved
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > into
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> "rejected
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > > >>> > alternatives,"
> > > > > >>> > > > >> > rather
> > > > > >>> > > > >> > > > than
> > > > > >>> > > > >> > > > > > > > >> appearing
> > > > > >>> > > > >> > > > > > > > >> > > > > inline.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Another
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > case
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> is
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > "Should
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > we
> > > > > >>> remove
> > > > > >>> > > topic
> > > > > >>> > > > >> name
> > > > > >>> > > > >> > > from
> > > > > >>> > > > >> > > > > the
> > > > > >>> > > > >> > > > > > > > >> protocol
> > > > > >>> > > > >> > > > > > > > >> > > > where
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > possible"
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> section.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > This
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > is
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > > clearly
> > > > > >>> > > > >> discussing a
> > > > > >>> > > > >> > > > design
> > > > > >>> > > > >> > > > > > > > >> > alternative
> > > > > >>> > > > >> > > > > > > > >> > > > that
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> we're
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > not
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> proposing
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > > >>> implement:
> > > > > >>> > > > >> removing
> > > > > >>> > > > >> > the
> > > > > >>> > > > >> > > > > topic
> > > > > >>> > > > >> > > > > > > > name
> > > > > >>> > > > >> > > > > > > > >> > from
> > > > > >>> > > > >> > > > > > > > >> > > > > those
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > protocols.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > Is
> > > > > >>> it
> > > > > >>> > > really
> > > > > >>> > > > >> > > > necessary
> > > > > >>> > > > >> > > > > to
> > > > > >>> > > > >> > > > > > > > have
> > > > > >>> > > > >> > > > > > > > >> a
> > > > > >>> > > > >> > > > > > > > >> > new
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > >>> > > /admin/delete_topics_by_id
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> path
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > in
> > > > > >>> > ZooKeeper?
> > > > > >>> > > > It
> > > > > >>> > > > >> > seems
> > > > > >>> > > > >> > > > > like
> > > > > >>> > > > >> > > > > > we
> > > > > >>> > > > >> > > > > > > > >> don't
> > > > > >>> > > > >> > > > > > > > >> > > > really
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> need
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > this.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Whenever
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> there
> > > is
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > a
> > > > new
> > > > > >>> > > > controller,
> > > > > >>> > > > >> > we'll
> > > > > >>> > > > >> > > > > send
> > > > > >>> > > > >> > > > > > > out
> > > > > >>> > > > >> > > > > > > > >> full
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> LeaderAndIsrRequests
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> which
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > will
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > > trigger
> > > > > >>> the
> > > > > >>> > > > stale
> > > > > >>> > > > >> > > topics
> > > > > >>> > > > >> > > > to
> > > > > >>> > > > >> > > > > > be
> > > > > >>> > > > >> > > > > > > > >> cleaned
> > > > > >>> > > > >> > > > > > > > >> > > up.
> > > > > >>> > > > >> > > > > > > > >> > > > > >  The
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > active
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> controller
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > will
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > also
> > > > > >>> send
> > > > > >>> > the
> > > > > >>> > > > full
> > > > > >>> > > > >> > > > > > > > >> LeaderAndIsrRequest
> > > > > >>> > > > >> > > > > > > > >> > > to
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> brokers
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > are
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> just
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > starting
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > up.
> > > > > >>> So we
> > > > > >>> > > > don't
> > > > > >>> > > > >> > > really
> > > > > >>> > > > >> > > > > > need
> > > > > >>> > > > >> > > > > > > > this
> > > > > >>> > > > >> > > > > > > > >> > kind
> > > > > >>> > > > >> > > > > > > > >> > > > of
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > two-phase
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > commit
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > (send
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > out
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > > >>> > > > >> StopReplicasRequest,
> > > > > >>> > > > >> > > get
> > > > > >>> > > > >> > > > > ACKs
> > > > > >>> > > > >> > > > > > > > from
> > > > > >>> > > > >> > > > > > > > >> all
> > > > > >>> > > > >> > > > > > > > >> > > > > nodes,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > commit
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > by
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> removing
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > > >>> > > > >> /admin/delete_topics
> > > > > >>> > > > >> > > > node)
> > > > > >>> > > > >> > > > > > any
> > > > > >>> > > > >> > > > > > > > >> more.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > > You
> > > > > >>> > > mention
> > > > > >>> > > > >> that
> > > > > >>> > > > >> > > > > > > FetchRequest
> > > > > >>> > > > >> > > > > > > > >> will
> > > > > >>> > > > >> > > > > > > > >> > > now
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> include
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > UUID
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> avoid
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> issues
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > where
> > > > > >>> > requests
> > > > > >>> > > > are
> > > > > >>> > > > >> > made
> > > > > >>> > > > >> > > > to
> > > > > >>> > > > >> > > > > > > stale
> > > > > >>> > > > >> > > > > > > > >> > > > partitions.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > However,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> adding a
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > UUID
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > > >>> > > MetadataRequest
> > > > > >>> > > > is
> > > > > >>> > > > >> > > listed
> > > > > >>> > > > >> > > > > as
> > > > > >>> > > > >> > > > > > > > future
> > > > > >>> > > > >> > > > > > > > >> > > work,
> > > > > >>> > > > >> > > > > > > > >> > > > > out
> > > > > >>> > > > >> > > > > > > > >> > > > > > of
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > scope
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > for
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> this
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > KIP.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> How
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > will
> > > > > the
> > > > > >>> > > client
> > > > > >>> > > > >> learn
> > > > > >>> > > > >> > > > what
> > > > > >>> > > > >> > > > > > the
> > > > > >>> > > > >> > > > > > > > >> topic
> > > > > >>> > > > >> > > > > > > > >> > > UUID
> > > > > >>> > > > >> > > > > > > > >> > > > > is,
> > > > > >>> > > > >> > > > > > > > >> > > > > > if
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > metadata
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> response
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > > doesn't
> > > > > >>> > > include
> > > > > >>> > > > >> that
> > > > > >>> > > > >> > > > > > > information?
> > > > > >>> > > > >> > > > > > > > >> It
> > > > > >>> > > > >> > > > > > > > >> > > > seems
> > > > > >>> > > > >> > > > > > > > >> > > > > > like
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > adding
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> UUID
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > > >>> > > MetadataResponse
> > > > > >>> > > > >> > would
> > > > > >>> > > > >> > > be
> > > > > >>> > > > >> > > > > an
> > > > > >>> > > > >> > > > > > > > >> > improvement
> > > > > >>> > > > >> > > > > > > > >> > > > > here
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> that
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > might
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > not
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> be
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > too
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> hard
> > > to
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > make.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > > >>> best,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > > >>> Colin
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > On
> > > > > >>> Mon,
> > > > > >>> > > Sep
> > > > > >>> > > > 9,
> > > > > >>> > > > >> > > 2019,
> > > > > >>> > > > >> > > > at
> > > > > >>> > > > >> > > > > > > > 17:48,
> > > > > >>> > > > >> > > > > > > > >> > > Ryanne
> > > > > >>> > > > >> > > > > > > > >> > > > > > Dolan
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > wrote:
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > >
> > > > > >>> Lucas,
> > > > > >>> > > > this
> > > > > >>> > > > >> > would
> > > > > >>> > > > >> > > > be
> > > > > >>> > > > >> > > > > > > great.
> > > > > >>> > > > >> > > > > > > > >> I've
> > > > > >>> > > > >> > > > > > > > >> > > run
> > > > > >>> > > > >> > > > > > > > >> > > > > > into
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > issues
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > with
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > topics
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> being
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > >
> > > > > >>> > > > resurrected
> > > > > >>> > > > >> > > > > > accidentally,
> > > > > >>> > > > >> > > > > > > > >> since
> > > > > >>> > > > >> > > > > > > > >> > a
> > > > > >>> > > > >> > > > > > > > >> > > > > client
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > cannot
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > easily
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > distinguish
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > > between
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > >
> > > > > a
> > > > > >>> > > deleted
> > > > > >>> > > > >> topic
> > > > > >>> > > > >> > > > and a
> > > > > >>> > > > >> > > > > > new
> > > > > >>> > > > >> > > > > > > > >> topic
> > > > > >>> > > > >> > > > > > > > >> > > with
> > > > > >>> > > > >> > > > > > > > >> > > > > the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> same
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > name.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I'd
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > need
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > ID
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > >
> > > > > >>> > > accessible
> > > > > >>> > > > >> from
> > > > > >>> > > > >> > > the
> > > > > >>> > > > >> > > > > > > client
> > > > > >>> > > > >> > > > > > > > to
> > > > > >>> > > > >> > > > > > > > >> > > solve
> > > > > >>> > > > >> > > > > > > > >> > > > > that
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > issue,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > but
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> this
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > is a
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> good
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > first
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > >
> > > > > >>> step.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > >
> > > > > >>> Ryanne
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > >
> > > > > On
> > > > > >>> > Wed,
> > > > > >>> > > > Sep
> > > > > >>> > > > >> 4,
> > > > > >>> > > > >> > > 2019
> > > > > >>> > > > >> > > > > at
> > > > > >>> > > > >> > > > > > > 1:41
> > > > > >>> > > > >> > > > > > > > >> PM
> > > > > >>> > > > >> > > > > > > > >> > > Lucas
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Bradstreet <
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > >>> > > lucas@confluent.io>
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > wrote:
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > >
> > > > > >
> > > > > >>> Hi
> > > > > >>> > > all,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > >
> > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > >
> > > > > >
> > > > > >>> I
> > > > > >>> > > would
> > > > > >>> > > > >> like
> > > > > >>> > > > >> > to
> > > > > >>> > > > >> > > > > kick
> > > > > >>> > > > >> > > > > > > off
> > > > > >>> > > > >> > > > > > > > >> > > > discussion
> > > > > >>> > > > >> > > > > > > > >> > > > > of
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > KIP-516,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > an
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > >>> implementation
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > of
> > > > > topic
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > >
> > > > > >
> > > > > >>> IDs
> > > > > >>> > > for
> > > > > >>> > > > >> > Kafka.
> > > > > >>> > > > >> > > > > Topic
> > > > > >>> > > > >> > > > > > > IDs
> > > > > >>> > > > >> > > > > > > > >> aim
> > > > > >>> > > > >> > > > > > > > >> > to
> > > > > >>> > > > >> > > > > > > > >> > > > > solve
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> topic
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> uniqueness
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > problems
> > > > > in
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > Kafka,
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > >
> > > > > >
> > > > > >>> > where
> > > > > >>> > > > >> > referring
> > > > > >>> > > > >> > > > to
> > > > > >>> > > > >> > > > > a
> > > > > >>> > > > >> > > > > > > > topic
> > > > > >>> > > > >> > > > > > > > >> by
> > > > > >>> > > > >> > > > > > > > >> > > name
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> alone is
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> insufficient.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Such
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > cases
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > >
> > > > > >
> > > > > >>> > > include
> > > > > >>> > > > >> when
> > > > > >>> > > > >> > a
> > > > > >>> > > > >> > > > > topic
> > > > > >>> > > > >> > > > > > > has
> > > > > >>> > > > >> > > > > > > > >> been
> > > > > >>> > > > >> > > > > > > > >> > > > > deleted
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > recreated
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> with
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > the
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> same
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > name.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > >
> > > > > >
> > > > > >>> > Unique
> > > > > >>> > > > >> > > > identifiers
> > > > > >>> > > > >> > > > > > will
> > > > > >>> > > > >> > > > > > > > >> help
> > > > > >>> > > > >> > > > > > > > >> > > > > simplify
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > improve
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Kafka's
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> topic
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > > deletion
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > >
> > > > > >
> > > > > >>> > > process,
> > > > > >>> > > > >> as
> > > > > >>> > > > >> > > well
> > > > > >>> > > > >> > > > as
> > > > > >>> > > > >> > > > > > > > prevent
> > > > > >>> > > > >> > > > > > > > >> > > cases
> > > > > >>> > > > >> > > > > > > > >> > > > > > where
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > brokers
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > may
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > incorrectly
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >
> > > > > interact
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > >
> > > > > >
> > > > > >>> with
> > > > > >>> > > > stale
> > > > > >>> > > > >> > > > versions
> > > > > >>> > > > >> > > > > > of
> > > > > >>> > > > >> > > > > > > > >> topics.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > >
> > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > >
> > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > >
> > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > >>> > > > >> > > > > > > > >> > >
> > > > > >>> > > > >> > > > > > > > >> >
> > > > > >>> > > > >> > > > > > > > >>
> > > > > >>> > > > >> > > > > > > >
> > > > > >>> > > > >> > > > > > >
> > > > > >>> > > > >> > > > > >
> > > > > >>> > > > >> > > > >
> > > > > >>> > > > >> > > >
> > > > > >>> > > > >> > >
> > > > > >>> > > > >> >
> > > > > >>> > > > >>
> > > > > >>> > > >
> > > > > >>> > >
> > > > > >>> >
> > > > > >>>
> > > > >
> > > >
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > >
> > > > > >
> > > > > >>> > > Looking
> > > > > >>> > > > >> > forward
> > > > > >>> > > > >> > > > to
> > > > > >>> > > > >> > > > > > your
> > > > > >>> > > > >> > > > > > > > >> > > thoughts.
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > >
> > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > >
> > > > > >
> > > > > >>> > Lucas
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >
> > > >
> > > > >
> > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > > >>> > > > >> > > > > > > > >> > > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > > >>> > > > >> > > > > > > > >> > > > >
> > > > > >>> > > > >> > > > > > > > >> > > >
> > > > > >>> > > > >> > > > > > > > >> > >
> > > > > >>> > > > >> > > > > > > > >> >
> > > > > >>> > > > >> > > > > > > > >>
> > > > > >>> > > > >> > > > > > > > >
> > > > > >>> > > > >> > > > > > > >
> > > > > >>> > > > >> > > > > > >
> > > > > >>> > > > >> > > > > >
> > > > > >>> > > > >> > > > >
> > > > > >>> > > > >> > > >
> > > > > >>> > > > >> > >
> > > > > >>> > > > >> >
> > > > > >>> > > > >>
> > > > > >>> > > > >
> > > > > >>> > > >
> > > > > >>> > >
> > > > > >>> >
> > > > > >>>
> > > > > >>
> > > > >
> > > >
> > >
>
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Satish Duggana <sa...@gmail.com>]

Hi Jun, Justine,
+1 to have TopicIdPartition in public packages. This is useful for
KIP-405 tiered storage too. We currently use TopicPartition for
pluggable log storage and metadata storage apis. We will be using
TopicIdPartition once the proposed changes in this KIP are merged.

Thanks,
Satish.

On Tue, Oct 6, 2020 at 5:21 AM Jun Rao <ju...@confluent.io> wrote:
>
> Hi, Justine,
>
> I guess we have to maintain the mapping between topic name and topicId in
> both the client and the server. If TopicIdPartition is meant for internal
> usage only, we can create it under a package
> like org/apache/kafka/common/internal and we don't need to document it in
> the KIP. If it's meant for external public usage (and potentially internal
> usage too), we can create it in org/apache/kafka/common and need to
> document it in the KIP.
>
> Since topicId is now part of the topic metadata, perhaps it's useful to
> expose it in TopicMetadataAndConfig too.
>
> Thanks,
>
> Jun
>
> On Mon, Oct 5, 2020 at 3:35 PM Justine Olshan <jo...@confluent.io> wrote:
>
> > Hi Jun,
> >
> > This KIP removes the topic name from many requests. A lot of requests keep
> > track of the partitions internally with a TopicPartition. So when we remove
> > the topic name, we can not differentiate unless we can also use topic ID.
> > The Fetch request relies heavily on this, but ListOffset,
> > OffsetsForLeaderEpoch, etc all use TopicPartition as a key value for
> > internal maps. Digging deeper into Fetch, I've noticed that the code path
> > runs into some public APIs that use the TopicPartition. Some examples are
> > Cluster.java or RecordTooLargeException.
> >
> > A related issue is that in some places like TopicAuthorizationException,
> > the topic name is required and without the name in the request, we won't
> > know it.
> >
> > I guess an alternative would be to have some mapping from topic name to
> > topic ID, but I would have to think about that more.
> >
> > The KIP mentions exposing TopicId in TopicDescription. (The font was
> > weirdly colored, so I've made it easier to read.)
> >
> > Would the idea for the TopicMetadataAndConfig be that the topic ID would be
> > returned as a result?
> > As for the other methods describe/deletetopics, it makes sense to include
> > them at some point, but I'm not sure how far we should go in terms of
> > supporting topic IDs everywhere.
> >
> > Thank you for continuing to look at the KIP!
> > Justine
> >
> >
> > On Mon, Oct 5, 2020 at 2:53 PM Jun Rao <ju...@confluent.io> wrote:
> >
> > > Hi, Justine,
> > >
> > > Adding a TopicIdPartition in the public interface may be useful. Where
> > > would you expect it to be used?
> > >
> > > Related to this, since we are exposing topicId in the tool, it seems that
> > > we may want to expose topicId in at least TopicDescription used by
> > > AdminClient.describeTopics() and TopicMetadataAndConfig used in
> > > AdminClient.createTopics(Collection<NewTopic> newTopics). I am also not
> > > sure if we plan to expose topicId in other AdminClient methods in this
> > KIP
> > > or in some future KIP. For example, it might be useful to support topicId
> > > in the following methods.
> > >
> > >     DescribeTopicsResult describeTopics(Collection<String> topicNames,
> > > DescribeTopicsOptions options);
> > >     DeleteTopicsResult deleteTopics(Collection<String> topics,
> > > DeleteTopicsOptions options);
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Mon, Oct 5, 2020 at 11:03 AM Justine Olshan <jo...@confluent.io>
> > > wrote:
> > >
> > > > Hi all,
> > > >
> > > > I was thinking about this KIP some more. We are removing the topic name
> > > > from many of these request types, and some of them (eg. Fetch) use
> > > > TopicPartition as keys in a map when handling the request. I was
> > thinking
> > > > it would make sense to make some sort of TopicIdPartition with topic
> > > name,
> > > > topic id, and partition fields (name and id would potentially be
> > optional
> > > > types since different request versions will contain only one of the
> > two)
> > > I
> > > > was wondering if it made sense to include this as a public facing API
> > > > similar to TopicPartition. If so, I will add this class to the KIP as
> > > well.
> > > >
> > > > Thanks,
> > > > Justine
> > > >
> > > > On Mon, Oct 5, 2020 at 8:52 AM Justine Olshan <jo...@confluent.io>
> > > > wrote:
> > > >
> > > > > Hi all,
> > > > > I've made a few minor changes to the UUID class. I've added a few
> > more
> > > > > methods to support UUIDs more generally, by including a constructor
> > > and a
> > > > > method to generate random UUIDs. Both of these will be useful for
> > > > testing.
> > > > >
> > > > > Let me know if there are any other areas for discussion.
> > > > >
> > > > > Thank you,
> > > > > Justine
> > > > >
> > > > > On Thu, Oct 1, 2020 at 5:29 PM Justine Olshan <jo...@confluent.io>
> > > > > wrote:
> > > > >
> > > > >> Hi Jun,
> > > > >> Thanks for looking at it again. Here's the new spec. (I fixed the
> > typo
> > > > in
> > > > >> it too.)
> > > > >> {"name": "id", "type": "string", "doc": option[UUID]}
> > > > >>
> > > > >> Justine
> > > > >>
> > > > >>
> > > > >> On Thu, Oct 1, 2020 at 5:03 PM Jun Rao <ju...@confluent.io> wrote:
> > > > >>
> > > > >>> Hi, Justine,
> > > > >>>
> > > > >>> Thanks for the update. The KIP looks good to me now. Just a minor
> > > > comment
> > > > >>> below.
> > > > >>>
> > > > >>> 30. Perhaps "option[UUID]" can be put in the doc.
> > > > >>>
> > > > >>> Jun
> > > > >>>
> > > > >>> On Thu, Oct 1, 2020 at 3:28 PM Justine Olshan <
> > jolshan@confluent.io>
> > > > >>> wrote:
> > > > >>>
> > > > >>> > Hi Jun,
> > > > >>> > Thanks for the response!
> > > > >>> >
> > > > >>> > 30. I think I might have changed this in between. The current
> > > version
> > > > >>> > says:  {"name":
> > > > >>> > "id", "type": "option[UUID]"}, "doc": topic id}
> > > > >>> > I have switched to the option type to cover the migration case
> > > where
> > > > a
> > > > >>> > TopicZNode does not yet have a topic ID.
> > > > >>> > I understand that due to json, this field is written as a string,
> > > so
> > > > >>> if I
> > > > >>> > should move the "option[uuid]" to the doc field and the type
> > should
> > > > be
> > > > >>> > "string" please let me know.
> > > > >>> >
> > > > >>> > 40. I've added a definition for UUID.
> > > > >>> > 41,42. Fixed
> > > > >>> >
> > > > >>> > Thank you,
> > > > >>> > Justine
> > > > >>> >
> > > > >>> > On Wed, Sep 30, 2020 at 1:15 PM Jun Rao <ju...@confluent.io>
> > wrote:
> > > > >>> >
> > > > >>> > > Hi, Justine,
> > > > >>> > >
> > > > >>> > > Thanks for the summary. Just a few minor comments blow.
> > > > >>> > >
> > > > >>> > > 30.  {"name": "id", "type": "string", "doc": "version id"}}:
> > The
> > > > doc
> > > > >>> > should
> > > > >>> > > say UUID. The issue seems unfixed.
> > > > >>> > >
> > > > >>> > > 40. Since UUID is public facing, could you include its
> > > definition?
> > > > >>> > >
> > > > >>> > > 41. StopReplicaResponse still includes the topic field.
> > > > >>> > >
> > > > >>> > > 42. "It is unnecessary to include the name of the topic in the
> > > > >>> following
> > > > >>> > > Request/Response calls" It would be useful to include all
> > > modified
> > > > >>> > requests
> > > > >>> > > (e.g. produce) in the list below.
> > > > >>> > >
> > > > >>> > > Thanks,
> > > > >>> > >
> > > > >>> > > Jun
> > > > >>> > >
> > > > >>> > >
> > > > >>> > > On Wed, Sep 30, 2020 at 10:17 AM Justine Olshan <
> > > > >>> jolshan@confluent.io>
> > > > >>> > > wrote:
> > > > >>> > >
> > > > >>> > > > Hello again,
> > > > >>> > > >
> > > > >>> > > > I've taken some time to discuss some of the remaining points
> > > > >>> brought up
> > > > >>> > > by
> > > > >>> > > > the previous emails offline. Here are some of the
> > conclusions.
> > > > >>> > > >
> > > > >>> > > > 1. Directory Structure:
> > > > >>> > > > There was some talk about whether the directory structure
> > > should
> > > > be
> > > > >>> > > changed
> > > > >>> > > > to replace all topic names with topic IDs.
> > > > >>> > > > This will be a useful change, but will prevent downgrades. It
> > > > will
> > > > >>> be
> > > > >>> > > best
> > > > >>> > > > to wait until a major release, likely alongside KIP-500
> > changes
> > > > >>> that
> > > > >>> > will
> > > > >>> > > > prevent downgrades. I've updated the KIP to include this
> > change
> > > > >>> with
> > > > >>> > the
> > > > >>> > > > note about migration and deprecation.
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-log.dirlayout
> > > > >>> > > >
> > > > >>> > > > 2. Partition Metadata file
> > > > >>> > > > There was some disagreement about the inclusion of the
> > > partition
> > > > >>> > metadata
> > > > >>> > > > file.
> > > > >>> > > > This file will be needed to persist the topic ID, especially
> > > > while
> > > > >>> we
> > > > >>> > > still
> > > > >>> > > > have the old directory structure. Even after the changes,
> > this
> > > > >>> file can
> > > > >>> > > be
> > > > >>> > > > useful for debugging and recovery.
> > > > >>> > > > Creating a single mapping file from topic names to topic IDs
> > > was
> > > > >>> > > > considered, but ultimately scrapped as it would not be as
> > easy
> > > to
> > > > >>> > > maintain.
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PartitionMetadatafile
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PersistingTopicIDs
> > > > >>> > > >
> > > > >>> > > > 3. Produce Protocols
> > > > >>> > > > After some further discussion, this replacing the topic name
> > > with
> > > > >>> topic
> > > > >>> > > ID
> > > > >>> > > > in these protocols has been added to the KIP.
> > > > >>> > > > This will cut down on the size of the protocol. Since changes
> > > to
> > > > >>> fetch
> > > > >>> > > are
> > > > >>> > > > included, it does make sense to update these protocols.
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-Produce
> > > > >>> > > >
> > > > >>> > > > 4. KIP-500 Compatibility
> > > > >>> > > > After some discussion with those involved with KIP-500, it
> > > seems
> > > > >>> best
> > > > >>> > to
> > > > >>> > > > use a sentinel topic ID for the metadata topic that is used
> > > > before
> > > > >>> the
> > > > >>> > > > first controller election.
> > > > >>> > > > We had an issue where this metadata topic may not be assigned
> > > an
> > > > ID
> > > > >>> > > before
> > > > >>> > > > utilizing the Vote and Fetch protocols. It was decided to
> > > > reserve a
> > > > >>> > > unique
> > > > >>> > > > ID for this topic to be used until the controller could give
> > > the
> > > > >>> topic
> > > > >>> > a
> > > > >>> > > > unique ID.
> > > > >>> > > > Having the topic keep the sentinel ID (not be reassigned to a
> > > > >>> unique
> > > > >>> > ID)
> > > > >>> > > > was considered, but it seemed like a good idea to leave the
> > > > option
> > > > >>> open
> > > > >>> > > for
> > > > >>> > > > the metadata topic to have a unique ID in cases where it
> > would
> > > > >>> need to
> > > > >>> > be
> > > > >>> > > > differentiated from other clusters' metadata topics. (ex.
> > > tiered
> > > > >>> > storage)
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-CompatibilitywithKIP-500
> > > > >>> > > >
> > > > >>> > > > I've also split up the KIP into sub-tasks on the JIRA.
> > > Hopefully
> > > > >>> this
> > > > >>> > > will
> > > > >>> > > > provide a better idea about what tasks we have, and
> > eventually
> > > > >>> provide
> > > > >>> > a
> > > > >>> > > > place to see what's done and what is left.
> > > > >>> > > > If there is a task I am missing, please let me know!
> > > > >>> > > > https://issues.apache.org/jira/browse/KAFKA-8872
> > > > >>> > > >
> > > > >>> > > > Of course, these decisions are not set in stone, and I would
> > > love
> > > > >>> to
> > > > >>> > hear
> > > > >>> > > > any feedback.
> > > > >>> > > >
> > > > >>> > > > Thanks,
> > > > >>> > > > Justine
> > > > >>> > > >
> > > > >>> > > > On Mon, Sep 28, 2020 at 11:38 AM Justine Olshan <
> > > > >>> jolshan@confluent.io>
> > > > >>> > > > wrote:
> > > > >>> > > >
> > > > >>> > > > > Hello all,
> > > > >>> > > > >
> > > > >>> > > > > I just wanted to follow up on this discussion. Did we come
> > to
> > > > an
> > > > >>> > > > > understanding about the directory structure?
> > > > >>> > > > >
> > > > >>> > > > > I think the biggest question here is what is acceptable to
> > > > leave
> > > > >>> out
> > > > >>> > > due
> > > > >>> > > > > to scope vs. what is considered to be too much tech debt.
> > > > >>> > > > > This KIP is already pretty large with the number of
> > changes,
> > > > but
> > > > >>> it
> > > > >>> > > also
> > > > >>> > > > > makes sense to do things correctly, so I'd love to hear
> > > > >>> everyone's
> > > > >>> > > > thoughts.
> > > > >>> > > > >
> > > > >>> > > > > Thanks,
> > > > >>> > > > > Justine
> > > > >>> > > > >
> > > > >>> > > > > On Fri, Sep 25, 2020 at 8:19 AM Lucas Bradstreet <
> > > > >>> lucas@confluent.io
> > > > >>> > >
> > > > >>> > > > > wrote:
> > > > >>> > > > >
> > > > >>> > > > >> Hi Ismael,
> > > > >>> > > > >>
> > > > >>> > > > >> If you do not store it in a metadata file or in the
> > > directory
> > > > >>> > > structure
> > > > >>> > > > >> would you then
> > > > >>> > > > >> require the LeaderAndIsrRequest following startup to give
> > > you
> > > > >>> some
> > > > >>> > > > notion
> > > > >>> > > > >> of
> > > > >>> > > > >> topic name in memory? We would still need this information
> > > for
> > > > >>> the
> > > > >>> > > older
> > > > >>> > > > >> protocols, but
> > > > >>> > > > >> perhaps this is what's meant by tech debt.
> > > > >>> > > > >>
> > > > >>> > > > >> Once we're free of the old non-topicID requests then I
> > think
> > > > you
> > > > >>> > > > wouldn't
> > > > >>> > > > >> need to retain the topic name.
> > > > >>> > > > >> I think the ability to easily look up topic names
> > associated
> > > > >>> with
> > > > >>> > > > >> partition
> > > > >>> > > > >> directories would still be missed
> > > > >>> > > > >> when diagnosing issues, though maybe it wouldn't be a deal
> > > > >>> breaker
> > > > >>> > > with
> > > > >>> > > > >> the
> > > > >>> > > > >> right tooling.
> > > > >>> > > > >>
> > > > >>> > > > >> Thanks,
> > > > >>> > > > >>
> > > > >>> > > > >> Lucas
> > > > >>> > > > >>
> > > > >>> > > > >> On Fri, Sep 25, 2020 at 7:55 AM Ismael Juma <
> > > > ismael@juma.me.uk>
> > > > >>> > > wrote:
> > > > >>> > > > >>
> > > > >>> > > > >> > Hi Lucas,
> > > > >>> > > > >> >
> > > > >>> > > > >> > Why would you include the name and id? I think you'd
> > want
> > > to
> > > > >>> > remove
> > > > >>> > > > the
> > > > >>> > > > >> > name from the directory name right? Jason's suggestion
> > was
> > > > >>> that if
> > > > >>> > > you
> > > > >>> > > > >> > remove the name from the directory, then why would you
> > > need
> > > > >>> the id
> > > > >>> > > > name
> > > > >>> > > > >> > mapping file?
> > > > >>> > > > >> >
> > > > >>> > > > >> > Ismael
> > > > >>> > > > >> >
> > > > >>> > > > >> > On Thu, Sep 24, 2020 at 4:24 PM Lucas Bradstreet <
> > > > >>> > > lucas@confluent.io>
> > > > >>> > > > >> > wrote:
> > > > >>> > > > >> >
> > > > >>> > > > >> > > > 2. Part of the usage of the file is to have
> > persistent
> > > > >>> storage
> > > > >>> > > of
> > > > >>> > > > >> the
> > > > >>> > > > >> > > topic
> > > > >>> > > > >> > > ID and use it to compare with the ID supplied in the
> > > > >>> > LeaderAndIsr
> > > > >>> > > > >> > Request.
> > > > >>> > > > >> > > There is some discussion in the KIP about changes to
> > the
> > > > >>> > directory
> > > > >>> > > > >> > > structure, but I believe directory changes were
> > > considered
> > > > >>> to be
> > > > >>> > > out
> > > > >>> > > > >> of
> > > > >>> > > > >> > > scope when the KIP was written.
> > > > >>> > > > >> > >
> > > > >>> > > > >> > >
> > > > >>> > > > >> > > Yeah, I was hoping to get a better understanding of
> > why
> > > it
> > > > >>> was
> > > > >>> > > taken
> > > > >>> > > > >> out
> > > > >>> > > > >> > of
> > > > >>> > > > >> > > scope. Perhaps Lucas Bradstreet might have more
> > insight
> > > > >>> about
> > > > >>> > the
> > > > >>> > > > >> > decision.
> > > > >>> > > > >> > > Basically my point is that we have to create
> > additional
> > > > >>> > > > infrastructure
> > > > >>> > > > >> > here
> > > > >>> > > > >> > > to support the name/id mapping, so I wanted to
> > > understand
> > > > >>> if we
> > > > >>> > > just
> > > > >>> > > > >> > > consider this a sort of tech debt. It would be useful
> > to
> > > > >>> cover
> > > > >>> > how
> > > > >>> > > > we
> > > > >>> > > > >> > > handle the case when this file gets corrupted. Seems
> > > like
> > > > we
> > > > >>> > just
> > > > >>> > > > >> have to
> > > > >>> > > > >> > > trust that it matches whatever the controller tells us
> > > and
> > > > >>> > rewrite
> > > > >>> > > > it?
> > > > >>> > > > >> > >
> > > > >>> > > > >> > >
> > > > >>> > > > >> > > Hi Jason, Justine,
> > > > >>> > > > >> > >
> > > > >>> > > > >> > > My thought process is that we will likely need the
> > > > metadata
> > > > >>> file
> > > > >>> > > > >> whether
> > > > >>> > > > >> > we
> > > > >>> > > > >> > > rename the directories or not.
> > > > >>> > > > >> > > Linux supports filenames of up to 255 bytes and that
> > > would
> > > > >>> not
> > > > >>> > be
> > > > >>> > > > >> enough
> > > > >>> > > > >> > to
> > > > >>> > > > >> > > support a directory name
> > > > >>> > > > >> > >  including both the name and topic ID. Given that
> > fact,
> > > it
> > > > >>> > seemed
> > > > >>> > > > >> > > reasonable to add the metadata file
> > > > >>> > > > >> > > and leave the directory rename until some time in the
> > > > future
> > > > >>> > > > (possibly
> > > > >>> > > > >> > > never).
> > > > >>> > > > >> > >
> > > > >>> > > > >> > > If the file does get corrupted I think you're right.
> > We
> > > > >>> would
> > > > >>> > > either
> > > > >>> > > > >> have
> > > > >>> > > > >> > > to trust it matches what the controller tells us
> > > > >>> > > > >> > >  or error out and let an administrator resolve it by
> > > > >>> checking
> > > > >>> > > across
> > > > >>> > > > >> > > replicas for consistency.
> > > > >>> > > > >> > >
> > > > >>> > > > >> > > Lucas
> > > > >>> > > > >> > >
> > > > >>> > > > >> > >
> > > > >>> > > > >> > > On Thu, Sep 24, 2020 at 3:41 PM Jason Gustafson <
> > > > >>> > > jason@confluent.io
> > > > >>> > > > >
> > > > >>> > > > >> > > wrote:
> > > > >>> > > > >> > >
> > > > >>> > > > >> > > > Thanks Justine. Responses below:
> > > > >>> > > > >> > > >
> > > > >>> > > > >> > > > > 1. Yes, the directory will still be based on the
> > > topic
> > > > >>> > names.
> > > > >>> > > > >> > > > LeaderAndIsrRequest is one of the few requests that
> > > will
> > > > >>> still
> > > > >>> > > > >> contain
> > > > >>> > > > >> > > the
> > > > >>> > > > >> > > > topic name. So I think we have this covered. Sorry
> > for
> > > > >>> > > confusion.
> > > > >>> > > > >> > > >
> > > > >>> > > > >> > > > Ah, you're right. My eyes passed right over the
> > field.
> > > > >>> > > > >> > > >
> > > > >>> > > > >> > > > > 2. Part of the usage of the file is to have
> > > persistent
> > > > >>> > storage
> > > > >>> > > > of
> > > > >>> > > > >> the
> > > > >>> > > > >> > > > topic
> > > > >>> > > > >> > > > ID and use it to compare with the ID supplied in the
> > > > >>> > > LeaderAndIsr
> > > > >>> > > > >> > > Request.
> > > > >>> > > > >> > > > There is some discussion in the KIP about changes to
> > > the
> > > > >>> > > directory
> > > > >>> > > > >> > > > structure, but I believe directory changes were
> > > > >>> considered to
> > > > >>> > be
> > > > >>> > > > >> out of
> > > > >>> > > > >> > > > scope when the KIP was written.
> > > > >>> > > > >> > > >
> > > > >>> > > > >> > > > Yeah, I was hoping to get a better understanding of
> > > why
> > > > >>> it was
> > > > >>> > > > taken
> > > > >>> > > > >> > out
> > > > >>> > > > >> > > of
> > > > >>> > > > >> > > > scope. Perhaps Lucas Bradstreet might have more
> > > insight
> > > > >>> about
> > > > >>> > > the
> > > > >>> > > > >> > > decision.
> > > > >>> > > > >> > > > Basically my point is that we have to create
> > > additional
> > > > >>> > > > >> infrastructure
> > > > >>> > > > >> > > here
> > > > >>> > > > >> > > > to support the name/id mapping, so I wanted to
> > > > understand
> > > > >>> if
> > > > >>> > we
> > > > >>> > > > just
> > > > >>> > > > >> > > > consider this a sort of tech debt. It would be
> > useful
> > > to
> > > > >>> cover
> > > > >>> > > how
> > > > >>> > > > >> we
> > > > >>> > > > >> > > > handle the case when this file gets corrupted. Seems
> > > > like
> > > > >>> we
> > > > >>> > > just
> > > > >>> > > > >> have
> > > > >>> > > > >> > to
> > > > >>> > > > >> > > > trust that it matches whatever the controller tells
> > us
> > > > and
> > > > >>> > > rewrite
> > > > >>> > > > >> it?
> > > > >>> > > > >> > > >
> > > > >>> > > > >> > > > > 3. I think this is a good point, but I again I
> > > wonder
> > > > >>> about
> > > > >>> > > the
> > > > >>> > > > >> scope
> > > > >>> > > > >> > > of
> > > > >>> > > > >> > > > the KIP. Most of the changes mentioned in the KIP
> > are
> > > > for
> > > > >>> > > > supporting
> > > > >>> > > > >> > > topic
> > > > >>> > > > >> > > > deletion and I believe that is why the produce
> > request
> > > > was
> > > > >>> > > listed
> > > > >>> > > > >> under
> > > > >>> > > > >> > > > future work.
> > > > >>> > > > >> > > >
> > > > >>> > > > >> > > > That's fair. I brought it up since `Fetch` is
> > already
> > > > >>> > included.
> > > > >>> > > If
> > > > >>> > > > >> > we've
> > > > >>> > > > >> > > > got `Metadata` and `Fetch`, seems we may as well do
> > > > >>> `Produce`
> > > > >>> > > and
> > > > >>> > > > >> save
> > > > >>> > > > >> > an
> > > > >>> > > > >> > > > extra kip. No strong objection though if you want to
> > > > >>> leave it
> > > > >>> > > out.
> > > > >>> > > > >> > > >
> > > > >>> > > > >> > > >
> > > > >>> > > > >> > > > -Jason
> > > > >>> > > > >> > > >
> > > > >>> > > > >> > > >
> > > > >>> > > > >> > > > On Thu, Sep 24, 2020 at 3:26 PM Justine Olshan <
> > > > >>> > > > >> jolshan@confluent.io>
> > > > >>> > > > >> > > > wrote:
> > > > >>> > > > >> > > >
> > > > >>> > > > >> > > > > Hi Jason,
> > > > >>> > > > >> > > > >
> > > > >>> > > > >> > > > > Thanks for your comments.
> > > > >>> > > > >> > > > >
> > > > >>> > > > >> > > > > 1. Yes, the directory will still be based on the
> > > topic
> > > > >>> > names.
> > > > >>> > > > >> > > > > LeaderAndIsrRequest is one of the few requests
> > that
> > > > will
> > > > >>> > still
> > > > >>> > > > >> > contain
> > > > >>> > > > >> > > > the
> > > > >>> > > > >> > > > > topic name. So I think we have this covered. Sorry
> > > for
> > > > >>> > > > confusion.
> > > > >>> > > > >> > > > >
> > > > >>> > > > >> > > > > 2. Part of the usage of the file is to have
> > > persistent
> > > > >>> > storage
> > > > >>> > > > of
> > > > >>> > > > >> the
> > > > >>> > > > >> > > > topic
> > > > >>> > > > >> > > > > ID and use it to compare with the ID supplied in
> > the
> > > > >>> > > > LeaderAndIsr
> > > > >>> > > > >> > > > Request.
> > > > >>> > > > >> > > > > There is some discussion in the KIP about changes
> > to
> > > > the
> > > > >>> > > > directory
> > > > >>> > > > >> > > > > structure, but I believe directory changes were
> > > > >>> considered
> > > > >>> > to
> > > > >>> > > be
> > > > >>> > > > >> out
> > > > >>> > > > >> > of
> > > > >>> > > > >> > > > > scope when the KIP was written.
> > > > >>> > > > >> > > > >
> > > > >>> > > > >> > > > > 3. I think this is a good point, but I again I
> > > wonder
> > > > >>> about
> > > > >>> > > the
> > > > >>> > > > >> scope
> > > > >>> > > > >> > > of
> > > > >>> > > > >> > > > > the KIP. Most of the changes mentioned in the KIP
> > > are
> > > > >>> for
> > > > >>> > > > >> supporting
> > > > >>> > > > >> > > > topic
> > > > >>> > > > >> > > > > deletion and I believe that is why the produce
> > > request
> > > > >>> was
> > > > >>> > > > listed
> > > > >>> > > > >> > under
> > > > >>> > > > >> > > > > future work.
> > > > >>> > > > >> > > > >
> > > > >>> > > > >> > > > > 4. This sounds like it might be a good solution,
> > > but I
> > > > >>> will
> > > > >>> > > need
> > > > >>> > > > >> to
> > > > >>> > > > >> > > > discuss
> > > > >>> > > > >> > > > > more with KIP-500 folks to get the details right.
> > > > >>> > > > >> > > > >
> > > > >>> > > > >> > > > > Thanks,
> > > > >>> > > > >> > > > > Justine
> > > > >>> > > > >> > > > >
> > > > >>> > > > >> > > > > On Thu, Sep 24, 2020 at 12:30 PM Jason Gustafson <
> > > > >>> > > > >> jason@confluent.io
> > > > >>> > > > >> > >
> > > > >>> > > > >> > > > > wrote:
> > > > >>> > > > >> > > > >
> > > > >>> > > > >> > > > > > Hi Justine,
> > > > >>> > > > >> > > > > >
> > > > >>> > > > >> > > > > > Thanks for picking up this work. I have a few
> > > > >>> > > > >> questions/comments:
> > > > >>> > > > >> > > > > >
> > > > >>> > > > >> > > > > > 1. It sounds like the directory structure is
> > still
> > > > >>> going
> > > > >>> > to
> > > > >>> > > be
> > > > >>> > > > >> > based
> > > > >>> > > > >> > > on
> > > > >>> > > > >> > > > > > topic names. Do I have that right? One
> > > complication
> > > > is
> > > > >>> > that
> > > > >>> > > > the
> > > > >>> > > > >> > > > > > LeaderAndIsr request does not include the topic
> > > name
> > > > >>> any
> > > > >>> > > > longer.
> > > > >>> > > > >> > This
> > > > >>> > > > >> > > > > means
> > > > >>> > > > >> > > > > > that a replica must wait for the UpdateMetadata
> > > > >>> request to
> > > > >>> > > > >> arrive
> > > > >>> > > > >> > > with
> > > > >>> > > > >> > > > > the
> > > > >>> > > > >> > > > > > topic name mapping before it can create the
> > > > >>> directory. I
> > > > >>> > > > wonder
> > > > >>> > > > >> if
> > > > >>> > > > >> > it
> > > > >>> > > > >> > > > > would
> > > > >>> > > > >> > > > > > be simpler to avoid assumptions on the ordering
> > of
> > > > >>> > > > >> UpdateMetadata
> > > > >>> > > > >> > and
> > > > >>> > > > >> > > > let
> > > > >>> > > > >> > > > > > LeaderAndIsr include the topic name as well.
> > Feels
> > > > >>> like we
> > > > >>> > > are
> > > > >>> > > > >> not
> > > > >>> > > > >> > > > saving
> > > > >>> > > > >> > > > > > that much by excluding it.
> > > > >>> > > > >> > > > > >
> > > > >>> > > > >> > > > > > 2. On a related note, it seems that the reason
> > we
> > > > >>> have the
> > > > >>> > > > >> > partition
> > > > >>> > > > >> > > > > > metadata file is because we are not changing the
> > > > >>> directory
> > > > >>> > > > >> > structure.
> > > > >>> > > > >> > > > We
> > > > >>> > > > >> > > > > > need it so that we remember which directories
> > map
> > > to
> > > > >>> which
> > > > >>> > > > topic
> > > > >>> > > > >> > id.
> > > > >>> > > > >> > > I
> > > > >>> > > > >> > > > > > think it would be useful to add some detail
> > about
> > > > why
> > > > >>> > > changing
> > > > >>> > > > >> the
> > > > >>> > > > >> > > > > > directory layout was rejected. Basically trying
> > to
> > > > >>> > > understand
> > > > >>> > > > >> how
> > > > >>> > > > >> > > much
> > > > >>> > > > >> > > > > > effort we are saving by skipping this step if we
> > > > have
> > > > >>> to
> > > > >>> > > > >> implement
> > > > >>> > > > >> > > this
> > > > >>> > > > >> > > > > > additional bookkeeping. One concern I have for
> > > > >>> example is
> > > > >>> > > that
> > > > >>> > > > >> the
> > > > >>> > > > >> > > > > > partition metadata file gets corrupt and then
> > all
> > > > >>> bets are
> > > > >>> > > off
> > > > >>> > > > >> as
> > > > >>> > > > >> > far
> > > > >>> > > > >> > > > as
> > > > >>> > > > >> > > > > > topic consistency.
> > > > >>> > > > >> > > > > >
> > > > >>> > > > >> > > > > > 3. Is it worth adding support now for topic ids
> > in
> > > > the
> > > > >>> > > > `Produce`
> > > > >>> > > > >> > > > request
> > > > >>> > > > >> > > > > > now? Overhead is mentioned as one of the
> > > motivations
> > > > >>> and
> > > > >>> > the
> > > > >>> > > > >> best
> > > > >>> > > > >> > > APIs
> > > > >>> > > > >> > > > to
> > > > >>> > > > >> > > > > > get savings from are `Produce` and `Fetch`.
> > > > >>> > > > >> > > > > >
> > > > >>> > > > >> > > > > > 4. When it comes to bootstrapping the metadata
> > > topic
> > > > >>> with
> > > > >>> > > > >> respect
> > > > >>> > > > >> > to
> > > > >>> > > > >> > > > > > KIP-500, one suggestion would be to reserve a
> > > > sentinel
> > > > >>> > topic
> > > > >>> > > > ID
> > > > >>> > > > >> > which
> > > > >>> > > > >> > > > can
> > > > >>> > > > >> > > > > > be used initially by new brokers. When the first
> > > > >>> > controller
> > > > >>> > > is
> > > > >>> > > > >> > > elected,
> > > > >>> > > > >> > > > > it
> > > > >>> > > > >> > > > > > can assign a topicId to the metadata topic. This
> > > is
> > > > a
> > > > >>> bit
> > > > >>> > > > >> similar
> > > > >>> > > > >> > to
> > > > >>> > > > >> > > > how
> > > > >>> > > > >> > > > > we
> > > > >>> > > > >> > > > > > were planning to generate the clusterId.
> > > > >>> > > > >> > > > > >
> > > > >>> > > > >> > > > > > Thanks,
> > > > >>> > > > >> > > > > > Jason
> > > > >>> > > > >> > > > > >
> > > > >>> > > > >> > > > > >
> > > > >>> > > > >> > > > > > On Thu, Sep 24, 2020 at 11:10 AM Jun Rao <
> > > > >>> > jun@confluent.io>
> > > > >>> > > > >> wrote:
> > > > >>> > > > >> > > > > >
> > > > >>> > > > >> > > > > > > Hi, Justine,
> > > > >>> > > > >> > > > > > >
> > > > >>> > > > >> > > > > > > Thanks for the updated KIP. A few more
> > comments
> > > > >>> below.
> > > > >>> > > > >> > > > > > >
> > > > >>> > > > >> > > > > > > 30.  {"name": "id", "type": "string", "doc":
> > > > >>> "version
> > > > >>> > > id"}}:
> > > > >>> > > > >> The
> > > > >>> > > > >> > > doc
> > > > >>> > > > >> > > > > > should
> > > > >>> > > > >> > > > > > > say UUID.
> > > > >>> > > > >> > > > > > >
> > > > >>> > > > >> > > > > > > 31. LeaderAndIsrResponse v5 and
> > > > StopReplicaResponse
> > > > >>> v4 :
> > > > >>> > > It
> > > > >>> > > > >> seems
> > > > >>> > > > >> > > > there
> > > > >>> > > > >> > > > > > is
> > > > >>> > > > >> > > > > > > no need to add topic_id at partitions level.
> > > > >>> > > > >> > > > > > >
> > > > >>> > > > >> > > > > > > 32. Regarding partition metadata file. Perhaps
> > > the
> > > > >>> key
> > > > >>> > can
> > > > >>> > > > be
> > > > >>> > > > >> a
> > > > >>> > > > >> > > > single
> > > > >>> > > > >> > > > > > > word, sth like the following.
> > > > >>> > > > >> > > > > > > version: 0
> > > > >>> > > > >> > > > > > > topic_id: 46bdb63f-9e8d-4a38-bf7b-ee4eb2a794e4
> > > > >>> > > > >> > > > > > >
> > > > >>> > > > >> > > > > > > 33. Another tricky thing that I realized is
> > how
> > > to
> > > > >>> > support
> > > > >>> > > > the
> > > > >>> > > > >> > > > metadata
> > > > >>> > > > >> > > > > > > topic introduced in KIP-595. It's a bit tricky
> > > to
> > > > >>> > assign a
> > > > >>> > > > >> UUID
> > > > >>> > > > >> > to
> > > > >>> > > > >> > > > the
> > > > >>> > > > >> > > > > > > metadata topic since we have a chicken-and-egg
> > > > >>> problem.
> > > > >>> > > The
> > > > >>> > > > >> > > > controller
> > > > >>> > > > >> > > > > > > needs to persist the UUID in the metadata
> > topic
> > > in
> > > > >>> order
> > > > >>> > > to
> > > > >>> > > > >> > assign
> > > > >>> > > > >> > > > one
> > > > >>> > > > >> > > > > > > successfully, but the metadata topic is needed
> > > to
> > > > >>> elect
> > > > >>> > a
> > > > >>> > > > >> > > controller
> > > > >>> > > > >> > > > > > > (KIP-631). So, this probably needs a bit more
> > > > >>> thought.
> > > > >>> > > > >> > > > > > >
> > > > >>> > > > >> > > > > > > Jun
> > > > >>> > > > >> > > > > > >
> > > > >>> > > > >> > > > > > > On Thu, Sep 24, 2020 at 3:04 AM Ismael Juma <
> > > > >>> > > > >> ismael@juma.me.uk>
> > > > >>> > > > >> > > > wrote:
> > > > >>> > > > >> > > > > > >
> > > > >>> > > > >> > > > > > > > Also, can we provide more details on how the
> > > > >>> Partition
> > > > >>> > > > >> Metadata
> > > > >>> > > > >> > > > file
> > > > >>> > > > >> > > > > > will
> > > > >>> > > > >> > > > > > > > be used?
> > > > >>> > > > >> > > > > > > >
> > > > >>> > > > >> > > > > > > > Ismael
> > > > >>> > > > >> > > > > > > >
> > > > >>> > > > >> > > > > > > > On Thu, Sep 24, 2020 at 3:01 AM Ismael Juma
> > <
> > > > >>> > > > >> ismael@juma.me.uk
> > > > >>> > > > >> > >
> > > > >>> > > > >> > > > > wrote:
> > > > >>> > > > >> > > > > > > >
> > > > >>> > > > >> > > > > > > > > Hi Justine,
> > > > >>> > > > >> > > > > > > > >
> > > > >>> > > > >> > > > > > > > > I think we need to update the "Rejected
> > > > >>> > Alternatives"
> > > > >>> > > > >> section
> > > > >>> > > > >> > > to
> > > > >>> > > > >> > > > > take
> > > > >>> > > > >> > > > > > > > into
> > > > >>> > > > >> > > > > > > > > account that the proposal now removes the
> > > > topic
> > > > >>> name
> > > > >>> > > > from
> > > > >>> > > > >> the
> > > > >>> > > > >> > > > fetch
> > > > >>> > > > >> > > > > > > > > request. Also, if we are removing it from
> > > the
> > > > >>> Fetch
> > > > >>> > > > >> request,
> > > > >>> > > > >> > > does
> > > > >>> > > > >> > > > > it
> > > > >>> > > > >> > > > > > > make
> > > > >>> > > > >> > > > > > > > > sense not to remove it from similar
> > requests
> > > > >>> like
> > > > >>> > > > >> > > > > ListOffsetRequest?
> > > > >>> > > > >> > > > > > > > >
> > > > >>> > > > >> > > > > > > > > Ismael
> > > > >>> > > > >> > > > > > > > >
> > > > >>> > > > >> > > > > > > > > On Thu, Sep 24, 2020 at 2:46 AM David
> > Jacot
> > > <
> > > > >>> > > > >> > > djacot@confluent.io
> > > > >>> > > > >> > > > >
> > > > >>> > > > >> > > > > > > wrote:
> > > > >>> > > > >> > > > > > > > >
> > > > >>> > > > >> > > > > > > > >> Hi Justine,
> > > > >>> > > > >> > > > > > > > >>
> > > > >>> > > > >> > > > > > > > >> Thanks for the KIP. I finally had time to
> > > > read
> > > > >>> it
> > > > >>> > :).
> > > > >>> > > > It
> > > > >>> > > > >> is
> > > > >>> > > > >> > a
> > > > >>> > > > >> > > > > great
> > > > >>> > > > >> > > > > > > > >> improvement.
> > > > >>> > > > >> > > > > > > > >>
> > > > >>> > > > >> > > > > > > > >> I have a few comments/questions:
> > > > >>> > > > >> > > > > > > > >>
> > > > >>> > > > >> > > > > > > > >> 1. It seems that the schema of the
> > > > >>> > StopReplicaRequest
> > > > >>> > > > is
> > > > >>> > > > >> > > > slightly
> > > > >>> > > > >> > > > > > > > >> outdated.
> > > > >>> > > > >> > > > > > > > >> We
> > > > >>> > > > >> > > > > > > > >> did some changes as part of KIP-570. V3
> > is
> > > > >>> already
> > > > >>> > > > >> organized
> > > > >>> > > > >> > > by
> > > > >>> > > > >> > > > > > > topics.
> > > > >>> > > > >> > > > > > > > >>
> > > > >>> > > > >> > > > > > > > >> 2. I just want to make sure that I
> > > understand
> > > > >>> the
> > > > >>> > > > >> > > reconciliation
> > > > >>> > > > >> > > > > > > > >> logic correctly. When
> > > > >>> > > > >> > > > > > > > >> an "INCREMENTAL" LeaderAndIsr Request is
> > > > >>> received,
> > > > >>> > > the
> > > > >>> > > > >> > broker
> > > > >>> > > > >> > > > will
> > > > >>> > > > >> > > > > > > also
> > > > >>> > > > >> > > > > > > > >> reconcile
> > > > >>> > > > >> > > > > > > > >> when the local uuid does not match the
> > uuid
> > > > in
> > > > >>> the
> > > > >>> > > > >> request,
> > > > >>> > > > >> > > > right?
> > > > >>> > > > >> > > > > > In
> > > > >>> > > > >> > > > > > > > this
> > > > >>> > > > >> > > > > > > > >> case, the
> > > > >>> > > > >> > > > > > > > >> local log is staged for deletion.
> > > > >>> > > > >> > > > > > > > >>
> > > > >>> > > > >> > > > > > > > >> 3. In the documentation of the `
> > > > >>> > > > >> delete.stale.topic.delay.ms
> > > > >>> > > > >> > `
> > > > >>> > > > >> > > > > > config,
> > > > >>> > > > >> > > > > > > it
> > > > >>> > > > >> > > > > > > > >> says "When a
> > > > >>> > > > >> > > > > > > > >> FULL LeaderAndIsrRequest is received..."
> > > but
> > > > I
> > > > >>> > > suppose
> > > > >>> > > > it
> > > > >>> > > > >> > > > applies
> > > > >>> > > > >> > > > > to
> > > > >>> > > > >> > > > > > > > both
> > > > >>> > > > >> > > > > > > > >> types.
> > > > >>> > > > >> > > > > > > > >>
> > > > >>> > > > >> > > > > > > > >> Best,
> > > > >>> > > > >> > > > > > > > >> David
> > > > >>> > > > >> > > > > > > > >>
> > > > >>> > > > >> > > > > > > > >> On Thu, Sep 24, 2020 at 1:40 AM Justine
> > > > Olshan
> > > > >>> <
> > > > >>> > > > >> > > > > > jolshan@confluent.io>
> > > > >>> > > > >> > > > > > > > >> wrote:
> > > > >>> > > > >> > > > > > > > >>
> > > > >>> > > > >> > > > > > > > >> > Hi Jun,
> > > > >>> > > > >> > > > > > > > >> >
> > > > >>> > > > >> > > > > > > > >> > Thanks for the comments. I apologize
> > for
> > > > >>> some of
> > > > >>> > > the
> > > > >>> > > > >> typos
> > > > >>> > > > >> > > and
> > > > >>> > > > >> > > > > > > > >> confusion.
> > > > >>> > > > >> > > > > > > > >> > I’ve updated the KIP to fix some of the
> > > > >>> issues
> > > > >>> > you
> > > > >>> > > > >> > > mentioned.
> > > > >>> > > > >> > > > > > > > >> >
> > > > >>> > > > >> > > > > > > > >> > 20.2 I’ve changed the type to String.
> > > > >>> > > > >> > > > > > > > >> > 20.1/3 I’ve updated the TopicZNode to
> > fix
> > > > >>> > > formatting
> > > > >>> > > > >> and
> > > > >>> > > > >> > > > reflect
> > > > >>> > > > >> > > > > > the
> > > > >>> > > > >> > > > > > > > >> latest
> > > > >>> > > > >> > > > > > > > >> > version before this change.
> > > > >>> > > > >> > > > > > > > >> >
> > > > >>> > > > >> > > > > > > > >> > 21. You are correct and I’ve removed
> > this
> > > > >>> line.
> > > > >>> > > I’ve
> > > > >>> > > > >> also
> > > > >>> > > > >> > > > added
> > > > >>> > > > >> > > > > a
> > > > >>> > > > >> > > > > > > line
> > > > >>> > > > >> > > > > > > > >> > mentioning an IBP bump is necessary for
> > > > >>> migration
> > > > >>> > > > >> > > > > > > > >> >
> > > > >>> > > > >> > > > > > > > >> > 22. I think the wording was unclear but
> > > > your
> > > > >>> > > summary
> > > > >>> > > > is
> > > > >>> > > > >> > what
> > > > >>> > > > >> > > > was
> > > > >>> > > > >> > > > > > > > >> intended
> > > > >>> > > > >> > > > > > > > >> > by this line. I’ve updated to make this
> > > > point
> > > > >>> > more
> > > > >>> > > > >> clear.
> > > > >>> > > > >> > > > > “Remove
> > > > >>> > > > >> > > > > > > > >> deleted
> > > > >>> > > > >> > > > > > > > >> > topics from replicas by sending
> > > > >>> > StopReplicaRequest
> > > > >>> > > V3
> > > > >>> > > > >> > before
> > > > >>> > > > >> > > > the
> > > > >>> > > > >> > > > > > IBP
> > > > >>> > > > >> > > > > > > > >> bump
> > > > >>> > > > >> > > > > > > > >> > using the old logic, and using V4 and
> > the
> > > > new
> > > > >>> > logic
> > > > >>> > > > >> with
> > > > >>> > > > >> > > topic
> > > > >>> > > > >> > > > > IDs
> > > > >>> > > > >> > > > > > > > after
> > > > >>> > > > >> > > > > > > > >> > the IBP bump.”
> > > > >>> > > > >> > > > > > > > >> >
> > > > >>> > > > >> > > > > > > > >> > 23. I’ve removed the unspecified type
> > > from
> > > > >>> the
> > > > >>> > KIP
> > > > >>> > > > and
> > > > >>> > > > >> > > mention
> > > > >>> > > > >> > > > > > that
> > > > >>> > > > >> > > > > > > > IBP
> > > > >>> > > > >> > > > > > > > >> > will be used to handle this request.
> > “IBP
> > > > >>> will be
> > > > >>> > > > used
> > > > >>> > > > >> to
> > > > >>> > > > >> > > > > > determine
> > > > >>> > > > >> > > > > > > > >> whether
> > > > >>> > > > >> > > > > > > > >> > this new form of the request will be
> > > used.
> > > > >>> For
> > > > >>> > > older
> > > > >>> > > > >> > > requests,
> > > > >>> > > > >> > > > > we
> > > > >>> > > > >> > > > > > > will
> > > > >>> > > > >> > > > > > > > >> > ignore this field and default to
> > previous
> > > > >>> > > behavior.”
> > > > >>> > > > >> > > > > > > > >> >
> > > > >>> > > > >> > > > > > > > >> > 24. I’ve fixed this typo.
> > > > >>> > > > >> > > > > > > > >> >
> > > > >>> > > > >> > > > > > > > >> > 25. I've added a topics at a higher
> > level
> > > > for
> > > > >>> > > > >> > > > > LeaderAndIsrResponse
> > > > >>> > > > >> > > > > > > v5,
> > > > >>> > > > >> > > > > > > > >> > StopReplicaResponse v4. I've also
> > changed
> > > > >>> > > > >> > StopReplicaRequest
> > > > >>> > > > >> > > > v4
> > > > >>> > > > >> > > > > to
> > > > >>> > > > >> > > > > > > > have
> > > > >>> > > > >> > > > > > > > >> > topics at a higher level.
> > > > >>> > > > >> > > > > > > > >> >
> > > > >>> > > > >> > > > > > > > >> > 26. I’ve updated
> > > > >>> forgotten_topics_data--added the
> > > > >>> > > > >> topic ID
> > > > >>> > > > >> > > and
> > > > >>> > > > >> > > > > > > removed
> > > > >>> > > > >> > > > > > > > >> the
> > > > >>> > > > >> > > > > > > > >> > topic name
> > > > >>> > > > >> > > > > > > > >> >
> > > > >>> > > > >> > > > > > > > >> > 27. I’ve decided on plain text, and
> > I’ve
> > > > >>> added an
> > > > >>> > > > >> example.
> > > > >>> > > > >> > > > > > > > >> >
> > > > >>> > > > >> > > > > > > > >> > 28. I’ve added this idea to future
> > work.
> > > > >>> > > > >> > > > > > > > >> >
> > > > >>> > > > >> > > > > > > > >> > Thanks again for taking a look,
> > > > >>> > > > >> > > > > > > > >> >
> > > > >>> > > > >> > > > > > > > >> > Justine
> > > > >>> > > > >> > > > > > > > >> >
> > > > >>> > > > >> > > > > > > > >> > On Wed, Sep 23, 2020 at 10:28 AM Jun
> > Rao
> > > <
> > > > >>> > > > >> > jun@confluent.io>
> > > > >>> > > > >> > > > > > wrote:
> > > > >>> > > > >> > > > > > > > >> >
> > > > >>> > > > >> > > > > > > > >> > > Hi, Justine,
> > > > >>> > > > >> > > > > > > > >> > >
> > > > >>> > > > >> > > > > > > > >> > > Thanks for the response. Made another
> > > > >>> pass. A
> > > > >>> > few
> > > > >>> > > > >> more
> > > > >>> > > > >> > > > > comments
> > > > >>> > > > >> > > > > > > > below.
> > > > >>> > > > >> > > > > > > > >> > >
> > > > >>> > > > >> > > > > > > > >> > > 20. znode schema:
> > > > >>> > > > >> > > > > > > > >> > > 20.1 It seems that {"name":
> > "version",
> > > > >>> "type":
> > > > >>> > > > "int",
> > > > >>> > > > >> > > "id":
> > > > >>> > > > >> > > > > > > "UUID",
> > > > >>> > > > >> > > > > > > > >> > "doc":
> > > > >>> > > > >> > > > > > > > >> > > "version id"} should be {"name":
> > > > "version",
> > > > >>> > > "type":
> > > > >>> > > > >> > > "int"},
> > > > >>> > > > >> > > > > > > {"name":
> > > > >>> > > > >> > > > > > > > >> > "id",
> > > > >>> > > > >> > > > > > > > >> > > "type": "UUID", "doc": "version id"}.
> > > > >>> > > > >> > > > > > > > >> > > 20.2 The znode format is JSON which
> > > > doesn't
> > > > >>> > have
> > > > >>> > > > UUID
> > > > >>> > > > >> > > type.
> > > > >>> > > > >> > > > So
> > > > >>> > > > >> > > > > > the
> > > > >>> > > > >> > > > > > > > >> type
> > > > >>> > > > >> > > > > > > > >> > > probably should be string?
> > > > >>> > > > >> > > > > > > > >> > > 20.3 Also, the existing format used
> > > seems
> > > > >>> > > outdated.
> > > > >>> > > > >> It
> > > > >>> > > > >> > > > should
> > > > >>> > > > >> > > > > > have
> > > > >>> > > > >> > > > > > > > the
> > > > >>> > > > >> > > > > > > > >> > > following format.
> > > > >>> > > > >> > > > > > > > >> > >     Json.encodeAsBytes(Map(
> > > > >>> > > > >> > > > > > > > >> > >       "version" -> 2,
> > > > >>> > > > >> > > > > > > > >> > >       "partitions" ->
> > > > >>> > > replicaAssignmentJson.asJava,
> > > > >>> > > > >> > > > > > > > >> > >       "adding_replicas" ->
> > > > >>> > > > >> > > > > addingReplicasAssignmentJson.asJava,
> > > > >>> > > > >> > > > > > > > >> > >       "removing_replicas" ->
> > > > >>> > > > >> > > > > > removingReplicasAssignmentJson.asJava
> > > > >>> > > > >> > > > > > > > >> > >     ).asJava)
> > > > >>> > > > >> > > > > > > > >> > >   }
> > > > >>> > > > >> > > > > > > > >> > >
> > > > >>> > > > >> > > > > > > > >> > > 21. Migration: The KIP says "The
> > > > migration
> > > > >>> > > process
> > > > >>> > > > >> can
> > > > >>> > > > >> > > take
> > > > >>> > > > >> > > > > > place
> > > > >>> > > > >> > > > > > > > >> without
> > > > >>> > > > >> > > > > > > > >> > > an inter-broker protocol bump, as the
> > > > >>> format
> > > > >>> > > stored
> > > > >>> > > > >> in
> > > > >>> > > > >> > > > > > > > >> > > /brokers/topics/[topic] will be
> > > > compatible
> > > > >>> with
> > > > >>> > > > older
> > > > >>> > > > >> > > broker
> > > > >>> > > > >> > > > > > > > >> versions."
> > > > >>> > > > >> > > > > > > > >> > > However, since we are bumping up the
> > > > >>> version of
> > > > >>> > > > >> > > inter-broker
> > > > >>> > > > >> > > > > > > > >> requests, it
> > > > >>> > > > >> > > > > > > > >> > > seems that we need to use IBP for
> > > > >>> migration.
> > > > >>> > > > >> > > > > > > > >> > >
> > > > >>> > > > >> > > > > > > > >> > > 22. The KIP says "Remove deleted
> > topics
> > > > >>> from
> > > > >>> > > > >> replicas by
> > > > >>> > > > >> > > > > sending
> > > > >>> > > > >> > > > > > > > >> > > StopReplicaRequest V3 for any topics
> > > > which
> > > > >>> do
> > > > >>> > not
> > > > >>> > > > >> > contain
> > > > >>> > > > >> > > a
> > > > >>> > > > >> > > > > > topic
> > > > >>> > > > >> > > > > > > > ID,
> > > > >>> > > > >> > > > > > > > >> and
> > > > >>> > > > >> > > > > > > > >> > > V4 for any topics which do contain a
> > > > topic
> > > > >>> > ID.".
> > > > >>> > > > >> > However,
> > > > >>> > > > >> > > if
> > > > >>> > > > >> > > > > we
> > > > >>> > > > >> > > > > > > use
> > > > >>> > > > >> > > > > > > > >> IBP,
> > > > >>> > > > >> > > > > > > > >> > it
> > > > >>> > > > >> > > > > > > > >> > > seems that the controller will either
> > > > send
> > > > >>> > > > >> > > > StopReplicaRequest
> > > > >>> > > > >> > > > > V3
> > > > >>> > > > >> > > > > > > > >> > > or StopReplicaRequest V4, but never
> > > mixed
> > > > >>> V3
> > > > >>> > and
> > > > >>> > > V4
> > > > >>> > > > >> for
> > > > >>> > > > >> > > > > > different
> > > > >>> > > > >> > > > > > > > >> topics.
> > > > >>> > > > >> > > > > > > > >> > > Basically, before the IBP bump, V3
> > will
> > > > be
> > > > >>> > used.
> > > > >>> > > > >> After
> > > > >>> > > > >> > the
> > > > >>> > > > >> > > > IBP
> > > > >>> > > > >> > > > > > > bump,
> > > > >>> > > > >> > > > > > > > >> > > topicId will be created and V4 will
> > be
> > > > >>> used.
> > > > >>> > > > >> > > > > > > > >> > >
> > > > >>> > > > >> > > > > > > > >> > > 23. Given that we depend on IBP, do
> > we
> > > > >>> still
> > > > >>> > need
> > > > >>> > > > "0
> > > > >>> > > > >> > > > > > UNSPECIFIED"
> > > > >>> > > > >> > > > > > > > >> > > in LeaderAndIsr?
> > > > >>> > > > >> > > > > > > > >> > >
> > > > >>> > > > >> > > > > > > > >> > > 24. LeaderAndIsrResponse v5 : It
> > still
> > > > has
> > > > >>> the
> > > > >>> > > > topic
> > > > >>> > > > >> > > field.
> > > > >>> > > > >> > > > > > > > >> > >
> > > > >>> > > > >> > > > > > > > >> > > 25. LeaderAndIsrResponse v5,
> > > > >>> > StopReplicaResponse
> > > > >>> > > > v4:
> > > > >>> > > > >> > Could
> > > > >>> > > > >> > > > we
> > > > >>> > > > >> > > > > > use
> > > > >>> > > > >> > > > > > > > this
> > > > >>> > > > >> > > > > > > > >> > > opportunity to organize the response
> > > in 2
> > > > >>> > levels,
> > > > >>> > > > >> first
> > > > >>> > > > >> > by
> > > > >>> > > > >> > > > > > topic,
> > > > >>> > > > >> > > > > > > > >> then by
> > > > >>> > > > >> > > > > > > > >> > > partition, as most other
> > > > >>> requests/responses?
> > > > >>> > > > >> > > > > > > > >> > >
> > > > >>> > > > >> > > > > > > > >> > > 26. FetchRequest v13 : Should
> > > > >>> > > forgotten_topics_data
> > > > >>> > > > >> use
> > > > >>> > > > >> > > > > topicId
> > > > >>> > > > >> > > > > > > too?
> > > > >>> > > > >> > > > > > > > >> > >
> > > > >>> > > > >> > > > > > > > >> > > 27. "This file can either be plain
> > text
> > > > >>> > > (key/value
> > > > >>> > > > >> > pairs)
> > > > >>> > > > >> > > or
> > > > >>> > > > >> > > > > > > JSON."
> > > > >>> > > > >> > > > > > > > >> Have
> > > > >>> > > > >> > > > > > > > >> > we
> > > > >>> > > > >> > > > > > > > >> > > decided which one to use? Also, it
> > > would
> > > > be
> > > > >>> > > helpful
> > > > >>> > > > >> to
> > > > >>> > > > >> > > > provide
> > > > >>> > > > >> > > > > > an
> > > > >>> > > > >> > > > > > > > >> > example.
> > > > >>> > > > >> > > > > > > > >> > >
> > > > >>> > > > >> > > > > > > > >> > > 28. Future improvement: Another
> > future
> > > > >>> > > improvement
> > > > >>> > > > >> > > > opportunity
> > > > >>> > > > >> > > > > > is
> > > > >>> > > > >> > > > > > > to
> > > > >>> > > > >> > > > > > > > >> use
> > > > >>> > > > >> > > > > > > > >> > > topicId in
> > > > >>> GroupMetadataManager.offsetCommitKey
> > > > >>> > > in
> > > > >>> > > > >> the
> > > > >>> > > > >> > > > > > > offset_commit
> > > > >>> > > > >> > > > > > > > >> > topic.
> > > > >>> > > > >> > > > > > > > >> > > This may save some space.
> > > > >>> > > > >> > > > > > > > >> > >
> > > > >>> > > > >> > > > > > > > >> > > Thanks,
> > > > >>> > > > >> > > > > > > > >> > >
> > > > >>> > > > >> > > > > > > > >> > > Jun
> > > > >>> > > > >> > > > > > > > >> > >
> > > > >>> > > > >> > > > > > > > >> > > On Wed, Sep 23, 2020 at 8:50 AM
> > Justine
> > > > >>> Olshan
> > > > >>> > <
> > > > >>> > > > >> > > > > > > > jolshan@confluent.io>
> > > > >>> > > > >> > > > > > > > >> > > wrote:
> > > > >>> > > > >> > > > > > > > >> > >
> > > > >>> > > > >> > > > > > > > >> > > > Hi Tom,
> > > > >>> > > > >> > > > > > > > >> > > >
> > > > >>> > > > >> > > > > > > > >> > > > Thanks for the comment. I think
> > this
> > > > is a
> > > > >>> > > really
> > > > >>> > > > >> good
> > > > >>> > > > >> > > idea
> > > > >>> > > > >> > > > > and
> > > > >>> > > > >> > > > > > > it
> > > > >>> > > > >> > > > > > > > >> has
> > > > >>> > > > >> > > > > > > > >> > > been
> > > > >>> > > > >> > > > > > > > >> > > > added to the KIP under the newly
> > > added
> > > > >>> > tooling
> > > > >>> > > > >> > section.
> > > > >>> > > > >> > > > > > > > >> > > >
> > > > >>> > > > >> > > > > > > > >> > > > Thanks again,
> > > > >>> > > > >> > > > > > > > >> > > > Justine
> > > > >>> > > > >> > > > > > > > >> > > >
> > > > >>> > > > >> > > > > > > > >> > > > On Wed, Sep 23, 2020 at 3:17 AM Tom
> > > > >>> Bentley <
> > > > >>> > > > >> > > > > > > tbentley@redhat.com>
> > > > >>> > > > >> > > > > > > > >> > wrote:
> > > > >>> > > > >> > > > > > > > >> > > >
> > > > >>> > > > >> > > > > > > > >> > > > > Hi Justine,
> > > > >>> > > > >> > > > > > > > >> > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > I know you started the vote
> > thread,
> > > > >>> but on
> > > > >>> > > > >> > re-reading
> > > > >>> > > > >> > > > the
> > > > >>> > > > >> > > > > > KIP
> > > > >>> > > > >> > > > > > > I
> > > > >>> > > > >> > > > > > > > >> > noticed
> > > > >>> > > > >> > > > > > > > >> > > > > that although the topic id is
> > > > included
> > > > >>> in
> > > > >>> > the
> > > > >>> > > > >> > > > > > MetadataResponse
> > > > >>> > > > >> > > > > > > > >> it's
> > > > >>> > > > >> > > > > > > > >> > not
> > > > >>> > > > >> > > > > > > > >> > > > > surfaced in the output from
> > > > >>> > `kafka-topics.sh
> > > > >>> > > > >> > > > --describe`.
> > > > >>> > > > >> > > > > > > Maybe
> > > > >>> > > > >> > > > > > > > >> that
> > > > >>> > > > >> > > > > > > > >> > > was
> > > > >>> > > > >> > > > > > > > >> > > > > intentional because ids are
> > > > >>> intentionally
> > > > >>> > not
> > > > >>> > > > >> really
> > > > >>> > > > >> > > > > > something
> > > > >>> > > > >> > > > > > > > the
> > > > >>> > > > >> > > > > > > > >> > user
> > > > >>> > > > >> > > > > > > > >> > > > > should care deeply about, but it
> > > > would
> > > > >>> also
> > > > >>> > > > make
> > > > >>> > > > >> > life
> > > > >>> > > > >> > > > > harder
> > > > >>> > > > >> > > > > > > for
> > > > >>> > > > >> > > > > > > > >> > anyone
> > > > >>> > > > >> > > > > > > > >> > > > > debugging Kafka and this would
> > > likely
> > > > >>> get
> > > > >>> > > worse
> > > > >>> > > > >> the
> > > > >>> > > > >> > > more
> > > > >>> > > > >> > > > > > topic
> > > > >>> > > > >> > > > > > > > ids
> > > > >>> > > > >> > > > > > > > >> > got
> > > > >>> > > > >> > > > > > > > >> > > > > rolled out across the protocols,
> > > > >>> clients
> > > > >>> > etc.
> > > > >>> > > > It
> > > > >>> > > > >> > seems
> > > > >>> > > > >> > > > > > likely
> > > > >>> > > > >> > > > > > > > that
> > > > >>> > > > >> > > > > > > > >> > > > > `kafka-topics.sh` will eventually
> > > > need
> > > > >>> the
> > > > >>> > > > >> ability
> > > > >>> > > > >> > to
> > > > >>> > > > >> > > > show
> > > > >>> > > > >> > > > > > the
> > > > >>> > > > >> > > > > > > > id
> > > > >>> > > > >> > > > > > > > >> of
> > > > >>> > > > >> > > > > > > > >> > a
> > > > >>> > > > >> > > > > > > > >> > > > > topic and perhaps find a topic
> > name
> > > > >>> given
> > > > >>> > an
> > > > >>> > > > id.
> > > > >>> > > > >> Is
> > > > >>> > > > >> > > > there
> > > > >>> > > > >> > > > > > any
> > > > >>> > > > >> > > > > > > > >> reason
> > > > >>> > > > >> > > > > > > > >> > > not
> > > > >>> > > > >> > > > > > > > >> > > > to
> > > > >>> > > > >> > > > > > > > >> > > > > implement that in this KIP?
> > > > >>> > > > >> > > > > > > > >> > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > Many thanks,
> > > > >>> > > > >> > > > > > > > >> > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > Tom
> > > > >>> > > > >> > > > > > > > >> > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > On Mon, Sep 21, 2020 at 9:54 PM
> > > > Justine
> > > > >>> > > Olshan
> > > > >>> > > > <
> > > > >>> > > > >> > > > > > > > >> jolshan@confluent.io
> > > > >>> > > > >> > > > > > > > >> > >
> > > > >>> > > > >> > > > > > > > >> > > > > wrote:
> > > > >>> > > > >> > > > > > > > >> > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > Hi all,
> > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > After thinking about it, I've
> > > > >>> decided to
> > > > >>> > > > remove
> > > > >>> > > > >> > the
> > > > >>> > > > >> > > > > topic
> > > > >>> > > > >> > > > > > > name
> > > > >>> > > > >> > > > > > > > >> from
> > > > >>> > > > >> > > > > > > > >> > > the
> > > > >>> > > > >> > > > > > > > >> > > > > > Fetch Request and Response
> > after
> > > > all.
> > > > >>> > Since
> > > > >>> > > > >> there
> > > > >>> > > > >> > > are
> > > > >>> > > > >> > > > so
> > > > >>> > > > >> > > > > > > many
> > > > >>> > > > >> > > > > > > > of
> > > > >>> > > > >> > > > > > > > >> > > these
> > > > >>> > > > >> > > > > > > > >> > > > > > requests per second, it is
> > worth
> > > > >>> removing
> > > > >>> > > the
> > > > >>> > > > >> > extra
> > > > >>> > > > >> > > > > > > > information.
> > > > >>> > > > >> > > > > > > > >> > I've
> > > > >>> > > > >> > > > > > > > >> > > > > > updated the KIP to reflect this
> > > > >>> change.
> > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > Please let me know if there is
> > > > >>> anything
> > > > >>> > > else
> > > > >>> > > > we
> > > > >>> > > > >> > > should
> > > > >>> > > > >> > > > > > > discuss
> > > > >>> > > > >> > > > > > > > >> > before
> > > > >>> > > > >> > > > > > > > >> > > > > > voting.
> > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > Thank you,
> > > > >>> > > > >> > > > > > > > >> > > > > > Justine
> > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > On Fri, Sep 18, 2020 at 9:46 AM
> > > > >>> Justine
> > > > >>> > > > Olshan
> > > > >>> > > > >> <
> > > > >>> > > > >> > > > > > > > >> > jolshan@confluent.io
> > > > >>> > > > >> > > > > > > > >> > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > wrote:
> > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > > Hi Jun,
> > > > >>> > > > >> > > > > > > > >> > > > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > > I see what you are saying.
> > For
> > > > now
> > > > >>> we
> > > > >>> > can
> > > > >>> > > > >> remove
> > > > >>> > > > >> > > the
> > > > >>> > > > >> > > > > > extra
> > > > >>> > > > >> > > > > > > > >> > > > information.
> > > > >>> > > > >> > > > > > > > >> > > > > > > I'll leave the option to add
> > > more
> > > > >>> > fields
> > > > >>> > > to
> > > > >>> > > > >> the
> > > > >>> > > > >> > > file
> > > > >>> > > > >> > > > > in
> > > > >>> > > > >> > > > > > > the
> > > > >>> > > > >> > > > > > > > >> > future.
> > > > >>> > > > >> > > > > > > > >> > > > The
> > > > >>> > > > >> > > > > > > > >> > > > > > KIP
> > > > >>> > > > >> > > > > > > > >> > > > > > > has been updated to reflect
> > > this
> > > > >>> > change.
> > > > >>> > > > >> > > > > > > > >> > > > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > > Thanks,
> > > > >>> > > > >> > > > > > > > >> > > > > > > Justine
> > > > >>> > > > >> > > > > > > > >> > > > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > > On Fri, Sep 18, 2020 at 8:46
> > AM
> > > > Jun
> > > > >>> > Rao <
> > > > >>> > > > >> > > > > > jun@confluent.io
> > > > >>> > > > >> > > > > > > >
> > > > >>> > > > >> > > > > > > > >> > wrote:
> > > > >>> > > > >> > > > > > > > >> > > > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> Hi, Justine,
> > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > >>> > > > >> > > > > > > > >> > > > > > >> Thanks for the reply.
> > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > >>> > > > >> > > > > > > > >> > > > > > >> 13. If the log directory is
> > > the
> > > > >>> source
> > > > >>> > > of
> > > > >>> > > > >> > truth,
> > > > >>> > > > >> > > it
> > > > >>> > > > >> > > > > > means
> > > > >>> > > > >> > > > > > > > >> that
> > > > >>> > > > >> > > > > > > > >> > the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> redundant info in the
> > metadata
> > > > >>> file
> > > > >>> > will
> > > > >>> > > > be
> > > > >>> > > > >> > > > ignored.
> > > > >>> > > > >> > > > > > Then
> > > > >>> > > > >> > > > > > > > the
> > > > >>> > > > >> > > > > > > > >> > > > question
> > > > >>> > > > >> > > > > > > > >> > > > > > is
> > > > >>> > > > >> > > > > > > > >> > > > > > >> why do we need to put the
> > > > >>> redundant
> > > > >>> > info
> > > > >>> > > > in
> > > > >>> > > > >> the
> > > > >>> > > > >> > > > > > metadata
> > > > >>> > > > >> > > > > > > > file
> > > > >>> > > > >> > > > > > > > >> > now?
> > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > >>> > > > >> > > > > > > > >> > > > > > >> Thanks,
> > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > >>> > > > >> > > > > > > > >> > > > > > >> Jun
> > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > >>> > > > >> > > > > > > > >> > > > > > >> On Thu, Sep 17, 2020 at 5:07
> > > PM
> > > > >>> > Justine
> > > > >>> > > > >> Olshan
> > > > >>> > > > >> > <
> > > > >>> > > > >> > > > > > > > >> > > > jolshan@confluent.io>
> > > > >>> > > > >> > > > > > > > >> > > > > > >> wrote:
> > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > Hi Jun,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > Thanks for the quick
> > > response!
> > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > 12. I've decided to bump
> > up
> > > > the
> > > > >>> > > versions
> > > > >>> > > > >> on
> > > > >>> > > > >> > the
> > > > >>> > > > >> > > > > > > requests
> > > > >>> > > > >> > > > > > > > >> and
> > > > >>> > > > >> > > > > > > > >> > > > updated
> > > > >>> > > > >> > > > > > > > >> > > > > > the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > KIP. I think it's good we
> > > > >>> thoroughly
> > > > >>> > > > >> > discussed
> > > > >>> > > > >> > > > the
> > > > >>> > > > >> > > > > > > > options
> > > > >>> > > > >> > > > > > > > >> > here,
> > > > >>> > > > >> > > > > > > > >> > > > so
> > > > >>> > > > >> > > > > > > > >> > > > > we
> > > > >>> > > > >> > > > > > > > >> > > > > > >> know
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > we made a good choice. :)
> > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > 13. This is an interesting
> > > > >>> > situation.
> > > > >>> > > I
> > > > >>> > > > >> think
> > > > >>> > > > >> > > if
> > > > >>> > > > >> > > > > this
> > > > >>> > > > >> > > > > > > > does
> > > > >>> > > > >> > > > > > > > >> > occur
> > > > >>> > > > >> > > > > > > > >> > > > we
> > > > >>> > > > >> > > > > > > > >> > > > > > >> should
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > give a warning. I agree
> > that
> > > > >>> it's
> > > > >>> > hard
> > > > >>> > > > to
> > > > >>> > > > >> > know
> > > > >>> > > > >> > > > the
> > > > >>> > > > >> > > > > > > source
> > > > >>> > > > >> > > > > > > > >> of
> > > > >>> > > > >> > > > > > > > >> > > truth
> > > > >>> > > > >> > > > > > > > >> > > > > for
> > > > >>> > > > >> > > > > > > > >> > > > > > >> sure
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > since the directory or the
> > > > file
> > > > >>> > could
> > > > >>> > > be
> > > > >>> > > > >> > > manually
> > > > >>> > > > >> > > > > > > > >> modified. I
> > > > >>> > > > >> > > > > > > > >> > > > guess
> > > > >>> > > > >> > > > > > > > >> > > > > > the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > directory could be used as
> > > the
> > > > >>> > source
> > > > >>> > > of
> > > > >>> > > > >> > truth.
> > > > >>> > > > >> > > > To
> > > > >>> > > > >> > > > > be
> > > > >>> > > > >> > > > > > > > >> honest,
> > > > >>> > > > >> > > > > > > > >> > > I'm
> > > > >>> > > > >> > > > > > > > >> > > > > not
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > really sure what happens
> > in
> > > > >>> kafka
> > > > >>> > when
> > > > >>> > > > the
> > > > >>> > > > >> > log
> > > > >>> > > > >> > > > > > > directory
> > > > >>> > > > >> > > > > > > > is
> > > > >>> > > > >> > > > > > > > >> > > > renamed
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > manually in such a way.
> > I'm
> > > > also
> > > > >>> > > > >> wondering if
> > > > >>> > > > >> > > the
> > > > >>> > > > >> > > > > > > > >> situation is
> > > > >>> > > > >> > > > > > > > >> > > > > > >> recoverable
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > in this scenario.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > Thanks,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > Justine
> > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > On Thu, Sep 17, 2020 at
> > 4:28
> > > > PM
> > > > >>> Jun
> > > > >>> > > Rao
> > > > >>> > > > <
> > > > >>> > > > >> > > > > > > > jun@confluent.io>
> > > > >>> > > > >> > > > > > > > >> > > wrote:
> > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Hi, Justine,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Thanks for the reply.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > 12. I don't have a
> > strong
> > > > >>> > preference
> > > > >>> > > > >> > either.
> > > > >>> > > > >> > > > > > However,
> > > > >>> > > > >> > > > > > > > if
> > > > >>> > > > >> > > > > > > > >> we
> > > > >>> > > > >> > > > > > > > >> > > need
> > > > >>> > > > >> > > > > > > > >> > > > > IBP
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > anyway, maybe it's
> > easier
> > > to
> > > > >>> just
> > > > >>> > > bump
> > > > >>> > > > >> up
> > > > >>> > > > >> > the
> > > > >>> > > > >> > > > > > version
> > > > >>> > > > >> > > > > > > > for
> > > > >>> > > > >> > > > > > > > >> > all
> > > > >>> > > > >> > > > > > > > >> > > > > inter
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > broker
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > requests and add the
> > topic
> > > > id
> > > > >>> > field
> > > > >>> > > > as a
> > > > >>> > > > >> > > > regular
> > > > >>> > > > >> > > > > > > > field. A
> > > > >>> > > > >> > > > > > > > >> > > > regular
> > > > >>> > > > >> > > > > > > > >> > > > > > >> field
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > is
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > a bit more concise in
> > wire
> > > > >>> > transfer
> > > > >>> > > > >> than a
> > > > >>> > > > >> > > > > flexible
> > > > >>> > > > >> > > > > > > > >> field.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > 13. The confusion that I
> > > was
> > > > >>> > > referring
> > > > >>> > > > >> to
> > > > >>> > > > >> > is
> > > > >>> > > > >> > > > > > between
> > > > >>> > > > >> > > > > > > > the
> > > > >>> > > > >> > > > > > > > >> > topic
> > > > >>> > > > >> > > > > > > > >> > > > > name
> > > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > partition number between
> > > the
> > > > >>> log
> > > > >>> > dir
> > > > >>> > > > and
> > > > >>> > > > >> > the
> > > > >>> > > > >> > > > > > metadata
> > > > >>> > > > >> > > > > > > > >> file.
> > > > >>> > > > >> > > > > > > > >> > > For
> > > > >>> > > > >> > > > > > > > >> > > > > > >> example,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > if
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > the log dir is topicA-1
> > > and
> > > > >>> the
> > > > >>> > > > metadata
> > > > >>> > > > >> > file
> > > > >>> > > > >> > > > in
> > > > >>> > > > >> > > > > it
> > > > >>> > > > >> > > > > > > has
> > > > >>> > > > >> > > > > > > > >> > topicB
> > > > >>> > > > >> > > > > > > > >> > > > and
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > partition 0 (say due to
> > a
> > > > bug
> > > > >>> or
> > > > >>> > > > manual
> > > > >>> > > > >> > > > > > > modification),
> > > > >>> > > > >> > > > > > > > >> which
> > > > >>> > > > >> > > > > > > > >> > > one
> > > > >>> > > > >> > > > > > > > >> > > > > do
> > > > >>> > > > >> > > > > > > > >> > > > > > we
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > use
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > as the source of truth?
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Jun
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > On Thu, Sep 17, 2020 at
> > > 3:43
> > > > >>> PM
> > > > >>> > > > Justine
> > > > >>> > > > >> > > Olshan
> > > > >>> > > > >> > > > <
> > > > >>> > > > >> > > > > > > > >> > > > > > jolshan@confluent.io>
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > wrote:
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > Hi Jun,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > Thanks for the
> > comments.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > 12. I bumped the
> > > > >>> > > LeaderAndIsrRequest
> > > > >>> > > > >> > > because
> > > > >>> > > > >> > > > I
> > > > >>> > > > >> > > > > > > > removed
> > > > >>> > > > >> > > > > > > > >> the
> > > > >>> > > > >> > > > > > > > >> > > > topic
> > > > >>> > > > >> > > > > > > > >> > > > > > >> name
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > field
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > in the response. It
> > may
> > > be
> > > > >>> > > possible
> > > > >>> > > > to
> > > > >>> > > > >> > > avoid
> > > > >>> > > > >> > > > > > > bumping
> > > > >>> > > > >> > > > > > > > >> the
> > > > >>> > > > >> > > > > > > > >> > > > version
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > without
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > that change, but I may
> > > be
> > > > >>> > missing
> > > > >>> > > > >> > > something.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > I believe StopReplica
> > is
> > > > >>> > actually
> > > > >>> > > on
> > > > >>> > > > >> > > version
> > > > >>> > > > >> > > > 3
> > > > >>> > > > >> > > > > > now,
> > > > >>> > > > >> > > > > > > > but
> > > > >>> > > > >> > > > > > > > >> > > > because
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > version 2
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > is flexible, I kept
> > that
> > > > >>> listed
> > > > >>> > as
> > > > >>> > > > >> > version
> > > > >>> > > > >> > > 2
> > > > >>> > > > >> > > > on
> > > > >>> > > > >> > > > > > the
> > > > >>> > > > >> > > > > > > > KIP
> > > > >>> > > > >> > > > > > > > >> > > page.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> However,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > you
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > may be right in that
> > we
> > > > may
> > > > >>> need
> > > > >>> > > to
> > > > >>> > > > >> bump
> > > > >>> > > > >> > > the
> > > > >>> > > > >> > > > > > > version
> > > > >>> > > > >> > > > > > > > on
> > > > >>> > > > >> > > > > > > > >> > > > > > StopReplica
> > > > >>> > > > >> > > > > > > > >> > > > > > >> to
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > deal
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > with deletion
> > > differently
> > > > as
> > > > >>> > > > mentioned
> > > > >>> > > > >> > > > above. I
> > > > >>> > > > >> > > > > > > don't
> > > > >>> > > > >> > > > > > > > >> know
> > > > >>> > > > >> > > > > > > > >> > > if
> > > > >>> > > > >> > > > > > > > >> > > > I
> > > > >>> > > > >> > > > > > > > >> > > > > > >> have a
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > big
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > preference over used
> > > > tagged
> > > > >>> > fields
> > > > >>> > > > or
> > > > >>> > > > >> > not.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > 13. I was thinking
> > that
> > > in
> > > > >>> the
> > > > >>> > > case
> > > > >>> > > > >> where
> > > > >>> > > > >> > > the
> > > > >>> > > > >> > > > > > file
> > > > >>> > > > >> > > > > > > > and
> > > > >>> > > > >> > > > > > > > >> the
> > > > >>> > > > >> > > > > > > > >> > > > > request
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > topic
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > ids don't match, it
> > > means
> > > > >>> that
> > > > >>> > the
> > > > >>> > > > >> > broker's
> > > > >>> > > > >> > > > > > > topic/the
> > > > >>> > > > >> > > > > > > > >> one
> > > > >>> > > > >> > > > > > > > >> > in
> > > > >>> > > > >> > > > > > > > >> > > > the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> file
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > has
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > been deleted. In that
> > > > case,
> > > > >>> we
> > > > >>> > > would
> > > > >>> > > > >> need
> > > > >>> > > > >> > > to
> > > > >>> > > > >> > > > > > delete
> > > > >>> > > > >> > > > > > > > the
> > > > >>> > > > >> > > > > > > > >> > old
> > > > >>> > > > >> > > > > > > > >> > > > > topic
> > > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > start
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > receiving the new
> > > version.
> > > > >>> If
> > > > >>> > the
> > > > >>> > > > >> topic
> > > > >>> > > > >> > > name
> > > > >>> > > > >> > > > > were
> > > > >>> > > > >> > > > > > > to
> > > > >>> > > > >> > > > > > > > >> > change,
> > > > >>> > > > >> > > > > > > > >> > > > but
> > > > >>> > > > >> > > > > > > > >> > > > > > the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > ids
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > still matched, the
> > file
> > > > >>> would
> > > > >>> > also
> > > > >>> > > > >> need
> > > > >>> > > > >> > to
> > > > >>> > > > >> > > > > > update.
> > > > >>> > > > >> > > > > > > > Am I
> > > > >>> > > > >> > > > > > > > >> > > > missing
> > > > >>> > > > >> > > > > > > > >> > > > > a
> > > > >>> > > > >> > > > > > > > >> > > > > > >> case
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > where the file would
> > be
> > > > >>> correct
> > > > >>> > > and
> > > > >>> > > > >> not
> > > > >>> > > > >> > the
> > > > >>> > > > >> > > > > > > request?
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > Thanks,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > Justine
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > On Thu, Sep 17, 2020
> > at
> > > > >>> 3:18 PM
> > > > >>> > > Jun
> > > > >>> > > > >> Rao <
> > > > >>> > > > >> > > > > > > > >> jun@confluent.io
> > > > >>> > > > >> > > > > > > > >> > >
> > > > >>> > > > >> > > > > > > > >> > > > > wrote:
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Hi, Justine,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Thanks for the
> > reply.
> > > A
> > > > >>> couple
> > > > >>> > > of
> > > > >>> > > > >> more
> > > > >>> > > > >> > > > > comments
> > > > >>> > > > >> > > > > > > > >> below.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > 12. ListOffset and
> > > > >>> > > OffsetForLeader
> > > > >>> > > > >> > > > currently
> > > > >>> > > > >> > > > > > > don't
> > > > >>> > > > >> > > > > > > > >> > support
> > > > >>> > > > >> > > > > > > > >> > > > > > >> flexible
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > fields.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > So, we have to bump
> > up
> > > > the
> > > > >>> > > version
> > > > >>> > > > >> > number
> > > > >>> > > > >> > > > and
> > > > >>> > > > >> > > > > > use
> > > > >>> > > > >> > > > > > > > >> IBP at
> > > > >>> > > > >> > > > > > > > >> > > > least
> > > > >>> > > > >> > > > > > > > >> > > > > > for
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > these
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > two requests. Note
> > > that
> > > > it
> > > > >>> > seems
> > > > >>> > > > >> 2.7.0
> > > > >>> > > > >> > > will
> > > > >>> > > > >> > > > > > > require
> > > > >>> > > > >> > > > > > > > >> IBP
> > > > >>> > > > >> > > > > > > > >> > > > anyway
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > because
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > of
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > changes in
> > > KAFKA-10435.
> > > > >>> Also,
> > > > >>> > it
> > > > >>> > > > >> seems
> > > > >>> > > > >> > > that
> > > > >>> > > > >> > > > > the
> > > > >>> > > > >> > > > > > > > >> version
> > > > >>> > > > >> > > > > > > > >> > > for
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > LeaderAndIsrRequest
> > > and
> > > > >>> > > > StopReplica
> > > > >>> > > > >> are
> > > > >>> > > > >> > > > > bumped
> > > > >>> > > > >> > > > > > > even
> > > > >>> > > > >> > > > > > > > >> > though
> > > > >>> > > > >> > > > > > > > >> > > > we
> > > > >>> > > > >> > > > > > > > >> > > > > > only
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > added
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > a
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > tagged field. But
> > > since
> > > > >>> IBP is
> > > > >>> > > > >> needed
> > > > >>> > > > >> > > > anyway,
> > > > >>> > > > >> > > > > > we
> > > > >>> > > > >> > > > > > > > may
> > > > >>> > > > >> > > > > > > > >> > want
> > > > >>> > > > >> > > > > > > > >> > > to
> > > > >>> > > > >> > > > > > > > >> > > > > > >> revisit
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > overall tagged field
> > > > >>> choice.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > 13. The only
> > downside
> > > is
> > > > >>> the
> > > > >>> > > > >> potential
> > > > >>> > > > >> > > > > > confusion
> > > > >>> > > > >> > > > > > > on
> > > > >>> > > > >> > > > > > > > >> > which
> > > > >>> > > > >> > > > > > > > >> > > > one
> > > > >>> > > > >> > > > > > > > >> > > > > is
> > > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > source
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > of truth if they
> > don't
> > > > >>> match.
> > > > >>> > > > >> Another
> > > > >>> > > > >> > > > option
> > > > >>> > > > >> > > > > is
> > > > >>> > > > >> > > > > > > to
> > > > >>> > > > >> > > > > > > > >> > include
> > > > >>> > > > >> > > > > > > > >> > > > > those
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > fields
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > in
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the metadata file
> > when
> > > > we
> > > > >>> > > actually
> > > > >>> > > > >> > change
> > > > >>> > > > >> > > > the
> > > > >>> > > > >> > > > > > > > >> directory
> > > > >>> > > > >> > > > > > > > >> > > > > > structure.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Thanks,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Jun
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > On Thu, Sep 17, 2020
> > > at
> > > > >>> 2:01
> > > > >>> > PM
> > > > >>> > > > >> Justine
> > > > >>> > > > >> > > > > Olshan
> > > > >>> > > > >> > > > > > <
> > > > >>> > > > >> > > > > > > > >> > > > > > >> jolshan@confluent.io
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > wrote:
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > Hello all,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I've thought some
> > > more
> > > > >>> about
> > > > >>> > > > >> removing
> > > > >>> > > > >> > > the
> > > > >>> > > > >> > > > > > topic
> > > > >>> > > > >> > > > > > > > >> name
> > > > >>> > > > >> > > > > > > > >> > > field
> > > > >>> > > > >> > > > > > > > >> > > > > > from
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > some
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > of
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > requests. On
> > closer
> > > > >>> > inspection
> > > > >>> > > > of
> > > > >>> > > > >> the
> > > > >>> > > > >> > > > > > > > >> > > requests/responses,
> > > > >>> > > > >> > > > > > > > >> > > > it
> > > > >>> > > > >> > > > > > > > >> > > > > > >> seems
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > internal changes
> > > would
> > > > >>> be
> > > > >>> > much
> > > > >>> > > > >> larger
> > > > >>> > > > >> > > > than
> > > > >>> > > > >> > > > > I
> > > > >>> > > > >> > > > > > > > >> expected.
> > > > >>> > > > >> > > > > > > > >> > > > Some
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > protocols
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > involve clients,
> > so
> > > > they
> > > > >>> > would
> > > > >>> > > > >> > require
> > > > >>> > > > >> > > > > > changes
> > > > >>> > > > >> > > > > > > > too.
> > > > >>> > > > >> > > > > > > > >> > I'm
> > > > >>> > > > >> > > > > > > > >> > > > > > thinking
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > that
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > for
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > now, removing the
> > > > topic
> > > > >>> name
> > > > >>> > > > from
> > > > >>> > > > >> > these
> > > > >>> > > > >> > > > > > > requests
> > > > >>> > > > >> > > > > > > > >> and
> > > > >>> > > > >> > > > > > > > >> > > > > responses
> > > > >>> > > > >> > > > > > > > >> > > > > > >> are
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > out
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > scope.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I have decided to
> > > just
> > > > >>> keep
> > > > >>> > > the
> > > > >>> > > > >> > change
> > > > >>> > > > >> > > > > > > > >> > > > LeaderAndIsrResponse
> > > > >>> > > > >> > > > > > > > >> > > > > to
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > remove
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > topic name, and
> > have
> > > > >>> updated
> > > > >>> > > the
> > > > >>> > > > >> KIP
> > > > >>> > > > >> > to
> > > > >>> > > > >> > > > > > reflect
> > > > >>> > > > >> > > > > > > > >> this
> > > > >>> > > > >> > > > > > > > >> > > > > change. I
> > > > >>> > > > >> > > > > > > > >> > > > > > >> have
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > also
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > mentioned the
> > other
> > > > >>> requests
> > > > >>> > > and
> > > > >>> > > > >> > > > responses
> > > > >>> > > > >> > > > > in
> > > > >>> > > > >> > > > > > > > >> future
> > > > >>> > > > >> > > > > > > > >> > > work.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I'm hoping to
> > start
> > > > the
> > > > >>> > voting
> > > > >>> > > > >> > process
> > > > >>> > > > >> > > > > soon,
> > > > >>> > > > >> > > > > > so
> > > > >>> > > > >> > > > > > > > >> let me
> > > > >>> > > > >> > > > > > > > >> > > > know
> > > > >>> > > > >> > > > > > > > >> > > > > if
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > there
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > is
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > anything else we
> > > > should
> > > > >>> > > discuss.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > Thank you,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > Justine
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > On Tue, Sep 15,
> > 2020
> > > > at
> > > > >>> 3:57
> > > > >>> > > PM
> > > > >>> > > > >> > Justine
> > > > >>> > > > >> > > > > > Olshan
> > > > >>> > > > >> > > > > > > <
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > jolshan@confluent.io
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > wrote:
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > Hello again,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > To follow up on
> > > some
> > > > >>> of
> > > > >>> > the
> > > > >>> > > > >> other
> > > > >>> > > > >> > > > > comments:
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 10/11) We can
> > > remove
> > > > >>> the
> > > > >>> > > topic
> > > > >>> > > > >> name
> > > > >>> > > > >> > > > from
> > > > >>> > > > >> > > > > > > these
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > requests/responses,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > and
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > that means we
> > will
> > > > >>> just
> > > > >>> > have
> > > > >>> > > > to
> > > > >>> > > > >> > make
> > > > >>> > > > >> > > a
> > > > >>> > > > >> > > > > few
> > > > >>> > > > >> > > > > > > > >> internal
> > > > >>> > > > >> > > > > > > > >> > > > > changes
> > > > >>> > > > >> > > > > > > > >> > > > > > to
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > make
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > partitions
> > > > accessible
> > > > >>> by
> > > > >>> > > topic
> > > > >>> > > > >> id
> > > > >>> > > > >> > and
> > > > >>> > > > >> > > > > > > > partition.
> > > > >>> > > > >> > > > > > > > >> I
> > > > >>> > > > >> > > > > > > > >> > can
> > > > >>> > > > >> > > > > > > > >> > > > > > update
> > > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > KIP
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > remove them
> > unless
> > > > >>> anyone
> > > > >>> > > > thinks
> > > > >>> > > > >> > they
> > > > >>> > > > >> > > > > > should
> > > > >>> > > > >> > > > > > > > >> stay.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 12) Addressed in
> > > the
> > > > >>> > > previous
> > > > >>> > > > >> > email.
> > > > >>> > > > >> > > > I've
> > > > >>> > > > >> > > > > > > > updated
> > > > >>> > > > >> > > > > > > > >> > the
> > > > >>> > > > >> > > > > > > > >> > > > KIP
> > > > >>> > > > >> > > > > > > > >> > > > > to
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > include
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > tagged fields
> > for
> > > > the
> > > > >>> > > requests
> > > > >>> > > > >> and
> > > > >>> > > > >> > > > > > responses.
> > > > >>> > > > >> > > > > > > > >> (More
> > > > >>> > > > >> > > > > > > > >> > on
> > > > >>> > > > >> > > > > > > > >> > > > > that
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > below)
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 13) I think part
> > > of
> > > > >>> the
> > > > >>> > idea
> > > > >>> > > > for
> > > > >>> > > > >> > > > > including
> > > > >>> > > > >> > > > > > > this
> > > > >>> > > > >> > > > > > > > >> > > > > information
> > > > >>> > > > >> > > > > > > > >> > > > > > >> is to
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > prepare
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > for future
> > > changes.
> > > > >>> > Perhaps
> > > > >>> > > > the
> > > > >>> > > > >> > > > directory
> > > > >>> > > > >> > > > > > > > >> structure
> > > > >>> > > > >> > > > > > > > >> > > > might
> > > > >>> > > > >> > > > > > > > >> > > > > > >> change
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > from
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > >>> topicName_partitionNumber
> > > > >>> > to
> > > > >>> > > > >> > > something
> > > > >>> > > > >> > > > > like
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > topicID_partitionNumber.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Then
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > it would be
> > useful
> > > > to
> > > > >>> have
> > > > >>> > > the
> > > > >>> > > > >> > topic
> > > > >>> > > > >> > > > name
> > > > >>> > > > >> > > > > > in
> > > > >>> > > > >> > > > > > > > the
> > > > >>> > > > >> > > > > > > > >> > file
> > > > >>> > > > >> > > > > > > > >> > > > > since
> > > > >>> > > > >> > > > > > > > >> > > > > > it
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > would
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > not
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > be
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > in the directory
> > > > >>> > structure.
> > > > >>> > > > >> > > Supporting
> > > > >>> > > > >> > > > > > topic
> > > > >>> > > > >> > > > > > > > >> renames
> > > > >>> > > > >> > > > > > > > >> > > > might
> > > > >>> > > > >> > > > > > > > >> > > > > > be
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > easier
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > if
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > other fields are
> > > > >>> included.
> > > > >>> > > > Would
> > > > >>> > > > >> > > there
> > > > >>> > > > >> > > > be
> > > > >>> > > > >> > > > > > any
> > > > >>> > > > >> > > > > > > > >> > > downsides
> > > > >>> > > > >> > > > > > > > >> > > > to
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > including
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > this
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > information?
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 14)  Yes, we
> > would
> > > > >>> need to
> > > > >>> > > > copy
> > > > >>> > > > >> the
> > > > >>> > > > >> > > > > > partition
> > > > >>> > > > >> > > > > > > > >> > metadata
> > > > >>> > > > >> > > > > > > > >> > > > > file
> > > > >>> > > > >> > > > > > > > >> > > > > > in
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > this
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > process. I've
> > > > updated
> > > > >>> the
> > > > >>> > > KIP
> > > > >>> > > > to
> > > > >>> > > > >> > > > include
> > > > >>> > > > >> > > > > > > this.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 15) I believe
> > > Lucas
> > > > >>> meant
> > > > >>> > v1
> > > > >>> > > > >> and v2
> > > > >>> > > > >> > > > here.
> > > > >>> > > > >> > > > > > He
> > > > >>> > > > >> > > > > > > > was
> > > > >>> > > > >> > > > > > > > >> > > > referring
> > > > >>> > > > >> > > > > > > > >> > > > > > to
> > > > >>> > > > >> > > > > > > > >> > > > > > >> how
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > requests would
> > > fall
> > > > >>> under
> > > > >>> > > > >> different
> > > > >>> > > > >> > > IBP
> > > > >>> > > > >> > > > > and
> > > > >>> > > > >> > > > > > > > meant
> > > > >>> > > > >> > > > > > > > >> > that
> > > > >>> > > > >> > > > > > > > >> > > > > older
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > brokers
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > would
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > have to use the
> > > > older
> > > > >>> > > version
> > > > >>> > > > of
> > > > >>> > > > >> > the
> > > > >>> > > > >> > > > > > request
> > > > >>> > > > >> > > > > > > > and
> > > > >>> > > > >> > > > > > > > >> the
> > > > >>> > > > >> > > > > > > > >> > > > > > existing
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > topic
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > deletion
> > process.
> > > At
> > > > >>> > first,
> > > > >>> > > it
> > > > >>> > > > >> > seemed
> > > > >>> > > > >> > > > > like
> > > > >>> > > > >> > > > > > > > tagged
> > > > >>> > > > >> > > > > > > > >> > > fields
> > > > >>> > > > >> > > > > > > > >> > > > > > would
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > resolve
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > the IBP issue.
> > > > >>> However, we
> > > > >>> > > may
> > > > >>> > > > >> need
> > > > >>> > > > >> > > IBP
> > > > >>> > > > >> > > > > for
> > > > >>> > > > >> > > > > > > > this
> > > > >>> > > > >> > > > > > > > >> > > request
> > > > >>> > > > >> > > > > > > > >> > > > > > after
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > all
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > since
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > the controller
> > > > >>> handles the
> > > > >>> > > > topic
> > > > >>> > > > >> > > > deletion
> > > > >>> > > > >> > > > > > > > >> > differently
> > > > >>> > > > >> > > > > > > > >> > > > > > >> depending
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > on
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > IBP
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > version. In an
> > > older
> > > > >>> > > version,
> > > > >>> > > > we
> > > > >>> > > > >> > > can't
> > > > >>> > > > >> > > > > just
> > > > >>> > > > >> > > > > > > > send
> > > > >>> > > > >> > > > > > > > >> a
> > > > >>> > > > >> > > > > > > > >> > > > > > StopReplica
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > delete
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > topic
> > immediately
> > > > like
> > > > >>> > we'd
> > > > >>> > > > >> want to
> > > > >>> > > > >> > > for
> > > > >>> > > > >> > > > > > this
> > > > >>> > > > >> > > > > > > > KIP.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > This makes me
> > > wonder
> > > > >>> if we
> > > > >>> > > > want
> > > > >>> > > > >> > > tagged
> > > > >>> > > > >> > > > > > fields
> > > > >>> > > > >> > > > > > > > on
> > > > >>> > > > >> > > > > > > > >> all
> > > > >>> > > > >> > > > > > > > >> > > the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> requests
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > after
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > all. Let me know
> > > > your
> > > > >>> > > > thoughts!
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > Justine
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > On Tue, Sep 15,
> > > 2020
> > > > >>> at
> > > > >>> > 1:03
> > > > >>> > > > PM
> > > > >>> > > > >> > > Justine
> > > > >>> > > > >> > > > > > > Olshan
> > > > >>> > > > >> > > > > > > > <
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > jolshan@confluent.io
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > wrote:
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Hi all,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Jun brought up
> > a
> > > > good
> > > > >>> > point
> > > > >>> > > > in
> > > > >>> > > > >> his
> > > > >>> > > > >> > > > last
> > > > >>> > > > >> > > > > > > email
> > > > >>> > > > >> > > > > > > > >> about
> > > > >>> > > > >> > > > > > > > >> > > > > tagged
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > fields,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > and
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> I've updated
> > the
> > > > KIP
> > > > >>> to
> > > > >>> > > > reflect
> > > > >>> > > > >> > that
> > > > >>> > > > >> > > > the
> > > > >>> > > > >> > > > > > > > >> changes to
> > > > >>> > > > >> > > > > > > > >> > > > > > requests
> > > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > responses
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> will be in the
> > > form
> > > > >>> of
> > > > >>> > > tagged
> > > > >>> > > > >> > fields
> > > > >>> > > > >> > > > to
> > > > >>> > > > >> > > > > > > avoid
> > > > >>> > > > >> > > > > > > > >> > > changing
> > > > >>> > > > >> > > > > > > > >> > > > > IBP.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Jun: I plan on
> > > > >>> sending a
> > > > >>> > > > >> followup
> > > > >>> > > > >> > > > email
> > > > >>> > > > >> > > > > to
> > > > >>> > > > >> > > > > > > > >> address
> > > > >>> > > > >> > > > > > > > >> > > some
> > > > >>> > > > >> > > > > > > > >> > > > > of
> > > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > other
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> points.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Thanks,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Justine
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> On Mon, Sep 14,
> > > > 2020
> > > > >>> at
> > > > >>> > > 4:25
> > > > >>> > > > PM
> > > > >>> > > > >> > Jun
> > > > >>> > > > >> > > > Rao
> > > > >>> > > > >> > > > > <
> > > > >>> > > > >> > > > > > > > >> > > > > jun@confluent.io>
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > wrote:
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Hi, Justine,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Thanks for the
> > > > >>> updated
> > > > >>> > > KIP.
> > > > >>> > > > A
> > > > >>> > > > >> few
> > > > >>> > > > >> > > > > > comments
> > > > >>> > > > >> > > > > > > > >> below.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 10.
> > LeaderAndIsr
> > > > >>> > Response:
> > > > >>> > > > Do
> > > > >>> > > > >> we
> > > > >>> > > > >> > > need
> > > > >>> > > > >> > > > > the
> > > > >>> > > > >> > > > > > > > topic
> > > > >>> > > > >> > > > > > > > >> > > name?
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 11. For the
> > > > changed
> > > > >>> > > > >> > > request/response,
> > > > >>> > > > >> > > > > > other
> > > > >>> > > > >> > > > > > > > >> than
> > > > >>> > > > >> > > > > > > > >> > > > > > >> LeaderAndIsr,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > UpdateMetadata,
> > > > >>> > Metadata,
> > > > >>> > > do
> > > > >>> > > > >> we
> > > > >>> > > > >> > > need
> > > > >>> > > > >> > > > to
> > > > >>> > > > >> > > > > > > > include
> > > > >>> > > > >> > > > > > > > >> > the
> > > > >>> > > > >> > > > > > > > >> > > > > topic
> > > > >>> > > > >> > > > > > > > >> > > > > > >> name?
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 12. It seems
> > > that
> > > > >>> > upgrades
> > > > >>> > > > >> don't
> > > > >>> > > > >> > > > > require
> > > > >>> > > > >> > > > > > > IBP.
> > > > >>> > > > >> > > > > > > > >> Does
> > > > >>> > > > >> > > > > > > > >> > > > that
> > > > >>> > > > >> > > > > > > > >> > > > > > mean
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > new
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> fields
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> in all the
> > > > >>> > > request/response
> > > > >>> > > > >> are
> > > > >>> > > > >> > > added
> > > > >>> > > > >> > > > > as
> > > > >>> > > > >> > > > > > > > tagged
> > > > >>> > > > >> > > > > > > > >> > > fields
> > > > >>> > > > >> > > > > > > > >> > > > > > >> without
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > bumping
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > up
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> the request
> > > > >>> version? It
> > > > >>> > > > would
> > > > >>> > > > >> be
> > > > >>> > > > >> > > > useful
> > > > >>> > > > >> > > > > > to
> > > > >>> > > > >> > > > > > > > make
> > > > >>> > > > >> > > > > > > > >> > that
> > > > >>> > > > >> > > > > > > > >> > > > > > clear.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 13. Partition
> > > > >>> Metadata
> > > > >>> > > file:
> > > > >>> > > > >> Do
> > > > >>> > > > >> > we
> > > > >>> > > > >> > > > need
> > > > >>> > > > >> > > > > > to
> > > > >>> > > > >> > > > > > > > >> include
> > > > >>> > > > >> > > > > > > > >> > > the
> > > > >>> > > > >> > > > > > > > >> > > > > > topic
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > name
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > and
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> partition id
> > > since
> > > > >>> they
> > > > >>> > > are
> > > > >>> > > > >> > implied
> > > > >>> > > > >> > > > in
> > > > >>> > > > >> > > > > > the
> > > > >>> > > > >> > > > > > > > >> > directory
> > > > >>> > > > >> > > > > > > > >> > > > > name?
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 14. In the
> > JBOD
> > > > >>> mode, we
> > > > >>> > > > >> support
> > > > >>> > > > >> > > > > moving a
> > > > >>> > > > >> > > > > > > > >> > > partition's
> > > > >>> > > > >> > > > > > > > >> > > > > data
> > > > >>> > > > >> > > > > > > > >> > > > > > >> from
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > one
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > disk
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> to
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> another. Will
> > > the
> > > > >>> new
> > > > >>> > > > >> partition
> > > > >>> > > > >> > > > > metadata
> > > > >>> > > > >> > > > > > > file
> > > > >>> > > > >> > > > > > > > >> be
> > > > >>> > > > >> > > > > > > > >> > > > copied
> > > > >>> > > > >> > > > > > > > >> > > > > > >> during
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> process?
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 15. The KIP
> > says
> > > > >>> "Remove
> > > > >>> > > > >> deleted
> > > > >>> > > > >> > > > topics
> > > > >>> > > > >> > > > > > > from
> > > > >>> > > > >> > > > > > > > >> > > replicas
> > > > >>> > > > >> > > > > > > > >> > > > by
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > sending
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > StopReplicaRequest
> > > > >>> V2
> > > > >>> > for
> > > > >>> > > > any
> > > > >>> > > > >> > > topics
> > > > >>> > > > >> > > > > > which
> > > > >>> > > > >> > > > > > > do
> > > > >>> > > > >> > > > > > > > >> not
> > > > >>> > > > >> > > > > > > > >> > > > > contain
> > > > >>> > > > >> > > > > > > > >> > > > > > a
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > topic
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > ID,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > and
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> V3 for any
> > > topics
> > > > >>> which
> > > > >>> > do
> > > > >>> > > > >> > contain
> > > > >>> > > > >> > > a
> > > > >>> > > > >> > > > > > topic
> > > > >>> > > > >> > > > > > > > >> ID.".
> > > > >>> > > > >> > > > > > > > >> > > > > However,
> > > > >>> > > > >> > > > > > > > >> > > > > > it
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > seems
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> updated
> > > controller
> > > > >>> will
> > > > >>> > > > create
> > > > >>> > > > >> > all
> > > > >>> > > > >> > > > > > missing
> > > > >>> > > > >> > > > > > > > >> topic
> > > > >>> > > > >> > > > > > > > >> > IDs
> > > > >>> > > > >> > > > > > > > >> > > > > first
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > before
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > doing
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> other actions.
> > > So,
> > > > >>> is
> > > > >>> > > > >> > > > > StopReplicaRequest
> > > > >>> > > > >> > > > > > V2
> > > > >>> > > > >> > > > > > > > >> > needed?
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Jun
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> On Fri, Sep
> > 11,
> > > > >>> 2020 at
> > > > >>> > > > 10:31
> > > > >>> > > > >> AM
> > > > >>> > > > >> > > John
> > > > >>> > > > >> > > > > > > > Roesler <
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > vvcephei@apache.org
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> wrote:
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > Thanks,
> > > Justine!
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > Your
> > response
> > > > >>> seems
> > > > >>> > > > >> compelling
> > > > >>> > > > >> > to
> > > > >>> > > > >> > > > me.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > -John
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > On Fri,
> > > > >>> 2020-09-11 at
> > > > >>> > > > 10:17
> > > > >>> > > > >> > > -0700,
> > > > >>> > > > >> > > > > > > Justine
> > > > >>> > > > >> > > > > > > > >> > Olshan
> > > > >>> > > > >> > > > > > > > >> > > > > wrote:
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Hello all,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Thanks for
> > > > >>> > continuing
> > > > >>> > > > the
> > > > >>> > > > >> > > > > > discussion! I
> > > > >>> > > > >> > > > > > > > >> have a
> > > > >>> > > > >> > > > > > > > >> > > few
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > responses
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > to
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > your
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > points.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Tom: You
> > are
> > > > >>> correct
> > > > >>> > > in
> > > > >>> > > > >> that
> > > > >>> > > > >> > > this
> > > > >>> > > > >> > > > > KIP
> > > > >>> > > > >> > > > > > > has
> > > > >>> > > > >> > > > > > > > >> not
> > > > >>> > > > >> > > > > > > > >> > > > > > mentioned
> > > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > >>> > DeleteTopicsRequest. I
> > > > >>> > > > >> think
> > > > >>> > > > >> > > that
> > > > >>> > > > >> > > > > > this
> > > > >>> > > > >> > > > > > > > >> would
> > > > >>> > > > >> > > > > > > > >> > be
> > > > >>> > > > >> > > > > > > > >> > > > out
> > > > >>> > > > >> > > > > > > > >> > > > > of
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > scope
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > for
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> now, but
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > may be
> > > > something
> > > > >>> > worth
> > > > >>> > > > >> adding
> > > > >>> > > > >> > > in
> > > > >>> > > > >> > > > > the
> > > > >>> > > > >> > > > > > > > >> future.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > John: We
> > did
> > > > >>> > consider
> > > > >>> > > > >> > sequence
> > > > >>> > > > >> > > > ids,
> > > > >>> > > > >> > > > > > but
> > > > >>> > > > >> > > > > > > > >> there
> > > > >>> > > > >> > > > > > > > >> > > are
> > > > >>> > > > >> > > > > > > > >> > > > a
> > > > >>> > > > >> > > > > > > > >> > > > > > few
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > reasons
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> favor
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > UUIDs.
> > There
> > > > are
> > > > >>> > > several
> > > > >>> > > > >> > cases
> > > > >>> > > > >> > > > > where
> > > > >>> > > > >> > > > > > > > topics
> > > > >>> > > > >> > > > > > > > >> > from
> > > > >>> > > > >> > > > > > > > >> > > > > > >> different
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > clusters
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> may
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > interact
> > now
> > > > >>> and in
> > > > >>> > > the
> > > > >>> > > > >> > future.
> > > > >>> > > > >> > > > For
> > > > >>> > > > >> > > > > > > > >> example,
> > > > >>> > > > >> > > > > > > > >> > > > Mirror
> > > > >>> > > > >> > > > > > > > >> > > > > > >> Maker 2
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > may
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> benefit
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > from being
> > > > able
> > > > >>> to
> > > > >>> > > > detect
> > > > >>> > > > >> > when
> > > > >>> > > > >> > > a
> > > > >>> > > > >> > > > > > > cluster
> > > > >>> > > > >> > > > > > > > >> being
> > > > >>> > > > >> > > > > > > > >> > > > > > mirrored
> > > > >>> > > > >> > > > > > > > >> > > > > > >> is
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > deleted
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> and
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > recreated
> > > and
> > > > >>> > globally
> > > > >>> > > > >> unique
> > > > >>> > > > >> > > > > > > identifiers
> > > > >>> > > > >> > > > > > > > >> > would
> > > > >>> > > > >> > > > > > > > >> > > > make
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > resolving
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > issues
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > easier
> > than
> > > > >>> sequence
> > > > >>> > > IDs
> > > > >>> > > > >> > which
> > > > >>> > > > >> > > > may
> > > > >>> > > > >> > > > > > > > collide
> > > > >>> > > > >> > > > > > > > >> > > between
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > clusters.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > KIP-405
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > (tiered
> > > > storage)
> > > > >>> > will
> > > > >>> > > > also
> > > > >>> > > > >> > > > benefit
> > > > >>> > > > >> > > > > > from
> > > > >>> > > > >> > > > > > > > >> > globally
> > > > >>> > > > >> > > > > > > > >> > > > > > unique
> > > > >>> > > > >> > > > > > > > >> > > > > > >> IDs
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > as
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > shared
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > buckets
> > may
> > > be
> > > > >>> used
> > > > >>> > > > >> between
> > > > >>> > > > >> > > > > clusters.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Globally
> > > > unique
> > > > >>> IDs
> > > > >>> > > > would
> > > > >>> > > > >> > also
> > > > >>> > > > >> > > > make
> > > > >>> > > > >> > > > > > > > >> > > functionality
> > > > >>> > > > >> > > > > > > > >> > > > > like
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > moving
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > topics
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > between
> > > > >>> disparate
> > > > >>> > > > clusters
> > > > >>> > > > >> > > easier
> > > > >>> > > > >> > > > > in
> > > > >>> > > > >> > > > > > > the
> > > > >>> > > > >> > > > > > > > >> > future,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> simplify
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > any
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > future
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > implementations
> > > > >>> of
> > > > >>> > > > backups
> > > > >>> > > > >> > and
> > > > >>> > > > >> > > > > > > restores,
> > > > >>> > > > >> > > > > > > > >> and
> > > > >>> > > > >> > > > > > > > >> > > more.
> > > > >>> > > > >> > > > > > > > >> > > > > In
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > general,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> unique IDs
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > would
> > ensure
> > > > >>> that
> > > > >>> > the
> > > > >>> > > > >> source
> > > > >>> > > > >> > > > > cluster
> > > > >>> > > > >> > > > > > > > >> topics do
> > > > >>> > > > >> > > > > > > > >> > > not
> > > > >>> > > > >> > > > > > > > >> > > > > > >> conflict
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > with
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > destination
> > > > >>> cluster
> > > > >>> > > > >> topics.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > If we were
> > > to
> > > > >>> use
> > > > >>> > > > sequence
> > > > >>> > > > >> > ids,
> > > > >>> > > > >> > > > we
> > > > >>> > > > >> > > > > > > would
> > > > >>> > > > >> > > > > > > > >> need
> > > > >>> > > > >> > > > > > > > >> > > > > > >> sufficiently
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > large
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> cluster
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > ids to be
> > > > stored
> > > > >>> > with
> > > > >>> > > > the
> > > > >>> > > > >> > topic
> > > > >>> > > > >> > > > > > > > >> identifiers or
> > > > >>> > > > >> > > > > > > > >> > > we
> > > > >>> > > > >> > > > > > > > >> > > > > run
> > > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > risk
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > collisions.
> > > > This
> > > > >>> > will
> > > > >>> > > > >> give up
> > > > >>> > > > >> > > any
> > > > >>> > > > >> > > > > > > > >> advantage in
> > > > >>> > > > >> > > > > > > > >> > > > > > >> compactness
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> sequence
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > numbers
> > may
> > > > >>> bring.
> > > > >>> > > Given
> > > > >>> > > > >> > these
> > > > >>> > > > >> > > > > > > > advantages I
> > > > >>> > > > >> > > > > > > > >> > > think
> > > > >>> > > > >> > > > > > > > >> > > > it
> > > > >>> > > > >> > > > > > > > >> > > > > > >> makes
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > sense
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> use
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > UUIDs.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Gokul:
> > This
> > > is
> > > > >>> an
> > > > >>> > > > >> interesting
> > > > >>> > > > >> > > > idea,
> > > > >>> > > > >> > > > > > but
> > > > >>> > > > >> > > > > > > > >> this
> > > > >>> > > > >> > > > > > > > >> > is
> > > > >>> > > > >> > > > > > > > >> > > a
> > > > >>> > > > >> > > > > > > > >> > > > > > >> breaking
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > change.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Out of
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > scope for
> > > now,
> > > > >>> but
> > > > >>> > > maybe
> > > > >>> > > > >> > worth
> > > > >>> > > > >> > > > > > > discussing
> > > > >>> > > > >> > > > > > > > >> in
> > > > >>> > > > >> > > > > > > > >> > the
> > > > >>> > > > >> > > > > > > > >> > > > > > future.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Hope this
> > > > >>> explains
> > > > >>> > > some
> > > > >>> > > > of
> > > > >>> > > > >> > the
> > > > >>> > > > >> > > > > > > decisions,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Justine
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > On Fri,
> > Sep
> > > > 11,
> > > > >>> 2020
> > > > >>> > > at
> > > > >>> > > > >> 8:27
> > > > >>> > > > >> > AM
> > > > >>> > > > >> > > > > Gokul
> > > > >>> > > > >> > > > > > > > >> Ramanan
> > > > >>> > > > >> > > > > > > > >> > > > > > >> Subramanian <
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > >>> > gokul2411s@gmail.com>
> > > > >>> > > > >> wrote:
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Hi.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Thanks
> > for
> > > > the
> > > > >>> > KIP.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Have you
> > > > >>> thought
> > > > >>> > > about
> > > > >>> > > > >> > > whether
> > > > >>> > > > >> > > > it
> > > > >>> > > > >> > > > > > > makes
> > > > >>> > > > >> > > > > > > > >> > sense
> > > > >>> > > > >> > > > > > > > >> > > to
> > > > >>> > > > >> > > > > > > > >> > > > > > >> support
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> authorizing a
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > principal
> > > > for
> > > > >>> a
> > > > >>> > > topic
> > > > >>> > > > ID
> > > > >>> > > > >> > > rather
> > > > >>> > > > >> > > > > > than
> > > > >>> > > > >> > > > > > > a
> > > > >>> > > > >> > > > > > > > >> topic
> > > > >>> > > > >> > > > > > > > >> > > > name
> > > > >>> > > > >> > > > > > > > >> > > > > to
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > achieve
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> tighter
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > security?
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Or is
> > the
> > > > >>> topic ID
> > > > >>> > > > >> > > > fundamentally
> > > > >>> > > > >> > > > > an
> > > > >>> > > > >> > > > > > > > >> internal
> > > > >>> > > > >> > > > > > > > >> > > > > detail
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > similar
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > to
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> epochs
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > used
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > in a
> > bunch
> > > > of
> > > > >>> > other
> > > > >>> > > > >> places
> > > > >>> > > > >> > in
> > > > >>> > > > >> > > > > > Kafka?
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Thanks.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > On Fri,
> > > Sep
> > > > >>> 11,
> > > > >>> > 2020
> > > > >>> > > > at
> > > > >>> > > > >> > 4:06
> > > > >>> > > > >> > > PM
> > > > >>> > > > >> > > > > > John
> > > > >>> > > > >> > > > > > > > >> > Roesler <
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > vvcephei@apache.org
> > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > wrote:
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Hello
> > > > >>> Justine,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Thanks
> > > for
> > > > >>> the
> > > > >>> > > KIP!
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > I
> > happen
> > > > to
> > > > >>> have
> > > > >>> > > > been
> > > > >>> > > > >> > > > > confronted
> > > > >>> > > > >> > > > > > > > >> recently
> > > > >>> > > > >> > > > > > > > >> > > with
> > > > >>> > > > >> > > > > > > > >> > > > > the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> need
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > to
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > keep
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > track of
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > a
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > large
> > > > >>> number of
> > > > >>> > > > >> topics as
> > > > >>> > > > >> > > > > > compactly
> > > > >>> > > > >> > > > > > > > as
> > > > >>> > > > >> > > > > > > > >> > > > > possible. I
> > > > >>> > > > >> > > > > > > > >> > > > > > >> was
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > going
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> come
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > up
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > with
> > > some
> > > > >>> way to
> > > > >>> > > > >> > dictionary
> > > > >>> > > > >> > > > > > encode
> > > > >>> > > > >> > > > > > > > the
> > > > >>> > > > >> > > > > > > > >> > topic
> > > > >>> > > > >> > > > > > > > >> > > > > names
> > > > >>> > > > >> > > > > > > > >> > > > > > >> as
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > integers,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> but
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > this
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > seems
> > > much
> > > > >>> > better!
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > Apologies
> > > > if
> > > > >>> > this
> > > > >>> > > > has
> > > > >>> > > > >> > been
> > > > >>> > > > >> > > > > raised
> > > > >>> > > > >> > > > > > > > >> before,
> > > > >>> > > > >> > > > > > > > >> > > but
> > > > >>> > > > >> > > > > > > > >> > > > > I’m
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > wondering
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> about the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > choice
> > > of
> > > > >>> UUID
> > > > >>> > vs
> > > > >>> > > > >> > sequence
> > > > >>> > > > >> > > > > number
> > > > >>> > > > >> > > > > > > for
> > > > >>> > > > >> > > > > > > > >> the
> > > > >>> > > > >> > > > > > > > >> > > ids.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > Typically,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > I’ve
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> seen
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > UUIDs
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > in two
> > > > >>> > situations:
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 1.
> > When
> > > > >>> > processes
> > > > >>> > > > >> need to
> > > > >>> > > > >> > > > > > generate
> > > > >>> > > > >> > > > > > > > >> > > > non-colliding
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > identifiers
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> without
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > >>> coordination.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 2.
> > When
> > > > the
> > > > >>> > > > identifier
> > > > >>> > > > >> > > needs
> > > > >>> > > > >> > > > to
> > > > >>> > > > >> > > > > > be
> > > > >>> > > > >> > > > > > > > >> > > > “universally
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > unique”;
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > I.e.,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > identifier
> > > > >>> needs
> > > > >>> > > to
> > > > >>> > > > >> > > > distinguish
> > > > >>> > > > >> > > > > > the
> > > > >>> > > > >> > > > > > > > >> entity
> > > > >>> > > > >> > > > > > > > >> > > > from
> > > > >>> > > > >> > > > > > > > >> > > > > > all
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > other
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> entities
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > that
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > could
> > > ever
> > > > >>> > exist.
> > > > >>> > > > >> This is
> > > > >>> > > > >> > > > > useful
> > > > >>> > > > >> > > > > > in
> > > > >>> > > > >> > > > > > > > >> cases
> > > > >>> > > > >> > > > > > > > >> > > > where
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > entities
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > from
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > all
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > kinds
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > of
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > systems
> > > > get
> > > > >>> > mixed
> > > > >>> > > > >> > together,
> > > > >>> > > > >> > > > > such
> > > > >>> > > > >> > > > > > as
> > > > >>> > > > >> > > > > > > > >> when
> > > > >>> > > > >> > > > > > > > >> > > > dumping
> > > > >>> > > > >> > > > > > > > >> > > > > > >> logs
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > from
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > all
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > processes
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > in
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a
> > > company
> > > > >>> into a
> > > > >>> > > > >> common
> > > > >>> > > > >> > > > system.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Maybe
> > > I’m
> > > > >>> being
> > > > >>> > > > >> > > > short-sighted,
> > > > >>> > > > >> > > > > > but
> > > > >>> > > > >> > > > > > > it
> > > > >>> > > > >> > > > > > > > >> > > doesn’t
> > > > >>> > > > >> > > > > > > > >> > > > > seem
> > > > >>> > > > >> > > > > > > > >> > > > > > >> like
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > either
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> really
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > applies
> > > > >>> here. It
> > > > >>> > > > seems
> > > > >>> > > > >> > like
> > > > >>> > > > >> > > > the
> > > > >>> > > > >> > > > > > > > brokers
> > > > >>> > > > >> > > > > > > > >> > > could
> > > > >>> > > > >> > > > > > > > >> > > > > and
> > > > >>> > > > >> > > > > > > > >> > > > > > >> would
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > achieve
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > consensus
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > when
> > > > >>> creating a
> > > > >>> > > > topic
> > > > >>> > > > >> > > anyway,
> > > > >>> > > > >> > > > > > which
> > > > >>> > > > >> > > > > > > > is
> > > > >>> > > > >> > > > > > > > >> all
> > > > >>> > > > >> > > > > > > > >> > > > > that’s
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > required
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > to
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > generate
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > >>> non-colliding
> > > > >>> > > > sequence
> > > > >>> > > > >> > ids.
> > > > >>> > > > >> > > > For
> > > > >>> > > > >> > > > > > the
> > > > >>> > > > >> > > > > > > > >> > second,
> > > > >>> > > > >> > > > > > > > >> > > as
> > > > >>> > > > >> > > > > > > > >> > > > > you
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > mention,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > we
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> could
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > assign
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a UUID
> > > to
> > > > >>> the
> > > > >>> > > > cluster
> > > > >>> > > > >> as
> > > > >>> > > > >> > a
> > > > >>> > > > >> > > > > whole,
> > > > >>> > > > >> > > > > > > > which
> > > > >>> > > > >> > > > > > > > >> > > would
> > > > >>> > > > >> > > > > > > > >> > > > > > render
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > any
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > resource
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > scoped
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > the
> > > broker
> > > > >>> > > > universally
> > > > >>> > > > >> > > unique
> > > > >>> > > > >> > > > > as
> > > > >>> > > > >> > > > > > > > well.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > The
> > > > reason I
> > > > >>> > > mention
> > > > >>> > > > >> this
> > > > >>> > > > >> > > is
> > > > >>> > > > >> > > > > > that,
> > > > >>> > > > >> > > > > > > > >> > although
> > > > >>> > > > >> > > > > > > > >> > > a
> > > > >>> > > > >> > > > > > > > >> > > > > UUID
> > > > >>> > > > >> > > > > > > > >> > > > > > >> is
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > way
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > more
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > compact
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > than
> > > topic
> > > > >>> > names,
> > > > >>> > > > it’s
> > > > >>> > > > >> > > still
> > > > >>> > > > >> > > > 16
> > > > >>> > > > >> > > > > > > > bytes.
> > > > >>> > > > >> > > > > > > > >> In
> > > > >>> > > > >> > > > > > > > >> > > > > > contrast,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> a
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > 4-byte
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> integer
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > sequence
> > > > id
> > > > >>> > would
> > > > >>> > > > give
> > > > >>> > > > >> > us 4
> > > > >>> > > > >> > > > > > billion
> > > > >>> > > > >> > > > > > > > >> unique
> > > > >>> > > > >> > > > > > > > >> > > > > topics
> > > > >>> > > > >> > > > > > > > >> > > > > > >> per
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > cluster,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> which
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > seems
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > like
> > > > enough
> > > > >>> ;)
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > Considering
> > > > >>> the
> > > > >>> > > > >> number of
> > > > >>> > > > >> > > > > > different
> > > > >>> > > > >> > > > > > > > >> times
> > > > >>> > > > >> > > > > > > > >> > > > these
> > > > >>> > > > >> > > > > > > > >> > > > > > >> topic
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> identifiers
> > are
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > sent
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > over
> > the
> > > > >>> wire or
> > > > >>> > > > >> stored
> > > > >>> > > > >> > in
> > > > >>> > > > >> > > > > > memory,
> > > > >>> > > > >> > > > > > > it
> > > > >>> > > > >> > > > > > > > >> > seems
> > > > >>> > > > >> > > > > > > > >> > > > like
> > > > >>> > > > >> > > > > > > > >> > > > > > it
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > might
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > be
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> worth
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > additional
> > > > >>> 4x
> > > > >>> > > space
> > > > >>> > > > >> > > savings.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > What
> > do
> > > > you
> > > > >>> > think
> > > > >>> > > > >> about
> > > > >>> > > > >> > > this?
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > Thanks,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > John
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > On
> > Fri,
> > > > Sep
> > > > >>> 11,
> > > > >>> > > > 2020,
> > > > >>> > > > >> at
> > > > >>> > > > >> > > > 03:20,
> > > > >>> > > > >> > > > > > Tom
> > > > >>> > > > >> > > > > > > > >> > Bentley
> > > > >>> > > > >> > > > > > > > >> > > > > wrote:
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Hi
> > > > >>> Justine,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > This
> > > > looks
> > > > >>> > like
> > > > >>> > > a
> > > > >>> > > > >> very
> > > > >>> > > > >> > > > > welcome
> > > > >>> > > > >> > > > > > > > >> > > improvement.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> Thanks!
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > Maybe
> > > I
> > > > >>> missed
> > > > >>> > > it,
> > > > >>> > > > >> but
> > > > >>> > > > >> > > the
> > > > >>> > > > >> > > > > KIP
> > > > >>> > > > >> > > > > > > > >> doesn't
> > > > >>> > > > >> > > > > > > > >> > > seem
> > > > >>> > > > >> > > > > > > > >> > > > to
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > mention
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > changing
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > >>> > > > DeleteTopicsRequest
> > > > >>> > > > >> to
> > > > >>> > > > >> > > > > identify
> > > > >>> > > > >> > > > > > > the
> > > > >>> > > > >> > > > > > > > >> > topic
> > > > >>> > > > >> > > > > > > > >> > > > > using
> > > > >>> > > > >> > > > > > > > >> > > > > > an
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > id.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Maybe
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > that's out
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > of
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > scope,
> > > > but
> > > > >>> > > > >> > > > > DeleteTopicsRequest
> > > > >>> > > > >> > > > > > is
> > > > >>> > > > >> > > > > > > > not
> > > > >>> > > > >> > > > > > > > >> > > listed
> > > > >>> > > > >> > > > > > > > >> > > > > > among
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Future
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Work
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > APIs
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > either.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Kind
> > > > >>> regards,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Tom
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > On
> > > Thu,
> > > > >>> Sep
> > > > >>> > 10,
> > > > >>> > > > >> 2020 at
> > > > >>> > > > >> > > > 3:59
> > > > >>> > > > >> > > > > PM
> > > > >>> > > > >> > > > > > > > >> Satish
> > > > >>> > > > >> > > > > > > > >> > > > > Duggana <
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > >>> > > > >> satish.duggana@gmail.com
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > wrote:
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > Thanks
> > > > >>> > > > >> Lucas/Justine
> > > > >>> > > > >> > > for
> > > > >>> > > > >> > > > > the
> > > > >>> > > > >> > > > > > > nice
> > > > >>> > > > >> > > > > > > > >> KIP.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > It
> > > has
> > > > >>> > several
> > > > >>> > > > >> > benefits
> > > > >>> > > > >> > > > > which
> > > > >>> > > > >> > > > > > > > also
> > > > >>> > > > >> > > > > > > > >> > > include
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > simplifying
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> topic
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > deletion
> > > > >>> > > process
> > > > >>> > > > >> by
> > > > >>> > > > >> > > > > > controller
> > > > >>> > > > >> > > > > > > > and
> > > > >>> > > > >> > > > > > > > >> > logs
> > > > >>> > > > >> > > > > > > > >> > > > > > cleanup
> > > > >>> > > > >> > > > > > > > >> > > > > > >> by
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > brokers
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > in
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > corner
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > cases.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > Best,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > Satish.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > On
> > > > Wed,
> > > > >>> Sep
> > > > >>> > 9,
> > > > >>> > > > >> 2020
> > > > >>> > > > >> > at
> > > > >>> > > > >> > > > > 10:07
> > > > >>> > > > >> > > > > > PM
> > > > >>> > > > >> > > > > > > > >> > Justine
> > > > >>> > > > >> > > > > > > > >> > > > > > Olshan <
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > >>> jolshan@confluent.io
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > wrote:
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > > > Hello
> > > > >>> all,
> > > > >>> > > > it's
> > > > >>> > > > >> > been
> > > > >>> > > > >> > > > > > almost a
> > > > >>> > > > >> > > > > > > > >> year!
> > > > >>> > > > >> > > > > > > > >> > > I've
> > > > >>> > > > >> > > > > > > > >> > > > > > made
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > some
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > changes
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> to
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > this
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > KIP
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > and
> > > > >>> hope to
> > > > >>> > > > >> continue
> > > > >>> > > > >> > > the
> > > > >>> > > > >> > > > > > > > >> discussion.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > > One
> > > > >>> of the
> > > > >>> > > > main
> > > > >>> > > > >> > > changes
> > > > >>> > > > >> > > > > > I've
> > > > >>> > > > >> > > > > > > > >> added
> > > > >>> > > > >> > > > > > > > >> > is
> > > > >>> > > > >> > > > > > > > >> > > > now
> > > > >>> > > > >> > > > > > > > >> > > > > > the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > metadata
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> response
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > will
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > include
> > > > >>> the
> > > > >>> > > > topic
> > > > >>> > > > >> ID
> > > > >>> > > > >> > > (as
> > > > >>> > > > >> > > > > > Colin
> > > > >>> > > > >> > > > > > > > >> > > suggested).
> > > > >>> > > > >> > > > > > > > >> > > > > > >> Clients
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > can
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> obtain the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > topicID
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > of a
> > > > >>> given
> > > > >>> > > topic
> > > > >>> > > > >> > > through
> > > > >>> > > > >> > > > a
> > > > >>> > > > >> > > > > > > > >> > > > TopicDescription.
> > > > >>> > > > >> > > > > > > > >> > > > > > The
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > topicId
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > will
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > also be
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > included
> > > > >>> > with
> > > > >>> > > > the
> > > > >>> > > > >> > > > > > > UpdateMetadata
> > > > >>> > > > >> > > > > > > > >> > > request.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > > Let
> > > > me
> > > > >>> > know
> > > > >>> > > > what
> > > > >>> > > > >> > you
> > > > >>> > > > >> > > > all
> > > > >>> > > > >> > > > > > > think.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > > > Thank
> > > > >>> you,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > > > >>> Justine
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > On
> > > > >>> > > 2019/09/13
> > > > >>> > > > >> > > 16:38:26,
> > > > >>> > > > >> > > > > > > "Colin
> > > > >>> > > > >> > > > > > > > >> > > McCabe" <
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > cmccabe@apache.org
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > wrote:
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > Hi
> > > > >>> > Lucas,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > > >>> Thanks
> > > > >>> > for
> > > > >>> > > > >> > tackling
> > > > >>> > > > >> > > > > this.
> > > > >>> > > > >> > > > > > > > >> Topic
> > > > >>> > > > >> > > > > > > > >> > IDs
> > > > >>> > > > >> > > > > > > > >> > > > > are a
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > great
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > idea,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> and
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > this
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > is
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > really
> > > > >>> good
> > > > >>> > > > >> writeup.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > > For
> > > > >>> > > > >> > > > > > > /brokers/topics/[topic],
> > > > >>> > > > >> > > > > > > > >> the
> > > > >>> > > > >> > > > > > > > >> > > > schema
> > > > >>> > > > >> > > > > > > > >> > > > > > >> version
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > should
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > be
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > bumped
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > version
> > > > >>> 3,
> > > > >>> > > > rather
> > > > >>> > > > >> > than
> > > > >>> > > > >> > > 2.
> > > > >>> > > > >> > > > > > > > KIP-455
> > > > >>> > > > >> > > > > > > > >> > > bumped
> > > > >>> > > > >> > > > > > > > >> > > > > the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > version
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > this
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > znode
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 2
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > already
> > > > >>> :)
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > > >>> Given
> > > > >>> > that
> > > > >>> > > > >> we're
> > > > >>> > > > >> > > > going
> > > > >>> > > > >> > > > > to
> > > > >>> > > > >> > > > > > > be
> > > > >>> > > > >> > > > > > > > >> > seeing
> > > > >>> > > > >> > > > > > > > >> > > > > these
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > things
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > as
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> strings
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > as
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > lot
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > (in
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > logs,
> > > > in
> > > > >>> > > > >> ZooKeeper,
> > > > >>> > > > >> > on
> > > > >>> > > > >> > > > the
> > > > >>> > > > >> > > > > > > > >> > command-line,
> > > > >>> > > > >> > > > > > > > >> > > > > > etc.),
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > does
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > it
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > make
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > sense to
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > use
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > base64
> > > > >>> when
> > > > >>> > > > >> > converting
> > > > >>> > > > >> > > > them
> > > > >>> > > > >> > > > > > to
> > > > >>> > > > >> > > > > > > > >> > strings?
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > > >>> Here is
> > > > >>> > an
> > > > >>> > > > >> > example
> > > > >>> > > > >> > > of
> > > > >>> > > > >> > > > > the
> > > > >>> > > > >> > > > > > > hex
> > > > >>> > > > >> > > > > > > > >> > > > > > >> representation:
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > > >>> > > > >> > > > > > > > >> > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > > And
> > > > >>> here
> > > > >>> > > is
> > > > >>> > > > an
> > > > >>> > > > >> > > > example
> > > > >>> > > > >> > > > > in
> > > > >>> > > > >> > > > > > > > >> base64.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > > >>> > > > >> > > > b8tRS7h4TJ2Vt43Dp85v2A
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > > The
> > > > >>> > base64
> > > > >>> > > > >> > version
> > > > >>> > > > >> > > > > saves
> > > > >>> > > > >> > > > > > 15
> > > > >>> > > > >> > > > > > > > >> > letters
> > > > >>> > > > >> > > > > > > > >> > > > (to
> > > > >>> > > > >> > > > > > > > >> > > > > be
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > fair,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > 4
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> those
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > were
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > dashes
> > > > >>> that
> > > > >>> > we
> > > > >>> > > > >> could
> > > > >>> > > > >> > > have
> > > > >>> > > > >> > > > > > > elided
> > > > >>> > > > >> > > > > > > > in
> > > > >>> > > > >> > > > > > > > >> > the
> > > > >>> > > > >> > > > > > > > >> > > > hex
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > representation.)
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > > >>> Another
> > > > >>> > > > thing
> > > > >>> > > > >> to
> > > > >>> > > > >> > > > > consider
> > > > >>> > > > >> > > > > > > is
> > > > >>> > > > >> > > > > > > > >> that
> > > > >>> > > > >> > > > > > > > >> > we
> > > > >>> > > > >> > > > > > > > >> > > > > > should
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > specify
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > that
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > >>> all-zeroes
> > > > >>> > > UUID
> > > > >>> > > > is
> > > > >>> > > > >> > not
> > > > >>> > > > >> > > a
> > > > >>> > > > >> > > > > > valid
> > > > >>> > > > >> > > > > > > > >> topic
> > > > >>> > > > >> > > > > > > > >> > > UUID.
> > > > >>> > > > >> > > > > > > > >> > > > > >  We
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > can't
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > use
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> null
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > for
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > this
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > because
> > > > >>> we
> > > > >>> > > can't
> > > > >>> > > > >> > pass a
> > > > >>> > > > >> > > > > null
> > > > >>> > > > >> > > > > > > UUID
> > > > >>> > > > >> > > > > > > > >> over
> > > > >>> > > > >> > > > > > > > >> > > the
> > > > >>> > > > >> > > > > > > > >> > > > > RPC
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > protocol
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> (there
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > is no
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > special
> > > > >>> > > pattern
> > > > >>> > > > >> for
> > > > >>> > > > >> > > null,
> > > > >>> > > > >> > > > > nor
> > > > >>> > > > >> > > > > > > do
> > > > >>> > > > >> > > > > > > > we
> > > > >>> > > > >> > > > > > > > >> > want
> > > > >>> > > > >> > > > > > > > >> > > > to
> > > > >>> > > > >> > > > > > > > >> > > > > > >> waste
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > space
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> reserving
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > such
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > >>> pattern.)
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > > >>> Maybe I
> > > > >>> > > > missed
> > > > >>> > > > >> > it,
> > > > >>> > > > >> > > > but
> > > > >>> > > > >> > > > > > did
> > > > >>> > > > >> > > > > > > > you
> > > > >>> > > > >> > > > > > > > >> > > > describe
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > "migration
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > of...
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > existing
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > topic[s]
> > > > >>> > > without
> > > > >>> > > > >> > topic
> > > > >>> > > > >> > > > IDs"
> > > > >>> > > > >> > > > > > in
> > > > >>> > > > >> > > > > > > > >> detail
> > > > >>> > > > >> > > > > > > > >> > in
> > > > >>> > > > >> > > > > > > > >> > > > any
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > section?
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > It
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> seems
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > like
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > when
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > the
> > > > new
> > > > >>> > > > controller
> > > > >>> > > > >> > > > becomes
> > > > >>> > > > >> > > > > > > > active,
> > > > >>> > > > >> > > > > > > > >> it
> > > > >>> > > > >> > > > > > > > >> > > > should
> > > > >>> > > > >> > > > > > > > >> > > > > > >> just
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > generate
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> random
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > UUIDs
> > > for
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > these,
> > > > >>> and
> > > > >>> > > write
> > > > >>> > > > >> the
> > > > >>> > > > >> > > > random
> > > > >>> > > > >> > > > > > > UUIDs
> > > > >>> > > > >> > > > > > > > >> back
> > > > >>> > > > >> > > > > > > > >> > > to
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > ZooKeeper.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > It
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> would be
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > good
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > to
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > spell
> > > > >>> that
> > > > >>> > > out.
> > > > >>> > > > >> We
> > > > >>> > > > >> > > > should
> > > > >>> > > > >> > > > > > make
> > > > >>> > > > >> > > > > > > > it
> > > > >>> > > > >> > > > > > > > >> > clear
> > > > >>> > > > >> > > > > > > > >> > > > > that
> > > > >>> > > > >> > > > > > > > >> > > > > > >> this
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > happens
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > regardless
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > of
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > the
> > > > >>> > > inter-broker
> > > > >>> > > > >> > > protocol
> > > > >>> > > > >> > > > > > > version
> > > > >>> > > > >> > > > > > > > >> > (it's
> > > > >>> > > > >> > > > > > > > >> > > a
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > compatible
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > change).
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > > >>> > > > >> > > "LeaderAndIsrRequests
> > > > >>> > > > >> > > > > > > > >> including an
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > is_every_partition
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> flag"
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > seems a
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > bit
> > > > >>> wordy.
> > > > >>> > > Can
> > > > >>> > > > we
> > > > >>> > > > >> > just
> > > > >>> > > > >> > > > > call
> > > > >>> > > > >> > > > > > > > these
> > > > >>> > > > >> > > > > > > > >> > "full
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > >>> LeaderAndIsrRequests"?
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > Then
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > RPC
> > > > >>> field
> > > > >>> > > could
> > > > >>> > > > be
> > > > >>> > > > >> > > named
> > > > >>> > > > >> > > > > > > "full".
> > > > >>> > > > >> > > > > > > > >> > Also,
> > > > >>> > > > >> > > > > > > > >> > > it
> > > > >>> > > > >> > > > > > > > >> > > > > > would
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > probably
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > be
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > better
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > for
> > the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > RPC
> > > > >>> field to
> > > > >>> > > be
> > > > >>> > > > an
> > > > >>> > > > >> > enum
> > > > >>> > > > >> > > > of
> > > > >>> > > > >> > > > > {
> > > > >>> > > > >> > > > > > > > >> > > UNSPECIFIED,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > INCREMENTAL,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > FULL
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> }, so
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > that
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > we
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > can
> > > > >>> cleanly
> > > > >>> > > > handle
> > > > >>> > > > >> > old
> > > > >>> > > > >> > > > > > versions
> > > > >>> > > > >> > > > > > > > (by
> > > > >>> > > > >> > > > > > > > >> > > > treating
> > > > >>> > > > >> > > > > > > > >> > > > > > >> them
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > as
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> UNSPECIFIED)
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > In
> > > > >>> the
> > > > >>> > > > >> > > > > > LeaderAndIsrRequest
> > > > >>> > > > >> > > > > > > > >> > section,
> > > > >>> > > > >> > > > > > > > >> > > > you
> > > > >>> > > > >> > > > > > > > >> > > > > > >> write
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > "A
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > final
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > deletion
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > event
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > will
> > > > be
> > > > >>> > > > secheduled
> > > > >>> > > > >> > for
> > > > >>> > > > >> > > X
> > > > >>> > > > >> > > > ms
> > > > >>> > > > >> > > > > > > after
> > > > >>> > > > >> > > > > > > > >> the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > LeaderAndIsrRequest
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > was
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > first
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > >>> received..."
> > > > >>> > > I
> > > > >>> > > > >> guess
> > > > >>> > > > >> > > > the X
> > > > >>> > > > >> > > > > > > was a
> > > > >>> > > > >> > > > > > > > >> > > > > placeholder
> > > > >>> > > > >> > > > > > > > >> > > > > > >> that
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > you
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> intended
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > replace
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > before
> > > > >>> > > posting?
> > > > >>> > > > :)
> > > > >>> > > > >> > In
> > > > >>> > > > >> > > > any
> > > > >>> > > > >> > > > > > > case,
> > > > >>> > > > >> > > > > > > > >> this
> > > > >>> > > > >> > > > > > > > >> > > > seems
> > > > >>> > > > >> > > > > > > > >> > > > > > like
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > kind
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > of
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > thing
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > we'd
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > want a
> > > > >>> > > > >> configuration
> > > > >>> > > > >> > > for.
> > > > >>> > > > >> > > > > > > Let's
> > > > >>> > > > >> > > > > > > > >> > > describe
> > > > >>> > > > >> > > > > > > > >> > > > > that
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > configuration
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> key
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > somewhere
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > in
> > > > this
> > > > >>> KIP,
> > > > >>> > > > >> > including
> > > > >>> > > > >> > > > what
> > > > >>> > > > >> > > > > > its
> > > > >>> > > > >> > > > > > > > >> > default
> > > > >>> > > > >> > > > > > > > >> > > > > value
> > > > >>> > > > >> > > > > > > > >> > > > > > >> is.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > We
> > > > >>> > should
> > > > >>> > > > >> > probably
> > > > >>> > > > >> > > > also
> > > > >>> > > > >> > > > > > > log a
> > > > >>> > > > >> > > > > > > > >> > bunch
> > > > >>> > > > >> > > > > > > > >> > > of
> > > > >>> > > > >> > > > > > > > >> > > > > > >> messages
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > at
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > WARN
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> level
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > when
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > >>> something is
> > > > >>> > > > >> > scheduled
> > > > >>> > > > >> > > > for
> > > > >>> > > > >> > > > > > > > >> deletion,
> > > > >>> > > > >> > > > > > > > >> > as
> > > > >>> > > > >> > > > > > > > >> > > > > well.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > (Maybe
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > this
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> was
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > assumed,
> > > > but
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > it
> > > > >>> would be
> > > > >>> > > good
> > > > >>> > > > >> to
> > > > >>> > > > >> > > > mention
> > > > >>> > > > >> > > > > > > it).
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > I
> > > > >>> feel
> > > > >>> > > like
> > > > >>> > > > >> there
> > > > >>> > > > >> > > > are a
> > > > >>> > > > >> > > > > > few
> > > > >>> > > > >> > > > > > > > >> > sections
> > > > >>> > > > >> > > > > > > > >> > > > > that
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > should
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > be
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> moved to
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > "rejected
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > >>> > alternatives."
> > > > >>> > > > >> For
> > > > >>> > > > >> > > > > example,
> > > > >>> > > > >> > > > > > in
> > > > >>> > > > >> > > > > > > > the
> > > > >>> > > > >> > > > > > > > >> > > > > > DeleteTopics
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > section,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> since
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > we're
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > not
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > going
> > > > >>> to do
> > > > >>> > > > >> option 1
> > > > >>> > > > >> > or
> > > > >>> > > > >> > > > 2,
> > > > >>> > > > >> > > > > > > these
> > > > >>> > > > >> > > > > > > > >> > should
> > > > >>> > > > >> > > > > > > > >> > > be
> > > > >>> > > > >> > > > > > > > >> > > > > > moved
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > into
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> "rejected
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > >>> > alternatives,"
> > > > >>> > > > >> > rather
> > > > >>> > > > >> > > > than
> > > > >>> > > > >> > > > > > > > >> appearing
> > > > >>> > > > >> > > > > > > > >> > > > > inline.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Another
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > case
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> is
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > "Should
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > we
> > > > >>> remove
> > > > >>> > > topic
> > > > >>> > > > >> name
> > > > >>> > > > >> > > from
> > > > >>> > > > >> > > > > the
> > > > >>> > > > >> > > > > > > > >> protocol
> > > > >>> > > > >> > > > > > > > >> > > > where
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > possible"
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> section.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > This
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > is
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > clearly
> > > > >>> > > > >> discussing a
> > > > >>> > > > >> > > > design
> > > > >>> > > > >> > > > > > > > >> > alternative
> > > > >>> > > > >> > > > > > > > >> > > > that
> > > > >>> > > > >> > > > > > > > >> > > > > > >> we're
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > not
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> proposing
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > >>> implement:
> > > > >>> > > > >> removing
> > > > >>> > > > >> > the
> > > > >>> > > > >> > > > > topic
> > > > >>> > > > >> > > > > > > > name
> > > > >>> > > > >> > > > > > > > >> > from
> > > > >>> > > > >> > > > > > > > >> > > > > those
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > protocols.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > Is
> > > > >>> it
> > > > >>> > > really
> > > > >>> > > > >> > > > necessary
> > > > >>> > > > >> > > > > to
> > > > >>> > > > >> > > > > > > > have
> > > > >>> > > > >> > > > > > > > >> a
> > > > >>> > > > >> > > > > > > > >> > new
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > >>> > > /admin/delete_topics_by_id
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > path
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > in
> > > > >>> > ZooKeeper?
> > > > >>> > > > It
> > > > >>> > > > >> > seems
> > > > >>> > > > >> > > > > like
> > > > >>> > > > >> > > > > > we
> > > > >>> > > > >> > > > > > > > >> don't
> > > > >>> > > > >> > > > > > > > >> > > > really
> > > > >>> > > > >> > > > > > > > >> > > > > > >> need
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > this.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Whenever
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > there
> > is
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > a
> > > new
> > > > >>> > > > controller,
> > > > >>> > > > >> > we'll
> > > > >>> > > > >> > > > > send
> > > > >>> > > > >> > > > > > > out
> > > > >>> > > > >> > > > > > > > >> full
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > LeaderAndIsrRequests
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> which
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > will
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > trigger
> > > > >>> the
> > > > >>> > > > stale
> > > > >>> > > > >> > > topics
> > > > >>> > > > >> > > > to
> > > > >>> > > > >> > > > > > be
> > > > >>> > > > >> > > > > > > > >> cleaned
> > > > >>> > > > >> > > > > > > > >> > > up.
> > > > >>> > > > >> > > > > > > > >> > > > > >  The
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > active
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > controller
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > will
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > also
> > > > >>> send
> > > > >>> > the
> > > > >>> > > > full
> > > > >>> > > > >> > > > > > > > >> LeaderAndIsrRequest
> > > > >>> > > > >> > > > > > > > >> > > to
> > > > >>> > > > >> > > > > > > > >> > > > > > >> brokers
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > are
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> just
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > starting
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > up.
> > > > >>> So we
> > > > >>> > > > don't
> > > > >>> > > > >> > > really
> > > > >>> > > > >> > > > > > need
> > > > >>> > > > >> > > > > > > > this
> > > > >>> > > > >> > > > > > > > >> > kind
> > > > >>> > > > >> > > > > > > > >> > > > of
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > two-phase
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > commit
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > (send
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > out
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > >>> > > > >> StopReplicasRequest,
> > > > >>> > > > >> > > get
> > > > >>> > > > >> > > > > ACKs
> > > > >>> > > > >> > > > > > > > from
> > > > >>> > > > >> > > > > > > > >> all
> > > > >>> > > > >> > > > > > > > >> > > > > nodes,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > commit
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > by
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> removing
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > >>> > > > >> /admin/delete_topics
> > > > >>> > > > >> > > > node)
> > > > >>> > > > >> > > > > > any
> > > > >>> > > > >> > > > > > > > >> more.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > > You
> > > > >>> > > mention
> > > > >>> > > > >> that
> > > > >>> > > > >> > > > > > > FetchRequest
> > > > >>> > > > >> > > > > > > > >> will
> > > > >>> > > > >> > > > > > > > >> > > now
> > > > >>> > > > >> > > > > > > > >> > > > > > >> include
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > UUID
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> avoid
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > issues
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > where
> > > > >>> > requests
> > > > >>> > > > are
> > > > >>> > > > >> > made
> > > > >>> > > > >> > > > to
> > > > >>> > > > >> > > > > > > stale
> > > > >>> > > > >> > > > > > > > >> > > > partitions.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > However,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> adding a
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > UUID
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > >>> > > MetadataRequest
> > > > >>> > > > is
> > > > >>> > > > >> > > listed
> > > > >>> > > > >> > > > > as
> > > > >>> > > > >> > > > > > > > future
> > > > >>> > > > >> > > > > > > > >> > > work,
> > > > >>> > > > >> > > > > > > > >> > > > > out
> > > > >>> > > > >> > > > > > > > >> > > > > > of
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > scope
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > for
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> this
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > KIP.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > How
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > will
> > > > the
> > > > >>> > > client
> > > > >>> > > > >> learn
> > > > >>> > > > >> > > > what
> > > > >>> > > > >> > > > > > the
> > > > >>> > > > >> > > > > > > > >> topic
> > > > >>> > > > >> > > > > > > > >> > > UUID
> > > > >>> > > > >> > > > > > > > >> > > > > is,
> > > > >>> > > > >> > > > > > > > >> > > > > > if
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > metadata
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > response
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > doesn't
> > > > >>> > > include
> > > > >>> > > > >> that
> > > > >>> > > > >> > > > > > > information?
> > > > >>> > > > >> > > > > > > > >> It
> > > > >>> > > > >> > > > > > > > >> > > > seems
> > > > >>> > > > >> > > > > > > > >> > > > > > like
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > adding
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> UUID
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > >>> > > MetadataResponse
> > > > >>> > > > >> > would
> > > > >>> > > > >> > > be
> > > > >>> > > > >> > > > > an
> > > > >>> > > > >> > > > > > > > >> > improvement
> > > > >>> > > > >> > > > > > > > >> > > > > here
> > > > >>> > > > >> > > > > > > > >> > > > > > >> that
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > might
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > not
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> be
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > too
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > hard
> > to
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > make.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > > >>> best,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > > >>> Colin
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > On
> > > > >>> Mon,
> > > > >>> > > Sep
> > > > >>> > > > 9,
> > > > >>> > > > >> > > 2019,
> > > > >>> > > > >> > > > at
> > > > >>> > > > >> > > > > > > > 17:48,
> > > > >>> > > > >> > > > > > > > >> > > Ryanne
> > > > >>> > > > >> > > > > > > > >> > > > > > Dolan
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > wrote:
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > >
> > > > >>> Lucas,
> > > > >>> > > > this
> > > > >>> > > > >> > would
> > > > >>> > > > >> > > > be
> > > > >>> > > > >> > > > > > > great.
> > > > >>> > > > >> > > > > > > > >> I've
> > > > >>> > > > >> > > > > > > > >> > > run
> > > > >>> > > > >> > > > > > > > >> > > > > > into
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > issues
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > with
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > topics
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > being
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > >
> > > > >>> > > > resurrected
> > > > >>> > > > >> > > > > > accidentally,
> > > > >>> > > > >> > > > > > > > >> since
> > > > >>> > > > >> > > > > > > > >> > a
> > > > >>> > > > >> > > > > > > > >> > > > > client
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > cannot
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > easily
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > distinguish
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > between
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > >
> > > > a
> > > > >>> > > deleted
> > > > >>> > > > >> topic
> > > > >>> > > > >> > > > and a
> > > > >>> > > > >> > > > > > new
> > > > >>> > > > >> > > > > > > > >> topic
> > > > >>> > > > >> > > > > > > > >> > > with
> > > > >>> > > > >> > > > > > > > >> > > > > the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> same
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > name.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I'd
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > need
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > ID
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > >
> > > > >>> > > accessible
> > > > >>> > > > >> from
> > > > >>> > > > >> > > the
> > > > >>> > > > >> > > > > > > client
> > > > >>> > > > >> > > > > > > > to
> > > > >>> > > > >> > > > > > > > >> > > solve
> > > > >>> > > > >> > > > > > > > >> > > > > that
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > issue,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > but
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> this
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > is a
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > good
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > first
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > >
> > > > >>> step.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > >
> > > > >>> Ryanne
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > >
> > > > On
> > > > >>> > Wed,
> > > > >>> > > > Sep
> > > > >>> > > > >> 4,
> > > > >>> > > > >> > > 2019
> > > > >>> > > > >> > > > > at
> > > > >>> > > > >> > > > > > > 1:41
> > > > >>> > > > >> > > > > > > > >> PM
> > > > >>> > > > >> > > > > > > > >> > > Lucas
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > Bradstreet <
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > >>> > > lucas@confluent.io>
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > wrote:
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > >
> > > > >
> > > > >>> Hi
> > > > >>> > > all,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > >
> > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > >
> > > > >
> > > > >>> I
> > > > >>> > > would
> > > > >>> > > > >> like
> > > > >>> > > > >> > to
> > > > >>> > > > >> > > > > kick
> > > > >>> > > > >> > > > > > > off
> > > > >>> > > > >> > > > > > > > >> > > > discussion
> > > > >>> > > > >> > > > > > > > >> > > > > of
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > KIP-516,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > an
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > >>> implementation
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > of
> > > > topic
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > >
> > > > >
> > > > >>> IDs
> > > > >>> > > for
> > > > >>> > > > >> > Kafka.
> > > > >>> > > > >> > > > > Topic
> > > > >>> > > > >> > > > > > > IDs
> > > > >>> > > > >> > > > > > > > >> aim
> > > > >>> > > > >> > > > > > > > >> > to
> > > > >>> > > > >> > > > > > > > >> > > > > solve
> > > > >>> > > > >> > > > > > > > >> > > > > > >> topic
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> uniqueness
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > problems
> > > > in
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > Kafka,
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > >
> > > > >
> > > > >>> > where
> > > > >>> > > > >> > referring
> > > > >>> > > > >> > > > to
> > > > >>> > > > >> > > > > a
> > > > >>> > > > >> > > > > > > > topic
> > > > >>> > > > >> > > > > > > > >> by
> > > > >>> > > > >> > > > > > > > >> > > name
> > > > >>> > > > >> > > > > > > > >> > > > > > >> alone is
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> insufficient.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Such
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > cases
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > >
> > > > >
> > > > >>> > > include
> > > > >>> > > > >> when
> > > > >>> > > > >> > a
> > > > >>> > > > >> > > > > topic
> > > > >>> > > > >> > > > > > > has
> > > > >>> > > > >> > > > > > > > >> been
> > > > >>> > > > >> > > > > > > > >> > > > > deleted
> > > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > recreated
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> with
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > the
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > same
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > name.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > >
> > > > >
> > > > >>> > Unique
> > > > >>> > > > >> > > > identifiers
> > > > >>> > > > >> > > > > > will
> > > > >>> > > > >> > > > > > > > >> help
> > > > >>> > > > >> > > > > > > > >> > > > > simplify
> > > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > improve
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Kafka's
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > topic
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > deletion
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > >
> > > > >
> > > > >>> > > process,
> > > > >>> > > > >> as
> > > > >>> > > > >> > > well
> > > > >>> > > > >> > > > as
> > > > >>> > > > >> > > > > > > > prevent
> > > > >>> > > > >> > > > > > > > >> > > cases
> > > > >>> > > > >> > > > > > > > >> > > > > > where
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > brokers
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > may
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > incorrectly
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > interact
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > >
> > > > >
> > > > >>> with
> > > > >>> > > > stale
> > > > >>> > > > >> > > > versions
> > > > >>> > > > >> > > > > > of
> > > > >>> > > > >> > > > > > > > >> topics.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > >
> > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > >
> > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > >
> > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > >>> > > > >> > > > > > > > >> > > > >
> > > > >>> > > > >> > > > > > > > >> > > >
> > > > >>> > > > >> > > > > > > > >> > >
> > > > >>> > > > >> > > > > > > > >> >
> > > > >>> > > > >> > > > > > > > >>
> > > > >>> > > > >> > > > > > > >
> > > > >>> > > > >> > > > > > >
> > > > >>> > > > >> > > > > >
> > > > >>> > > > >> > > > >
> > > > >>> > > > >> > > >
> > > > >>> > > > >> > >
> > > > >>> > > > >> >
> > > > >>> > > > >>
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > >
> > > > >
> > > > >>> > > Looking
> > > > >>> > > > >> > forward
> > > > >>> > > > >> > > > to
> > > > >>> > > > >> > > > > > your
> > > > >>> > > > >> > > > > > > > >> > > thoughts.
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > >
> > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > >
> > > > >
> > > > >>> > Lucas
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >
> > > >
> > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > > >>> > > > >> > > > > > > > >> > > > > > >
> > > > >>> > > > >> > > > > > > > >> > > > > >
> > > > >>> > > > >> > > > > > > > >> > > > >
> > > > >>> > > > >> > > > > > > > >> > > >
> > > > >>> > > > >> > > > > > > > >> > >
> > > > >>> > > > >> > > > > > > > >> >
> > > > >>> > > > >> > > > > > > > >>
> > > > >>> > > > >> > > > > > > > >
> > > > >>> > > > >> > > > > > > >
> > > > >>> > > > >> > > > > > >
> > > > >>> > > > >> > > > > >
> > > > >>> > > > >> > > > >
> > > > >>> > > > >> > > >
> > > > >>> > > > >> > >
> > > > >>> > > > >> >
> > > > >>> > > > >>
> > > > >>> > > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > > >>
> > > >
> > >
> >

Re: [DISCUSS] KIP-516: Topic Identifiers

[Jun Rao <ju...@confluent.io>]

Hi, Justine,

I guess we have to maintain the mapping between topic name and topicId in
both the client and the server. If TopicIdPartition is meant for internal
usage only, we can create it under a package
like org/apache/kafka/common/internal and we don't need to document it in
the KIP. If it's meant for external public usage (and potentially internal
usage too), we can create it in org/apache/kafka/common and need to
document it in the KIP.

Since topicId is now part of the topic metadata, perhaps it's useful to
expose it in TopicMetadataAndConfig too.

Thanks,

Jun

On Mon, Oct 5, 2020 at 3:35 PM Justine Olshan <jo...@confluent.io> wrote:

> Hi Jun,
>
> This KIP removes the topic name from many requests. A lot of requests keep
> track of the partitions internally with a TopicPartition. So when we remove
> the topic name, we can not differentiate unless we can also use topic ID.
> The Fetch request relies heavily on this, but ListOffset,
> OffsetsForLeaderEpoch, etc all use TopicPartition as a key value for
> internal maps. Digging deeper into Fetch, I've noticed that the code path
> runs into some public APIs that use the TopicPartition. Some examples are
> Cluster.java or RecordTooLargeException.
>
> A related issue is that in some places like TopicAuthorizationException,
> the topic name is required and without the name in the request, we won't
> know it.
>
> I guess an alternative would be to have some mapping from topic name to
> topic ID, but I would have to think about that more.
>
> The KIP mentions exposing TopicId in TopicDescription. (The font was
> weirdly colored, so I've made it easier to read.)
>
> Would the idea for the TopicMetadataAndConfig be that the topic ID would be
> returned as a result?
> As for the other methods describe/deletetopics, it makes sense to include
> them at some point, but I'm not sure how far we should go in terms of
> supporting topic IDs everywhere.
>
> Thank you for continuing to look at the KIP!
> Justine
>
>
> On Mon, Oct 5, 2020 at 2:53 PM Jun Rao <ju...@confluent.io> wrote:
>
> > Hi, Justine,
> >
> > Adding a TopicIdPartition in the public interface may be useful. Where
> > would you expect it to be used?
> >
> > Related to this, since we are exposing topicId in the tool, it seems that
> > we may want to expose topicId in at least TopicDescription used by
> > AdminClient.describeTopics() and TopicMetadataAndConfig used in
> > AdminClient.createTopics(Collection<NewTopic> newTopics). I am also not
> > sure if we plan to expose topicId in other AdminClient methods in this
> KIP
> > or in some future KIP. For example, it might be useful to support topicId
> > in the following methods.
> >
> >     DescribeTopicsResult describeTopics(Collection<String> topicNames,
> > DescribeTopicsOptions options);
> >     DeleteTopicsResult deleteTopics(Collection<String> topics,
> > DeleteTopicsOptions options);
> >
> > Thanks,
> >
> > Jun
> >
> > On Mon, Oct 5, 2020 at 11:03 AM Justine Olshan <jo...@confluent.io>
> > wrote:
> >
> > > Hi all,
> > >
> > > I was thinking about this KIP some more. We are removing the topic name
> > > from many of these request types, and some of them (eg. Fetch) use
> > > TopicPartition as keys in a map when handling the request. I was
> thinking
> > > it would make sense to make some sort of TopicIdPartition with topic
> > name,
> > > topic id, and partition fields (name and id would potentially be
> optional
> > > types since different request versions will contain only one of the
> two)
> > I
> > > was wondering if it made sense to include this as a public facing API
> > > similar to TopicPartition. If so, I will add this class to the KIP as
> > well.
> > >
> > > Thanks,
> > > Justine
> > >
> > > On Mon, Oct 5, 2020 at 8:52 AM Justine Olshan <jo...@confluent.io>
> > > wrote:
> > >
> > > > Hi all,
> > > > I've made a few minor changes to the UUID class. I've added a few
> more
> > > > methods to support UUIDs more generally, by including a constructor
> > and a
> > > > method to generate random UUIDs. Both of these will be useful for
> > > testing.
> > > >
> > > > Let me know if there are any other areas for discussion.
> > > >
> > > > Thank you,
> > > > Justine
> > > >
> > > > On Thu, Oct 1, 2020 at 5:29 PM Justine Olshan <jo...@confluent.io>
> > > > wrote:
> > > >
> > > >> Hi Jun,
> > > >> Thanks for looking at it again. Here's the new spec. (I fixed the
> typo
> > > in
> > > >> it too.)
> > > >> {"name": "id", "type": "string", "doc": option[UUID]}
> > > >>
> > > >> Justine
> > > >>
> > > >>
> > > >> On Thu, Oct 1, 2020 at 5:03 PM Jun Rao <ju...@confluent.io> wrote:
> > > >>
> > > >>> Hi, Justine,
> > > >>>
> > > >>> Thanks for the update. The KIP looks good to me now. Just a minor
> > > comment
> > > >>> below.
> > > >>>
> > > >>> 30. Perhaps "option[UUID]" can be put in the doc.
> > > >>>
> > > >>> Jun
> > > >>>
> > > >>> On Thu, Oct 1, 2020 at 3:28 PM Justine Olshan <
> jolshan@confluent.io>
> > > >>> wrote:
> > > >>>
> > > >>> > Hi Jun,
> > > >>> > Thanks for the response!
> > > >>> >
> > > >>> > 30. I think I might have changed this in between. The current
> > version
> > > >>> > says:  {"name":
> > > >>> > "id", "type": "option[UUID]"}, "doc": topic id}
> > > >>> > I have switched to the option type to cover the migration case
> > where
> > > a
> > > >>> > TopicZNode does not yet have a topic ID.
> > > >>> > I understand that due to json, this field is written as a string,
> > so
> > > >>> if I
> > > >>> > should move the "option[uuid]" to the doc field and the type
> should
> > > be
> > > >>> > "string" please let me know.
> > > >>> >
> > > >>> > 40. I've added a definition for UUID.
> > > >>> > 41,42. Fixed
> > > >>> >
> > > >>> > Thank you,
> > > >>> > Justine
> > > >>> >
> > > >>> > On Wed, Sep 30, 2020 at 1:15 PM Jun Rao <ju...@confluent.io>
> wrote:
> > > >>> >
> > > >>> > > Hi, Justine,
> > > >>> > >
> > > >>> > > Thanks for the summary. Just a few minor comments blow.
> > > >>> > >
> > > >>> > > 30.  {"name": "id", "type": "string", "doc": "version id"}}:
> The
> > > doc
> > > >>> > should
> > > >>> > > say UUID. The issue seems unfixed.
> > > >>> > >
> > > >>> > > 40. Since UUID is public facing, could you include its
> > definition?
> > > >>> > >
> > > >>> > > 41. StopReplicaResponse still includes the topic field.
> > > >>> > >
> > > >>> > > 42. "It is unnecessary to include the name of the topic in the
> > > >>> following
> > > >>> > > Request/Response calls" It would be useful to include all
> > modified
> > > >>> > requests
> > > >>> > > (e.g. produce) in the list below.
> > > >>> > >
> > > >>> > > Thanks,
> > > >>> > >
> > > >>> > > Jun
> > > >>> > >
> > > >>> > >
> > > >>> > > On Wed, Sep 30, 2020 at 10:17 AM Justine Olshan <
> > > >>> jolshan@confluent.io>
> > > >>> > > wrote:
> > > >>> > >
> > > >>> > > > Hello again,
> > > >>> > > >
> > > >>> > > > I've taken some time to discuss some of the remaining points
> > > >>> brought up
> > > >>> > > by
> > > >>> > > > the previous emails offline. Here are some of the
> conclusions.
> > > >>> > > >
> > > >>> > > > 1. Directory Structure:
> > > >>> > > > There was some talk about whether the directory structure
> > should
> > > be
> > > >>> > > changed
> > > >>> > > > to replace all topic names with topic IDs.
> > > >>> > > > This will be a useful change, but will prevent downgrades. It
> > > will
> > > >>> be
> > > >>> > > best
> > > >>> > > > to wait until a major release, likely alongside KIP-500
> changes
> > > >>> that
> > > >>> > will
> > > >>> > > > prevent downgrades. I've updated the KIP to include this
> change
> > > >>> with
> > > >>> > the
> > > >>> > > > note about migration and deprecation.
> > > >>> > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-log.dirlayout
> > > >>> > > >
> > > >>> > > > 2. Partition Metadata file
> > > >>> > > > There was some disagreement about the inclusion of the
> > partition
> > > >>> > metadata
> > > >>> > > > file.
> > > >>> > > > This file will be needed to persist the topic ID, especially
> > > while
> > > >>> we
> > > >>> > > still
> > > >>> > > > have the old directory structure. Even after the changes,
> this
> > > >>> file can
> > > >>> > > be
> > > >>> > > > useful for debugging and recovery.
> > > >>> > > > Creating a single mapping file from topic names to topic IDs
> > was
> > > >>> > > > considered, but ultimately scrapped as it would not be as
> easy
> > to
> > > >>> > > maintain.
> > > >>> > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PartitionMetadatafile
> > > >>> > > >
> > > >>> > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PersistingTopicIDs
> > > >>> > > >
> > > >>> > > > 3. Produce Protocols
> > > >>> > > > After some further discussion, this replacing the topic name
> > with
> > > >>> topic
> > > >>> > > ID
> > > >>> > > > in these protocols has been added to the KIP.
> > > >>> > > > This will cut down on the size of the protocol. Since changes
> > to
> > > >>> fetch
> > > >>> > > are
> > > >>> > > > included, it does make sense to update these protocols.
> > > >>> > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-Produce
> > > >>> > > >
> > > >>> > > > 4. KIP-500 Compatibility
> > > >>> > > > After some discussion with those involved with KIP-500, it
> > seems
> > > >>> best
> > > >>> > to
> > > >>> > > > use a sentinel topic ID for the metadata topic that is used
> > > before
> > > >>> the
> > > >>> > > > first controller election.
> > > >>> > > > We had an issue where this metadata topic may not be assigned
> > an
> > > ID
> > > >>> > > before
> > > >>> > > > utilizing the Vote and Fetch protocols. It was decided to
> > > reserve a
> > > >>> > > unique
> > > >>> > > > ID for this topic to be used until the controller could give
> > the
> > > >>> topic
> > > >>> > a
> > > >>> > > > unique ID.
> > > >>> > > > Having the topic keep the sentinel ID (not be reassigned to a
> > > >>> unique
> > > >>> > ID)
> > > >>> > > > was considered, but it seemed like a good idea to leave the
> > > option
> > > >>> open
> > > >>> > > for
> > > >>> > > > the metadata topic to have a unique ID in cases where it
> would
> > > >>> need to
> > > >>> > be
> > > >>> > > > differentiated from other clusters' metadata topics. (ex.
> > tiered
> > > >>> > storage)
> > > >>> > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-CompatibilitywithKIP-500
> > > >>> > > >
> > > >>> > > > I've also split up the KIP into sub-tasks on the JIRA.
> > Hopefully
> > > >>> this
> > > >>> > > will
> > > >>> > > > provide a better idea about what tasks we have, and
> eventually
> > > >>> provide
> > > >>> > a
> > > >>> > > > place to see what's done and what is left.
> > > >>> > > > If there is a task I am missing, please let me know!
> > > >>> > > > https://issues.apache.org/jira/browse/KAFKA-8872
> > > >>> > > >
> > > >>> > > > Of course, these decisions are not set in stone, and I would
> > love
> > > >>> to
> > > >>> > hear
> > > >>> > > > any feedback.
> > > >>> > > >
> > > >>> > > > Thanks,
> > > >>> > > > Justine
> > > >>> > > >
> > > >>> > > > On Mon, Sep 28, 2020 at 11:38 AM Justine Olshan <
> > > >>> jolshan@confluent.io>
> > > >>> > > > wrote:
> > > >>> > > >
> > > >>> > > > > Hello all,
> > > >>> > > > >
> > > >>> > > > > I just wanted to follow up on this discussion. Did we come
> to
> > > an
> > > >>> > > > > understanding about the directory structure?
> > > >>> > > > >
> > > >>> > > > > I think the biggest question here is what is acceptable to
> > > leave
> > > >>> out
> > > >>> > > due
> > > >>> > > > > to scope vs. what is considered to be too much tech debt.
> > > >>> > > > > This KIP is already pretty large with the number of
> changes,
> > > but
> > > >>> it
> > > >>> > > also
> > > >>> > > > > makes sense to do things correctly, so I'd love to hear
> > > >>> everyone's
> > > >>> > > > thoughts.
> > > >>> > > > >
> > > >>> > > > > Thanks,
> > > >>> > > > > Justine
> > > >>> > > > >
> > > >>> > > > > On Fri, Sep 25, 2020 at 8:19 AM Lucas Bradstreet <
> > > >>> lucas@confluent.io
> > > >>> > >
> > > >>> > > > > wrote:
> > > >>> > > > >
> > > >>> > > > >> Hi Ismael,
> > > >>> > > > >>
> > > >>> > > > >> If you do not store it in a metadata file or in the
> > directory
> > > >>> > > structure
> > > >>> > > > >> would you then
> > > >>> > > > >> require the LeaderAndIsrRequest following startup to give
> > you
> > > >>> some
> > > >>> > > > notion
> > > >>> > > > >> of
> > > >>> > > > >> topic name in memory? We would still need this information
> > for
> > > >>> the
> > > >>> > > older
> > > >>> > > > >> protocols, but
> > > >>> > > > >> perhaps this is what's meant by tech debt.
> > > >>> > > > >>
> > > >>> > > > >> Once we're free of the old non-topicID requests then I
> think
> > > you
> > > >>> > > > wouldn't
> > > >>> > > > >> need to retain the topic name.
> > > >>> > > > >> I think the ability to easily look up topic names
> associated
> > > >>> with
> > > >>> > > > >> partition
> > > >>> > > > >> directories would still be missed
> > > >>> > > > >> when diagnosing issues, though maybe it wouldn't be a deal
> > > >>> breaker
> > > >>> > > with
> > > >>> > > > >> the
> > > >>> > > > >> right tooling.
> > > >>> > > > >>
> > > >>> > > > >> Thanks,
> > > >>> > > > >>
> > > >>> > > > >> Lucas
> > > >>> > > > >>
> > > >>> > > > >> On Fri, Sep 25, 2020 at 7:55 AM Ismael Juma <
> > > ismael@juma.me.uk>
> > > >>> > > wrote:
> > > >>> > > > >>
> > > >>> > > > >> > Hi Lucas,
> > > >>> > > > >> >
> > > >>> > > > >> > Why would you include the name and id? I think you'd
> want
> > to
> > > >>> > remove
> > > >>> > > > the
> > > >>> > > > >> > name from the directory name right? Jason's suggestion
> was
> > > >>> that if
> > > >>> > > you
> > > >>> > > > >> > remove the name from the directory, then why would you
> > need
> > > >>> the id
> > > >>> > > > name
> > > >>> > > > >> > mapping file?
> > > >>> > > > >> >
> > > >>> > > > >> > Ismael
> > > >>> > > > >> >
> > > >>> > > > >> > On Thu, Sep 24, 2020 at 4:24 PM Lucas Bradstreet <
> > > >>> > > lucas@confluent.io>
> > > >>> > > > >> > wrote:
> > > >>> > > > >> >
> > > >>> > > > >> > > > 2. Part of the usage of the file is to have
> persistent
> > > >>> storage
> > > >>> > > of
> > > >>> > > > >> the
> > > >>> > > > >> > > topic
> > > >>> > > > >> > > ID and use it to compare with the ID supplied in the
> > > >>> > LeaderAndIsr
> > > >>> > > > >> > Request.
> > > >>> > > > >> > > There is some discussion in the KIP about changes to
> the
> > > >>> > directory
> > > >>> > > > >> > > structure, but I believe directory changes were
> > considered
> > > >>> to be
> > > >>> > > out
> > > >>> > > > >> of
> > > >>> > > > >> > > scope when the KIP was written.
> > > >>> > > > >> > >
> > > >>> > > > >> > >
> > > >>> > > > >> > > Yeah, I was hoping to get a better understanding of
> why
> > it
> > > >>> was
> > > >>> > > taken
> > > >>> > > > >> out
> > > >>> > > > >> > of
> > > >>> > > > >> > > scope. Perhaps Lucas Bradstreet might have more
> insight
> > > >>> about
> > > >>> > the
> > > >>> > > > >> > decision.
> > > >>> > > > >> > > Basically my point is that we have to create
> additional
> > > >>> > > > infrastructure
> > > >>> > > > >> > here
> > > >>> > > > >> > > to support the name/id mapping, so I wanted to
> > understand
> > > >>> if we
> > > >>> > > just
> > > >>> > > > >> > > consider this a sort of tech debt. It would be useful
> to
> > > >>> cover
> > > >>> > how
> > > >>> > > > we
> > > >>> > > > >> > > handle the case when this file gets corrupted. Seems
> > like
> > > we
> > > >>> > just
> > > >>> > > > >> have to
> > > >>> > > > >> > > trust that it matches whatever the controller tells us
> > and
> > > >>> > rewrite
> > > >>> > > > it?
> > > >>> > > > >> > >
> > > >>> > > > >> > >
> > > >>> > > > >> > > Hi Jason, Justine,
> > > >>> > > > >> > >
> > > >>> > > > >> > > My thought process is that we will likely need the
> > > metadata
> > > >>> file
> > > >>> > > > >> whether
> > > >>> > > > >> > we
> > > >>> > > > >> > > rename the directories or not.
> > > >>> > > > >> > > Linux supports filenames of up to 255 bytes and that
> > would
> > > >>> not
> > > >>> > be
> > > >>> > > > >> enough
> > > >>> > > > >> > to
> > > >>> > > > >> > > support a directory name
> > > >>> > > > >> > >  including both the name and topic ID. Given that
> fact,
> > it
> > > >>> > seemed
> > > >>> > > > >> > > reasonable to add the metadata file
> > > >>> > > > >> > > and leave the directory rename until some time in the
> > > future
> > > >>> > > > (possibly
> > > >>> > > > >> > > never).
> > > >>> > > > >> > >
> > > >>> > > > >> > > If the file does get corrupted I think you're right.
> We
> > > >>> would
> > > >>> > > either
> > > >>> > > > >> have
> > > >>> > > > >> > > to trust it matches what the controller tells us
> > > >>> > > > >> > >  or error out and let an administrator resolve it by
> > > >>> checking
> > > >>> > > across
> > > >>> > > > >> > > replicas for consistency.
> > > >>> > > > >> > >
> > > >>> > > > >> > > Lucas
> > > >>> > > > >> > >
> > > >>> > > > >> > >
> > > >>> > > > >> > > On Thu, Sep 24, 2020 at 3:41 PM Jason Gustafson <
> > > >>> > > jason@confluent.io
> > > >>> > > > >
> > > >>> > > > >> > > wrote:
> > > >>> > > > >> > >
> > > >>> > > > >> > > > Thanks Justine. Responses below:
> > > >>> > > > >> > > >
> > > >>> > > > >> > > > > 1. Yes, the directory will still be based on the
> > topic
> > > >>> > names.
> > > >>> > > > >> > > > LeaderAndIsrRequest is one of the few requests that
> > will
> > > >>> still
> > > >>> > > > >> contain
> > > >>> > > > >> > > the
> > > >>> > > > >> > > > topic name. So I think we have this covered. Sorry
> for
> > > >>> > > confusion.
> > > >>> > > > >> > > >
> > > >>> > > > >> > > > Ah, you're right. My eyes passed right over the
> field.
> > > >>> > > > >> > > >
> > > >>> > > > >> > > > > 2. Part of the usage of the file is to have
> > persistent
> > > >>> > storage
> > > >>> > > > of
> > > >>> > > > >> the
> > > >>> > > > >> > > > topic
> > > >>> > > > >> > > > ID and use it to compare with the ID supplied in the
> > > >>> > > LeaderAndIsr
> > > >>> > > > >> > > Request.
> > > >>> > > > >> > > > There is some discussion in the KIP about changes to
> > the
> > > >>> > > directory
> > > >>> > > > >> > > > structure, but I believe directory changes were
> > > >>> considered to
> > > >>> > be
> > > >>> > > > >> out of
> > > >>> > > > >> > > > scope when the KIP was written.
> > > >>> > > > >> > > >
> > > >>> > > > >> > > > Yeah, I was hoping to get a better understanding of
> > why
> > > >>> it was
> > > >>> > > > taken
> > > >>> > > > >> > out
> > > >>> > > > >> > > of
> > > >>> > > > >> > > > scope. Perhaps Lucas Bradstreet might have more
> > insight
> > > >>> about
> > > >>> > > the
> > > >>> > > > >> > > decision.
> > > >>> > > > >> > > > Basically my point is that we have to create
> > additional
> > > >>> > > > >> infrastructure
> > > >>> > > > >> > > here
> > > >>> > > > >> > > > to support the name/id mapping, so I wanted to
> > > understand
> > > >>> if
> > > >>> > we
> > > >>> > > > just
> > > >>> > > > >> > > > consider this a sort of tech debt. It would be
> useful
> > to
> > > >>> cover
> > > >>> > > how
> > > >>> > > > >> we
> > > >>> > > > >> > > > handle the case when this file gets corrupted. Seems
> > > like
> > > >>> we
> > > >>> > > just
> > > >>> > > > >> have
> > > >>> > > > >> > to
> > > >>> > > > >> > > > trust that it matches whatever the controller tells
> us
> > > and
> > > >>> > > rewrite
> > > >>> > > > >> it?
> > > >>> > > > >> > > >
> > > >>> > > > >> > > > > 3. I think this is a good point, but I again I
> > wonder
> > > >>> about
> > > >>> > > the
> > > >>> > > > >> scope
> > > >>> > > > >> > > of
> > > >>> > > > >> > > > the KIP. Most of the changes mentioned in the KIP
> are
> > > for
> > > >>> > > > supporting
> > > >>> > > > >> > > topic
> > > >>> > > > >> > > > deletion and I believe that is why the produce
> request
> > > was
> > > >>> > > listed
> > > >>> > > > >> under
> > > >>> > > > >> > > > future work.
> > > >>> > > > >> > > >
> > > >>> > > > >> > > > That's fair. I brought it up since `Fetch` is
> already
> > > >>> > included.
> > > >>> > > If
> > > >>> > > > >> > we've
> > > >>> > > > >> > > > got `Metadata` and `Fetch`, seems we may as well do
> > > >>> `Produce`
> > > >>> > > and
> > > >>> > > > >> save
> > > >>> > > > >> > an
> > > >>> > > > >> > > > extra kip. No strong objection though if you want to
> > > >>> leave it
> > > >>> > > out.
> > > >>> > > > >> > > >
> > > >>> > > > >> > > >
> > > >>> > > > >> > > > -Jason
> > > >>> > > > >> > > >
> > > >>> > > > >> > > >
> > > >>> > > > >> > > > On Thu, Sep 24, 2020 at 3:26 PM Justine Olshan <
> > > >>> > > > >> jolshan@confluent.io>
> > > >>> > > > >> > > > wrote:
> > > >>> > > > >> > > >
> > > >>> > > > >> > > > > Hi Jason,
> > > >>> > > > >> > > > >
> > > >>> > > > >> > > > > Thanks for your comments.
> > > >>> > > > >> > > > >
> > > >>> > > > >> > > > > 1. Yes, the directory will still be based on the
> > topic
> > > >>> > names.
> > > >>> > > > >> > > > > LeaderAndIsrRequest is one of the few requests
> that
> > > will
> > > >>> > still
> > > >>> > > > >> > contain
> > > >>> > > > >> > > > the
> > > >>> > > > >> > > > > topic name. So I think we have this covered. Sorry
> > for
> > > >>> > > > confusion.
> > > >>> > > > >> > > > >
> > > >>> > > > >> > > > > 2. Part of the usage of the file is to have
> > persistent
> > > >>> > storage
> > > >>> > > > of
> > > >>> > > > >> the
> > > >>> > > > >> > > > topic
> > > >>> > > > >> > > > > ID and use it to compare with the ID supplied in
> the
> > > >>> > > > LeaderAndIsr
> > > >>> > > > >> > > > Request.
> > > >>> > > > >> > > > > There is some discussion in the KIP about changes
> to
> > > the
> > > >>> > > > directory
> > > >>> > > > >> > > > > structure, but I believe directory changes were
> > > >>> considered
> > > >>> > to
> > > >>> > > be
> > > >>> > > > >> out
> > > >>> > > > >> > of
> > > >>> > > > >> > > > > scope when the KIP was written.
> > > >>> > > > >> > > > >
> > > >>> > > > >> > > > > 3. I think this is a good point, but I again I
> > wonder
> > > >>> about
> > > >>> > > the
> > > >>> > > > >> scope
> > > >>> > > > >> > > of
> > > >>> > > > >> > > > > the KIP. Most of the changes mentioned in the KIP
> > are
> > > >>> for
> > > >>> > > > >> supporting
> > > >>> > > > >> > > > topic
> > > >>> > > > >> > > > > deletion and I believe that is why the produce
> > request
> > > >>> was
> > > >>> > > > listed
> > > >>> > > > >> > under
> > > >>> > > > >> > > > > future work.
> > > >>> > > > >> > > > >
> > > >>> > > > >> > > > > 4. This sounds like it might be a good solution,
> > but I
> > > >>> will
> > > >>> > > need
> > > >>> > > > >> to
> > > >>> > > > >> > > > discuss
> > > >>> > > > >> > > > > more with KIP-500 folks to get the details right.
> > > >>> > > > >> > > > >
> > > >>> > > > >> > > > > Thanks,
> > > >>> > > > >> > > > > Justine
> > > >>> > > > >> > > > >
> > > >>> > > > >> > > > > On Thu, Sep 24, 2020 at 12:30 PM Jason Gustafson <
> > > >>> > > > >> jason@confluent.io
> > > >>> > > > >> > >
> > > >>> > > > >> > > > > wrote:
> > > >>> > > > >> > > > >
> > > >>> > > > >> > > > > > Hi Justine,
> > > >>> > > > >> > > > > >
> > > >>> > > > >> > > > > > Thanks for picking up this work. I have a few
> > > >>> > > > >> questions/comments:
> > > >>> > > > >> > > > > >
> > > >>> > > > >> > > > > > 1. It sounds like the directory structure is
> still
> > > >>> going
> > > >>> > to
> > > >>> > > be
> > > >>> > > > >> > based
> > > >>> > > > >> > > on
> > > >>> > > > >> > > > > > topic names. Do I have that right? One
> > complication
> > > is
> > > >>> > that
> > > >>> > > > the
> > > >>> > > > >> > > > > > LeaderAndIsr request does not include the topic
> > name
> > > >>> any
> > > >>> > > > longer.
> > > >>> > > > >> > This
> > > >>> > > > >> > > > > means
> > > >>> > > > >> > > > > > that a replica must wait for the UpdateMetadata
> > > >>> request to
> > > >>> > > > >> arrive
> > > >>> > > > >> > > with
> > > >>> > > > >> > > > > the
> > > >>> > > > >> > > > > > topic name mapping before it can create the
> > > >>> directory. I
> > > >>> > > > wonder
> > > >>> > > > >> if
> > > >>> > > > >> > it
> > > >>> > > > >> > > > > would
> > > >>> > > > >> > > > > > be simpler to avoid assumptions on the ordering
> of
> > > >>> > > > >> UpdateMetadata
> > > >>> > > > >> > and
> > > >>> > > > >> > > > let
> > > >>> > > > >> > > > > > LeaderAndIsr include the topic name as well.
> Feels
> > > >>> like we
> > > >>> > > are
> > > >>> > > > >> not
> > > >>> > > > >> > > > saving
> > > >>> > > > >> > > > > > that much by excluding it.
> > > >>> > > > >> > > > > >
> > > >>> > > > >> > > > > > 2. On a related note, it seems that the reason
> we
> > > >>> have the
> > > >>> > > > >> > partition
> > > >>> > > > >> > > > > > metadata file is because we are not changing the
> > > >>> directory
> > > >>> > > > >> > structure.
> > > >>> > > > >> > > > We
> > > >>> > > > >> > > > > > need it so that we remember which directories
> map
> > to
> > > >>> which
> > > >>> > > > topic
> > > >>> > > > >> > id.
> > > >>> > > > >> > > I
> > > >>> > > > >> > > > > > think it would be useful to add some detail
> about
> > > why
> > > >>> > > changing
> > > >>> > > > >> the
> > > >>> > > > >> > > > > > directory layout was rejected. Basically trying
> to
> > > >>> > > understand
> > > >>> > > > >> how
> > > >>> > > > >> > > much
> > > >>> > > > >> > > > > > effort we are saving by skipping this step if we
> > > have
> > > >>> to
> > > >>> > > > >> implement
> > > >>> > > > >> > > this
> > > >>> > > > >> > > > > > additional bookkeeping. One concern I have for
> > > >>> example is
> > > >>> > > that
> > > >>> > > > >> the
> > > >>> > > > >> > > > > > partition metadata file gets corrupt and then
> all
> > > >>> bets are
> > > >>> > > off
> > > >>> > > > >> as
> > > >>> > > > >> > far
> > > >>> > > > >> > > > as
> > > >>> > > > >> > > > > > topic consistency.
> > > >>> > > > >> > > > > >
> > > >>> > > > >> > > > > > 3. Is it worth adding support now for topic ids
> in
> > > the
> > > >>> > > > `Produce`
> > > >>> > > > >> > > > request
> > > >>> > > > >> > > > > > now? Overhead is mentioned as one of the
> > motivations
> > > >>> and
> > > >>> > the
> > > >>> > > > >> best
> > > >>> > > > >> > > APIs
> > > >>> > > > >> > > > to
> > > >>> > > > >> > > > > > get savings from are `Produce` and `Fetch`.
> > > >>> > > > >> > > > > >
> > > >>> > > > >> > > > > > 4. When it comes to bootstrapping the metadata
> > topic
> > > >>> with
> > > >>> > > > >> respect
> > > >>> > > > >> > to
> > > >>> > > > >> > > > > > KIP-500, one suggestion would be to reserve a
> > > sentinel
> > > >>> > topic
> > > >>> > > > ID
> > > >>> > > > >> > which
> > > >>> > > > >> > > > can
> > > >>> > > > >> > > > > > be used initially by new brokers. When the first
> > > >>> > controller
> > > >>> > > is
> > > >>> > > > >> > > elected,
> > > >>> > > > >> > > > > it
> > > >>> > > > >> > > > > > can assign a topicId to the metadata topic. This
> > is
> > > a
> > > >>> bit
> > > >>> > > > >> similar
> > > >>> > > > >> > to
> > > >>> > > > >> > > > how
> > > >>> > > > >> > > > > we
> > > >>> > > > >> > > > > > were planning to generate the clusterId.
> > > >>> > > > >> > > > > >
> > > >>> > > > >> > > > > > Thanks,
> > > >>> > > > >> > > > > > Jason
> > > >>> > > > >> > > > > >
> > > >>> > > > >> > > > > >
> > > >>> > > > >> > > > > > On Thu, Sep 24, 2020 at 11:10 AM Jun Rao <
> > > >>> > jun@confluent.io>
> > > >>> > > > >> wrote:
> > > >>> > > > >> > > > > >
> > > >>> > > > >> > > > > > > Hi, Justine,
> > > >>> > > > >> > > > > > >
> > > >>> > > > >> > > > > > > Thanks for the updated KIP. A few more
> comments
> > > >>> below.
> > > >>> > > > >> > > > > > >
> > > >>> > > > >> > > > > > > 30.  {"name": "id", "type": "string", "doc":
> > > >>> "version
> > > >>> > > id"}}:
> > > >>> > > > >> The
> > > >>> > > > >> > > doc
> > > >>> > > > >> > > > > > should
> > > >>> > > > >> > > > > > > say UUID.
> > > >>> > > > >> > > > > > >
> > > >>> > > > >> > > > > > > 31. LeaderAndIsrResponse v5 and
> > > StopReplicaResponse
> > > >>> v4 :
> > > >>> > > It
> > > >>> > > > >> seems
> > > >>> > > > >> > > > there
> > > >>> > > > >> > > > > > is
> > > >>> > > > >> > > > > > > no need to add topic_id at partitions level.
> > > >>> > > > >> > > > > > >
> > > >>> > > > >> > > > > > > 32. Regarding partition metadata file. Perhaps
> > the
> > > >>> key
> > > >>> > can
> > > >>> > > > be
> > > >>> > > > >> a
> > > >>> > > > >> > > > single
> > > >>> > > > >> > > > > > > word, sth like the following.
> > > >>> > > > >> > > > > > > version: 0
> > > >>> > > > >> > > > > > > topic_id: 46bdb63f-9e8d-4a38-bf7b-ee4eb2a794e4
> > > >>> > > > >> > > > > > >
> > > >>> > > > >> > > > > > > 33. Another tricky thing that I realized is
> how
> > to
> > > >>> > support
> > > >>> > > > the
> > > >>> > > > >> > > > metadata
> > > >>> > > > >> > > > > > > topic introduced in KIP-595. It's a bit tricky
> > to
> > > >>> > assign a
> > > >>> > > > >> UUID
> > > >>> > > > >> > to
> > > >>> > > > >> > > > the
> > > >>> > > > >> > > > > > > metadata topic since we have a chicken-and-egg
> > > >>> problem.
> > > >>> > > The
> > > >>> > > > >> > > > controller
> > > >>> > > > >> > > > > > > needs to persist the UUID in the metadata
> topic
> > in
> > > >>> order
> > > >>> > > to
> > > >>> > > > >> > assign
> > > >>> > > > >> > > > one
> > > >>> > > > >> > > > > > > successfully, but the metadata topic is needed
> > to
> > > >>> elect
> > > >>> > a
> > > >>> > > > >> > > controller
> > > >>> > > > >> > > > > > > (KIP-631). So, this probably needs a bit more
> > > >>> thought.
> > > >>> > > > >> > > > > > >
> > > >>> > > > >> > > > > > > Jun
> > > >>> > > > >> > > > > > >
> > > >>> > > > >> > > > > > > On Thu, Sep 24, 2020 at 3:04 AM Ismael Juma <
> > > >>> > > > >> ismael@juma.me.uk>
> > > >>> > > > >> > > > wrote:
> > > >>> > > > >> > > > > > >
> > > >>> > > > >> > > > > > > > Also, can we provide more details on how the
> > > >>> Partition
> > > >>> > > > >> Metadata
> > > >>> > > > >> > > > file
> > > >>> > > > >> > > > > > will
> > > >>> > > > >> > > > > > > > be used?
> > > >>> > > > >> > > > > > > >
> > > >>> > > > >> > > > > > > > Ismael
> > > >>> > > > >> > > > > > > >
> > > >>> > > > >> > > > > > > > On Thu, Sep 24, 2020 at 3:01 AM Ismael Juma
> <
> > > >>> > > > >> ismael@juma.me.uk
> > > >>> > > > >> > >
> > > >>> > > > >> > > > > wrote:
> > > >>> > > > >> > > > > > > >
> > > >>> > > > >> > > > > > > > > Hi Justine,
> > > >>> > > > >> > > > > > > > >
> > > >>> > > > >> > > > > > > > > I think we need to update the "Rejected
> > > >>> > Alternatives"
> > > >>> > > > >> section
> > > >>> > > > >> > > to
> > > >>> > > > >> > > > > take
> > > >>> > > > >> > > > > > > > into
> > > >>> > > > >> > > > > > > > > account that the proposal now removes the
> > > topic
> > > >>> name
> > > >>> > > > from
> > > >>> > > > >> the
> > > >>> > > > >> > > > fetch
> > > >>> > > > >> > > > > > > > > request. Also, if we are removing it from
> > the
> > > >>> Fetch
> > > >>> > > > >> request,
> > > >>> > > > >> > > does
> > > >>> > > > >> > > > > it
> > > >>> > > > >> > > > > > > make
> > > >>> > > > >> > > > > > > > > sense not to remove it from similar
> requests
> > > >>> like
> > > >>> > > > >> > > > > ListOffsetRequest?
> > > >>> > > > >> > > > > > > > >
> > > >>> > > > >> > > > > > > > > Ismael
> > > >>> > > > >> > > > > > > > >
> > > >>> > > > >> > > > > > > > > On Thu, Sep 24, 2020 at 2:46 AM David
> Jacot
> > <
> > > >>> > > > >> > > djacot@confluent.io
> > > >>> > > > >> > > > >
> > > >>> > > > >> > > > > > > wrote:
> > > >>> > > > >> > > > > > > > >
> > > >>> > > > >> > > > > > > > >> Hi Justine,
> > > >>> > > > >> > > > > > > > >>
> > > >>> > > > >> > > > > > > > >> Thanks for the KIP. I finally had time to
> > > read
> > > >>> it
> > > >>> > :).
> > > >>> > > > It
> > > >>> > > > >> is
> > > >>> > > > >> > a
> > > >>> > > > >> > > > > great
> > > >>> > > > >> > > > > > > > >> improvement.
> > > >>> > > > >> > > > > > > > >>
> > > >>> > > > >> > > > > > > > >> I have a few comments/questions:
> > > >>> > > > >> > > > > > > > >>
> > > >>> > > > >> > > > > > > > >> 1. It seems that the schema of the
> > > >>> > StopReplicaRequest
> > > >>> > > > is
> > > >>> > > > >> > > > slightly
> > > >>> > > > >> > > > > > > > >> outdated.
> > > >>> > > > >> > > > > > > > >> We
> > > >>> > > > >> > > > > > > > >> did some changes as part of KIP-570. V3
> is
> > > >>> already
> > > >>> > > > >> organized
> > > >>> > > > >> > > by
> > > >>> > > > >> > > > > > > topics.
> > > >>> > > > >> > > > > > > > >>
> > > >>> > > > >> > > > > > > > >> 2. I just want to make sure that I
> > understand
> > > >>> the
> > > >>> > > > >> > > reconciliation
> > > >>> > > > >> > > > > > > > >> logic correctly. When
> > > >>> > > > >> > > > > > > > >> an "INCREMENTAL" LeaderAndIsr Request is
> > > >>> received,
> > > >>> > > the
> > > >>> > > > >> > broker
> > > >>> > > > >> > > > will
> > > >>> > > > >> > > > > > > also
> > > >>> > > > >> > > > > > > > >> reconcile
> > > >>> > > > >> > > > > > > > >> when the local uuid does not match the
> uuid
> > > in
> > > >>> the
> > > >>> > > > >> request,
> > > >>> > > > >> > > > right?
> > > >>> > > > >> > > > > > In
> > > >>> > > > >> > > > > > > > this
> > > >>> > > > >> > > > > > > > >> case, the
> > > >>> > > > >> > > > > > > > >> local log is staged for deletion.
> > > >>> > > > >> > > > > > > > >>
> > > >>> > > > >> > > > > > > > >> 3. In the documentation of the `
> > > >>> > > > >> delete.stale.topic.delay.ms
> > > >>> > > > >> > `
> > > >>> > > > >> > > > > > config,
> > > >>> > > > >> > > > > > > it
> > > >>> > > > >> > > > > > > > >> says "When a
> > > >>> > > > >> > > > > > > > >> FULL LeaderAndIsrRequest is received..."
> > but
> > > I
> > > >>> > > suppose
> > > >>> > > > it
> > > >>> > > > >> > > > applies
> > > >>> > > > >> > > > > to
> > > >>> > > > >> > > > > > > > both
> > > >>> > > > >> > > > > > > > >> types.
> > > >>> > > > >> > > > > > > > >>
> > > >>> > > > >> > > > > > > > >> Best,
> > > >>> > > > >> > > > > > > > >> David
> > > >>> > > > >> > > > > > > > >>
> > > >>> > > > >> > > > > > > > >> On Thu, Sep 24, 2020 at 1:40 AM Justine
> > > Olshan
> > > >>> <
> > > >>> > > > >> > > > > > jolshan@confluent.io>
> > > >>> > > > >> > > > > > > > >> wrote:
> > > >>> > > > >> > > > > > > > >>
> > > >>> > > > >> > > > > > > > >> > Hi Jun,
> > > >>> > > > >> > > > > > > > >> >
> > > >>> > > > >> > > > > > > > >> > Thanks for the comments. I apologize
> for
> > > >>> some of
> > > >>> > > the
> > > >>> > > > >> typos
> > > >>> > > > >> > > and
> > > >>> > > > >> > > > > > > > >> confusion.
> > > >>> > > > >> > > > > > > > >> > I’ve updated the KIP to fix some of the
> > > >>> issues
> > > >>> > you
> > > >>> > > > >> > > mentioned.
> > > >>> > > > >> > > > > > > > >> >
> > > >>> > > > >> > > > > > > > >> > 20.2 I’ve changed the type to String.
> > > >>> > > > >> > > > > > > > >> > 20.1/3 I’ve updated the TopicZNode to
> fix
> > > >>> > > formatting
> > > >>> > > > >> and
> > > >>> > > > >> > > > reflect
> > > >>> > > > >> > > > > > the
> > > >>> > > > >> > > > > > > > >> latest
> > > >>> > > > >> > > > > > > > >> > version before this change.
> > > >>> > > > >> > > > > > > > >> >
> > > >>> > > > >> > > > > > > > >> > 21. You are correct and I’ve removed
> this
> > > >>> line.
> > > >>> > > I’ve
> > > >>> > > > >> also
> > > >>> > > > >> > > > added
> > > >>> > > > >> > > > > a
> > > >>> > > > >> > > > > > > line
> > > >>> > > > >> > > > > > > > >> > mentioning an IBP bump is necessary for
> > > >>> migration
> > > >>> > > > >> > > > > > > > >> >
> > > >>> > > > >> > > > > > > > >> > 22. I think the wording was unclear but
> > > your
> > > >>> > > summary
> > > >>> > > > is
> > > >>> > > > >> > what
> > > >>> > > > >> > > > was
> > > >>> > > > >> > > > > > > > >> intended
> > > >>> > > > >> > > > > > > > >> > by this line. I’ve updated to make this
> > > point
> > > >>> > more
> > > >>> > > > >> clear.
> > > >>> > > > >> > > > > “Remove
> > > >>> > > > >> > > > > > > > >> deleted
> > > >>> > > > >> > > > > > > > >> > topics from replicas by sending
> > > >>> > StopReplicaRequest
> > > >>> > > V3
> > > >>> > > > >> > before
> > > >>> > > > >> > > > the
> > > >>> > > > >> > > > > > IBP
> > > >>> > > > >> > > > > > > > >> bump
> > > >>> > > > >> > > > > > > > >> > using the old logic, and using V4 and
> the
> > > new
> > > >>> > logic
> > > >>> > > > >> with
> > > >>> > > > >> > > topic
> > > >>> > > > >> > > > > IDs
> > > >>> > > > >> > > > > > > > after
> > > >>> > > > >> > > > > > > > >> > the IBP bump.”
> > > >>> > > > >> > > > > > > > >> >
> > > >>> > > > >> > > > > > > > >> > 23. I’ve removed the unspecified type
> > from
> > > >>> the
> > > >>> > KIP
> > > >>> > > > and
> > > >>> > > > >> > > mention
> > > >>> > > > >> > > > > > that
> > > >>> > > > >> > > > > > > > IBP
> > > >>> > > > >> > > > > > > > >> > will be used to handle this request.
> “IBP
> > > >>> will be
> > > >>> > > > used
> > > >>> > > > >> to
> > > >>> > > > >> > > > > > determine
> > > >>> > > > >> > > > > > > > >> whether
> > > >>> > > > >> > > > > > > > >> > this new form of the request will be
> > used.
> > > >>> For
> > > >>> > > older
> > > >>> > > > >> > > requests,
> > > >>> > > > >> > > > > we
> > > >>> > > > >> > > > > > > will
> > > >>> > > > >> > > > > > > > >> > ignore this field and default to
> previous
> > > >>> > > behavior.”
> > > >>> > > > >> > > > > > > > >> >
> > > >>> > > > >> > > > > > > > >> > 24. I’ve fixed this typo.
> > > >>> > > > >> > > > > > > > >> >
> > > >>> > > > >> > > > > > > > >> > 25. I've added a topics at a higher
> level
> > > for
> > > >>> > > > >> > > > > LeaderAndIsrResponse
> > > >>> > > > >> > > > > > > v5,
> > > >>> > > > >> > > > > > > > >> > StopReplicaResponse v4. I've also
> changed
> > > >>> > > > >> > StopReplicaRequest
> > > >>> > > > >> > > > v4
> > > >>> > > > >> > > > > to
> > > >>> > > > >> > > > > > > > have
> > > >>> > > > >> > > > > > > > >> > topics at a higher level.
> > > >>> > > > >> > > > > > > > >> >
> > > >>> > > > >> > > > > > > > >> > 26. I’ve updated
> > > >>> forgotten_topics_data--added the
> > > >>> > > > >> topic ID
> > > >>> > > > >> > > and
> > > >>> > > > >> > > > > > > removed
> > > >>> > > > >> > > > > > > > >> the
> > > >>> > > > >> > > > > > > > >> > topic name
> > > >>> > > > >> > > > > > > > >> >
> > > >>> > > > >> > > > > > > > >> > 27. I’ve decided on plain text, and
> I’ve
> > > >>> added an
> > > >>> > > > >> example.
> > > >>> > > > >> > > > > > > > >> >
> > > >>> > > > >> > > > > > > > >> > 28. I’ve added this idea to future
> work.
> > > >>> > > > >> > > > > > > > >> >
> > > >>> > > > >> > > > > > > > >> > Thanks again for taking a look,
> > > >>> > > > >> > > > > > > > >> >
> > > >>> > > > >> > > > > > > > >> > Justine
> > > >>> > > > >> > > > > > > > >> >
> > > >>> > > > >> > > > > > > > >> > On Wed, Sep 23, 2020 at 10:28 AM Jun
> Rao
> > <
> > > >>> > > > >> > jun@confluent.io>
> > > >>> > > > >> > > > > > wrote:
> > > >>> > > > >> > > > > > > > >> >
> > > >>> > > > >> > > > > > > > >> > > Hi, Justine,
> > > >>> > > > >> > > > > > > > >> > >
> > > >>> > > > >> > > > > > > > >> > > Thanks for the response. Made another
> > > >>> pass. A
> > > >>> > few
> > > >>> > > > >> more
> > > >>> > > > >> > > > > comments
> > > >>> > > > >> > > > > > > > below.
> > > >>> > > > >> > > > > > > > >> > >
> > > >>> > > > >> > > > > > > > >> > > 20. znode schema:
> > > >>> > > > >> > > > > > > > >> > > 20.1 It seems that {"name":
> "version",
> > > >>> "type":
> > > >>> > > > "int",
> > > >>> > > > >> > > "id":
> > > >>> > > > >> > > > > > > "UUID",
> > > >>> > > > >> > > > > > > > >> > "doc":
> > > >>> > > > >> > > > > > > > >> > > "version id"} should be {"name":
> > > "version",
> > > >>> > > "type":
> > > >>> > > > >> > > "int"},
> > > >>> > > > >> > > > > > > {"name":
> > > >>> > > > >> > > > > > > > >> > "id",
> > > >>> > > > >> > > > > > > > >> > > "type": "UUID", "doc": "version id"}.
> > > >>> > > > >> > > > > > > > >> > > 20.2 The znode format is JSON which
> > > doesn't
> > > >>> > have
> > > >>> > > > UUID
> > > >>> > > > >> > > type.
> > > >>> > > > >> > > > So
> > > >>> > > > >> > > > > > the
> > > >>> > > > >> > > > > > > > >> type
> > > >>> > > > >> > > > > > > > >> > > probably should be string?
> > > >>> > > > >> > > > > > > > >> > > 20.3 Also, the existing format used
> > seems
> > > >>> > > outdated.
> > > >>> > > > >> It
> > > >>> > > > >> > > > should
> > > >>> > > > >> > > > > > have
> > > >>> > > > >> > > > > > > > the
> > > >>> > > > >> > > > > > > > >> > > following format.
> > > >>> > > > >> > > > > > > > >> > >     Json.encodeAsBytes(Map(
> > > >>> > > > >> > > > > > > > >> > >       "version" -> 2,
> > > >>> > > > >> > > > > > > > >> > >       "partitions" ->
> > > >>> > > replicaAssignmentJson.asJava,
> > > >>> > > > >> > > > > > > > >> > >       "adding_replicas" ->
> > > >>> > > > >> > > > > addingReplicasAssignmentJson.asJava,
> > > >>> > > > >> > > > > > > > >> > >       "removing_replicas" ->
> > > >>> > > > >> > > > > > removingReplicasAssignmentJson.asJava
> > > >>> > > > >> > > > > > > > >> > >     ).asJava)
> > > >>> > > > >> > > > > > > > >> > >   }
> > > >>> > > > >> > > > > > > > >> > >
> > > >>> > > > >> > > > > > > > >> > > 21. Migration: The KIP says "The
> > > migration
> > > >>> > > process
> > > >>> > > > >> can
> > > >>> > > > >> > > take
> > > >>> > > > >> > > > > > place
> > > >>> > > > >> > > > > > > > >> without
> > > >>> > > > >> > > > > > > > >> > > an inter-broker protocol bump, as the
> > > >>> format
> > > >>> > > stored
> > > >>> > > > >> in
> > > >>> > > > >> > > > > > > > >> > > /brokers/topics/[topic] will be
> > > compatible
> > > >>> with
> > > >>> > > > older
> > > >>> > > > >> > > broker
> > > >>> > > > >> > > > > > > > >> versions."
> > > >>> > > > >> > > > > > > > >> > > However, since we are bumping up the
> > > >>> version of
> > > >>> > > > >> > > inter-broker
> > > >>> > > > >> > > > > > > > >> requests, it
> > > >>> > > > >> > > > > > > > >> > > seems that we need to use IBP for
> > > >>> migration.
> > > >>> > > > >> > > > > > > > >> > >
> > > >>> > > > >> > > > > > > > >> > > 22. The KIP says "Remove deleted
> topics
> > > >>> from
> > > >>> > > > >> replicas by
> > > >>> > > > >> > > > > sending
> > > >>> > > > >> > > > > > > > >> > > StopReplicaRequest V3 for any topics
> > > which
> > > >>> do
> > > >>> > not
> > > >>> > > > >> > contain
> > > >>> > > > >> > > a
> > > >>> > > > >> > > > > > topic
> > > >>> > > > >> > > > > > > > ID,
> > > >>> > > > >> > > > > > > > >> and
> > > >>> > > > >> > > > > > > > >> > > V4 for any topics which do contain a
> > > topic
> > > >>> > ID.".
> > > >>> > > > >> > However,
> > > >>> > > > >> > > if
> > > >>> > > > >> > > > > we
> > > >>> > > > >> > > > > > > use
> > > >>> > > > >> > > > > > > > >> IBP,
> > > >>> > > > >> > > > > > > > >> > it
> > > >>> > > > >> > > > > > > > >> > > seems that the controller will either
> > > send
> > > >>> > > > >> > > > StopReplicaRequest
> > > >>> > > > >> > > > > V3
> > > >>> > > > >> > > > > > > > >> > > or StopReplicaRequest V4, but never
> > mixed
> > > >>> V3
> > > >>> > and
> > > >>> > > V4
> > > >>> > > > >> for
> > > >>> > > > >> > > > > > different
> > > >>> > > > >> > > > > > > > >> topics.
> > > >>> > > > >> > > > > > > > >> > > Basically, before the IBP bump, V3
> will
> > > be
> > > >>> > used.
> > > >>> > > > >> After
> > > >>> > > > >> > the
> > > >>> > > > >> > > > IBP
> > > >>> > > > >> > > > > > > bump,
> > > >>> > > > >> > > > > > > > >> > > topicId will be created and V4 will
> be
> > > >>> used.
> > > >>> > > > >> > > > > > > > >> > >
> > > >>> > > > >> > > > > > > > >> > > 23. Given that we depend on IBP, do
> we
> > > >>> still
> > > >>> > need
> > > >>> > > > "0
> > > >>> > > > >> > > > > > UNSPECIFIED"
> > > >>> > > > >> > > > > > > > >> > > in LeaderAndIsr?
> > > >>> > > > >> > > > > > > > >> > >
> > > >>> > > > >> > > > > > > > >> > > 24. LeaderAndIsrResponse v5 : It
> still
> > > has
> > > >>> the
> > > >>> > > > topic
> > > >>> > > > >> > > field.
> > > >>> > > > >> > > > > > > > >> > >
> > > >>> > > > >> > > > > > > > >> > > 25. LeaderAndIsrResponse v5,
> > > >>> > StopReplicaResponse
> > > >>> > > > v4:
> > > >>> > > > >> > Could
> > > >>> > > > >> > > > we
> > > >>> > > > >> > > > > > use
> > > >>> > > > >> > > > > > > > this
> > > >>> > > > >> > > > > > > > >> > > opportunity to organize the response
> > in 2
> > > >>> > levels,
> > > >>> > > > >> first
> > > >>> > > > >> > by
> > > >>> > > > >> > > > > > topic,
> > > >>> > > > >> > > > > > > > >> then by
> > > >>> > > > >> > > > > > > > >> > > partition, as most other
> > > >>> requests/responses?
> > > >>> > > > >> > > > > > > > >> > >
> > > >>> > > > >> > > > > > > > >> > > 26. FetchRequest v13 : Should
> > > >>> > > forgotten_topics_data
> > > >>> > > > >> use
> > > >>> > > > >> > > > > topicId
> > > >>> > > > >> > > > > > > too?
> > > >>> > > > >> > > > > > > > >> > >
> > > >>> > > > >> > > > > > > > >> > > 27. "This file can either be plain
> text
> > > >>> > > (key/value
> > > >>> > > > >> > pairs)
> > > >>> > > > >> > > or
> > > >>> > > > >> > > > > > > JSON."
> > > >>> > > > >> > > > > > > > >> Have
> > > >>> > > > >> > > > > > > > >> > we
> > > >>> > > > >> > > > > > > > >> > > decided which one to use? Also, it
> > would
> > > be
> > > >>> > > helpful
> > > >>> > > > >> to
> > > >>> > > > >> > > > provide
> > > >>> > > > >> > > > > > an
> > > >>> > > > >> > > > > > > > >> > example.
> > > >>> > > > >> > > > > > > > >> > >
> > > >>> > > > >> > > > > > > > >> > > 28. Future improvement: Another
> future
> > > >>> > > improvement
> > > >>> > > > >> > > > opportunity
> > > >>> > > > >> > > > > > is
> > > >>> > > > >> > > > > > > to
> > > >>> > > > >> > > > > > > > >> use
> > > >>> > > > >> > > > > > > > >> > > topicId in
> > > >>> GroupMetadataManager.offsetCommitKey
> > > >>> > > in
> > > >>> > > > >> the
> > > >>> > > > >> > > > > > > offset_commit
> > > >>> > > > >> > > > > > > > >> > topic.
> > > >>> > > > >> > > > > > > > >> > > This may save some space.
> > > >>> > > > >> > > > > > > > >> > >
> > > >>> > > > >> > > > > > > > >> > > Thanks,
> > > >>> > > > >> > > > > > > > >> > >
> > > >>> > > > >> > > > > > > > >> > > Jun
> > > >>> > > > >> > > > > > > > >> > >
> > > >>> > > > >> > > > > > > > >> > > On Wed, Sep 23, 2020 at 8:50 AM
> Justine
> > > >>> Olshan
> > > >>> > <
> > > >>> > > > >> > > > > > > > jolshan@confluent.io>
> > > >>> > > > >> > > > > > > > >> > > wrote:
> > > >>> > > > >> > > > > > > > >> > >
> > > >>> > > > >> > > > > > > > >> > > > Hi Tom,
> > > >>> > > > >> > > > > > > > >> > > >
> > > >>> > > > >> > > > > > > > >> > > > Thanks for the comment. I think
> this
> > > is a
> > > >>> > > really
> > > >>> > > > >> good
> > > >>> > > > >> > > idea
> > > >>> > > > >> > > > > and
> > > >>> > > > >> > > > > > > it
> > > >>> > > > >> > > > > > > > >> has
> > > >>> > > > >> > > > > > > > >> > > been
> > > >>> > > > >> > > > > > > > >> > > > added to the KIP under the newly
> > added
> > > >>> > tooling
> > > >>> > > > >> > section.
> > > >>> > > > >> > > > > > > > >> > > >
> > > >>> > > > >> > > > > > > > >> > > > Thanks again,
> > > >>> > > > >> > > > > > > > >> > > > Justine
> > > >>> > > > >> > > > > > > > >> > > >
> > > >>> > > > >> > > > > > > > >> > > > On Wed, Sep 23, 2020 at 3:17 AM Tom
> > > >>> Bentley <
> > > >>> > > > >> > > > > > > tbentley@redhat.com>
> > > >>> > > > >> > > > > > > > >> > wrote:
> > > >>> > > > >> > > > > > > > >> > > >
> > > >>> > > > >> > > > > > > > >> > > > > Hi Justine,
> > > >>> > > > >> > > > > > > > >> > > > >
> > > >>> > > > >> > > > > > > > >> > > > > I know you started the vote
> thread,
> > > >>> but on
> > > >>> > > > >> > re-reading
> > > >>> > > > >> > > > the
> > > >>> > > > >> > > > > > KIP
> > > >>> > > > >> > > > > > > I
> > > >>> > > > >> > > > > > > > >> > noticed
> > > >>> > > > >> > > > > > > > >> > > > > that although the topic id is
> > > included
> > > >>> in
> > > >>> > the
> > > >>> > > > >> > > > > > MetadataResponse
> > > >>> > > > >> > > > > > > > >> it's
> > > >>> > > > >> > > > > > > > >> > not
> > > >>> > > > >> > > > > > > > >> > > > > surfaced in the output from
> > > >>> > `kafka-topics.sh
> > > >>> > > > >> > > > --describe`.
> > > >>> > > > >> > > > > > > Maybe
> > > >>> > > > >> > > > > > > > >> that
> > > >>> > > > >> > > > > > > > >> > > was
> > > >>> > > > >> > > > > > > > >> > > > > intentional because ids are
> > > >>> intentionally
> > > >>> > not
> > > >>> > > > >> really
> > > >>> > > > >> > > > > > something
> > > >>> > > > >> > > > > > > > the
> > > >>> > > > >> > > > > > > > >> > user
> > > >>> > > > >> > > > > > > > >> > > > > should care deeply about, but it
> > > would
> > > >>> also
> > > >>> > > > make
> > > >>> > > > >> > life
> > > >>> > > > >> > > > > harder
> > > >>> > > > >> > > > > > > for
> > > >>> > > > >> > > > > > > > >> > anyone
> > > >>> > > > >> > > > > > > > >> > > > > debugging Kafka and this would
> > likely
> > > >>> get
> > > >>> > > worse
> > > >>> > > > >> the
> > > >>> > > > >> > > more
> > > >>> > > > >> > > > > > topic
> > > >>> > > > >> > > > > > > > ids
> > > >>> > > > >> > > > > > > > >> > got
> > > >>> > > > >> > > > > > > > >> > > > > rolled out across the protocols,
> > > >>> clients
> > > >>> > etc.
> > > >>> > > > It
> > > >>> > > > >> > seems
> > > >>> > > > >> > > > > > likely
> > > >>> > > > >> > > > > > > > that
> > > >>> > > > >> > > > > > > > >> > > > > `kafka-topics.sh` will eventually
> > > need
> > > >>> the
> > > >>> > > > >> ability
> > > >>> > > > >> > to
> > > >>> > > > >> > > > show
> > > >>> > > > >> > > > > > the
> > > >>> > > > >> > > > > > > > id
> > > >>> > > > >> > > > > > > > >> of
> > > >>> > > > >> > > > > > > > >> > a
> > > >>> > > > >> > > > > > > > >> > > > > topic and perhaps find a topic
> name
> > > >>> given
> > > >>> > an
> > > >>> > > > id.
> > > >>> > > > >> Is
> > > >>> > > > >> > > > there
> > > >>> > > > >> > > > > > any
> > > >>> > > > >> > > > > > > > >> reason
> > > >>> > > > >> > > > > > > > >> > > not
> > > >>> > > > >> > > > > > > > >> > > > to
> > > >>> > > > >> > > > > > > > >> > > > > implement that in this KIP?
> > > >>> > > > >> > > > > > > > >> > > > >
> > > >>> > > > >> > > > > > > > >> > > > > Many thanks,
> > > >>> > > > >> > > > > > > > >> > > > >
> > > >>> > > > >> > > > > > > > >> > > > > Tom
> > > >>> > > > >> > > > > > > > >> > > > >
> > > >>> > > > >> > > > > > > > >> > > > > On Mon, Sep 21, 2020 at 9:54 PM
> > > Justine
> > > >>> > > Olshan
> > > >>> > > > <
> > > >>> > > > >> > > > > > > > >> jolshan@confluent.io
> > > >>> > > > >> > > > > > > > >> > >
> > > >>> > > > >> > > > > > > > >> > > > > wrote:
> > > >>> > > > >> > > > > > > > >> > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > Hi all,
> > > >>> > > > >> > > > > > > > >> > > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > After thinking about it, I've
> > > >>> decided to
> > > >>> > > > remove
> > > >>> > > > >> > the
> > > >>> > > > >> > > > > topic
> > > >>> > > > >> > > > > > > name
> > > >>> > > > >> > > > > > > > >> from
> > > >>> > > > >> > > > > > > > >> > > the
> > > >>> > > > >> > > > > > > > >> > > > > > Fetch Request and Response
> after
> > > all.
> > > >>> > Since
> > > >>> > > > >> there
> > > >>> > > > >> > > are
> > > >>> > > > >> > > > so
> > > >>> > > > >> > > > > > > many
> > > >>> > > > >> > > > > > > > of
> > > >>> > > > >> > > > > > > > >> > > these
> > > >>> > > > >> > > > > > > > >> > > > > > requests per second, it is
> worth
> > > >>> removing
> > > >>> > > the
> > > >>> > > > >> > extra
> > > >>> > > > >> > > > > > > > information.
> > > >>> > > > >> > > > > > > > >> > I've
> > > >>> > > > >> > > > > > > > >> > > > > > updated the KIP to reflect this
> > > >>> change.
> > > >>> > > > >> > > > > > > > >> > > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > Please let me know if there is
> > > >>> anything
> > > >>> > > else
> > > >>> > > > we
> > > >>> > > > >> > > should
> > > >>> > > > >> > > > > > > discuss
> > > >>> > > > >> > > > > > > > >> > before
> > > >>> > > > >> > > > > > > > >> > > > > > voting.
> > > >>> > > > >> > > > > > > > >> > > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > Thank you,
> > > >>> > > > >> > > > > > > > >> > > > > > Justine
> > > >>> > > > >> > > > > > > > >> > > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > On Fri, Sep 18, 2020 at 9:46 AM
> > > >>> Justine
> > > >>> > > > Olshan
> > > >>> > > > >> <
> > > >>> > > > >> > > > > > > > >> > jolshan@confluent.io
> > > >>> > > > >> > > > > > > > >> > > >
> > > >>> > > > >> > > > > > > > >> > > > > > wrote:
> > > >>> > > > >> > > > > > > > >> > > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > > Hi Jun,
> > > >>> > > > >> > > > > > > > >> > > > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > > I see what you are saying.
> For
> > > now
> > > >>> we
> > > >>> > can
> > > >>> > > > >> remove
> > > >>> > > > >> > > the
> > > >>> > > > >> > > > > > extra
> > > >>> > > > >> > > > > > > > >> > > > information.
> > > >>> > > > >> > > > > > > > >> > > > > > > I'll leave the option to add
> > more
> > > >>> > fields
> > > >>> > > to
> > > >>> > > > >> the
> > > >>> > > > >> > > file
> > > >>> > > > >> > > > > in
> > > >>> > > > >> > > > > > > the
> > > >>> > > > >> > > > > > > > >> > future.
> > > >>> > > > >> > > > > > > > >> > > > The
> > > >>> > > > >> > > > > > > > >> > > > > > KIP
> > > >>> > > > >> > > > > > > > >> > > > > > > has been updated to reflect
> > this
> > > >>> > change.
> > > >>> > > > >> > > > > > > > >> > > > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > > Thanks,
> > > >>> > > > >> > > > > > > > >> > > > > > > Justine
> > > >>> > > > >> > > > > > > > >> > > > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > > On Fri, Sep 18, 2020 at 8:46
> AM
> > > Jun
> > > >>> > Rao <
> > > >>> > > > >> > > > > > jun@confluent.io
> > > >>> > > > >> > > > > > > >
> > > >>> > > > >> > > > > > > > >> > wrote:
> > > >>> > > > >> > > > > > > > >> > > > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> Hi, Justine,
> > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > >>> > > > >> > > > > > > > >> > > > > > >> Thanks for the reply.
> > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > >>> > > > >> > > > > > > > >> > > > > > >> 13. If the log directory is
> > the
> > > >>> source
> > > >>> > > of
> > > >>> > > > >> > truth,
> > > >>> > > > >> > > it
> > > >>> > > > >> > > > > > means
> > > >>> > > > >> > > > > > > > >> that
> > > >>> > > > >> > > > > > > > >> > the
> > > >>> > > > >> > > > > > > > >> > > > > > >> redundant info in the
> metadata
> > > >>> file
> > > >>> > will
> > > >>> > > > be
> > > >>> > > > >> > > > ignored.
> > > >>> > > > >> > > > > > Then
> > > >>> > > > >> > > > > > > > the
> > > >>> > > > >> > > > > > > > >> > > > question
> > > >>> > > > >> > > > > > > > >> > > > > > is
> > > >>> > > > >> > > > > > > > >> > > > > > >> why do we need to put the
> > > >>> redundant
> > > >>> > info
> > > >>> > > > in
> > > >>> > > > >> the
> > > >>> > > > >> > > > > > metadata
> > > >>> > > > >> > > > > > > > file
> > > >>> > > > >> > > > > > > > >> > now?
> > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > >>> > > > >> > > > > > > > >> > > > > > >> Thanks,
> > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > >>> > > > >> > > > > > > > >> > > > > > >> Jun
> > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > >>> > > > >> > > > > > > > >> > > > > > >> On Thu, Sep 17, 2020 at 5:07
> > PM
> > > >>> > Justine
> > > >>> > > > >> Olshan
> > > >>> > > > >> > <
> > > >>> > > > >> > > > > > > > >> > > > jolshan@confluent.io>
> > > >>> > > > >> > > > > > > > >> > > > > > >> wrote:
> > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > >>> > > > >> > > > > > > > >> > > > > > >> > Hi Jun,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > Thanks for the quick
> > response!
> > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > 12. I've decided to bump
> up
> > > the
> > > >>> > > versions
> > > >>> > > > >> on
> > > >>> > > > >> > the
> > > >>> > > > >> > > > > > > requests
> > > >>> > > > >> > > > > > > > >> and
> > > >>> > > > >> > > > > > > > >> > > > updated
> > > >>> > > > >> > > > > > > > >> > > > > > the
> > > >>> > > > >> > > > > > > > >> > > > > > >> > KIP. I think it's good we
> > > >>> thoroughly
> > > >>> > > > >> > discussed
> > > >>> > > > >> > > > the
> > > >>> > > > >> > > > > > > > options
> > > >>> > > > >> > > > > > > > >> > here,
> > > >>> > > > >> > > > > > > > >> > > > so
> > > >>> > > > >> > > > > > > > >> > > > > we
> > > >>> > > > >> > > > > > > > >> > > > > > >> know
> > > >>> > > > >> > > > > > > > >> > > > > > >> > we made a good choice. :)
> > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > 13. This is an interesting
> > > >>> > situation.
> > > >>> > > I
> > > >>> > > > >> think
> > > >>> > > > >> > > if
> > > >>> > > > >> > > > > this
> > > >>> > > > >> > > > > > > > does
> > > >>> > > > >> > > > > > > > >> > occur
> > > >>> > > > >> > > > > > > > >> > > > we
> > > >>> > > > >> > > > > > > > >> > > > > > >> should
> > > >>> > > > >> > > > > > > > >> > > > > > >> > give a warning. I agree
> that
> > > >>> it's
> > > >>> > hard
> > > >>> > > > to
> > > >>> > > > >> > know
> > > >>> > > > >> > > > the
> > > >>> > > > >> > > > > > > source
> > > >>> > > > >> > > > > > > > >> of
> > > >>> > > > >> > > > > > > > >> > > truth
> > > >>> > > > >> > > > > > > > >> > > > > for
> > > >>> > > > >> > > > > > > > >> > > > > > >> sure
> > > >>> > > > >> > > > > > > > >> > > > > > >> > since the directory or the
> > > file
> > > >>> > could
> > > >>> > > be
> > > >>> > > > >> > > manually
> > > >>> > > > >> > > > > > > > >> modified. I
> > > >>> > > > >> > > > > > > > >> > > > guess
> > > >>> > > > >> > > > > > > > >> > > > > > the
> > > >>> > > > >> > > > > > > > >> > > > > > >> > directory could be used as
> > the
> > > >>> > source
> > > >>> > > of
> > > >>> > > > >> > truth.
> > > >>> > > > >> > > > To
> > > >>> > > > >> > > > > be
> > > >>> > > > >> > > > > > > > >> honest,
> > > >>> > > > >> > > > > > > > >> > > I'm
> > > >>> > > > >> > > > > > > > >> > > > > not
> > > >>> > > > >> > > > > > > > >> > > > > > >> > really sure what happens
> in
> > > >>> kafka
> > > >>> > when
> > > >>> > > > the
> > > >>> > > > >> > log
> > > >>> > > > >> > > > > > > directory
> > > >>> > > > >> > > > > > > > is
> > > >>> > > > >> > > > > > > > >> > > > renamed
> > > >>> > > > >> > > > > > > > >> > > > > > >> > manually in such a way.
> I'm
> > > also
> > > >>> > > > >> wondering if
> > > >>> > > > >> > > the
> > > >>> > > > >> > > > > > > > >> situation is
> > > >>> > > > >> > > > > > > > >> > > > > > >> recoverable
> > > >>> > > > >> > > > > > > > >> > > > > > >> > in this scenario.
> > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > Thanks,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > Justine
> > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > On Thu, Sep 17, 2020 at
> 4:28
> > > PM
> > > >>> Jun
> > > >>> > > Rao
> > > >>> > > > <
> > > >>> > > > >> > > > > > > > jun@confluent.io>
> > > >>> > > > >> > > > > > > > >> > > wrote:
> > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > Hi, Justine,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > Thanks for the reply.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > 12. I don't have a
> strong
> > > >>> > preference
> > > >>> > > > >> > either.
> > > >>> > > > >> > > > > > However,
> > > >>> > > > >> > > > > > > > if
> > > >>> > > > >> > > > > > > > >> we
> > > >>> > > > >> > > > > > > > >> > > need
> > > >>> > > > >> > > > > > > > >> > > > > IBP
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > anyway, maybe it's
> easier
> > to
> > > >>> just
> > > >>> > > bump
> > > >>> > > > >> up
> > > >>> > > > >> > the
> > > >>> > > > >> > > > > > version
> > > >>> > > > >> > > > > > > > for
> > > >>> > > > >> > > > > > > > >> > all
> > > >>> > > > >> > > > > > > > >> > > > > inter
> > > >>> > > > >> > > > > > > > >> > > > > > >> > broker
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > requests and add the
> topic
> > > id
> > > >>> > field
> > > >>> > > > as a
> > > >>> > > > >> > > > regular
> > > >>> > > > >> > > > > > > > field. A
> > > >>> > > > >> > > > > > > > >> > > > regular
> > > >>> > > > >> > > > > > > > >> > > > > > >> field
> > > >>> > > > >> > > > > > > > >> > > > > > >> > is
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > a bit more concise in
> wire
> > > >>> > transfer
> > > >>> > > > >> than a
> > > >>> > > > >> > > > > flexible
> > > >>> > > > >> > > > > > > > >> field.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > 13. The confusion that I
> > was
> > > >>> > > referring
> > > >>> > > > >> to
> > > >>> > > > >> > is
> > > >>> > > > >> > > > > > between
> > > >>> > > > >> > > > > > > > the
> > > >>> > > > >> > > > > > > > >> > topic
> > > >>> > > > >> > > > > > > > >> > > > > name
> > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > partition number between
> > the
> > > >>> log
> > > >>> > dir
> > > >>> > > > and
> > > >>> > > > >> > the
> > > >>> > > > >> > > > > > metadata
> > > >>> > > > >> > > > > > > > >> file.
> > > >>> > > > >> > > > > > > > >> > > For
> > > >>> > > > >> > > > > > > > >> > > > > > >> example,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > if
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > the log dir is topicA-1
> > and
> > > >>> the
> > > >>> > > > metadata
> > > >>> > > > >> > file
> > > >>> > > > >> > > > in
> > > >>> > > > >> > > > > it
> > > >>> > > > >> > > > > > > has
> > > >>> > > > >> > > > > > > > >> > topicB
> > > >>> > > > >> > > > > > > > >> > > > and
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > partition 0 (say due to
> a
> > > bug
> > > >>> or
> > > >>> > > > manual
> > > >>> > > > >> > > > > > > modification),
> > > >>> > > > >> > > > > > > > >> which
> > > >>> > > > >> > > > > > > > >> > > one
> > > >>> > > > >> > > > > > > > >> > > > > do
> > > >>> > > > >> > > > > > > > >> > > > > > we
> > > >>> > > > >> > > > > > > > >> > > > > > >> > use
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > as the source of truth?
> > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > Jun
> > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > On Thu, Sep 17, 2020 at
> > 3:43
> > > >>> PM
> > > >>> > > > Justine
> > > >>> > > > >> > > Olshan
> > > >>> > > > >> > > > <
> > > >>> > > > >> > > > > > > > >> > > > > > jolshan@confluent.io>
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > wrote:
> > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > Hi Jun,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > Thanks for the
> comments.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > 12. I bumped the
> > > >>> > > LeaderAndIsrRequest
> > > >>> > > > >> > > because
> > > >>> > > > >> > > > I
> > > >>> > > > >> > > > > > > > removed
> > > >>> > > > >> > > > > > > > >> the
> > > >>> > > > >> > > > > > > > >> > > > topic
> > > >>> > > > >> > > > > > > > >> > > > > > >> name
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > field
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > in the response. It
> may
> > be
> > > >>> > > possible
> > > >>> > > > to
> > > >>> > > > >> > > avoid
> > > >>> > > > >> > > > > > > bumping
> > > >>> > > > >> > > > > > > > >> the
> > > >>> > > > >> > > > > > > > >> > > > version
> > > >>> > > > >> > > > > > > > >> > > > > > >> > without
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > that change, but I may
> > be
> > > >>> > missing
> > > >>> > > > >> > > something.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > I believe StopReplica
> is
> > > >>> > actually
> > > >>> > > on
> > > >>> > > > >> > > version
> > > >>> > > > >> > > > 3
> > > >>> > > > >> > > > > > now,
> > > >>> > > > >> > > > > > > > but
> > > >>> > > > >> > > > > > > > >> > > > because
> > > >>> > > > >> > > > > > > > >> > > > > > >> > version 2
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > is flexible, I kept
> that
> > > >>> listed
> > > >>> > as
> > > >>> > > > >> > version
> > > >>> > > > >> > > 2
> > > >>> > > > >> > > > on
> > > >>> > > > >> > > > > > the
> > > >>> > > > >> > > > > > > > KIP
> > > >>> > > > >> > > > > > > > >> > > page.
> > > >>> > > > >> > > > > > > > >> > > > > > >> However,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > you
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > may be right in that
> we
> > > may
> > > >>> need
> > > >>> > > to
> > > >>> > > > >> bump
> > > >>> > > > >> > > the
> > > >>> > > > >> > > > > > > version
> > > >>> > > > >> > > > > > > > on
> > > >>> > > > >> > > > > > > > >> > > > > > StopReplica
> > > >>> > > > >> > > > > > > > >> > > > > > >> to
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > deal
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > with deletion
> > differently
> > > as
> > > >>> > > > mentioned
> > > >>> > > > >> > > > above. I
> > > >>> > > > >> > > > > > > don't
> > > >>> > > > >> > > > > > > > >> know
> > > >>> > > > >> > > > > > > > >> > > if
> > > >>> > > > >> > > > > > > > >> > > > I
> > > >>> > > > >> > > > > > > > >> > > > > > >> have a
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > big
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > preference over used
> > > tagged
> > > >>> > fields
> > > >>> > > > or
> > > >>> > > > >> > not.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > 13. I was thinking
> that
> > in
> > > >>> the
> > > >>> > > case
> > > >>> > > > >> where
> > > >>> > > > >> > > the
> > > >>> > > > >> > > > > > file
> > > >>> > > > >> > > > > > > > and
> > > >>> > > > >> > > > > > > > >> the
> > > >>> > > > >> > > > > > > > >> > > > > request
> > > >>> > > > >> > > > > > > > >> > > > > > >> > topic
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > ids don't match, it
> > means
> > > >>> that
> > > >>> > the
> > > >>> > > > >> > broker's
> > > >>> > > > >> > > > > > > topic/the
> > > >>> > > > >> > > > > > > > >> one
> > > >>> > > > >> > > > > > > > >> > in
> > > >>> > > > >> > > > > > > > >> > > > the
> > > >>> > > > >> > > > > > > > >> > > > > > >> file
> > > >>> > > > >> > > > > > > > >> > > > > > >> > has
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > been deleted. In that
> > > case,
> > > >>> we
> > > >>> > > would
> > > >>> > > > >> need
> > > >>> > > > >> > > to
> > > >>> > > > >> > > > > > delete
> > > >>> > > > >> > > > > > > > the
> > > >>> > > > >> > > > > > > > >> > old
> > > >>> > > > >> > > > > > > > >> > > > > topic
> > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > start
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > receiving the new
> > version.
> > > >>> If
> > > >>> > the
> > > >>> > > > >> topic
> > > >>> > > > >> > > name
> > > >>> > > > >> > > > > were
> > > >>> > > > >> > > > > > > to
> > > >>> > > > >> > > > > > > > >> > change,
> > > >>> > > > >> > > > > > > > >> > > > but
> > > >>> > > > >> > > > > > > > >> > > > > > the
> > > >>> > > > >> > > > > > > > >> > > > > > >> > ids
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > still matched, the
> file
> > > >>> would
> > > >>> > also
> > > >>> > > > >> need
> > > >>> > > > >> > to
> > > >>> > > > >> > > > > > update.
> > > >>> > > > >> > > > > > > > Am I
> > > >>> > > > >> > > > > > > > >> > > > missing
> > > >>> > > > >> > > > > > > > >> > > > > a
> > > >>> > > > >> > > > > > > > >> > > > > > >> case
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > where the file would
> be
> > > >>> correct
> > > >>> > > and
> > > >>> > > > >> not
> > > >>> > > > >> > the
> > > >>> > > > >> > > > > > > request?
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > Thanks,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > Justine
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > On Thu, Sep 17, 2020
> at
> > > >>> 3:18 PM
> > > >>> > > Jun
> > > >>> > > > >> Rao <
> > > >>> > > > >> > > > > > > > >> jun@confluent.io
> > > >>> > > > >> > > > > > > > >> > >
> > > >>> > > > >> > > > > > > > >> > > > > wrote:
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Hi, Justine,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Thanks for the
> reply.
> > A
> > > >>> couple
> > > >>> > > of
> > > >>> > > > >> more
> > > >>> > > > >> > > > > comments
> > > >>> > > > >> > > > > > > > >> below.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > 12. ListOffset and
> > > >>> > > OffsetForLeader
> > > >>> > > > >> > > > currently
> > > >>> > > > >> > > > > > > don't
> > > >>> > > > >> > > > > > > > >> > support
> > > >>> > > > >> > > > > > > > >> > > > > > >> flexible
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > fields.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > So, we have to bump
> up
> > > the
> > > >>> > > version
> > > >>> > > > >> > number
> > > >>> > > > >> > > > and
> > > >>> > > > >> > > > > > use
> > > >>> > > > >> > > > > > > > >> IBP at
> > > >>> > > > >> > > > > > > > >> > > > least
> > > >>> > > > >> > > > > > > > >> > > > > > for
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > these
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > two requests. Note
> > that
> > > it
> > > >>> > seems
> > > >>> > > > >> 2.7.0
> > > >>> > > > >> > > will
> > > >>> > > > >> > > > > > > require
> > > >>> > > > >> > > > > > > > >> IBP
> > > >>> > > > >> > > > > > > > >> > > > anyway
> > > >>> > > > >> > > > > > > > >> > > > > > >> > because
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > of
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > changes in
> > KAFKA-10435.
> > > >>> Also,
> > > >>> > it
> > > >>> > > > >> seems
> > > >>> > > > >> > > that
> > > >>> > > > >> > > > > the
> > > >>> > > > >> > > > > > > > >> version
> > > >>> > > > >> > > > > > > > >> > > for
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > LeaderAndIsrRequest
> > and
> > > >>> > > > StopReplica
> > > >>> > > > >> are
> > > >>> > > > >> > > > > bumped
> > > >>> > > > >> > > > > > > even
> > > >>> > > > >> > > > > > > > >> > though
> > > >>> > > > >> > > > > > > > >> > > > we
> > > >>> > > > >> > > > > > > > >> > > > > > only
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > added
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > a
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > tagged field. But
> > since
> > > >>> IBP is
> > > >>> > > > >> needed
> > > >>> > > > >> > > > anyway,
> > > >>> > > > >> > > > > > we
> > > >>> > > > >> > > > > > > > may
> > > >>> > > > >> > > > > > > > >> > want
> > > >>> > > > >> > > > > > > > >> > > to
> > > >>> > > > >> > > > > > > > >> > > > > > >> revisit
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > the
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > overall tagged field
> > > >>> choice.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > 13. The only
> downside
> > is
> > > >>> the
> > > >>> > > > >> potential
> > > >>> > > > >> > > > > > confusion
> > > >>> > > > >> > > > > > > on
> > > >>> > > > >> > > > > > > > >> > which
> > > >>> > > > >> > > > > > > > >> > > > one
> > > >>> > > > >> > > > > > > > >> > > > > is
> > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > source
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > of truth if they
> don't
> > > >>> match.
> > > >>> > > > >> Another
> > > >>> > > > >> > > > option
> > > >>> > > > >> > > > > is
> > > >>> > > > >> > > > > > > to
> > > >>> > > > >> > > > > > > > >> > include
> > > >>> > > > >> > > > > > > > >> > > > > those
> > > >>> > > > >> > > > > > > > >> > > > > > >> > fields
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > in
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the metadata file
> when
> > > we
> > > >>> > > actually
> > > >>> > > > >> > change
> > > >>> > > > >> > > > the
> > > >>> > > > >> > > > > > > > >> directory
> > > >>> > > > >> > > > > > > > >> > > > > > structure.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Thanks,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Jun
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > On Thu, Sep 17, 2020
> > at
> > > >>> 2:01
> > > >>> > PM
> > > >>> > > > >> Justine
> > > >>> > > > >> > > > > Olshan
> > > >>> > > > >> > > > > > <
> > > >>> > > > >> > > > > > > > >> > > > > > >> jolshan@confluent.io
> > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > wrote:
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > Hello all,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I've thought some
> > more
> > > >>> about
> > > >>> > > > >> removing
> > > >>> > > > >> > > the
> > > >>> > > > >> > > > > > topic
> > > >>> > > > >> > > > > > > > >> name
> > > >>> > > > >> > > > > > > > >> > > field
> > > >>> > > > >> > > > > > > > >> > > > > > from
> > > >>> > > > >> > > > > > > > >> > > > > > >> > some
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > of
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > requests. On
> closer
> > > >>> > inspection
> > > >>> > > > of
> > > >>> > > > >> the
> > > >>> > > > >> > > > > > > > >> > > requests/responses,
> > > >>> > > > >> > > > > > > > >> > > > it
> > > >>> > > > >> > > > > > > > >> > > > > > >> seems
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > internal changes
> > would
> > > >>> be
> > > >>> > much
> > > >>> > > > >> larger
> > > >>> > > > >> > > > than
> > > >>> > > > >> > > > > I
> > > >>> > > > >> > > > > > > > >> expected.
> > > >>> > > > >> > > > > > > > >> > > > Some
> > > >>> > > > >> > > > > > > > >> > > > > > >> > protocols
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > involve clients,
> so
> > > they
> > > >>> > would
> > > >>> > > > >> > require
> > > >>> > > > >> > > > > > changes
> > > >>> > > > >> > > > > > > > too.
> > > >>> > > > >> > > > > > > > >> > I'm
> > > >>> > > > >> > > > > > > > >> > > > > > thinking
> > > >>> > > > >> > > > > > > > >> > > > > > >> > that
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > for
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > now, removing the
> > > topic
> > > >>> name
> > > >>> > > > from
> > > >>> > > > >> > these
> > > >>> > > > >> > > > > > > requests
> > > >>> > > > >> > > > > > > > >> and
> > > >>> > > > >> > > > > > > > >> > > > > responses
> > > >>> > > > >> > > > > > > > >> > > > > > >> are
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > out
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > scope.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I have decided to
> > just
> > > >>> keep
> > > >>> > > the
> > > >>> > > > >> > change
> > > >>> > > > >> > > > > > > > >> > > > LeaderAndIsrResponse
> > > >>> > > > >> > > > > > > > >> > > > > to
> > > >>> > > > >> > > > > > > > >> > > > > > >> > remove
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > the
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > topic name, and
> have
> > > >>> updated
> > > >>> > > the
> > > >>> > > > >> KIP
> > > >>> > > > >> > to
> > > >>> > > > >> > > > > > reflect
> > > >>> > > > >> > > > > > > > >> this
> > > >>> > > > >> > > > > > > > >> > > > > change. I
> > > >>> > > > >> > > > > > > > >> > > > > > >> have
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > also
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > mentioned the
> other
> > > >>> requests
> > > >>> > > and
> > > >>> > > > >> > > > responses
> > > >>> > > > >> > > > > in
> > > >>> > > > >> > > > > > > > >> future
> > > >>> > > > >> > > > > > > > >> > > work.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I'm hoping to
> start
> > > the
> > > >>> > voting
> > > >>> > > > >> > process
> > > >>> > > > >> > > > > soon,
> > > >>> > > > >> > > > > > so
> > > >>> > > > >> > > > > > > > >> let me
> > > >>> > > > >> > > > > > > > >> > > > know
> > > >>> > > > >> > > > > > > > >> > > > > if
> > > >>> > > > >> > > > > > > > >> > > > > > >> > there
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > is
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > anything else we
> > > should
> > > >>> > > discuss.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > Thank you,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > Justine
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > On Tue, Sep 15,
> 2020
> > > at
> > > >>> 3:57
> > > >>> > > PM
> > > >>> > > > >> > Justine
> > > >>> > > > >> > > > > > Olshan
> > > >>> > > > >> > > > > > > <
> > > >>> > > > >> > > > > > > > >> > > > > > >> > jolshan@confluent.io
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > wrote:
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > Hello again,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > To follow up on
> > some
> > > >>> of
> > > >>> > the
> > > >>> > > > >> other
> > > >>> > > > >> > > > > comments:
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 10/11) We can
> > remove
> > > >>> the
> > > >>> > > topic
> > > >>> > > > >> name
> > > >>> > > > >> > > > from
> > > >>> > > > >> > > > > > > these
> > > >>> > > > >> > > > > > > > >> > > > > > >> > requests/responses,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > and
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > that means we
> will
> > > >>> just
> > > >>> > have
> > > >>> > > > to
> > > >>> > > > >> > make
> > > >>> > > > >> > > a
> > > >>> > > > >> > > > > few
> > > >>> > > > >> > > > > > > > >> internal
> > > >>> > > > >> > > > > > > > >> > > > > changes
> > > >>> > > > >> > > > > > > > >> > > > > > to
> > > >>> > > > >> > > > > > > > >> > > > > > >> > make
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > partitions
> > > accessible
> > > >>> by
> > > >>> > > topic
> > > >>> > > > >> id
> > > >>> > > > >> > and
> > > >>> > > > >> > > > > > > > partition.
> > > >>> > > > >> > > > > > > > >> I
> > > >>> > > > >> > > > > > > > >> > can
> > > >>> > > > >> > > > > > > > >> > > > > > update
> > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > KIP
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > remove them
> unless
> > > >>> anyone
> > > >>> > > > thinks
> > > >>> > > > >> > they
> > > >>> > > > >> > > > > > should
> > > >>> > > > >> > > > > > > > >> stay.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 12) Addressed in
> > the
> > > >>> > > previous
> > > >>> > > > >> > email.
> > > >>> > > > >> > > > I've
> > > >>> > > > >> > > > > > > > updated
> > > >>> > > > >> > > > > > > > >> > the
> > > >>> > > > >> > > > > > > > >> > > > KIP
> > > >>> > > > >> > > > > > > > >> > > > > to
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > include
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > tagged fields
> for
> > > the
> > > >>> > > requests
> > > >>> > > > >> and
> > > >>> > > > >> > > > > > responses.
> > > >>> > > > >> > > > > > > > >> (More
> > > >>> > > > >> > > > > > > > >> > on
> > > >>> > > > >> > > > > > > > >> > > > > that
> > > >>> > > > >> > > > > > > > >> > > > > > >> > below)
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 13) I think part
> > of
> > > >>> the
> > > >>> > idea
> > > >>> > > > for
> > > >>> > > > >> > > > > including
> > > >>> > > > >> > > > > > > this
> > > >>> > > > >> > > > > > > > >> > > > > information
> > > >>> > > > >> > > > > > > > >> > > > > > >> is to
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > prepare
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > for future
> > changes.
> > > >>> > Perhaps
> > > >>> > > > the
> > > >>> > > > >> > > > directory
> > > >>> > > > >> > > > > > > > >> structure
> > > >>> > > > >> > > > > > > > >> > > > might
> > > >>> > > > >> > > > > > > > >> > > > > > >> change
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > from
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > >>> topicName_partitionNumber
> > > >>> > to
> > > >>> > > > >> > > something
> > > >>> > > > >> > > > > like
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > topicID_partitionNumber.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Then
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > it would be
> useful
> > > to
> > > >>> have
> > > >>> > > the
> > > >>> > > > >> > topic
> > > >>> > > > >> > > > name
> > > >>> > > > >> > > > > > in
> > > >>> > > > >> > > > > > > > the
> > > >>> > > > >> > > > > > > > >> > file
> > > >>> > > > >> > > > > > > > >> > > > > since
> > > >>> > > > >> > > > > > > > >> > > > > > it
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > would
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > not
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > be
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > in the directory
> > > >>> > structure.
> > > >>> > > > >> > > Supporting
> > > >>> > > > >> > > > > > topic
> > > >>> > > > >> > > > > > > > >> renames
> > > >>> > > > >> > > > > > > > >> > > > might
> > > >>> > > > >> > > > > > > > >> > > > > > be
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > easier
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > if
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > the
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > other fields are
> > > >>> included.
> > > >>> > > > Would
> > > >>> > > > >> > > there
> > > >>> > > > >> > > > be
> > > >>> > > > >> > > > > > any
> > > >>> > > > >> > > > > > > > >> > > downsides
> > > >>> > > > >> > > > > > > > >> > > > to
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > including
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > this
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > information?
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 14)  Yes, we
> would
> > > >>> need to
> > > >>> > > > copy
> > > >>> > > > >> the
> > > >>> > > > >> > > > > > partition
> > > >>> > > > >> > > > > > > > >> > metadata
> > > >>> > > > >> > > > > > > > >> > > > > file
> > > >>> > > > >> > > > > > > > >> > > > > > in
> > > >>> > > > >> > > > > > > > >> > > > > > >> > this
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > process. I've
> > > updated
> > > >>> the
> > > >>> > > KIP
> > > >>> > > > to
> > > >>> > > > >> > > > include
> > > >>> > > > >> > > > > > > this.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 15) I believe
> > Lucas
> > > >>> meant
> > > >>> > v1
> > > >>> > > > >> and v2
> > > >>> > > > >> > > > here.
> > > >>> > > > >> > > > > > He
> > > >>> > > > >> > > > > > > > was
> > > >>> > > > >> > > > > > > > >> > > > referring
> > > >>> > > > >> > > > > > > > >> > > > > > to
> > > >>> > > > >> > > > > > > > >> > > > > > >> how
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > the
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > requests would
> > fall
> > > >>> under
> > > >>> > > > >> different
> > > >>> > > > >> > > IBP
> > > >>> > > > >> > > > > and
> > > >>> > > > >> > > > > > > > meant
> > > >>> > > > >> > > > > > > > >> > that
> > > >>> > > > >> > > > > > > > >> > > > > older
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > brokers
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > would
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > have to use the
> > > older
> > > >>> > > version
> > > >>> > > > of
> > > >>> > > > >> > the
> > > >>> > > > >> > > > > > request
> > > >>> > > > >> > > > > > > > and
> > > >>> > > > >> > > > > > > > >> the
> > > >>> > > > >> > > > > > > > >> > > > > > existing
> > > >>> > > > >> > > > > > > > >> > > > > > >> > topic
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > deletion
> process.
> > At
> > > >>> > first,
> > > >>> > > it
> > > >>> > > > >> > seemed
> > > >>> > > > >> > > > > like
> > > >>> > > > >> > > > > > > > tagged
> > > >>> > > > >> > > > > > > > >> > > fields
> > > >>> > > > >> > > > > > > > >> > > > > > would
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > resolve
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > the IBP issue.
> > > >>> However, we
> > > >>> > > may
> > > >>> > > > >> need
> > > >>> > > > >> > > IBP
> > > >>> > > > >> > > > > for
> > > >>> > > > >> > > > > > > > this
> > > >>> > > > >> > > > > > > > >> > > request
> > > >>> > > > >> > > > > > > > >> > > > > > after
> > > >>> > > > >> > > > > > > > >> > > > > > >> > all
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > since
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > the controller
> > > >>> handles the
> > > >>> > > > topic
> > > >>> > > > >> > > > deletion
> > > >>> > > > >> > > > > > > > >> > differently
> > > >>> > > > >> > > > > > > > >> > > > > > >> depending
> > > >>> > > > >> > > > > > > > >> > > > > > >> > on
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > the
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > IBP
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > version. In an
> > older
> > > >>> > > version,
> > > >>> > > > we
> > > >>> > > > >> > > can't
> > > >>> > > > >> > > > > just
> > > >>> > > > >> > > > > > > > send
> > > >>> > > > >> > > > > > > > >> a
> > > >>> > > > >> > > > > > > > >> > > > > > StopReplica
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > delete
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > topic
> immediately
> > > like
> > > >>> > we'd
> > > >>> > > > >> want to
> > > >>> > > > >> > > for
> > > >>> > > > >> > > > > > this
> > > >>> > > > >> > > > > > > > KIP.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > This makes me
> > wonder
> > > >>> if we
> > > >>> > > > want
> > > >>> > > > >> > > tagged
> > > >>> > > > >> > > > > > fields
> > > >>> > > > >> > > > > > > > on
> > > >>> > > > >> > > > > > > > >> all
> > > >>> > > > >> > > > > > > > >> > > the
> > > >>> > > > >> > > > > > > > >> > > > > > >> requests
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > after
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > all. Let me know
> > > your
> > > >>> > > > thoughts!
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > Justine
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > On Tue, Sep 15,
> > 2020
> > > >>> at
> > > >>> > 1:03
> > > >>> > > > PM
> > > >>> > > > >> > > Justine
> > > >>> > > > >> > > > > > > Olshan
> > > >>> > > > >> > > > > > > > <
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > jolshan@confluent.io
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > wrote:
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Hi all,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Jun brought up
> a
> > > good
> > > >>> > point
> > > >>> > > > in
> > > >>> > > > >> his
> > > >>> > > > >> > > > last
> > > >>> > > > >> > > > > > > email
> > > >>> > > > >> > > > > > > > >> about
> > > >>> > > > >> > > > > > > > >> > > > > tagged
> > > >>> > > > >> > > > > > > > >> > > > > > >> > fields,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > and
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> I've updated
> the
> > > KIP
> > > >>> to
> > > >>> > > > reflect
> > > >>> > > > >> > that
> > > >>> > > > >> > > > the
> > > >>> > > > >> > > > > > > > >> changes to
> > > >>> > > > >> > > > > > > > >> > > > > > requests
> > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > responses
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> will be in the
> > form
> > > >>> of
> > > >>> > > tagged
> > > >>> > > > >> > fields
> > > >>> > > > >> > > > to
> > > >>> > > > >> > > > > > > avoid
> > > >>> > > > >> > > > > > > > >> > > changing
> > > >>> > > > >> > > > > > > > >> > > > > IBP.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Jun: I plan on
> > > >>> sending a
> > > >>> > > > >> followup
> > > >>> > > > >> > > > email
> > > >>> > > > >> > > > > to
> > > >>> > > > >> > > > > > > > >> address
> > > >>> > > > >> > > > > > > > >> > > some
> > > >>> > > > >> > > > > > > > >> > > > > of
> > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > other
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> points.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Thanks,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Justine
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> On Mon, Sep 14,
> > > 2020
> > > >>> at
> > > >>> > > 4:25
> > > >>> > > > PM
> > > >>> > > > >> > Jun
> > > >>> > > > >> > > > Rao
> > > >>> > > > >> > > > > <
> > > >>> > > > >> > > > > > > > >> > > > > jun@confluent.io>
> > > >>> > > > >> > > > > > > > >> > > > > > >> > wrote:
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Hi, Justine,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Thanks for the
> > > >>> updated
> > > >>> > > KIP.
> > > >>> > > > A
> > > >>> > > > >> few
> > > >>> > > > >> > > > > > comments
> > > >>> > > > >> > > > > > > > >> below.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 10.
> LeaderAndIsr
> > > >>> > Response:
> > > >>> > > > Do
> > > >>> > > > >> we
> > > >>> > > > >> > > need
> > > >>> > > > >> > > > > the
> > > >>> > > > >> > > > > > > > topic
> > > >>> > > > >> > > > > > > > >> > > name?
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 11. For the
> > > changed
> > > >>> > > > >> > > request/response,
> > > >>> > > > >> > > > > > other
> > > >>> > > > >> > > > > > > > >> than
> > > >>> > > > >> > > > > > > > >> > > > > > >> LeaderAndIsr,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> UpdateMetadata,
> > > >>> > Metadata,
> > > >>> > > do
> > > >>> > > > >> we
> > > >>> > > > >> > > need
> > > >>> > > > >> > > > to
> > > >>> > > > >> > > > > > > > include
> > > >>> > > > >> > > > > > > > >> > the
> > > >>> > > > >> > > > > > > > >> > > > > topic
> > > >>> > > > >> > > > > > > > >> > > > > > >> name?
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 12. It seems
> > that
> > > >>> > upgrades
> > > >>> > > > >> don't
> > > >>> > > > >> > > > > require
> > > >>> > > > >> > > > > > > IBP.
> > > >>> > > > >> > > > > > > > >> Does
> > > >>> > > > >> > > > > > > > >> > > > that
> > > >>> > > > >> > > > > > > > >> > > > > > mean
> > > >>> > > > >> > > > > > > > >> > > > > > >> > the
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > new
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> fields
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> in all the
> > > >>> > > request/response
> > > >>> > > > >> are
> > > >>> > > > >> > > added
> > > >>> > > > >> > > > > as
> > > >>> > > > >> > > > > > > > tagged
> > > >>> > > > >> > > > > > > > >> > > fields
> > > >>> > > > >> > > > > > > > >> > > > > > >> without
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > bumping
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > up
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> the request
> > > >>> version? It
> > > >>> > > > would
> > > >>> > > > >> be
> > > >>> > > > >> > > > useful
> > > >>> > > > >> > > > > > to
> > > >>> > > > >> > > > > > > > make
> > > >>> > > > >> > > > > > > > >> > that
> > > >>> > > > >> > > > > > > > >> > > > > > clear.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 13. Partition
> > > >>> Metadata
> > > >>> > > file:
> > > >>> > > > >> Do
> > > >>> > > > >> > we
> > > >>> > > > >> > > > need
> > > >>> > > > >> > > > > > to
> > > >>> > > > >> > > > > > > > >> include
> > > >>> > > > >> > > > > > > > >> > > the
> > > >>> > > > >> > > > > > > > >> > > > > > topic
> > > >>> > > > >> > > > > > > > >> > > > > > >> > name
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > and
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > the
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> partition id
> > since
> > > >>> they
> > > >>> > > are
> > > >>> > > > >> > implied
> > > >>> > > > >> > > > in
> > > >>> > > > >> > > > > > the
> > > >>> > > > >> > > > > > > > >> > directory
> > > >>> > > > >> > > > > > > > >> > > > > name?
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 14. In the
> JBOD
> > > >>> mode, we
> > > >>> > > > >> support
> > > >>> > > > >> > > > > moving a
> > > >>> > > > >> > > > > > > > >> > > partition's
> > > >>> > > > >> > > > > > > > >> > > > > data
> > > >>> > > > >> > > > > > > > >> > > > > > >> from
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > one
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > disk
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> to
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> another. Will
> > the
> > > >>> new
> > > >>> > > > >> partition
> > > >>> > > > >> > > > > metadata
> > > >>> > > > >> > > > > > > file
> > > >>> > > > >> > > > > > > > >> be
> > > >>> > > > >> > > > > > > > >> > > > copied
> > > >>> > > > >> > > > > > > > >> > > > > > >> during
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> process?
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 15. The KIP
> says
> > > >>> "Remove
> > > >>> > > > >> deleted
> > > >>> > > > >> > > > topics
> > > >>> > > > >> > > > > > > from
> > > >>> > > > >> > > > > > > > >> > > replicas
> > > >>> > > > >> > > > > > > > >> > > > by
> > > >>> > > > >> > > > > > > > >> > > > > > >> > sending
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > StopReplicaRequest
> > > >>> V2
> > > >>> > for
> > > >>> > > > any
> > > >>> > > > >> > > topics
> > > >>> > > > >> > > > > > which
> > > >>> > > > >> > > > > > > do
> > > >>> > > > >> > > > > > > > >> not
> > > >>> > > > >> > > > > > > > >> > > > > contain
> > > >>> > > > >> > > > > > > > >> > > > > > a
> > > >>> > > > >> > > > > > > > >> > > > > > >> > topic
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > ID,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > and
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> V3 for any
> > topics
> > > >>> which
> > > >>> > do
> > > >>> > > > >> > contain
> > > >>> > > > >> > > a
> > > >>> > > > >> > > > > > topic
> > > >>> > > > >> > > > > > > > >> ID.".
> > > >>> > > > >> > > > > > > > >> > > > > However,
> > > >>> > > > >> > > > > > > > >> > > > > > it
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > seems
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> updated
> > controller
> > > >>> will
> > > >>> > > > create
> > > >>> > > > >> > all
> > > >>> > > > >> > > > > > missing
> > > >>> > > > >> > > > > > > > >> topic
> > > >>> > > > >> > > > > > > > >> > IDs
> > > >>> > > > >> > > > > > > > >> > > > > first
> > > >>> > > > >> > > > > > > > >> > > > > > >> > before
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > doing
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> other actions.
> > So,
> > > >>> is
> > > >>> > > > >> > > > > StopReplicaRequest
> > > >>> > > > >> > > > > > V2
> > > >>> > > > >> > > > > > > > >> > needed?
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Jun
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> On Fri, Sep
> 11,
> > > >>> 2020 at
> > > >>> > > > 10:31
> > > >>> > > > >> AM
> > > >>> > > > >> > > John
> > > >>> > > > >> > > > > > > > Roesler <
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > vvcephei@apache.org
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> wrote:
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > Thanks,
> > Justine!
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > Your
> response
> > > >>> seems
> > > >>> > > > >> compelling
> > > >>> > > > >> > to
> > > >>> > > > >> > > > me.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > -John
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > On Fri,
> > > >>> 2020-09-11 at
> > > >>> > > > 10:17
> > > >>> > > > >> > > -0700,
> > > >>> > > > >> > > > > > > Justine
> > > >>> > > > >> > > > > > > > >> > Olshan
> > > >>> > > > >> > > > > > > > >> > > > > wrote:
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Hello all,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Thanks for
> > > >>> > continuing
> > > >>> > > > the
> > > >>> > > > >> > > > > > discussion! I
> > > >>> > > > >> > > > > > > > >> have a
> > > >>> > > > >> > > > > > > > >> > > few
> > > >>> > > > >> > > > > > > > >> > > > > > >> > responses
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > to
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > your
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > points.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Tom: You
> are
> > > >>> correct
> > > >>> > > in
> > > >>> > > > >> that
> > > >>> > > > >> > > this
> > > >>> > > > >> > > > > KIP
> > > >>> > > > >> > > > > > > has
> > > >>> > > > >> > > > > > > > >> not
> > > >>> > > > >> > > > > > > > >> > > > > > mentioned
> > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > >>> > DeleteTopicsRequest. I
> > > >>> > > > >> think
> > > >>> > > > >> > > that
> > > >>> > > > >> > > > > > this
> > > >>> > > > >> > > > > > > > >> would
> > > >>> > > > >> > > > > > > > >> > be
> > > >>> > > > >> > > > > > > > >> > > > out
> > > >>> > > > >> > > > > > > > >> > > > > of
> > > >>> > > > >> > > > > > > > >> > > > > > >> > scope
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > for
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> now, but
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > may be
> > > something
> > > >>> > worth
> > > >>> > > > >> adding
> > > >>> > > > >> > > in
> > > >>> > > > >> > > > > the
> > > >>> > > > >> > > > > > > > >> future.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > John: We
> did
> > > >>> > consider
> > > >>> > > > >> > sequence
> > > >>> > > > >> > > > ids,
> > > >>> > > > >> > > > > > but
> > > >>> > > > >> > > > > > > > >> there
> > > >>> > > > >> > > > > > > > >> > > are
> > > >>> > > > >> > > > > > > > >> > > > a
> > > >>> > > > >> > > > > > > > >> > > > > > few
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > reasons
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> favor
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > UUIDs.
> There
> > > are
> > > >>> > > several
> > > >>> > > > >> > cases
> > > >>> > > > >> > > > > where
> > > >>> > > > >> > > > > > > > topics
> > > >>> > > > >> > > > > > > > >> > from
> > > >>> > > > >> > > > > > > > >> > > > > > >> different
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > clusters
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> may
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > interact
> now
> > > >>> and in
> > > >>> > > the
> > > >>> > > > >> > future.
> > > >>> > > > >> > > > For
> > > >>> > > > >> > > > > > > > >> example,
> > > >>> > > > >> > > > > > > > >> > > > Mirror
> > > >>> > > > >> > > > > > > > >> > > > > > >> Maker 2
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > may
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> benefit
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > from being
> > > able
> > > >>> to
> > > >>> > > > detect
> > > >>> > > > >> > when
> > > >>> > > > >> > > a
> > > >>> > > > >> > > > > > > cluster
> > > >>> > > > >> > > > > > > > >> being
> > > >>> > > > >> > > > > > > > >> > > > > > mirrored
> > > >>> > > > >> > > > > > > > >> > > > > > >> is
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > deleted
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> and
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > recreated
> > and
> > > >>> > globally
> > > >>> > > > >> unique
> > > >>> > > > >> > > > > > > identifiers
> > > >>> > > > >> > > > > > > > >> > would
> > > >>> > > > >> > > > > > > > >> > > > make
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > resolving
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > issues
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > easier
> than
> > > >>> sequence
> > > >>> > > IDs
> > > >>> > > > >> > which
> > > >>> > > > >> > > > may
> > > >>> > > > >> > > > > > > > collide
> > > >>> > > > >> > > > > > > > >> > > between
> > > >>> > > > >> > > > > > > > >> > > > > > >> > clusters.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > KIP-405
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > (tiered
> > > storage)
> > > >>> > will
> > > >>> > > > also
> > > >>> > > > >> > > > benefit
> > > >>> > > > >> > > > > > from
> > > >>> > > > >> > > > > > > > >> > globally
> > > >>> > > > >> > > > > > > > >> > > > > > unique
> > > >>> > > > >> > > > > > > > >> > > > > > >> IDs
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > as
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > shared
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > buckets
> may
> > be
> > > >>> used
> > > >>> > > > >> between
> > > >>> > > > >> > > > > clusters.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Globally
> > > unique
> > > >>> IDs
> > > >>> > > > would
> > > >>> > > > >> > also
> > > >>> > > > >> > > > make
> > > >>> > > > >> > > > > > > > >> > > functionality
> > > >>> > > > >> > > > > > > > >> > > > > like
> > > >>> > > > >> > > > > > > > >> > > > > > >> > moving
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > topics
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > between
> > > >>> disparate
> > > >>> > > > clusters
> > > >>> > > > >> > > easier
> > > >>> > > > >> > > > > in
> > > >>> > > > >> > > > > > > the
> > > >>> > > > >> > > > > > > > >> > future,
> > > >>> > > > >> > > > > > > > >> > > > > > >> simplify
> > > >>> > > > >> > > > > > > > >> > > > > > >> > any
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > future
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > implementations
> > > >>> of
> > > >>> > > > backups
> > > >>> > > > >> > and
> > > >>> > > > >> > > > > > > restores,
> > > >>> > > > >> > > > > > > > >> and
> > > >>> > > > >> > > > > > > > >> > > more.
> > > >>> > > > >> > > > > > > > >> > > > > In
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > general,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> unique IDs
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > would
> ensure
> > > >>> that
> > > >>> > the
> > > >>> > > > >> source
> > > >>> > > > >> > > > > cluster
> > > >>> > > > >> > > > > > > > >> topics do
> > > >>> > > > >> > > > > > > > >> > > not
> > > >>> > > > >> > > > > > > > >> > > > > > >> conflict
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > with
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > the
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> destination
> > > >>> cluster
> > > >>> > > > >> topics.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > If we were
> > to
> > > >>> use
> > > >>> > > > sequence
> > > >>> > > > >> > ids,
> > > >>> > > > >> > > > we
> > > >>> > > > >> > > > > > > would
> > > >>> > > > >> > > > > > > > >> need
> > > >>> > > > >> > > > > > > > >> > > > > > >> sufficiently
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > large
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> cluster
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > ids to be
> > > stored
> > > >>> > with
> > > >>> > > > the
> > > >>> > > > >> > topic
> > > >>> > > > >> > > > > > > > >> identifiers or
> > > >>> > > > >> > > > > > > > >> > > we
> > > >>> > > > >> > > > > > > > >> > > > > run
> > > >>> > > > >> > > > > > > > >> > > > > > >> the
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > risk
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> collisions.
> > > This
> > > >>> > will
> > > >>> > > > >> give up
> > > >>> > > > >> > > any
> > > >>> > > > >> > > > > > > > >> advantage in
> > > >>> > > > >> > > > > > > > >> > > > > > >> compactness
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> sequence
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > numbers
> may
> > > >>> bring.
> > > >>> > > Given
> > > >>> > > > >> > these
> > > >>> > > > >> > > > > > > > advantages I
> > > >>> > > > >> > > > > > > > >> > > think
> > > >>> > > > >> > > > > > > > >> > > > it
> > > >>> > > > >> > > > > > > > >> > > > > > >> makes
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > sense
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> use
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > UUIDs.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Gokul:
> This
> > is
> > > >>> an
> > > >>> > > > >> interesting
> > > >>> > > > >> > > > idea,
> > > >>> > > > >> > > > > > but
> > > >>> > > > >> > > > > > > > >> this
> > > >>> > > > >> > > > > > > > >> > is
> > > >>> > > > >> > > > > > > > >> > > a
> > > >>> > > > >> > > > > > > > >> > > > > > >> breaking
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > change.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Out of
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > scope for
> > now,
> > > >>> but
> > > >>> > > maybe
> > > >>> > > > >> > worth
> > > >>> > > > >> > > > > > > discussing
> > > >>> > > > >> > > > > > > > >> in
> > > >>> > > > >> > > > > > > > >> > the
> > > >>> > > > >> > > > > > > > >> > > > > > future.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Hope this
> > > >>> explains
> > > >>> > > some
> > > >>> > > > of
> > > >>> > > > >> > the
> > > >>> > > > >> > > > > > > decisions,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Justine
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > On Fri,
> Sep
> > > 11,
> > > >>> 2020
> > > >>> > > at
> > > >>> > > > >> 8:27
> > > >>> > > > >> > AM
> > > >>> > > > >> > > > > Gokul
> > > >>> > > > >> > > > > > > > >> Ramanan
> > > >>> > > > >> > > > > > > > >> > > > > > >> Subramanian <
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > >>> > gokul2411s@gmail.com>
> > > >>> > > > >> wrote:
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Hi.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Thanks
> for
> > > the
> > > >>> > KIP.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Have you
> > > >>> thought
> > > >>> > > about
> > > >>> > > > >> > > whether
> > > >>> > > > >> > > > it
> > > >>> > > > >> > > > > > > makes
> > > >>> > > > >> > > > > > > > >> > sense
> > > >>> > > > >> > > > > > > > >> > > to
> > > >>> > > > >> > > > > > > > >> > > > > > >> support
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> authorizing a
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> principal
> > > for
> > > >>> a
> > > >>> > > topic
> > > >>> > > > ID
> > > >>> > > > >> > > rather
> > > >>> > > > >> > > > > > than
> > > >>> > > > >> > > > > > > a
> > > >>> > > > >> > > > > > > > >> topic
> > > >>> > > > >> > > > > > > > >> > > > name
> > > >>> > > > >> > > > > > > > >> > > > > to
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > achieve
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> tighter
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> security?
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Or is
> the
> > > >>> topic ID
> > > >>> > > > >> > > > fundamentally
> > > >>> > > > >> > > > > an
> > > >>> > > > >> > > > > > > > >> internal
> > > >>> > > > >> > > > > > > > >> > > > > detail
> > > >>> > > > >> > > > > > > > >> > > > > > >> > similar
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > to
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> epochs
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > used
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > in a
> bunch
> > > of
> > > >>> > other
> > > >>> > > > >> places
> > > >>> > > > >> > in
> > > >>> > > > >> > > > > > Kafka?
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Thanks.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > On Fri,
> > Sep
> > > >>> 11,
> > > >>> > 2020
> > > >>> > > > at
> > > >>> > > > >> > 4:06
> > > >>> > > > >> > > PM
> > > >>> > > > >> > > > > > John
> > > >>> > > > >> > > > > > > > >> > Roesler <
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> vvcephei@apache.org
> > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > wrote:
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Hello
> > > >>> Justine,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Thanks
> > for
> > > >>> the
> > > >>> > > KIP!
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > I
> happen
> > > to
> > > >>> have
> > > >>> > > > been
> > > >>> > > > >> > > > > confronted
> > > >>> > > > >> > > > > > > > >> recently
> > > >>> > > > >> > > > > > > > >> > > with
> > > >>> > > > >> > > > > > > > >> > > > > the
> > > >>> > > > >> > > > > > > > >> > > > > > >> need
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > to
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > keep
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > track of
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > a
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > large
> > > >>> number of
> > > >>> > > > >> topics as
> > > >>> > > > >> > > > > > compactly
> > > >>> > > > >> > > > > > > > as
> > > >>> > > > >> > > > > > > > >> > > > > possible. I
> > > >>> > > > >> > > > > > > > >> > > > > > >> was
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > going
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> come
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > up
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > with
> > some
> > > >>> way to
> > > >>> > > > >> > dictionary
> > > >>> > > > >> > > > > > encode
> > > >>> > > > >> > > > > > > > the
> > > >>> > > > >> > > > > > > > >> > topic
> > > >>> > > > >> > > > > > > > >> > > > > names
> > > >>> > > > >> > > > > > > > >> > > > > > >> as
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > integers,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> but
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > this
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > seems
> > much
> > > >>> > better!
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > Apologies
> > > if
> > > >>> > this
> > > >>> > > > has
> > > >>> > > > >> > been
> > > >>> > > > >> > > > > raised
> > > >>> > > > >> > > > > > > > >> before,
> > > >>> > > > >> > > > > > > > >> > > but
> > > >>> > > > >> > > > > > > > >> > > > > I’m
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > wondering
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> about the
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > choice
> > of
> > > >>> UUID
> > > >>> > vs
> > > >>> > > > >> > sequence
> > > >>> > > > >> > > > > number
> > > >>> > > > >> > > > > > > for
> > > >>> > > > >> > > > > > > > >> the
> > > >>> > > > >> > > > > > > > >> > > ids.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > Typically,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > I’ve
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> seen
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > UUIDs
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > in two
> > > >>> > situations:
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 1.
> When
> > > >>> > processes
> > > >>> > > > >> need to
> > > >>> > > > >> > > > > > generate
> > > >>> > > > >> > > > > > > > >> > > > non-colliding
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > identifiers
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> without
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > >>> coordination.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 2.
> When
> > > the
> > > >>> > > > identifier
> > > >>> > > > >> > > needs
> > > >>> > > > >> > > > to
> > > >>> > > > >> > > > > > be
> > > >>> > > > >> > > > > > > > >> > > > “universally
> > > >>> > > > >> > > > > > > > >> > > > > > >> > unique”;
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > I.e.,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> the
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > identifier
> > > >>> needs
> > > >>> > > to
> > > >>> > > > >> > > > distinguish
> > > >>> > > > >> > > > > > the
> > > >>> > > > >> > > > > > > > >> entity
> > > >>> > > > >> > > > > > > > >> > > > from
> > > >>> > > > >> > > > > > > > >> > > > > > all
> > > >>> > > > >> > > > > > > > >> > > > > > >> > other
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> entities
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > that
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > could
> > ever
> > > >>> > exist.
> > > >>> > > > >> This is
> > > >>> > > > >> > > > > useful
> > > >>> > > > >> > > > > > in
> > > >>> > > > >> > > > > > > > >> cases
> > > >>> > > > >> > > > > > > > >> > > > where
> > > >>> > > > >> > > > > > > > >> > > > > > >> > entities
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > from
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > all
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > kinds
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > of
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> systems
> > > get
> > > >>> > mixed
> > > >>> > > > >> > together,
> > > >>> > > > >> > > > > such
> > > >>> > > > >> > > > > > as
> > > >>> > > > >> > > > > > > > >> when
> > > >>> > > > >> > > > > > > > >> > > > dumping
> > > >>> > > > >> > > > > > > > >> > > > > > >> logs
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > from
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > all
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > processes
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > in
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a
> > company
> > > >>> into a
> > > >>> > > > >> common
> > > >>> > > > >> > > > system.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Maybe
> > I’m
> > > >>> being
> > > >>> > > > >> > > > short-sighted,
> > > >>> > > > >> > > > > > but
> > > >>> > > > >> > > > > > > it
> > > >>> > > > >> > > > > > > > >> > > doesn’t
> > > >>> > > > >> > > > > > > > >> > > > > seem
> > > >>> > > > >> > > > > > > > >> > > > > > >> like
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > either
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> really
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> applies
> > > >>> here. It
> > > >>> > > > seems
> > > >>> > > > >> > like
> > > >>> > > > >> > > > the
> > > >>> > > > >> > > > > > > > brokers
> > > >>> > > > >> > > > > > > > >> > > could
> > > >>> > > > >> > > > > > > > >> > > > > and
> > > >>> > > > >> > > > > > > > >> > > > > > >> would
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > achieve
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > consensus
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > when
> > > >>> creating a
> > > >>> > > > topic
> > > >>> > > > >> > > anyway,
> > > >>> > > > >> > > > > > which
> > > >>> > > > >> > > > > > > > is
> > > >>> > > > >> > > > > > > > >> all
> > > >>> > > > >> > > > > > > > >> > > > > that’s
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > required
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > to
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > generate
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > >>> non-colliding
> > > >>> > > > sequence
> > > >>> > > > >> > ids.
> > > >>> > > > >> > > > For
> > > >>> > > > >> > > > > > the
> > > >>> > > > >> > > > > > > > >> > second,
> > > >>> > > > >> > > > > > > > >> > > as
> > > >>> > > > >> > > > > > > > >> > > > > you
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > mention,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > we
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> could
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > assign
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a UUID
> > to
> > > >>> the
> > > >>> > > > cluster
> > > >>> > > > >> as
> > > >>> > > > >> > a
> > > >>> > > > >> > > > > whole,
> > > >>> > > > >> > > > > > > > which
> > > >>> > > > >> > > > > > > > >> > > would
> > > >>> > > > >> > > > > > > > >> > > > > > render
> > > >>> > > > >> > > > > > > > >> > > > > > >> > any
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > resource
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > scoped
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > the
> > broker
> > > >>> > > > universally
> > > >>> > > > >> > > unique
> > > >>> > > > >> > > > > as
> > > >>> > > > >> > > > > > > > well.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > The
> > > reason I
> > > >>> > > mention
> > > >>> > > > >> this
> > > >>> > > > >> > > is
> > > >>> > > > >> > > > > > that,
> > > >>> > > > >> > > > > > > > >> > although
> > > >>> > > > >> > > > > > > > >> > > a
> > > >>> > > > >> > > > > > > > >> > > > > UUID
> > > >>> > > > >> > > > > > > > >> > > > > > >> is
> > > >>> > > > >> > > > > > > > >> > > > > > >> > way
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > more
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > compact
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > than
> > topic
> > > >>> > names,
> > > >>> > > > it’s
> > > >>> > > > >> > > still
> > > >>> > > > >> > > > 16
> > > >>> > > > >> > > > > > > > bytes.
> > > >>> > > > >> > > > > > > > >> In
> > > >>> > > > >> > > > > > > > >> > > > > > contrast,
> > > >>> > > > >> > > > > > > > >> > > > > > >> a
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > 4-byte
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> integer
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> sequence
> > > id
> > > >>> > would
> > > >>> > > > give
> > > >>> > > > >> > us 4
> > > >>> > > > >> > > > > > billion
> > > >>> > > > >> > > > > > > > >> unique
> > > >>> > > > >> > > > > > > > >> > > > > topics
> > > >>> > > > >> > > > > > > > >> > > > > > >> per
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > cluster,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> which
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > seems
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > like
> > > enough
> > > >>> ;)
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > Considering
> > > >>> the
> > > >>> > > > >> number of
> > > >>> > > > >> > > > > > different
> > > >>> > > > >> > > > > > > > >> times
> > > >>> > > > >> > > > > > > > >> > > > these
> > > >>> > > > >> > > > > > > > >> > > > > > >> topic
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> identifiers
> are
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > sent
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > over
> the
> > > >>> wire or
> > > >>> > > > >> stored
> > > >>> > > > >> > in
> > > >>> > > > >> > > > > > memory,
> > > >>> > > > >> > > > > > > it
> > > >>> > > > >> > > > > > > > >> > seems
> > > >>> > > > >> > > > > > > > >> > > > like
> > > >>> > > > >> > > > > > > > >> > > > > > it
> > > >>> > > > >> > > > > > > > >> > > > > > >> > might
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > be
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> worth
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > the
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > additional
> > > >>> 4x
> > > >>> > > space
> > > >>> > > > >> > > savings.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > What
> do
> > > you
> > > >>> > think
> > > >>> > > > >> about
> > > >>> > > > >> > > this?
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> Thanks,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > John
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > On
> Fri,
> > > Sep
> > > >>> 11,
> > > >>> > > > 2020,
> > > >>> > > > >> at
> > > >>> > > > >> > > > 03:20,
> > > >>> > > > >> > > > > > Tom
> > > >>> > > > >> > > > > > > > >> > Bentley
> > > >>> > > > >> > > > > > > > >> > > > > wrote:
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Hi
> > > >>> Justine,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > This
> > > looks
> > > >>> > like
> > > >>> > > a
> > > >>> > > > >> very
> > > >>> > > > >> > > > > welcome
> > > >>> > > > >> > > > > > > > >> > > improvement.
> > > >>> > > > >> > > > > > > > >> > > > > > >> Thanks!
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> Maybe
> > I
> > > >>> missed
> > > >>> > > it,
> > > >>> > > > >> but
> > > >>> > > > >> > > the
> > > >>> > > > >> > > > > KIP
> > > >>> > > > >> > > > > > > > >> doesn't
> > > >>> > > > >> > > > > > > > >> > > seem
> > > >>> > > > >> > > > > > > > >> > > > to
> > > >>> > > > >> > > > > > > > >> > > > > > >> > mention
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > changing
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > >>> > > > DeleteTopicsRequest
> > > >>> > > > >> to
> > > >>> > > > >> > > > > identify
> > > >>> > > > >> > > > > > > the
> > > >>> > > > >> > > > > > > > >> > topic
> > > >>> > > > >> > > > > > > > >> > > > > using
> > > >>> > > > >> > > > > > > > >> > > > > > an
> > > >>> > > > >> > > > > > > > >> > > > > > >> > id.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Maybe
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > that's out
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > of
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> scope,
> > > but
> > > >>> > > > >> > > > > DeleteTopicsRequest
> > > >>> > > > >> > > > > > is
> > > >>> > > > >> > > > > > > > not
> > > >>> > > > >> > > > > > > > >> > > listed
> > > >>> > > > >> > > > > > > > >> > > > > > among
> > > >>> > > > >> > > > > > > > >> > > > > > >> > the
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Future
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Work
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > APIs
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > either.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Kind
> > > >>> regards,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Tom
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > On
> > Thu,
> > > >>> Sep
> > > >>> > 10,
> > > >>> > > > >> 2020 at
> > > >>> > > > >> > > > 3:59
> > > >>> > > > >> > > > > PM
> > > >>> > > > >> > > > > > > > >> Satish
> > > >>> > > > >> > > > > > > > >> > > > > Duggana <
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > >>> > > > >> satish.duggana@gmail.com
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> wrote:
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > Thanks
> > > >>> > > > >> Lucas/Justine
> > > >>> > > > >> > > for
> > > >>> > > > >> > > > > the
> > > >>> > > > >> > > > > > > nice
> > > >>> > > > >> > > > > > > > >> KIP.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > It
> > has
> > > >>> > several
> > > >>> > > > >> > benefits
> > > >>> > > > >> > > > > which
> > > >>> > > > >> > > > > > > > also
> > > >>> > > > >> > > > > > > > >> > > include
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > simplifying
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> topic
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > deletion
> > > >>> > > process
> > > >>> > > > >> by
> > > >>> > > > >> > > > > > controller
> > > >>> > > > >> > > > > > > > and
> > > >>> > > > >> > > > > > > > >> > logs
> > > >>> > > > >> > > > > > > > >> > > > > > cleanup
> > > >>> > > > >> > > > > > > > >> > > > > > >> by
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > brokers
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > in
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > corner
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > cases.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > Best,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > Satish.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > On
> > > Wed,
> > > >>> Sep
> > > >>> > 9,
> > > >>> > > > >> 2020
> > > >>> > > > >> > at
> > > >>> > > > >> > > > > 10:07
> > > >>> > > > >> > > > > > PM
> > > >>> > > > >> > > > > > > > >> > Justine
> > > >>> > > > >> > > > > > > > >> > > > > > Olshan <
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > >>> jolshan@confluent.io
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > wrote:
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > > Hello
> > > >>> all,
> > > >>> > > > it's
> > > >>> > > > >> > been
> > > >>> > > > >> > > > > > almost a
> > > >>> > > > >> > > > > > > > >> year!
> > > >>> > > > >> > > > > > > > >> > > I've
> > > >>> > > > >> > > > > > > > >> > > > > > made
> > > >>> > > > >> > > > > > > > >> > > > > > >> > some
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > changes
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> to
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > this
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > KIP
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> and
> > > >>> hope to
> > > >>> > > > >> continue
> > > >>> > > > >> > > the
> > > >>> > > > >> > > > > > > > >> discussion.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > One
> > > >>> of the
> > > >>> > > > main
> > > >>> > > > >> > > changes
> > > >>> > > > >> > > > > > I've
> > > >>> > > > >> > > > > > > > >> added
> > > >>> > > > >> > > > > > > > >> > is
> > > >>> > > > >> > > > > > > > >> > > > now
> > > >>> > > > >> > > > > > > > >> > > > > > the
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > metadata
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> response
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > will
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > include
> > > >>> the
> > > >>> > > > topic
> > > >>> > > > >> ID
> > > >>> > > > >> > > (as
> > > >>> > > > >> > > > > > Colin
> > > >>> > > > >> > > > > > > > >> > > suggested).
> > > >>> > > > >> > > > > > > > >> > > > > > >> Clients
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > can
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> obtain the
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> topicID
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> of a
> > > >>> given
> > > >>> > > topic
> > > >>> > > > >> > > through
> > > >>> > > > >> > > > a
> > > >>> > > > >> > > > > > > > >> > > > TopicDescription.
> > > >>> > > > >> > > > > > > > >> > > > > > The
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > topicId
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > will
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > also be
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > included
> > > >>> > with
> > > >>> > > > the
> > > >>> > > > >> > > > > > > UpdateMetadata
> > > >>> > > > >> > > > > > > > >> > > request.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > Let
> > > me
> > > >>> > know
> > > >>> > > > what
> > > >>> > > > >> > you
> > > >>> > > > >> > > > all
> > > >>> > > > >> > > > > > > think.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > > Thank
> > > >>> you,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > > >>> Justine
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> On
> > > >>> > > 2019/09/13
> > > >>> > > > >> > > 16:38:26,
> > > >>> > > > >> > > > > > > "Colin
> > > >>> > > > >> > > > > > > > >> > > McCabe" <
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> cmccabe@apache.org
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > wrote:
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > Hi
> > > >>> > Lucas,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > > >>> Thanks
> > > >>> > for
> > > >>> > > > >> > tackling
> > > >>> > > > >> > > > > this.
> > > >>> > > > >> > > > > > > > >> Topic
> > > >>> > > > >> > > > > > > > >> > IDs
> > > >>> > > > >> > > > > > > > >> > > > > are a
> > > >>> > > > >> > > > > > > > >> > > > > > >> > great
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > idea,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> and
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > this
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > is
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > really
> > > >>> good
> > > >>> > > > >> writeup.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > > For
> > > >>> > > > >> > > > > > > /brokers/topics/[topic],
> > > >>> > > > >> > > > > > > > >> the
> > > >>> > > > >> > > > > > > > >> > > > schema
> > > >>> > > > >> > > > > > > > >> > > > > > >> version
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > should
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > be
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > bumped
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > version
> > > >>> 3,
> > > >>> > > > rather
> > > >>> > > > >> > than
> > > >>> > > > >> > > 2.
> > > >>> > > > >> > > > > > > > KIP-455
> > > >>> > > > >> > > > > > > > >> > > bumped
> > > >>> > > > >> > > > > > > > >> > > > > the
> > > >>> > > > >> > > > > > > > >> > > > > > >> > version
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > this
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > znode
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 2
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > already
> > > >>> :)
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > > >>> Given
> > > >>> > that
> > > >>> > > > >> we're
> > > >>> > > > >> > > > going
> > > >>> > > > >> > > > > to
> > > >>> > > > >> > > > > > > be
> > > >>> > > > >> > > > > > > > >> > seeing
> > > >>> > > > >> > > > > > > > >> > > > > these
> > > >>> > > > >> > > > > > > > >> > > > > > >> > things
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > as
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> strings
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > as
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > lot
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > (in
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > logs,
> > > in
> > > >>> > > > >> ZooKeeper,
> > > >>> > > > >> > on
> > > >>> > > > >> > > > the
> > > >>> > > > >> > > > > > > > >> > command-line,
> > > >>> > > > >> > > > > > > > >> > > > > > etc.),
> > > >>> > > > >> > > > > > > > >> > > > > > >> > does
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > it
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > make
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > sense to
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > use
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > base64
> > > >>> when
> > > >>> > > > >> > converting
> > > >>> > > > >> > > > them
> > > >>> > > > >> > > > > > to
> > > >>> > > > >> > > > > > > > >> > strings?
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > > >>> Here is
> > > >>> > an
> > > >>> > > > >> > example
> > > >>> > > > >> > > of
> > > >>> > > > >> > > > > the
> > > >>> > > > >> > > > > > > hex
> > > >>> > > > >> > > > > > > > >> > > > > > >> representation:
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > > >>> > > > >> > > > > > > > >> > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > > And
> > > >>> here
> > > >>> > > is
> > > >>> > > > an
> > > >>> > > > >> > > > example
> > > >>> > > > >> > > > > in
> > > >>> > > > >> > > > > > > > >> base64.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > > >>> > > > >> > > > b8tRS7h4TJ2Vt43Dp85v2A
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > > The
> > > >>> > base64
> > > >>> > > > >> > version
> > > >>> > > > >> > > > > saves
> > > >>> > > > >> > > > > > 15
> > > >>> > > > >> > > > > > > > >> > letters
> > > >>> > > > >> > > > > > > > >> > > > (to
> > > >>> > > > >> > > > > > > > >> > > > > be
> > > >>> > > > >> > > > > > > > >> > > > > > >> > fair,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > 4
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> those
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > were
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > dashes
> > > >>> that
> > > >>> > we
> > > >>> > > > >> could
> > > >>> > > > >> > > have
> > > >>> > > > >> > > > > > > elided
> > > >>> > > > >> > > > > > > > in
> > > >>> > > > >> > > > > > > > >> > the
> > > >>> > > > >> > > > > > > > >> > > > hex
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > representation.)
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > > >>> Another
> > > >>> > > > thing
> > > >>> > > > >> to
> > > >>> > > > >> > > > > consider
> > > >>> > > > >> > > > > > > is
> > > >>> > > > >> > > > > > > > >> that
> > > >>> > > > >> > > > > > > > >> > we
> > > >>> > > > >> > > > > > > > >> > > > > > should
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > specify
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > that
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> the
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > >>> all-zeroes
> > > >>> > > UUID
> > > >>> > > > is
> > > >>> > > > >> > not
> > > >>> > > > >> > > a
> > > >>> > > > >> > > > > > valid
> > > >>> > > > >> > > > > > > > >> topic
> > > >>> > > > >> > > > > > > > >> > > UUID.
> > > >>> > > > >> > > > > > > > >> > > > > >  We
> > > >>> > > > >> > > > > > > > >> > > > > > >> > can't
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > use
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> null
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > for
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > this
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > because
> > > >>> we
> > > >>> > > can't
> > > >>> > > > >> > pass a
> > > >>> > > > >> > > > > null
> > > >>> > > > >> > > > > > > UUID
> > > >>> > > > >> > > > > > > > >> over
> > > >>> > > > >> > > > > > > > >> > > the
> > > >>> > > > >> > > > > > > > >> > > > > RPC
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > protocol
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> (there
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > is no
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > special
> > > >>> > > pattern
> > > >>> > > > >> for
> > > >>> > > > >> > > null,
> > > >>> > > > >> > > > > nor
> > > >>> > > > >> > > > > > > do
> > > >>> > > > >> > > > > > > > we
> > > >>> > > > >> > > > > > > > >> > want
> > > >>> > > > >> > > > > > > > >> > > > to
> > > >>> > > > >> > > > > > > > >> > > > > > >> waste
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > space
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> reserving
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > such
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > >>> pattern.)
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > > >>> Maybe I
> > > >>> > > > missed
> > > >>> > > > >> > it,
> > > >>> > > > >> > > > but
> > > >>> > > > >> > > > > > did
> > > >>> > > > >> > > > > > > > you
> > > >>> > > > >> > > > > > > > >> > > > describe
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > "migration
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > of...
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > existing
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > topic[s]
> > > >>> > > without
> > > >>> > > > >> > topic
> > > >>> > > > >> > > > IDs"
> > > >>> > > > >> > > > > > in
> > > >>> > > > >> > > > > > > > >> detail
> > > >>> > > > >> > > > > > > > >> > in
> > > >>> > > > >> > > > > > > > >> > > > any
> > > >>> > > > >> > > > > > > > >> > > > > > >> > section?
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > It
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> seems
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > like
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > when
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> the
> > > new
> > > >>> > > > controller
> > > >>> > > > >> > > > becomes
> > > >>> > > > >> > > > > > > > active,
> > > >>> > > > >> > > > > > > > >> it
> > > >>> > > > >> > > > > > > > >> > > > should
> > > >>> > > > >> > > > > > > > >> > > > > > >> just
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > generate
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> random
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > UUIDs
> > for
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > these,
> > > >>> and
> > > >>> > > write
> > > >>> > > > >> the
> > > >>> > > > >> > > > random
> > > >>> > > > >> > > > > > > UUIDs
> > > >>> > > > >> > > > > > > > >> back
> > > >>> > > > >> > > > > > > > >> > > to
> > > >>> > > > >> > > > > > > > >> > > > > > >> > ZooKeeper.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > It
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> would be
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > good
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > to
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > spell
> > > >>> that
> > > >>> > > out.
> > > >>> > > > >> We
> > > >>> > > > >> > > > should
> > > >>> > > > >> > > > > > make
> > > >>> > > > >> > > > > > > > it
> > > >>> > > > >> > > > > > > > >> > clear
> > > >>> > > > >> > > > > > > > >> > > > > that
> > > >>> > > > >> > > > > > > > >> > > > > > >> this
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > happens
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > regardless
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > of
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> the
> > > >>> > > inter-broker
> > > >>> > > > >> > > protocol
> > > >>> > > > >> > > > > > > version
> > > >>> > > > >> > > > > > > > >> > (it's
> > > >>> > > > >> > > > > > > > >> > > a
> > > >>> > > > >> > > > > > > > >> > > > > > >> > compatible
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > change).
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > > >>> > > > >> > > "LeaderAndIsrRequests
> > > >>> > > > >> > > > > > > > >> including an
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > is_every_partition
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> flag"
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > seems a
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> bit
> > > >>> wordy.
> > > >>> > > Can
> > > >>> > > > we
> > > >>> > > > >> > just
> > > >>> > > > >> > > > > call
> > > >>> > > > >> > > > > > > > these
> > > >>> > > > >> > > > > > > > >> > "full
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > >>> LeaderAndIsrRequests"?
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > Then
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > the
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> RPC
> > > >>> field
> > > >>> > > could
> > > >>> > > > be
> > > >>> > > > >> > > named
> > > >>> > > > >> > > > > > > "full".
> > > >>> > > > >> > > > > > > > >> > Also,
> > > >>> > > > >> > > > > > > > >> > > it
> > > >>> > > > >> > > > > > > > >> > > > > > would
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > probably
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > be
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > better
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > for
> the
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> RPC
> > > >>> field to
> > > >>> > > be
> > > >>> > > > an
> > > >>> > > > >> > enum
> > > >>> > > > >> > > > of
> > > >>> > > > >> > > > > {
> > > >>> > > > >> > > > > > > > >> > > UNSPECIFIED,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > INCREMENTAL,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > FULL
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> }, so
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > that
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > we
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> can
> > > >>> cleanly
> > > >>> > > > handle
> > > >>> > > > >> > old
> > > >>> > > > >> > > > > > versions
> > > >>> > > > >> > > > > > > > (by
> > > >>> > > > >> > > > > > > > >> > > > treating
> > > >>> > > > >> > > > > > > > >> > > > > > >> them
> > > >>> > > > >> > > > > > > > >> > > > > > >> > as
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> UNSPECIFIED)
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > In
> > > >>> the
> > > >>> > > > >> > > > > > LeaderAndIsrRequest
> > > >>> > > > >> > > > > > > > >> > section,
> > > >>> > > > >> > > > > > > > >> > > > you
> > > >>> > > > >> > > > > > > > >> > > > > > >> write
> > > >>> > > > >> > > > > > > > >> > > > > > >> > "A
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > final
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > deletion
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > event
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> will
> > > be
> > > >>> > > > secheduled
> > > >>> > > > >> > for
> > > >>> > > > >> > > X
> > > >>> > > > >> > > > ms
> > > >>> > > > >> > > > > > > after
> > > >>> > > > >> > > > > > > > >> the
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > LeaderAndIsrRequest
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > was
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > first
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > >>> received..."
> > > >>> > > I
> > > >>> > > > >> guess
> > > >>> > > > >> > > > the X
> > > >>> > > > >> > > > > > > was a
> > > >>> > > > >> > > > > > > > >> > > > > placeholder
> > > >>> > > > >> > > > > > > > >> > > > > > >> that
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > you
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> intended
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> replace
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > before
> > > >>> > > posting?
> > > >>> > > > :)
> > > >>> > > > >> > In
> > > >>> > > > >> > > > any
> > > >>> > > > >> > > > > > > case,
> > > >>> > > > >> > > > > > > > >> this
> > > >>> > > > >> > > > > > > > >> > > > seems
> > > >>> > > > >> > > > > > > > >> > > > > > like
> > > >>> > > > >> > > > > > > > >> > > > > > >> > the
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > kind
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > of
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > thing
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > we'd
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > want a
> > > >>> > > > >> configuration
> > > >>> > > > >> > > for.
> > > >>> > > > >> > > > > > > Let's
> > > >>> > > > >> > > > > > > > >> > > describe
> > > >>> > > > >> > > > > > > > >> > > > > that
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > configuration
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> key
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > somewhere
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > in
> > > this
> > > >>> KIP,
> > > >>> > > > >> > including
> > > >>> > > > >> > > > what
> > > >>> > > > >> > > > > > its
> > > >>> > > > >> > > > > > > > >> > default
> > > >>> > > > >> > > > > > > > >> > > > > value
> > > >>> > > > >> > > > > > > > >> > > > > > >> is.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > We
> > > >>> > should
> > > >>> > > > >> > probably
> > > >>> > > > >> > > > also
> > > >>> > > > >> > > > > > > log a
> > > >>> > > > >> > > > > > > > >> > bunch
> > > >>> > > > >> > > > > > > > >> > > of
> > > >>> > > > >> > > > > > > > >> > > > > > >> messages
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > at
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > WARN
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> level
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > when
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > >>> something is
> > > >>> > > > >> > scheduled
> > > >>> > > > >> > > > for
> > > >>> > > > >> > > > > > > > >> deletion,
> > > >>> > > > >> > > > > > > > >> > as
> > > >>> > > > >> > > > > > > > >> > > > > well.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > (Maybe
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > this
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> was
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> assumed,
> > > but
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > it
> > > >>> would be
> > > >>> > > good
> > > >>> > > > >> to
> > > >>> > > > >> > > > mention
> > > >>> > > > >> > > > > > > it).
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > I
> > > >>> feel
> > > >>> > > like
> > > >>> > > > >> there
> > > >>> > > > >> > > > are a
> > > >>> > > > >> > > > > > few
> > > >>> > > > >> > > > > > > > >> > sections
> > > >>> > > > >> > > > > > > > >> > > > > that
> > > >>> > > > >> > > > > > > > >> > > > > > >> > should
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > be
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> moved to
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > "rejected
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > >>> > alternatives."
> > > >>> > > > >> For
> > > >>> > > > >> > > > > example,
> > > >>> > > > >> > > > > > in
> > > >>> > > > >> > > > > > > > the
> > > >>> > > > >> > > > > > > > >> > > > > > DeleteTopics
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > section,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> since
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > we're
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > not
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > going
> > > >>> to do
> > > >>> > > > >> option 1
> > > >>> > > > >> > or
> > > >>> > > > >> > > > 2,
> > > >>> > > > >> > > > > > > these
> > > >>> > > > >> > > > > > > > >> > should
> > > >>> > > > >> > > > > > > > >> > > be
> > > >>> > > > >> > > > > > > > >> > > > > > moved
> > > >>> > > > >> > > > > > > > >> > > > > > >> > into
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> "rejected
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > >>> > alternatives,"
> > > >>> > > > >> > rather
> > > >>> > > > >> > > > than
> > > >>> > > > >> > > > > > > > >> appearing
> > > >>> > > > >> > > > > > > > >> > > > > inline.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > Another
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > case
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> is
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > the
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> "Should
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > we
> > > >>> remove
> > > >>> > > topic
> > > >>> > > > >> name
> > > >>> > > > >> > > from
> > > >>> > > > >> > > > > the
> > > >>> > > > >> > > > > > > > >> protocol
> > > >>> > > > >> > > > > > > > >> > > > where
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > possible"
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> section.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > This
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > is
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > clearly
> > > >>> > > > >> discussing a
> > > >>> > > > >> > > > design
> > > >>> > > > >> > > > > > > > >> > alternative
> > > >>> > > > >> > > > > > > > >> > > > that
> > > >>> > > > >> > > > > > > > >> > > > > > >> we're
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > not
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> proposing
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > >>> implement:
> > > >>> > > > >> removing
> > > >>> > > > >> > the
> > > >>> > > > >> > > > > topic
> > > >>> > > > >> > > > > > > > name
> > > >>> > > > >> > > > > > > > >> > from
> > > >>> > > > >> > > > > > > > >> > > > > those
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > protocols.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > Is
> > > >>> it
> > > >>> > > really
> > > >>> > > > >> > > > necessary
> > > >>> > > > >> > > > > to
> > > >>> > > > >> > > > > > > > have
> > > >>> > > > >> > > > > > > > >> a
> > > >>> > > > >> > > > > > > > >> > new
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > >>> > > /admin/delete_topics_by_id
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > path
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > in
> > > >>> > ZooKeeper?
> > > >>> > > > It
> > > >>> > > > >> > seems
> > > >>> > > > >> > > > > like
> > > >>> > > > >> > > > > > we
> > > >>> > > > >> > > > > > > > >> don't
> > > >>> > > > >> > > > > > > > >> > > > really
> > > >>> > > > >> > > > > > > > >> > > > > > >> need
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > this.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Whenever
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > there
> is
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > a
> > new
> > > >>> > > > controller,
> > > >>> > > > >> > we'll
> > > >>> > > > >> > > > > send
> > > >>> > > > >> > > > > > > out
> > > >>> > > > >> > > > > > > > >> full
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > LeaderAndIsrRequests
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> which
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > will
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > trigger
> > > >>> the
> > > >>> > > > stale
> > > >>> > > > >> > > topics
> > > >>> > > > >> > > > to
> > > >>> > > > >> > > > > > be
> > > >>> > > > >> > > > > > > > >> cleaned
> > > >>> > > > >> > > > > > > > >> > > up.
> > > >>> > > > >> > > > > > > > >> > > > > >  The
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > active
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > controller
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > will
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> also
> > > >>> send
> > > >>> > the
> > > >>> > > > full
> > > >>> > > > >> > > > > > > > >> LeaderAndIsrRequest
> > > >>> > > > >> > > > > > > > >> > > to
> > > >>> > > > >> > > > > > > > >> > > > > > >> brokers
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > that
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > are
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> just
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> starting
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> up.
> > > >>> So we
> > > >>> > > > don't
> > > >>> > > > >> > > really
> > > >>> > > > >> > > > > > need
> > > >>> > > > >> > > > > > > > this
> > > >>> > > > >> > > > > > > > >> > kind
> > > >>> > > > >> > > > > > > > >> > > > of
> > > >>> > > > >> > > > > > > > >> > > > > > >> > two-phase
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > commit
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > (send
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > out
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > >>> > > > >> StopReplicasRequest,
> > > >>> > > > >> > > get
> > > >>> > > > >> > > > > ACKs
> > > >>> > > > >> > > > > > > > from
> > > >>> > > > >> > > > > > > > >> all
> > > >>> > > > >> > > > > > > > >> > > > > nodes,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > commit
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > by
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> removing
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > >>> > > > >> /admin/delete_topics
> > > >>> > > > >> > > > node)
> > > >>> > > > >> > > > > > any
> > > >>> > > > >> > > > > > > > >> more.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > > You
> > > >>> > > mention
> > > >>> > > > >> that
> > > >>> > > > >> > > > > > > FetchRequest
> > > >>> > > > >> > > > > > > > >> will
> > > >>> > > > >> > > > > > > > >> > > now
> > > >>> > > > >> > > > > > > > >> > > > > > >> include
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > UUID
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> avoid
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > issues
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > where
> > > >>> > requests
> > > >>> > > > are
> > > >>> > > > >> > made
> > > >>> > > > >> > > > to
> > > >>> > > > >> > > > > > > stale
> > > >>> > > > >> > > > > > > > >> > > > partitions.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > However,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> adding a
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > UUID
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > >>> > > MetadataRequest
> > > >>> > > > is
> > > >>> > > > >> > > listed
> > > >>> > > > >> > > > > as
> > > >>> > > > >> > > > > > > > future
> > > >>> > > > >> > > > > > > > >> > > work,
> > > >>> > > > >> > > > > > > > >> > > > > out
> > > >>> > > > >> > > > > > > > >> > > > > > of
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > scope
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > for
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> this
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > KIP.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > How
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> will
> > > the
> > > >>> > > client
> > > >>> > > > >> learn
> > > >>> > > > >> > > > what
> > > >>> > > > >> > > > > > the
> > > >>> > > > >> > > > > > > > >> topic
> > > >>> > > > >> > > > > > > > >> > > UUID
> > > >>> > > > >> > > > > > > > >> > > > > is,
> > > >>> > > > >> > > > > > > > >> > > > > > if
> > > >>> > > > >> > > > > > > > >> > > > > > >> > the
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > metadata
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > response
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > doesn't
> > > >>> > > include
> > > >>> > > > >> that
> > > >>> > > > >> > > > > > > information?
> > > >>> > > > >> > > > > > > > >> It
> > > >>> > > > >> > > > > > > > >> > > > seems
> > > >>> > > > >> > > > > > > > >> > > > > > like
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > adding
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> UUID
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > >>> > > MetadataResponse
> > > >>> > > > >> > would
> > > >>> > > > >> > > be
> > > >>> > > > >> > > > > an
> > > >>> > > > >> > > > > > > > >> > improvement
> > > >>> > > > >> > > > > > > > >> > > > > here
> > > >>> > > > >> > > > > > > > >> > > > > > >> that
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > might
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > not
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> be
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > too
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > hard
> to
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > make.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > > >>> best,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > > >>> Colin
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > On
> > > >>> Mon,
> > > >>> > > Sep
> > > >>> > > > 9,
> > > >>> > > > >> > > 2019,
> > > >>> > > > >> > > > at
> > > >>> > > > >> > > > > > > > 17:48,
> > > >>> > > > >> > > > > > > > >> > > Ryanne
> > > >>> > > > >> > > > > > > > >> > > > > > Dolan
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > wrote:
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > >
> > > >>> Lucas,
> > > >>> > > > this
> > > >>> > > > >> > would
> > > >>> > > > >> > > > be
> > > >>> > > > >> > > > > > > great.
> > > >>> > > > >> > > > > > > > >> I've
> > > >>> > > > >> > > > > > > > >> > > run
> > > >>> > > > >> > > > > > > > >> > > > > > into
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > issues
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > with
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > topics
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > being
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > >
> > > >>> > > > resurrected
> > > >>> > > > >> > > > > > accidentally,
> > > >>> > > > >> > > > > > > > >> since
> > > >>> > > > >> > > > > > > > >> > a
> > > >>> > > > >> > > > > > > > >> > > > > client
> > > >>> > > > >> > > > > > > > >> > > > > > >> > cannot
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > easily
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > distinguish
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > between
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > >
> > > a
> > > >>> > > deleted
> > > >>> > > > >> topic
> > > >>> > > > >> > > > and a
> > > >>> > > > >> > > > > > new
> > > >>> > > > >> > > > > > > > >> topic
> > > >>> > > > >> > > > > > > > >> > > with
> > > >>> > > > >> > > > > > > > >> > > > > the
> > > >>> > > > >> > > > > > > > >> > > > > > >> same
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > name.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I'd
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > need
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > the
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > ID
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > >
> > > >>> > > accessible
> > > >>> > > > >> from
> > > >>> > > > >> > > the
> > > >>> > > > >> > > > > > > client
> > > >>> > > > >> > > > > > > > to
> > > >>> > > > >> > > > > > > > >> > > solve
> > > >>> > > > >> > > > > > > > >> > > > > that
> > > >>> > > > >> > > > > > > > >> > > > > > >> > issue,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > but
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> this
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > is a
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > good
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > first
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > >
> > > >>> step.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > >
> > > >>> Ryanne
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > >
> > > On
> > > >>> > Wed,
> > > >>> > > > Sep
> > > >>> > > > >> 4,
> > > >>> > > > >> > > 2019
> > > >>> > > > >> > > > > at
> > > >>> > > > >> > > > > > > 1:41
> > > >>> > > > >> > > > > > > > >> PM
> > > >>> > > > >> > > > > > > > >> > > Lucas
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > Bradstreet <
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > >>> > > lucas@confluent.io>
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > wrote:
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > >
> > > >
> > > >>> Hi
> > > >>> > > all,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > >
> > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > >
> > > >
> > > >>> I
> > > >>> > > would
> > > >>> > > > >> like
> > > >>> > > > >> > to
> > > >>> > > > >> > > > > kick
> > > >>> > > > >> > > > > > > off
> > > >>> > > > >> > > > > > > > >> > > > discussion
> > > >>> > > > >> > > > > > > > >> > > > > of
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > KIP-516,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > an
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > >>> implementation
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > of
> > > topic
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > >
> > > >
> > > >>> IDs
> > > >>> > > for
> > > >>> > > > >> > Kafka.
> > > >>> > > > >> > > > > Topic
> > > >>> > > > >> > > > > > > IDs
> > > >>> > > > >> > > > > > > > >> aim
> > > >>> > > > >> > > > > > > > >> > to
> > > >>> > > > >> > > > > > > > >> > > > > solve
> > > >>> > > > >> > > > > > > > >> > > > > > >> topic
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> uniqueness
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> problems
> > > in
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > Kafka,
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > >
> > > >
> > > >>> > where
> > > >>> > > > >> > referring
> > > >>> > > > >> > > > to
> > > >>> > > > >> > > > > a
> > > >>> > > > >> > > > > > > > topic
> > > >>> > > > >> > > > > > > > >> by
> > > >>> > > > >> > > > > > > > >> > > name
> > > >>> > > > >> > > > > > > > >> > > > > > >> alone is
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> insufficient.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Such
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > cases
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > >
> > > >
> > > >>> > > include
> > > >>> > > > >> when
> > > >>> > > > >> > a
> > > >>> > > > >> > > > > topic
> > > >>> > > > >> > > > > > > has
> > > >>> > > > >> > > > > > > > >> been
> > > >>> > > > >> > > > > > > > >> > > > > deleted
> > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > recreated
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> with
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > the
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > same
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > name.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > >
> > > >
> > > >>> > Unique
> > > >>> > > > >> > > > identifiers
> > > >>> > > > >> > > > > > will
> > > >>> > > > >> > > > > > > > >> help
> > > >>> > > > >> > > > > > > > >> > > > > simplify
> > > >>> > > > >> > > > > > > > >> > > > > > >> and
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > improve
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Kafka's
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > topic
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > deletion
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > >
> > > >
> > > >>> > > process,
> > > >>> > > > >> as
> > > >>> > > > >> > > well
> > > >>> > > > >> > > > as
> > > >>> > > > >> > > > > > > > prevent
> > > >>> > > > >> > > > > > > > >> > > cases
> > > >>> > > > >> > > > > > > > >> > > > > > where
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > brokers
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > may
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > incorrectly
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > interact
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > >
> > > >
> > > >>> with
> > > >>> > > > stale
> > > >>> > > > >> > > > versions
> > > >>> > > > >> > > > > > of
> > > >>> > > > >> > > > > > > > >> topics.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > >
> > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > >
> > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > >
> > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > >>> > > > >> > > > > > > > >> > > > > >
> > > >>> > > > >> > > > > > > > >> > > > >
> > > >>> > > > >> > > > > > > > >> > > >
> > > >>> > > > >> > > > > > > > >> > >
> > > >>> > > > >> > > > > > > > >> >
> > > >>> > > > >> > > > > > > > >>
> > > >>> > > > >> > > > > > > >
> > > >>> > > > >> > > > > > >
> > > >>> > > > >> > > > > >
> > > >>> > > > >> > > > >
> > > >>> > > > >> > > >
> > > >>> > > > >> > >
> > > >>> > > > >> >
> > > >>> > > > >>
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > >
> > > >
> > > >>> > > Looking
> > > >>> > > > >> > forward
> > > >>> > > > >> > > > to
> > > >>> > > > >> > > > > > your
> > > >>> > > > >> > > > > > > > >> > > thoughts.
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > >
> > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > >
> > > >
> > > >>> > Lucas
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >
> > >
> > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > > >>> > > > >> > > > > > > > >> > > > > > >> >
> > > >>> > > > >> > > > > > > > >> > > > > > >>
> > > >>> > > > >> > > > > > > > >> > > > > > >
> > > >>> > > > >> > > > > > > > >> > > > > >
> > > >>> > > > >> > > > > > > > >> > > > >
> > > >>> > > > >> > > > > > > > >> > > >
> > > >>> > > > >> > > > > > > > >> > >
> > > >>> > > > >> > > > > > > > >> >
> > > >>> > > > >> > > > > > > > >>
> > > >>> > > > >> > > > > > > > >
> > > >>> > > > >> > > > > > > >
> > > >>> > > > >> > > > > > >
> > > >>> > > > >> > > > > >
> > > >>> > > > >> > > > >
> > > >>> > > > >> > > >
> > > >>> > > > >> > >
> > > >>> > > > >> >
> > > >>> > > > >>
> > > >>> > > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > > >>
> > >
> >
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Justine Olshan <jo...@confluent.io>]

Hi Jun,

This KIP removes the topic name from many requests. A lot of requests keep
track of the partitions internally with a TopicPartition. So when we remove
the topic name, we can not differentiate unless we can also use topic ID.
The Fetch request relies heavily on this, but ListOffset,
OffsetsForLeaderEpoch, etc all use TopicPartition as a key value for
internal maps. Digging deeper into Fetch, I've noticed that the code path
runs into some public APIs that use the TopicPartition. Some examples are
Cluster.java or RecordTooLargeException.

A related issue is that in some places like TopicAuthorizationException,
the topic name is required and without the name in the request, we won't
know it.

I guess an alternative would be to have some mapping from topic name to
topic ID, but I would have to think about that more.

The KIP mentions exposing TopicId in TopicDescription. (The font was
weirdly colored, so I've made it easier to read.)

Would the idea for the TopicMetadataAndConfig be that the topic ID would be
returned as a result?
As for the other methods describe/deletetopics, it makes sense to include
them at some point, but I'm not sure how far we should go in terms of
supporting topic IDs everywhere.

Thank you for continuing to look at the KIP!
Justine


On Mon, Oct 5, 2020 at 2:53 PM Jun Rao <ju...@confluent.io> wrote:

> Hi, Justine,
>
> Adding a TopicIdPartition in the public interface may be useful. Where
> would you expect it to be used?
>
> Related to this, since we are exposing topicId in the tool, it seems that
> we may want to expose topicId in at least TopicDescription used by
> AdminClient.describeTopics() and TopicMetadataAndConfig used in
> AdminClient.createTopics(Collection<NewTopic> newTopics). I am also not
> sure if we plan to expose topicId in other AdminClient methods in this KIP
> or in some future KIP. For example, it might be useful to support topicId
> in the following methods.
>
>     DescribeTopicsResult describeTopics(Collection<String> topicNames,
> DescribeTopicsOptions options);
>     DeleteTopicsResult deleteTopics(Collection<String> topics,
> DeleteTopicsOptions options);
>
> Thanks,
>
> Jun
>
> On Mon, Oct 5, 2020 at 11:03 AM Justine Olshan <jo...@confluent.io>
> wrote:
>
> > Hi all,
> >
> > I was thinking about this KIP some more. We are removing the topic name
> > from many of these request types, and some of them (eg. Fetch) use
> > TopicPartition as keys in a map when handling the request. I was thinking
> > it would make sense to make some sort of TopicIdPartition with topic
> name,
> > topic id, and partition fields (name and id would potentially be optional
> > types since different request versions will contain only one of the two)
> I
> > was wondering if it made sense to include this as a public facing API
> > similar to TopicPartition. If so, I will add this class to the KIP as
> well.
> >
> > Thanks,
> > Justine
> >
> > On Mon, Oct 5, 2020 at 8:52 AM Justine Olshan <jo...@confluent.io>
> > wrote:
> >
> > > Hi all,
> > > I've made a few minor changes to the UUID class. I've added a few more
> > > methods to support UUIDs more generally, by including a constructor
> and a
> > > method to generate random UUIDs. Both of these will be useful for
> > testing.
> > >
> > > Let me know if there are any other areas for discussion.
> > >
> > > Thank you,
> > > Justine
> > >
> > > On Thu, Oct 1, 2020 at 5:29 PM Justine Olshan <jo...@confluent.io>
> > > wrote:
> > >
> > >> Hi Jun,
> > >> Thanks for looking at it again. Here's the new spec. (I fixed the typo
> > in
> > >> it too.)
> > >> {"name": "id", "type": "string", "doc": option[UUID]}
> > >>
> > >> Justine
> > >>
> > >>
> > >> On Thu, Oct 1, 2020 at 5:03 PM Jun Rao <ju...@confluent.io> wrote:
> > >>
> > >>> Hi, Justine,
> > >>>
> > >>> Thanks for the update. The KIP looks good to me now. Just a minor
> > comment
> > >>> below.
> > >>>
> > >>> 30. Perhaps "option[UUID]" can be put in the doc.
> > >>>
> > >>> Jun
> > >>>
> > >>> On Thu, Oct 1, 2020 at 3:28 PM Justine Olshan <jo...@confluent.io>
> > >>> wrote:
> > >>>
> > >>> > Hi Jun,
> > >>> > Thanks for the response!
> > >>> >
> > >>> > 30. I think I might have changed this in between. The current
> version
> > >>> > says:  {"name":
> > >>> > "id", "type": "option[UUID]"}, "doc": topic id}
> > >>> > I have switched to the option type to cover the migration case
> where
> > a
> > >>> > TopicZNode does not yet have a topic ID.
> > >>> > I understand that due to json, this field is written as a string,
> so
> > >>> if I
> > >>> > should move the "option[uuid]" to the doc field and the type should
> > be
> > >>> > "string" please let me know.
> > >>> >
> > >>> > 40. I've added a definition for UUID.
> > >>> > 41,42. Fixed
> > >>> >
> > >>> > Thank you,
> > >>> > Justine
> > >>> >
> > >>> > On Wed, Sep 30, 2020 at 1:15 PM Jun Rao <ju...@confluent.io> wrote:
> > >>> >
> > >>> > > Hi, Justine,
> > >>> > >
> > >>> > > Thanks for the summary. Just a few minor comments blow.
> > >>> > >
> > >>> > > 30.  {"name": "id", "type": "string", "doc": "version id"}}: The
> > doc
> > >>> > should
> > >>> > > say UUID. The issue seems unfixed.
> > >>> > >
> > >>> > > 40. Since UUID is public facing, could you include its
> definition?
> > >>> > >
> > >>> > > 41. StopReplicaResponse still includes the topic field.
> > >>> > >
> > >>> > > 42. "It is unnecessary to include the name of the topic in the
> > >>> following
> > >>> > > Request/Response calls" It would be useful to include all
> modified
> > >>> > requests
> > >>> > > (e.g. produce) in the list below.
> > >>> > >
> > >>> > > Thanks,
> > >>> > >
> > >>> > > Jun
> > >>> > >
> > >>> > >
> > >>> > > On Wed, Sep 30, 2020 at 10:17 AM Justine Olshan <
> > >>> jolshan@confluent.io>
> > >>> > > wrote:
> > >>> > >
> > >>> > > > Hello again,
> > >>> > > >
> > >>> > > > I've taken some time to discuss some of the remaining points
> > >>> brought up
> > >>> > > by
> > >>> > > > the previous emails offline. Here are some of the conclusions.
> > >>> > > >
> > >>> > > > 1. Directory Structure:
> > >>> > > > There was some talk about whether the directory structure
> should
> > be
> > >>> > > changed
> > >>> > > > to replace all topic names with topic IDs.
> > >>> > > > This will be a useful change, but will prevent downgrades. It
> > will
> > >>> be
> > >>> > > best
> > >>> > > > to wait until a major release, likely alongside KIP-500 changes
> > >>> that
> > >>> > will
> > >>> > > > prevent downgrades. I've updated the KIP to include this change
> > >>> with
> > >>> > the
> > >>> > > > note about migration and deprecation.
> > >>> > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-log.dirlayout
> > >>> > > >
> > >>> > > > 2. Partition Metadata file
> > >>> > > > There was some disagreement about the inclusion of the
> partition
> > >>> > metadata
> > >>> > > > file.
> > >>> > > > This file will be needed to persist the topic ID, especially
> > while
> > >>> we
> > >>> > > still
> > >>> > > > have the old directory structure. Even after the changes, this
> > >>> file can
> > >>> > > be
> > >>> > > > useful for debugging and recovery.
> > >>> > > > Creating a single mapping file from topic names to topic IDs
> was
> > >>> > > > considered, but ultimately scrapped as it would not be as easy
> to
> > >>> > > maintain.
> > >>> > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PartitionMetadatafile
> > >>> > > >
> > >>> > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PersistingTopicIDs
> > >>> > > >
> > >>> > > > 3. Produce Protocols
> > >>> > > > After some further discussion, this replacing the topic name
> with
> > >>> topic
> > >>> > > ID
> > >>> > > > in these protocols has been added to the KIP.
> > >>> > > > This will cut down on the size of the protocol. Since changes
> to
> > >>> fetch
> > >>> > > are
> > >>> > > > included, it does make sense to update these protocols.
> > >>> > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-Produce
> > >>> > > >
> > >>> > > > 4. KIP-500 Compatibility
> > >>> > > > After some discussion with those involved with KIP-500, it
> seems
> > >>> best
> > >>> > to
> > >>> > > > use a sentinel topic ID for the metadata topic that is used
> > before
> > >>> the
> > >>> > > > first controller election.
> > >>> > > > We had an issue where this metadata topic may not be assigned
> an
> > ID
> > >>> > > before
> > >>> > > > utilizing the Vote and Fetch protocols. It was decided to
> > reserve a
> > >>> > > unique
> > >>> > > > ID for this topic to be used until the controller could give
> the
> > >>> topic
> > >>> > a
> > >>> > > > unique ID.
> > >>> > > > Having the topic keep the sentinel ID (not be reassigned to a
> > >>> unique
> > >>> > ID)
> > >>> > > > was considered, but it seemed like a good idea to leave the
> > option
> > >>> open
> > >>> > > for
> > >>> > > > the metadata topic to have a unique ID in cases where it would
> > >>> need to
> > >>> > be
> > >>> > > > differentiated from other clusters' metadata topics. (ex.
> tiered
> > >>> > storage)
> > >>> > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-CompatibilitywithKIP-500
> > >>> > > >
> > >>> > > > I've also split up the KIP into sub-tasks on the JIRA.
> Hopefully
> > >>> this
> > >>> > > will
> > >>> > > > provide a better idea about what tasks we have, and eventually
> > >>> provide
> > >>> > a
> > >>> > > > place to see what's done and what is left.
> > >>> > > > If there is a task I am missing, please let me know!
> > >>> > > > https://issues.apache.org/jira/browse/KAFKA-8872
> > >>> > > >
> > >>> > > > Of course, these decisions are not set in stone, and I would
> love
> > >>> to
> > >>> > hear
> > >>> > > > any feedback.
> > >>> > > >
> > >>> > > > Thanks,
> > >>> > > > Justine
> > >>> > > >
> > >>> > > > On Mon, Sep 28, 2020 at 11:38 AM Justine Olshan <
> > >>> jolshan@confluent.io>
> > >>> > > > wrote:
> > >>> > > >
> > >>> > > > > Hello all,
> > >>> > > > >
> > >>> > > > > I just wanted to follow up on this discussion. Did we come to
> > an
> > >>> > > > > understanding about the directory structure?
> > >>> > > > >
> > >>> > > > > I think the biggest question here is what is acceptable to
> > leave
> > >>> out
> > >>> > > due
> > >>> > > > > to scope vs. what is considered to be too much tech debt.
> > >>> > > > > This KIP is already pretty large with the number of changes,
> > but
> > >>> it
> > >>> > > also
> > >>> > > > > makes sense to do things correctly, so I'd love to hear
> > >>> everyone's
> > >>> > > > thoughts.
> > >>> > > > >
> > >>> > > > > Thanks,
> > >>> > > > > Justine
> > >>> > > > >
> > >>> > > > > On Fri, Sep 25, 2020 at 8:19 AM Lucas Bradstreet <
> > >>> lucas@confluent.io
> > >>> > >
> > >>> > > > > wrote:
> > >>> > > > >
> > >>> > > > >> Hi Ismael,
> > >>> > > > >>
> > >>> > > > >> If you do not store it in a metadata file or in the
> directory
> > >>> > > structure
> > >>> > > > >> would you then
> > >>> > > > >> require the LeaderAndIsrRequest following startup to give
> you
> > >>> some
> > >>> > > > notion
> > >>> > > > >> of
> > >>> > > > >> topic name in memory? We would still need this information
> for
> > >>> the
> > >>> > > older
> > >>> > > > >> protocols, but
> > >>> > > > >> perhaps this is what's meant by tech debt.
> > >>> > > > >>
> > >>> > > > >> Once we're free of the old non-topicID requests then I think
> > you
> > >>> > > > wouldn't
> > >>> > > > >> need to retain the topic name.
> > >>> > > > >> I think the ability to easily look up topic names associated
> > >>> with
> > >>> > > > >> partition
> > >>> > > > >> directories would still be missed
> > >>> > > > >> when diagnosing issues, though maybe it wouldn't be a deal
> > >>> breaker
> > >>> > > with
> > >>> > > > >> the
> > >>> > > > >> right tooling.
> > >>> > > > >>
> > >>> > > > >> Thanks,
> > >>> > > > >>
> > >>> > > > >> Lucas
> > >>> > > > >>
> > >>> > > > >> On Fri, Sep 25, 2020 at 7:55 AM Ismael Juma <
> > ismael@juma.me.uk>
> > >>> > > wrote:
> > >>> > > > >>
> > >>> > > > >> > Hi Lucas,
> > >>> > > > >> >
> > >>> > > > >> > Why would you include the name and id? I think you'd want
> to
> > >>> > remove
> > >>> > > > the
> > >>> > > > >> > name from the directory name right? Jason's suggestion was
> > >>> that if
> > >>> > > you
> > >>> > > > >> > remove the name from the directory, then why would you
> need
> > >>> the id
> > >>> > > > name
> > >>> > > > >> > mapping file?
> > >>> > > > >> >
> > >>> > > > >> > Ismael
> > >>> > > > >> >
> > >>> > > > >> > On Thu, Sep 24, 2020 at 4:24 PM Lucas Bradstreet <
> > >>> > > lucas@confluent.io>
> > >>> > > > >> > wrote:
> > >>> > > > >> >
> > >>> > > > >> > > > 2. Part of the usage of the file is to have persistent
> > >>> storage
> > >>> > > of
> > >>> > > > >> the
> > >>> > > > >> > > topic
> > >>> > > > >> > > ID and use it to compare with the ID supplied in the
> > >>> > LeaderAndIsr
> > >>> > > > >> > Request.
> > >>> > > > >> > > There is some discussion in the KIP about changes to the
> > >>> > directory
> > >>> > > > >> > > structure, but I believe directory changes were
> considered
> > >>> to be
> > >>> > > out
> > >>> > > > >> of
> > >>> > > > >> > > scope when the KIP was written.
> > >>> > > > >> > >
> > >>> > > > >> > >
> > >>> > > > >> > > Yeah, I was hoping to get a better understanding of why
> it
> > >>> was
> > >>> > > taken
> > >>> > > > >> out
> > >>> > > > >> > of
> > >>> > > > >> > > scope. Perhaps Lucas Bradstreet might have more insight
> > >>> about
> > >>> > the
> > >>> > > > >> > decision.
> > >>> > > > >> > > Basically my point is that we have to create additional
> > >>> > > > infrastructure
> > >>> > > > >> > here
> > >>> > > > >> > > to support the name/id mapping, so I wanted to
> understand
> > >>> if we
> > >>> > > just
> > >>> > > > >> > > consider this a sort of tech debt. It would be useful to
> > >>> cover
> > >>> > how
> > >>> > > > we
> > >>> > > > >> > > handle the case when this file gets corrupted. Seems
> like
> > we
> > >>> > just
> > >>> > > > >> have to
> > >>> > > > >> > > trust that it matches whatever the controller tells us
> and
> > >>> > rewrite
> > >>> > > > it?
> > >>> > > > >> > >
> > >>> > > > >> > >
> > >>> > > > >> > > Hi Jason, Justine,
> > >>> > > > >> > >
> > >>> > > > >> > > My thought process is that we will likely need the
> > metadata
> > >>> file
> > >>> > > > >> whether
> > >>> > > > >> > we
> > >>> > > > >> > > rename the directories or not.
> > >>> > > > >> > > Linux supports filenames of up to 255 bytes and that
> would
> > >>> not
> > >>> > be
> > >>> > > > >> enough
> > >>> > > > >> > to
> > >>> > > > >> > > support a directory name
> > >>> > > > >> > >  including both the name and topic ID. Given that fact,
> it
> > >>> > seemed
> > >>> > > > >> > > reasonable to add the metadata file
> > >>> > > > >> > > and leave the directory rename until some time in the
> > future
> > >>> > > > (possibly
> > >>> > > > >> > > never).
> > >>> > > > >> > >
> > >>> > > > >> > > If the file does get corrupted I think you're right. We
> > >>> would
> > >>> > > either
> > >>> > > > >> have
> > >>> > > > >> > > to trust it matches what the controller tells us
> > >>> > > > >> > >  or error out and let an administrator resolve it by
> > >>> checking
> > >>> > > across
> > >>> > > > >> > > replicas for consistency.
> > >>> > > > >> > >
> > >>> > > > >> > > Lucas
> > >>> > > > >> > >
> > >>> > > > >> > >
> > >>> > > > >> > > On Thu, Sep 24, 2020 at 3:41 PM Jason Gustafson <
> > >>> > > jason@confluent.io
> > >>> > > > >
> > >>> > > > >> > > wrote:
> > >>> > > > >> > >
> > >>> > > > >> > > > Thanks Justine. Responses below:
> > >>> > > > >> > > >
> > >>> > > > >> > > > > 1. Yes, the directory will still be based on the
> topic
> > >>> > names.
> > >>> > > > >> > > > LeaderAndIsrRequest is one of the few requests that
> will
> > >>> still
> > >>> > > > >> contain
> > >>> > > > >> > > the
> > >>> > > > >> > > > topic name. So I think we have this covered. Sorry for
> > >>> > > confusion.
> > >>> > > > >> > > >
> > >>> > > > >> > > > Ah, you're right. My eyes passed right over the field.
> > >>> > > > >> > > >
> > >>> > > > >> > > > > 2. Part of the usage of the file is to have
> persistent
> > >>> > storage
> > >>> > > > of
> > >>> > > > >> the
> > >>> > > > >> > > > topic
> > >>> > > > >> > > > ID and use it to compare with the ID supplied in the
> > >>> > > LeaderAndIsr
> > >>> > > > >> > > Request.
> > >>> > > > >> > > > There is some discussion in the KIP about changes to
> the
> > >>> > > directory
> > >>> > > > >> > > > structure, but I believe directory changes were
> > >>> considered to
> > >>> > be
> > >>> > > > >> out of
> > >>> > > > >> > > > scope when the KIP was written.
> > >>> > > > >> > > >
> > >>> > > > >> > > > Yeah, I was hoping to get a better understanding of
> why
> > >>> it was
> > >>> > > > taken
> > >>> > > > >> > out
> > >>> > > > >> > > of
> > >>> > > > >> > > > scope. Perhaps Lucas Bradstreet might have more
> insight
> > >>> about
> > >>> > > the
> > >>> > > > >> > > decision.
> > >>> > > > >> > > > Basically my point is that we have to create
> additional
> > >>> > > > >> infrastructure
> > >>> > > > >> > > here
> > >>> > > > >> > > > to support the name/id mapping, so I wanted to
> > understand
> > >>> if
> > >>> > we
> > >>> > > > just
> > >>> > > > >> > > > consider this a sort of tech debt. It would be useful
> to
> > >>> cover
> > >>> > > how
> > >>> > > > >> we
> > >>> > > > >> > > > handle the case when this file gets corrupted. Seems
> > like
> > >>> we
> > >>> > > just
> > >>> > > > >> have
> > >>> > > > >> > to
> > >>> > > > >> > > > trust that it matches whatever the controller tells us
> > and
> > >>> > > rewrite
> > >>> > > > >> it?
> > >>> > > > >> > > >
> > >>> > > > >> > > > > 3. I think this is a good point, but I again I
> wonder
> > >>> about
> > >>> > > the
> > >>> > > > >> scope
> > >>> > > > >> > > of
> > >>> > > > >> > > > the KIP. Most of the changes mentioned in the KIP are
> > for
> > >>> > > > supporting
> > >>> > > > >> > > topic
> > >>> > > > >> > > > deletion and I believe that is why the produce request
> > was
> > >>> > > listed
> > >>> > > > >> under
> > >>> > > > >> > > > future work.
> > >>> > > > >> > > >
> > >>> > > > >> > > > That's fair. I brought it up since `Fetch` is already
> > >>> > included.
> > >>> > > If
> > >>> > > > >> > we've
> > >>> > > > >> > > > got `Metadata` and `Fetch`, seems we may as well do
> > >>> `Produce`
> > >>> > > and
> > >>> > > > >> save
> > >>> > > > >> > an
> > >>> > > > >> > > > extra kip. No strong objection though if you want to
> > >>> leave it
> > >>> > > out.
> > >>> > > > >> > > >
> > >>> > > > >> > > >
> > >>> > > > >> > > > -Jason
> > >>> > > > >> > > >
> > >>> > > > >> > > >
> > >>> > > > >> > > > On Thu, Sep 24, 2020 at 3:26 PM Justine Olshan <
> > >>> > > > >> jolshan@confluent.io>
> > >>> > > > >> > > > wrote:
> > >>> > > > >> > > >
> > >>> > > > >> > > > > Hi Jason,
> > >>> > > > >> > > > >
> > >>> > > > >> > > > > Thanks for your comments.
> > >>> > > > >> > > > >
> > >>> > > > >> > > > > 1. Yes, the directory will still be based on the
> topic
> > >>> > names.
> > >>> > > > >> > > > > LeaderAndIsrRequest is one of the few requests that
> > will
> > >>> > still
> > >>> > > > >> > contain
> > >>> > > > >> > > > the
> > >>> > > > >> > > > > topic name. So I think we have this covered. Sorry
> for
> > >>> > > > confusion.
> > >>> > > > >> > > > >
> > >>> > > > >> > > > > 2. Part of the usage of the file is to have
> persistent
> > >>> > storage
> > >>> > > > of
> > >>> > > > >> the
> > >>> > > > >> > > > topic
> > >>> > > > >> > > > > ID and use it to compare with the ID supplied in the
> > >>> > > > LeaderAndIsr
> > >>> > > > >> > > > Request.
> > >>> > > > >> > > > > There is some discussion in the KIP about changes to
> > the
> > >>> > > > directory
> > >>> > > > >> > > > > structure, but I believe directory changes were
> > >>> considered
> > >>> > to
> > >>> > > be
> > >>> > > > >> out
> > >>> > > > >> > of
> > >>> > > > >> > > > > scope when the KIP was written.
> > >>> > > > >> > > > >
> > >>> > > > >> > > > > 3. I think this is a good point, but I again I
> wonder
> > >>> about
> > >>> > > the
> > >>> > > > >> scope
> > >>> > > > >> > > of
> > >>> > > > >> > > > > the KIP. Most of the changes mentioned in the KIP
> are
> > >>> for
> > >>> > > > >> supporting
> > >>> > > > >> > > > topic
> > >>> > > > >> > > > > deletion and I believe that is why the produce
> request
> > >>> was
> > >>> > > > listed
> > >>> > > > >> > under
> > >>> > > > >> > > > > future work.
> > >>> > > > >> > > > >
> > >>> > > > >> > > > > 4. This sounds like it might be a good solution,
> but I
> > >>> will
> > >>> > > need
> > >>> > > > >> to
> > >>> > > > >> > > > discuss
> > >>> > > > >> > > > > more with KIP-500 folks to get the details right.
> > >>> > > > >> > > > >
> > >>> > > > >> > > > > Thanks,
> > >>> > > > >> > > > > Justine
> > >>> > > > >> > > > >
> > >>> > > > >> > > > > On Thu, Sep 24, 2020 at 12:30 PM Jason Gustafson <
> > >>> > > > >> jason@confluent.io
> > >>> > > > >> > >
> > >>> > > > >> > > > > wrote:
> > >>> > > > >> > > > >
> > >>> > > > >> > > > > > Hi Justine,
> > >>> > > > >> > > > > >
> > >>> > > > >> > > > > > Thanks for picking up this work. I have a few
> > >>> > > > >> questions/comments:
> > >>> > > > >> > > > > >
> > >>> > > > >> > > > > > 1. It sounds like the directory structure is still
> > >>> going
> > >>> > to
> > >>> > > be
> > >>> > > > >> > based
> > >>> > > > >> > > on
> > >>> > > > >> > > > > > topic names. Do I have that right? One
> complication
> > is
> > >>> > that
> > >>> > > > the
> > >>> > > > >> > > > > > LeaderAndIsr request does not include the topic
> name
> > >>> any
> > >>> > > > longer.
> > >>> > > > >> > This
> > >>> > > > >> > > > > means
> > >>> > > > >> > > > > > that a replica must wait for the UpdateMetadata
> > >>> request to
> > >>> > > > >> arrive
> > >>> > > > >> > > with
> > >>> > > > >> > > > > the
> > >>> > > > >> > > > > > topic name mapping before it can create the
> > >>> directory. I
> > >>> > > > wonder
> > >>> > > > >> if
> > >>> > > > >> > it
> > >>> > > > >> > > > > would
> > >>> > > > >> > > > > > be simpler to avoid assumptions on the ordering of
> > >>> > > > >> UpdateMetadata
> > >>> > > > >> > and
> > >>> > > > >> > > > let
> > >>> > > > >> > > > > > LeaderAndIsr include the topic name as well. Feels
> > >>> like we
> > >>> > > are
> > >>> > > > >> not
> > >>> > > > >> > > > saving
> > >>> > > > >> > > > > > that much by excluding it.
> > >>> > > > >> > > > > >
> > >>> > > > >> > > > > > 2. On a related note, it seems that the reason we
> > >>> have the
> > >>> > > > >> > partition
> > >>> > > > >> > > > > > metadata file is because we are not changing the
> > >>> directory
> > >>> > > > >> > structure.
> > >>> > > > >> > > > We
> > >>> > > > >> > > > > > need it so that we remember which directories map
> to
> > >>> which
> > >>> > > > topic
> > >>> > > > >> > id.
> > >>> > > > >> > > I
> > >>> > > > >> > > > > > think it would be useful to add some detail about
> > why
> > >>> > > changing
> > >>> > > > >> the
> > >>> > > > >> > > > > > directory layout was rejected. Basically trying to
> > >>> > > understand
> > >>> > > > >> how
> > >>> > > > >> > > much
> > >>> > > > >> > > > > > effort we are saving by skipping this step if we
> > have
> > >>> to
> > >>> > > > >> implement
> > >>> > > > >> > > this
> > >>> > > > >> > > > > > additional bookkeeping. One concern I have for
> > >>> example is
> > >>> > > that
> > >>> > > > >> the
> > >>> > > > >> > > > > > partition metadata file gets corrupt and then all
> > >>> bets are
> > >>> > > off
> > >>> > > > >> as
> > >>> > > > >> > far
> > >>> > > > >> > > > as
> > >>> > > > >> > > > > > topic consistency.
> > >>> > > > >> > > > > >
> > >>> > > > >> > > > > > 3. Is it worth adding support now for topic ids in
> > the
> > >>> > > > `Produce`
> > >>> > > > >> > > > request
> > >>> > > > >> > > > > > now? Overhead is mentioned as one of the
> motivations
> > >>> and
> > >>> > the
> > >>> > > > >> best
> > >>> > > > >> > > APIs
> > >>> > > > >> > > > to
> > >>> > > > >> > > > > > get savings from are `Produce` and `Fetch`.
> > >>> > > > >> > > > > >
> > >>> > > > >> > > > > > 4. When it comes to bootstrapping the metadata
> topic
> > >>> with
> > >>> > > > >> respect
> > >>> > > > >> > to
> > >>> > > > >> > > > > > KIP-500, one suggestion would be to reserve a
> > sentinel
> > >>> > topic
> > >>> > > > ID
> > >>> > > > >> > which
> > >>> > > > >> > > > can
> > >>> > > > >> > > > > > be used initially by new brokers. When the first
> > >>> > controller
> > >>> > > is
> > >>> > > > >> > > elected,
> > >>> > > > >> > > > > it
> > >>> > > > >> > > > > > can assign a topicId to the metadata topic. This
> is
> > a
> > >>> bit
> > >>> > > > >> similar
> > >>> > > > >> > to
> > >>> > > > >> > > > how
> > >>> > > > >> > > > > we
> > >>> > > > >> > > > > > were planning to generate the clusterId.
> > >>> > > > >> > > > > >
> > >>> > > > >> > > > > > Thanks,
> > >>> > > > >> > > > > > Jason
> > >>> > > > >> > > > > >
> > >>> > > > >> > > > > >
> > >>> > > > >> > > > > > On Thu, Sep 24, 2020 at 11:10 AM Jun Rao <
> > >>> > jun@confluent.io>
> > >>> > > > >> wrote:
> > >>> > > > >> > > > > >
> > >>> > > > >> > > > > > > Hi, Justine,
> > >>> > > > >> > > > > > >
> > >>> > > > >> > > > > > > Thanks for the updated KIP. A few more comments
> > >>> below.
> > >>> > > > >> > > > > > >
> > >>> > > > >> > > > > > > 30.  {"name": "id", "type": "string", "doc":
> > >>> "version
> > >>> > > id"}}:
> > >>> > > > >> The
> > >>> > > > >> > > doc
> > >>> > > > >> > > > > > should
> > >>> > > > >> > > > > > > say UUID.
> > >>> > > > >> > > > > > >
> > >>> > > > >> > > > > > > 31. LeaderAndIsrResponse v5 and
> > StopReplicaResponse
> > >>> v4 :
> > >>> > > It
> > >>> > > > >> seems
> > >>> > > > >> > > > there
> > >>> > > > >> > > > > > is
> > >>> > > > >> > > > > > > no need to add topic_id at partitions level.
> > >>> > > > >> > > > > > >
> > >>> > > > >> > > > > > > 32. Regarding partition metadata file. Perhaps
> the
> > >>> key
> > >>> > can
> > >>> > > > be
> > >>> > > > >> a
> > >>> > > > >> > > > single
> > >>> > > > >> > > > > > > word, sth like the following.
> > >>> > > > >> > > > > > > version: 0
> > >>> > > > >> > > > > > > topic_id: 46bdb63f-9e8d-4a38-bf7b-ee4eb2a794e4
> > >>> > > > >> > > > > > >
> > >>> > > > >> > > > > > > 33. Another tricky thing that I realized is how
> to
> > >>> > support
> > >>> > > > the
> > >>> > > > >> > > > metadata
> > >>> > > > >> > > > > > > topic introduced in KIP-595. It's a bit tricky
> to
> > >>> > assign a
> > >>> > > > >> UUID
> > >>> > > > >> > to
> > >>> > > > >> > > > the
> > >>> > > > >> > > > > > > metadata topic since we have a chicken-and-egg
> > >>> problem.
> > >>> > > The
> > >>> > > > >> > > > controller
> > >>> > > > >> > > > > > > needs to persist the UUID in the metadata topic
> in
> > >>> order
> > >>> > > to
> > >>> > > > >> > assign
> > >>> > > > >> > > > one
> > >>> > > > >> > > > > > > successfully, but the metadata topic is needed
> to
> > >>> elect
> > >>> > a
> > >>> > > > >> > > controller
> > >>> > > > >> > > > > > > (KIP-631). So, this probably needs a bit more
> > >>> thought.
> > >>> > > > >> > > > > > >
> > >>> > > > >> > > > > > > Jun
> > >>> > > > >> > > > > > >
> > >>> > > > >> > > > > > > On Thu, Sep 24, 2020 at 3:04 AM Ismael Juma <
> > >>> > > > >> ismael@juma.me.uk>
> > >>> > > > >> > > > wrote:
> > >>> > > > >> > > > > > >
> > >>> > > > >> > > > > > > > Also, can we provide more details on how the
> > >>> Partition
> > >>> > > > >> Metadata
> > >>> > > > >> > > > file
> > >>> > > > >> > > > > > will
> > >>> > > > >> > > > > > > > be used?
> > >>> > > > >> > > > > > > >
> > >>> > > > >> > > > > > > > Ismael
> > >>> > > > >> > > > > > > >
> > >>> > > > >> > > > > > > > On Thu, Sep 24, 2020 at 3:01 AM Ismael Juma <
> > >>> > > > >> ismael@juma.me.uk
> > >>> > > > >> > >
> > >>> > > > >> > > > > wrote:
> > >>> > > > >> > > > > > > >
> > >>> > > > >> > > > > > > > > Hi Justine,
> > >>> > > > >> > > > > > > > >
> > >>> > > > >> > > > > > > > > I think we need to update the "Rejected
> > >>> > Alternatives"
> > >>> > > > >> section
> > >>> > > > >> > > to
> > >>> > > > >> > > > > take
> > >>> > > > >> > > > > > > > into
> > >>> > > > >> > > > > > > > > account that the proposal now removes the
> > topic
> > >>> name
> > >>> > > > from
> > >>> > > > >> the
> > >>> > > > >> > > > fetch
> > >>> > > > >> > > > > > > > > request. Also, if we are removing it from
> the
> > >>> Fetch
> > >>> > > > >> request,
> > >>> > > > >> > > does
> > >>> > > > >> > > > > it
> > >>> > > > >> > > > > > > make
> > >>> > > > >> > > > > > > > > sense not to remove it from similar requests
> > >>> like
> > >>> > > > >> > > > > ListOffsetRequest?
> > >>> > > > >> > > > > > > > >
> > >>> > > > >> > > > > > > > > Ismael
> > >>> > > > >> > > > > > > > >
> > >>> > > > >> > > > > > > > > On Thu, Sep 24, 2020 at 2:46 AM David Jacot
> <
> > >>> > > > >> > > djacot@confluent.io
> > >>> > > > >> > > > >
> > >>> > > > >> > > > > > > wrote:
> > >>> > > > >> > > > > > > > >
> > >>> > > > >> > > > > > > > >> Hi Justine,
> > >>> > > > >> > > > > > > > >>
> > >>> > > > >> > > > > > > > >> Thanks for the KIP. I finally had time to
> > read
> > >>> it
> > >>> > :).
> > >>> > > > It
> > >>> > > > >> is
> > >>> > > > >> > a
> > >>> > > > >> > > > > great
> > >>> > > > >> > > > > > > > >> improvement.
> > >>> > > > >> > > > > > > > >>
> > >>> > > > >> > > > > > > > >> I have a few comments/questions:
> > >>> > > > >> > > > > > > > >>
> > >>> > > > >> > > > > > > > >> 1. It seems that the schema of the
> > >>> > StopReplicaRequest
> > >>> > > > is
> > >>> > > > >> > > > slightly
> > >>> > > > >> > > > > > > > >> outdated.
> > >>> > > > >> > > > > > > > >> We
> > >>> > > > >> > > > > > > > >> did some changes as part of KIP-570. V3 is
> > >>> already
> > >>> > > > >> organized
> > >>> > > > >> > > by
> > >>> > > > >> > > > > > > topics.
> > >>> > > > >> > > > > > > > >>
> > >>> > > > >> > > > > > > > >> 2. I just want to make sure that I
> understand
> > >>> the
> > >>> > > > >> > > reconciliation
> > >>> > > > >> > > > > > > > >> logic correctly. When
> > >>> > > > >> > > > > > > > >> an "INCREMENTAL" LeaderAndIsr Request is
> > >>> received,
> > >>> > > the
> > >>> > > > >> > broker
> > >>> > > > >> > > > will
> > >>> > > > >> > > > > > > also
> > >>> > > > >> > > > > > > > >> reconcile
> > >>> > > > >> > > > > > > > >> when the local uuid does not match the uuid
> > in
> > >>> the
> > >>> > > > >> request,
> > >>> > > > >> > > > right?
> > >>> > > > >> > > > > > In
> > >>> > > > >> > > > > > > > this
> > >>> > > > >> > > > > > > > >> case, the
> > >>> > > > >> > > > > > > > >> local log is staged for deletion.
> > >>> > > > >> > > > > > > > >>
> > >>> > > > >> > > > > > > > >> 3. In the documentation of the `
> > >>> > > > >> delete.stale.topic.delay.ms
> > >>> > > > >> > `
> > >>> > > > >> > > > > > config,
> > >>> > > > >> > > > > > > it
> > >>> > > > >> > > > > > > > >> says "When a
> > >>> > > > >> > > > > > > > >> FULL LeaderAndIsrRequest is received..."
> but
> > I
> > >>> > > suppose
> > >>> > > > it
> > >>> > > > >> > > > applies
> > >>> > > > >> > > > > to
> > >>> > > > >> > > > > > > > both
> > >>> > > > >> > > > > > > > >> types.
> > >>> > > > >> > > > > > > > >>
> > >>> > > > >> > > > > > > > >> Best,
> > >>> > > > >> > > > > > > > >> David
> > >>> > > > >> > > > > > > > >>
> > >>> > > > >> > > > > > > > >> On Thu, Sep 24, 2020 at 1:40 AM Justine
> > Olshan
> > >>> <
> > >>> > > > >> > > > > > jolshan@confluent.io>
> > >>> > > > >> > > > > > > > >> wrote:
> > >>> > > > >> > > > > > > > >>
> > >>> > > > >> > > > > > > > >> > Hi Jun,
> > >>> > > > >> > > > > > > > >> >
> > >>> > > > >> > > > > > > > >> > Thanks for the comments. I apologize for
> > >>> some of
> > >>> > > the
> > >>> > > > >> typos
> > >>> > > > >> > > and
> > >>> > > > >> > > > > > > > >> confusion.
> > >>> > > > >> > > > > > > > >> > I’ve updated the KIP to fix some of the
> > >>> issues
> > >>> > you
> > >>> > > > >> > > mentioned.
> > >>> > > > >> > > > > > > > >> >
> > >>> > > > >> > > > > > > > >> > 20.2 I’ve changed the type to String.
> > >>> > > > >> > > > > > > > >> > 20.1/3 I’ve updated the TopicZNode to fix
> > >>> > > formatting
> > >>> > > > >> and
> > >>> > > > >> > > > reflect
> > >>> > > > >> > > > > > the
> > >>> > > > >> > > > > > > > >> latest
> > >>> > > > >> > > > > > > > >> > version before this change.
> > >>> > > > >> > > > > > > > >> >
> > >>> > > > >> > > > > > > > >> > 21. You are correct and I’ve removed this
> > >>> line.
> > >>> > > I’ve
> > >>> > > > >> also
> > >>> > > > >> > > > added
> > >>> > > > >> > > > > a
> > >>> > > > >> > > > > > > line
> > >>> > > > >> > > > > > > > >> > mentioning an IBP bump is necessary for
> > >>> migration
> > >>> > > > >> > > > > > > > >> >
> > >>> > > > >> > > > > > > > >> > 22. I think the wording was unclear but
> > your
> > >>> > > summary
> > >>> > > > is
> > >>> > > > >> > what
> > >>> > > > >> > > > was
> > >>> > > > >> > > > > > > > >> intended
> > >>> > > > >> > > > > > > > >> > by this line. I’ve updated to make this
> > point
> > >>> > more
> > >>> > > > >> clear.
> > >>> > > > >> > > > > “Remove
> > >>> > > > >> > > > > > > > >> deleted
> > >>> > > > >> > > > > > > > >> > topics from replicas by sending
> > >>> > StopReplicaRequest
> > >>> > > V3
> > >>> > > > >> > before
> > >>> > > > >> > > > the
> > >>> > > > >> > > > > > IBP
> > >>> > > > >> > > > > > > > >> bump
> > >>> > > > >> > > > > > > > >> > using the old logic, and using V4 and the
> > new
> > >>> > logic
> > >>> > > > >> with
> > >>> > > > >> > > topic
> > >>> > > > >> > > > > IDs
> > >>> > > > >> > > > > > > > after
> > >>> > > > >> > > > > > > > >> > the IBP bump.”
> > >>> > > > >> > > > > > > > >> >
> > >>> > > > >> > > > > > > > >> > 23. I’ve removed the unspecified type
> from
> > >>> the
> > >>> > KIP
> > >>> > > > and
> > >>> > > > >> > > mention
> > >>> > > > >> > > > > > that
> > >>> > > > >> > > > > > > > IBP
> > >>> > > > >> > > > > > > > >> > will be used to handle this request. “IBP
> > >>> will be
> > >>> > > > used
> > >>> > > > >> to
> > >>> > > > >> > > > > > determine
> > >>> > > > >> > > > > > > > >> whether
> > >>> > > > >> > > > > > > > >> > this new form of the request will be
> used.
> > >>> For
> > >>> > > older
> > >>> > > > >> > > requests,
> > >>> > > > >> > > > > we
> > >>> > > > >> > > > > > > will
> > >>> > > > >> > > > > > > > >> > ignore this field and default to previous
> > >>> > > behavior.”
> > >>> > > > >> > > > > > > > >> >
> > >>> > > > >> > > > > > > > >> > 24. I’ve fixed this typo.
> > >>> > > > >> > > > > > > > >> >
> > >>> > > > >> > > > > > > > >> > 25. I've added a topics at a higher level
> > for
> > >>> > > > >> > > > > LeaderAndIsrResponse
> > >>> > > > >> > > > > > > v5,
> > >>> > > > >> > > > > > > > >> > StopReplicaResponse v4. I've also changed
> > >>> > > > >> > StopReplicaRequest
> > >>> > > > >> > > > v4
> > >>> > > > >> > > > > to
> > >>> > > > >> > > > > > > > have
> > >>> > > > >> > > > > > > > >> > topics at a higher level.
> > >>> > > > >> > > > > > > > >> >
> > >>> > > > >> > > > > > > > >> > 26. I’ve updated
> > >>> forgotten_topics_data--added the
> > >>> > > > >> topic ID
> > >>> > > > >> > > and
> > >>> > > > >> > > > > > > removed
> > >>> > > > >> > > > > > > > >> the
> > >>> > > > >> > > > > > > > >> > topic name
> > >>> > > > >> > > > > > > > >> >
> > >>> > > > >> > > > > > > > >> > 27. I’ve decided on plain text, and I’ve
> > >>> added an
> > >>> > > > >> example.
> > >>> > > > >> > > > > > > > >> >
> > >>> > > > >> > > > > > > > >> > 28. I’ve added this idea to future work.
> > >>> > > > >> > > > > > > > >> >
> > >>> > > > >> > > > > > > > >> > Thanks again for taking a look,
> > >>> > > > >> > > > > > > > >> >
> > >>> > > > >> > > > > > > > >> > Justine
> > >>> > > > >> > > > > > > > >> >
> > >>> > > > >> > > > > > > > >> > On Wed, Sep 23, 2020 at 10:28 AM Jun Rao
> <
> > >>> > > > >> > jun@confluent.io>
> > >>> > > > >> > > > > > wrote:
> > >>> > > > >> > > > > > > > >> >
> > >>> > > > >> > > > > > > > >> > > Hi, Justine,
> > >>> > > > >> > > > > > > > >> > >
> > >>> > > > >> > > > > > > > >> > > Thanks for the response. Made another
> > >>> pass. A
> > >>> > few
> > >>> > > > >> more
> > >>> > > > >> > > > > comments
> > >>> > > > >> > > > > > > > below.
> > >>> > > > >> > > > > > > > >> > >
> > >>> > > > >> > > > > > > > >> > > 20. znode schema:
> > >>> > > > >> > > > > > > > >> > > 20.1 It seems that {"name": "version",
> > >>> "type":
> > >>> > > > "int",
> > >>> > > > >> > > "id":
> > >>> > > > >> > > > > > > "UUID",
> > >>> > > > >> > > > > > > > >> > "doc":
> > >>> > > > >> > > > > > > > >> > > "version id"} should be {"name":
> > "version",
> > >>> > > "type":
> > >>> > > > >> > > "int"},
> > >>> > > > >> > > > > > > {"name":
> > >>> > > > >> > > > > > > > >> > "id",
> > >>> > > > >> > > > > > > > >> > > "type": "UUID", "doc": "version id"}.
> > >>> > > > >> > > > > > > > >> > > 20.2 The znode format is JSON which
> > doesn't
> > >>> > have
> > >>> > > > UUID
> > >>> > > > >> > > type.
> > >>> > > > >> > > > So
> > >>> > > > >> > > > > > the
> > >>> > > > >> > > > > > > > >> type
> > >>> > > > >> > > > > > > > >> > > probably should be string?
> > >>> > > > >> > > > > > > > >> > > 20.3 Also, the existing format used
> seems
> > >>> > > outdated.
> > >>> > > > >> It
> > >>> > > > >> > > > should
> > >>> > > > >> > > > > > have
> > >>> > > > >> > > > > > > > the
> > >>> > > > >> > > > > > > > >> > > following format.
> > >>> > > > >> > > > > > > > >> > >     Json.encodeAsBytes(Map(
> > >>> > > > >> > > > > > > > >> > >       "version" -> 2,
> > >>> > > > >> > > > > > > > >> > >       "partitions" ->
> > >>> > > replicaAssignmentJson.asJava,
> > >>> > > > >> > > > > > > > >> > >       "adding_replicas" ->
> > >>> > > > >> > > > > addingReplicasAssignmentJson.asJava,
> > >>> > > > >> > > > > > > > >> > >       "removing_replicas" ->
> > >>> > > > >> > > > > > removingReplicasAssignmentJson.asJava
> > >>> > > > >> > > > > > > > >> > >     ).asJava)
> > >>> > > > >> > > > > > > > >> > >   }
> > >>> > > > >> > > > > > > > >> > >
> > >>> > > > >> > > > > > > > >> > > 21. Migration: The KIP says "The
> > migration
> > >>> > > process
> > >>> > > > >> can
> > >>> > > > >> > > take
> > >>> > > > >> > > > > > place
> > >>> > > > >> > > > > > > > >> without
> > >>> > > > >> > > > > > > > >> > > an inter-broker protocol bump, as the
> > >>> format
> > >>> > > stored
> > >>> > > > >> in
> > >>> > > > >> > > > > > > > >> > > /brokers/topics/[topic] will be
> > compatible
> > >>> with
> > >>> > > > older
> > >>> > > > >> > > broker
> > >>> > > > >> > > > > > > > >> versions."
> > >>> > > > >> > > > > > > > >> > > However, since we are bumping up the
> > >>> version of
> > >>> > > > >> > > inter-broker
> > >>> > > > >> > > > > > > > >> requests, it
> > >>> > > > >> > > > > > > > >> > > seems that we need to use IBP for
> > >>> migration.
> > >>> > > > >> > > > > > > > >> > >
> > >>> > > > >> > > > > > > > >> > > 22. The KIP says "Remove deleted topics
> > >>> from
> > >>> > > > >> replicas by
> > >>> > > > >> > > > > sending
> > >>> > > > >> > > > > > > > >> > > StopReplicaRequest V3 for any topics
> > which
> > >>> do
> > >>> > not
> > >>> > > > >> > contain
> > >>> > > > >> > > a
> > >>> > > > >> > > > > > topic
> > >>> > > > >> > > > > > > > ID,
> > >>> > > > >> > > > > > > > >> and
> > >>> > > > >> > > > > > > > >> > > V4 for any topics which do contain a
> > topic
> > >>> > ID.".
> > >>> > > > >> > However,
> > >>> > > > >> > > if
> > >>> > > > >> > > > > we
> > >>> > > > >> > > > > > > use
> > >>> > > > >> > > > > > > > >> IBP,
> > >>> > > > >> > > > > > > > >> > it
> > >>> > > > >> > > > > > > > >> > > seems that the controller will either
> > send
> > >>> > > > >> > > > StopReplicaRequest
> > >>> > > > >> > > > > V3
> > >>> > > > >> > > > > > > > >> > > or StopReplicaRequest V4, but never
> mixed
> > >>> V3
> > >>> > and
> > >>> > > V4
> > >>> > > > >> for
> > >>> > > > >> > > > > > different
> > >>> > > > >> > > > > > > > >> topics.
> > >>> > > > >> > > > > > > > >> > > Basically, before the IBP bump, V3 will
> > be
> > >>> > used.
> > >>> > > > >> After
> > >>> > > > >> > the
> > >>> > > > >> > > > IBP
> > >>> > > > >> > > > > > > bump,
> > >>> > > > >> > > > > > > > >> > > topicId will be created and V4 will be
> > >>> used.
> > >>> > > > >> > > > > > > > >> > >
> > >>> > > > >> > > > > > > > >> > > 23. Given that we depend on IBP, do we
> > >>> still
> > >>> > need
> > >>> > > > "0
> > >>> > > > >> > > > > > UNSPECIFIED"
> > >>> > > > >> > > > > > > > >> > > in LeaderAndIsr?
> > >>> > > > >> > > > > > > > >> > >
> > >>> > > > >> > > > > > > > >> > > 24. LeaderAndIsrResponse v5 : It still
> > has
> > >>> the
> > >>> > > > topic
> > >>> > > > >> > > field.
> > >>> > > > >> > > > > > > > >> > >
> > >>> > > > >> > > > > > > > >> > > 25. LeaderAndIsrResponse v5,
> > >>> > StopReplicaResponse
> > >>> > > > v4:
> > >>> > > > >> > Could
> > >>> > > > >> > > > we
> > >>> > > > >> > > > > > use
> > >>> > > > >> > > > > > > > this
> > >>> > > > >> > > > > > > > >> > > opportunity to organize the response
> in 2
> > >>> > levels,
> > >>> > > > >> first
> > >>> > > > >> > by
> > >>> > > > >> > > > > > topic,
> > >>> > > > >> > > > > > > > >> then by
> > >>> > > > >> > > > > > > > >> > > partition, as most other
> > >>> requests/responses?
> > >>> > > > >> > > > > > > > >> > >
> > >>> > > > >> > > > > > > > >> > > 26. FetchRequest v13 : Should
> > >>> > > forgotten_topics_data
> > >>> > > > >> use
> > >>> > > > >> > > > > topicId
> > >>> > > > >> > > > > > > too?
> > >>> > > > >> > > > > > > > >> > >
> > >>> > > > >> > > > > > > > >> > > 27. "This file can either be plain text
> > >>> > > (key/value
> > >>> > > > >> > pairs)
> > >>> > > > >> > > or
> > >>> > > > >> > > > > > > JSON."
> > >>> > > > >> > > > > > > > >> Have
> > >>> > > > >> > > > > > > > >> > we
> > >>> > > > >> > > > > > > > >> > > decided which one to use? Also, it
> would
> > be
> > >>> > > helpful
> > >>> > > > >> to
> > >>> > > > >> > > > provide
> > >>> > > > >> > > > > > an
> > >>> > > > >> > > > > > > > >> > example.
> > >>> > > > >> > > > > > > > >> > >
> > >>> > > > >> > > > > > > > >> > > 28. Future improvement: Another future
> > >>> > > improvement
> > >>> > > > >> > > > opportunity
> > >>> > > > >> > > > > > is
> > >>> > > > >> > > > > > > to
> > >>> > > > >> > > > > > > > >> use
> > >>> > > > >> > > > > > > > >> > > topicId in
> > >>> GroupMetadataManager.offsetCommitKey
> > >>> > > in
> > >>> > > > >> the
> > >>> > > > >> > > > > > > offset_commit
> > >>> > > > >> > > > > > > > >> > topic.
> > >>> > > > >> > > > > > > > >> > > This may save some space.
> > >>> > > > >> > > > > > > > >> > >
> > >>> > > > >> > > > > > > > >> > > Thanks,
> > >>> > > > >> > > > > > > > >> > >
> > >>> > > > >> > > > > > > > >> > > Jun
> > >>> > > > >> > > > > > > > >> > >
> > >>> > > > >> > > > > > > > >> > > On Wed, Sep 23, 2020 at 8:50 AM Justine
> > >>> Olshan
> > >>> > <
> > >>> > > > >> > > > > > > > jolshan@confluent.io>
> > >>> > > > >> > > > > > > > >> > > wrote:
> > >>> > > > >> > > > > > > > >> > >
> > >>> > > > >> > > > > > > > >> > > > Hi Tom,
> > >>> > > > >> > > > > > > > >> > > >
> > >>> > > > >> > > > > > > > >> > > > Thanks for the comment. I think this
> > is a
> > >>> > > really
> > >>> > > > >> good
> > >>> > > > >> > > idea
> > >>> > > > >> > > > > and
> > >>> > > > >> > > > > > > it
> > >>> > > > >> > > > > > > > >> has
> > >>> > > > >> > > > > > > > >> > > been
> > >>> > > > >> > > > > > > > >> > > > added to the KIP under the newly
> added
> > >>> > tooling
> > >>> > > > >> > section.
> > >>> > > > >> > > > > > > > >> > > >
> > >>> > > > >> > > > > > > > >> > > > Thanks again,
> > >>> > > > >> > > > > > > > >> > > > Justine
> > >>> > > > >> > > > > > > > >> > > >
> > >>> > > > >> > > > > > > > >> > > > On Wed, Sep 23, 2020 at 3:17 AM Tom
> > >>> Bentley <
> > >>> > > > >> > > > > > > tbentley@redhat.com>
> > >>> > > > >> > > > > > > > >> > wrote:
> > >>> > > > >> > > > > > > > >> > > >
> > >>> > > > >> > > > > > > > >> > > > > Hi Justine,
> > >>> > > > >> > > > > > > > >> > > > >
> > >>> > > > >> > > > > > > > >> > > > > I know you started the vote thread,
> > >>> but on
> > >>> > > > >> > re-reading
> > >>> > > > >> > > > the
> > >>> > > > >> > > > > > KIP
> > >>> > > > >> > > > > > > I
> > >>> > > > >> > > > > > > > >> > noticed
> > >>> > > > >> > > > > > > > >> > > > > that although the topic id is
> > included
> > >>> in
> > >>> > the
> > >>> > > > >> > > > > > MetadataResponse
> > >>> > > > >> > > > > > > > >> it's
> > >>> > > > >> > > > > > > > >> > not
> > >>> > > > >> > > > > > > > >> > > > > surfaced in the output from
> > >>> > `kafka-topics.sh
> > >>> > > > >> > > > --describe`.
> > >>> > > > >> > > > > > > Maybe
> > >>> > > > >> > > > > > > > >> that
> > >>> > > > >> > > > > > > > >> > > was
> > >>> > > > >> > > > > > > > >> > > > > intentional because ids are
> > >>> intentionally
> > >>> > not
> > >>> > > > >> really
> > >>> > > > >> > > > > > something
> > >>> > > > >> > > > > > > > the
> > >>> > > > >> > > > > > > > >> > user
> > >>> > > > >> > > > > > > > >> > > > > should care deeply about, but it
> > would
> > >>> also
> > >>> > > > make
> > >>> > > > >> > life
> > >>> > > > >> > > > > harder
> > >>> > > > >> > > > > > > for
> > >>> > > > >> > > > > > > > >> > anyone
> > >>> > > > >> > > > > > > > >> > > > > debugging Kafka and this would
> likely
> > >>> get
> > >>> > > worse
> > >>> > > > >> the
> > >>> > > > >> > > more
> > >>> > > > >> > > > > > topic
> > >>> > > > >> > > > > > > > ids
> > >>> > > > >> > > > > > > > >> > got
> > >>> > > > >> > > > > > > > >> > > > > rolled out across the protocols,
> > >>> clients
> > >>> > etc.
> > >>> > > > It
> > >>> > > > >> > seems
> > >>> > > > >> > > > > > likely
> > >>> > > > >> > > > > > > > that
> > >>> > > > >> > > > > > > > >> > > > > `kafka-topics.sh` will eventually
> > need
> > >>> the
> > >>> > > > >> ability
> > >>> > > > >> > to
> > >>> > > > >> > > > show
> > >>> > > > >> > > > > > the
> > >>> > > > >> > > > > > > > id
> > >>> > > > >> > > > > > > > >> of
> > >>> > > > >> > > > > > > > >> > a
> > >>> > > > >> > > > > > > > >> > > > > topic and perhaps find a topic name
> > >>> given
> > >>> > an
> > >>> > > > id.
> > >>> > > > >> Is
> > >>> > > > >> > > > there
> > >>> > > > >> > > > > > any
> > >>> > > > >> > > > > > > > >> reason
> > >>> > > > >> > > > > > > > >> > > not
> > >>> > > > >> > > > > > > > >> > > > to
> > >>> > > > >> > > > > > > > >> > > > > implement that in this KIP?
> > >>> > > > >> > > > > > > > >> > > > >
> > >>> > > > >> > > > > > > > >> > > > > Many thanks,
> > >>> > > > >> > > > > > > > >> > > > >
> > >>> > > > >> > > > > > > > >> > > > > Tom
> > >>> > > > >> > > > > > > > >> > > > >
> > >>> > > > >> > > > > > > > >> > > > > On Mon, Sep 21, 2020 at 9:54 PM
> > Justine
> > >>> > > Olshan
> > >>> > > > <
> > >>> > > > >> > > > > > > > >> jolshan@confluent.io
> > >>> > > > >> > > > > > > > >> > >
> > >>> > > > >> > > > > > > > >> > > > > wrote:
> > >>> > > > >> > > > > > > > >> > > > >
> > >>> > > > >> > > > > > > > >> > > > > > Hi all,
> > >>> > > > >> > > > > > > > >> > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > After thinking about it, I've
> > >>> decided to
> > >>> > > > remove
> > >>> > > > >> > the
> > >>> > > > >> > > > > topic
> > >>> > > > >> > > > > > > name
> > >>> > > > >> > > > > > > > >> from
> > >>> > > > >> > > > > > > > >> > > the
> > >>> > > > >> > > > > > > > >> > > > > > Fetch Request and Response after
> > all.
> > >>> > Since
> > >>> > > > >> there
> > >>> > > > >> > > are
> > >>> > > > >> > > > so
> > >>> > > > >> > > > > > > many
> > >>> > > > >> > > > > > > > of
> > >>> > > > >> > > > > > > > >> > > these
> > >>> > > > >> > > > > > > > >> > > > > > requests per second, it is worth
> > >>> removing
> > >>> > > the
> > >>> > > > >> > extra
> > >>> > > > >> > > > > > > > information.
> > >>> > > > >> > > > > > > > >> > I've
> > >>> > > > >> > > > > > > > >> > > > > > updated the KIP to reflect this
> > >>> change.
> > >>> > > > >> > > > > > > > >> > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > Please let me know if there is
> > >>> anything
> > >>> > > else
> > >>> > > > we
> > >>> > > > >> > > should
> > >>> > > > >> > > > > > > discuss
> > >>> > > > >> > > > > > > > >> > before
> > >>> > > > >> > > > > > > > >> > > > > > voting.
> > >>> > > > >> > > > > > > > >> > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > Thank you,
> > >>> > > > >> > > > > > > > >> > > > > > Justine
> > >>> > > > >> > > > > > > > >> > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > On Fri, Sep 18, 2020 at 9:46 AM
> > >>> Justine
> > >>> > > > Olshan
> > >>> > > > >> <
> > >>> > > > >> > > > > > > > >> > jolshan@confluent.io
> > >>> > > > >> > > > > > > > >> > > >
> > >>> > > > >> > > > > > > > >> > > > > > wrote:
> > >>> > > > >> > > > > > > > >> > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > > Hi Jun,
> > >>> > > > >> > > > > > > > >> > > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > > I see what you are saying. For
> > now
> > >>> we
> > >>> > can
> > >>> > > > >> remove
> > >>> > > > >> > > the
> > >>> > > > >> > > > > > extra
> > >>> > > > >> > > > > > > > >> > > > information.
> > >>> > > > >> > > > > > > > >> > > > > > > I'll leave the option to add
> more
> > >>> > fields
> > >>> > > to
> > >>> > > > >> the
> > >>> > > > >> > > file
> > >>> > > > >> > > > > in
> > >>> > > > >> > > > > > > the
> > >>> > > > >> > > > > > > > >> > future.
> > >>> > > > >> > > > > > > > >> > > > The
> > >>> > > > >> > > > > > > > >> > > > > > KIP
> > >>> > > > >> > > > > > > > >> > > > > > > has been updated to reflect
> this
> > >>> > change.
> > >>> > > > >> > > > > > > > >> > > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > > Thanks,
> > >>> > > > >> > > > > > > > >> > > > > > > Justine
> > >>> > > > >> > > > > > > > >> > > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > > On Fri, Sep 18, 2020 at 8:46 AM
> > Jun
> > >>> > Rao <
> > >>> > > > >> > > > > > jun@confluent.io
> > >>> > > > >> > > > > > > >
> > >>> > > > >> > > > > > > > >> > wrote:
> > >>> > > > >> > > > > > > > >> > > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> Hi, Justine,
> > >>> > > > >> > > > > > > > >> > > > > > >>
> > >>> > > > >> > > > > > > > >> > > > > > >> Thanks for the reply.
> > >>> > > > >> > > > > > > > >> > > > > > >>
> > >>> > > > >> > > > > > > > >> > > > > > >> 13. If the log directory is
> the
> > >>> source
> > >>> > > of
> > >>> > > > >> > truth,
> > >>> > > > >> > > it
> > >>> > > > >> > > > > > means
> > >>> > > > >> > > > > > > > >> that
> > >>> > > > >> > > > > > > > >> > the
> > >>> > > > >> > > > > > > > >> > > > > > >> redundant info in the metadata
> > >>> file
> > >>> > will
> > >>> > > > be
> > >>> > > > >> > > > ignored.
> > >>> > > > >> > > > > > Then
> > >>> > > > >> > > > > > > > the
> > >>> > > > >> > > > > > > > >> > > > question
> > >>> > > > >> > > > > > > > >> > > > > > is
> > >>> > > > >> > > > > > > > >> > > > > > >> why do we need to put the
> > >>> redundant
> > >>> > info
> > >>> > > > in
> > >>> > > > >> the
> > >>> > > > >> > > > > > metadata
> > >>> > > > >> > > > > > > > file
> > >>> > > > >> > > > > > > > >> > now?
> > >>> > > > >> > > > > > > > >> > > > > > >>
> > >>> > > > >> > > > > > > > >> > > > > > >> Thanks,
> > >>> > > > >> > > > > > > > >> > > > > > >>
> > >>> > > > >> > > > > > > > >> > > > > > >> Jun
> > >>> > > > >> > > > > > > > >> > > > > > >>
> > >>> > > > >> > > > > > > > >> > > > > > >> On Thu, Sep 17, 2020 at 5:07
> PM
> > >>> > Justine
> > >>> > > > >> Olshan
> > >>> > > > >> > <
> > >>> > > > >> > > > > > > > >> > > > jolshan@confluent.io>
> > >>> > > > >> > > > > > > > >> > > > > > >> wrote:
> > >>> > > > >> > > > > > > > >> > > > > > >>
> > >>> > > > >> > > > > > > > >> > > > > > >> > Hi Jun,
> > >>> > > > >> > > > > > > > >> > > > > > >> > Thanks for the quick
> response!
> > >>> > > > >> > > > > > > > >> > > > > > >> >
> > >>> > > > >> > > > > > > > >> > > > > > >> > 12. I've decided to bump up
> > the
> > >>> > > versions
> > >>> > > > >> on
> > >>> > > > >> > the
> > >>> > > > >> > > > > > > requests
> > >>> > > > >> > > > > > > > >> and
> > >>> > > > >> > > > > > > > >> > > > updated
> > >>> > > > >> > > > > > > > >> > > > > > the
> > >>> > > > >> > > > > > > > >> > > > > > >> > KIP. I think it's good we
> > >>> thoroughly
> > >>> > > > >> > discussed
> > >>> > > > >> > > > the
> > >>> > > > >> > > > > > > > options
> > >>> > > > >> > > > > > > > >> > here,
> > >>> > > > >> > > > > > > > >> > > > so
> > >>> > > > >> > > > > > > > >> > > > > we
> > >>> > > > >> > > > > > > > >> > > > > > >> know
> > >>> > > > >> > > > > > > > >> > > > > > >> > we made a good choice. :)
> > >>> > > > >> > > > > > > > >> > > > > > >> >
> > >>> > > > >> > > > > > > > >> > > > > > >> > 13. This is an interesting
> > >>> > situation.
> > >>> > > I
> > >>> > > > >> think
> > >>> > > > >> > > if
> > >>> > > > >> > > > > this
> > >>> > > > >> > > > > > > > does
> > >>> > > > >> > > > > > > > >> > occur
> > >>> > > > >> > > > > > > > >> > > > we
> > >>> > > > >> > > > > > > > >> > > > > > >> should
> > >>> > > > >> > > > > > > > >> > > > > > >> > give a warning. I agree that
> > >>> it's
> > >>> > hard
> > >>> > > > to
> > >>> > > > >> > know
> > >>> > > > >> > > > the
> > >>> > > > >> > > > > > > source
> > >>> > > > >> > > > > > > > >> of
> > >>> > > > >> > > > > > > > >> > > truth
> > >>> > > > >> > > > > > > > >> > > > > for
> > >>> > > > >> > > > > > > > >> > > > > > >> sure
> > >>> > > > >> > > > > > > > >> > > > > > >> > since the directory or the
> > file
> > >>> > could
> > >>> > > be
> > >>> > > > >> > > manually
> > >>> > > > >> > > > > > > > >> modified. I
> > >>> > > > >> > > > > > > > >> > > > guess
> > >>> > > > >> > > > > > > > >> > > > > > the
> > >>> > > > >> > > > > > > > >> > > > > > >> > directory could be used as
> the
> > >>> > source
> > >>> > > of
> > >>> > > > >> > truth.
> > >>> > > > >> > > > To
> > >>> > > > >> > > > > be
> > >>> > > > >> > > > > > > > >> honest,
> > >>> > > > >> > > > > > > > >> > > I'm
> > >>> > > > >> > > > > > > > >> > > > > not
> > >>> > > > >> > > > > > > > >> > > > > > >> > really sure what happens in
> > >>> kafka
> > >>> > when
> > >>> > > > the
> > >>> > > > >> > log
> > >>> > > > >> > > > > > > directory
> > >>> > > > >> > > > > > > > is
> > >>> > > > >> > > > > > > > >> > > > renamed
> > >>> > > > >> > > > > > > > >> > > > > > >> > manually in such a way. I'm
> > also
> > >>> > > > >> wondering if
> > >>> > > > >> > > the
> > >>> > > > >> > > > > > > > >> situation is
> > >>> > > > >> > > > > > > > >> > > > > > >> recoverable
> > >>> > > > >> > > > > > > > >> > > > > > >> > in this scenario.
> > >>> > > > >> > > > > > > > >> > > > > > >> >
> > >>> > > > >> > > > > > > > >> > > > > > >> > Thanks,
> > >>> > > > >> > > > > > > > >> > > > > > >> > Justine
> > >>> > > > >> > > > > > > > >> > > > > > >> >
> > >>> > > > >> > > > > > > > >> > > > > > >> > On Thu, Sep 17, 2020 at 4:28
> > PM
> > >>> Jun
> > >>> > > Rao
> > >>> > > > <
> > >>> > > > >> > > > > > > > jun@confluent.io>
> > >>> > > > >> > > > > > > > >> > > wrote:
> > >>> > > > >> > > > > > > > >> > > > > > >> >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > Hi, Justine,
> > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > Thanks for the reply.
> > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > 12. I don't have a strong
> > >>> > preference
> > >>> > > > >> > either.
> > >>> > > > >> > > > > > However,
> > >>> > > > >> > > > > > > > if
> > >>> > > > >> > > > > > > > >> we
> > >>> > > > >> > > > > > > > >> > > need
> > >>> > > > >> > > > > > > > >> > > > > IBP
> > >>> > > > >> > > > > > > > >> > > > > > >> > > anyway, maybe it's easier
> to
> > >>> just
> > >>> > > bump
> > >>> > > > >> up
> > >>> > > > >> > the
> > >>> > > > >> > > > > > version
> > >>> > > > >> > > > > > > > for
> > >>> > > > >> > > > > > > > >> > all
> > >>> > > > >> > > > > > > > >> > > > > inter
> > >>> > > > >> > > > > > > > >> > > > > > >> > broker
> > >>> > > > >> > > > > > > > >> > > > > > >> > > requests and add the topic
> > id
> > >>> > field
> > >>> > > > as a
> > >>> > > > >> > > > regular
> > >>> > > > >> > > > > > > > field. A
> > >>> > > > >> > > > > > > > >> > > > regular
> > >>> > > > >> > > > > > > > >> > > > > > >> field
> > >>> > > > >> > > > > > > > >> > > > > > >> > is
> > >>> > > > >> > > > > > > > >> > > > > > >> > > a bit more concise in wire
> > >>> > transfer
> > >>> > > > >> than a
> > >>> > > > >> > > > > flexible
> > >>> > > > >> > > > > > > > >> field.
> > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > 13. The confusion that I
> was
> > >>> > > referring
> > >>> > > > >> to
> > >>> > > > >> > is
> > >>> > > > >> > > > > > between
> > >>> > > > >> > > > > > > > the
> > >>> > > > >> > > > > > > > >> > topic
> > >>> > > > >> > > > > > > > >> > > > > name
> > >>> > > > >> > > > > > > > >> > > > > > >> and
> > >>> > > > >> > > > > > > > >> > > > > > >> > > partition number between
> the
> > >>> log
> > >>> > dir
> > >>> > > > and
> > >>> > > > >> > the
> > >>> > > > >> > > > > > metadata
> > >>> > > > >> > > > > > > > >> file.
> > >>> > > > >> > > > > > > > >> > > For
> > >>> > > > >> > > > > > > > >> > > > > > >> example,
> > >>> > > > >> > > > > > > > >> > > > > > >> > if
> > >>> > > > >> > > > > > > > >> > > > > > >> > > the log dir is topicA-1
> and
> > >>> the
> > >>> > > > metadata
> > >>> > > > >> > file
> > >>> > > > >> > > > in
> > >>> > > > >> > > > > it
> > >>> > > > >> > > > > > > has
> > >>> > > > >> > > > > > > > >> > topicB
> > >>> > > > >> > > > > > > > >> > > > and
> > >>> > > > >> > > > > > > > >> > > > > > >> > > partition 0 (say due to a
> > bug
> > >>> or
> > >>> > > > manual
> > >>> > > > >> > > > > > > modification),
> > >>> > > > >> > > > > > > > >> which
> > >>> > > > >> > > > > > > > >> > > one
> > >>> > > > >> > > > > > > > >> > > > > do
> > >>> > > > >> > > > > > > > >> > > > > > we
> > >>> > > > >> > > > > > > > >> > > > > > >> > use
> > >>> > > > >> > > > > > > > >> > > > > > >> > > as the source of truth?
> > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > Jun
> > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > On Thu, Sep 17, 2020 at
> 3:43
> > >>> PM
> > >>> > > > Justine
> > >>> > > > >> > > Olshan
> > >>> > > > >> > > > <
> > >>> > > > >> > > > > > > > >> > > > > > jolshan@confluent.io>
> > >>> > > > >> > > > > > > > >> > > > > > >> > > wrote:
> > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > Hi Jun,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > Thanks for the comments.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > 12. I bumped the
> > >>> > > LeaderAndIsrRequest
> > >>> > > > >> > > because
> > >>> > > > >> > > > I
> > >>> > > > >> > > > > > > > removed
> > >>> > > > >> > > > > > > > >> the
> > >>> > > > >> > > > > > > > >> > > > topic
> > >>> > > > >> > > > > > > > >> > > > > > >> name
> > >>> > > > >> > > > > > > > >> > > > > > >> > > field
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > in the response. It may
> be
> > >>> > > possible
> > >>> > > > to
> > >>> > > > >> > > avoid
> > >>> > > > >> > > > > > > bumping
> > >>> > > > >> > > > > > > > >> the
> > >>> > > > >> > > > > > > > >> > > > version
> > >>> > > > >> > > > > > > > >> > > > > > >> > without
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > that change, but I may
> be
> > >>> > missing
> > >>> > > > >> > > something.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > I believe StopReplica is
> > >>> > actually
> > >>> > > on
> > >>> > > > >> > > version
> > >>> > > > >> > > > 3
> > >>> > > > >> > > > > > now,
> > >>> > > > >> > > > > > > > but
> > >>> > > > >> > > > > > > > >> > > > because
> > >>> > > > >> > > > > > > > >> > > > > > >> > version 2
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > is flexible, I kept that
> > >>> listed
> > >>> > as
> > >>> > > > >> > version
> > >>> > > > >> > > 2
> > >>> > > > >> > > > on
> > >>> > > > >> > > > > > the
> > >>> > > > >> > > > > > > > KIP
> > >>> > > > >> > > > > > > > >> > > page.
> > >>> > > > >> > > > > > > > >> > > > > > >> However,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > you
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > may be right in that we
> > may
> > >>> need
> > >>> > > to
> > >>> > > > >> bump
> > >>> > > > >> > > the
> > >>> > > > >> > > > > > > version
> > >>> > > > >> > > > > > > > on
> > >>> > > > >> > > > > > > > >> > > > > > StopReplica
> > >>> > > > >> > > > > > > > >> > > > > > >> to
> > >>> > > > >> > > > > > > > >> > > > > > >> > > deal
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > with deletion
> differently
> > as
> > >>> > > > mentioned
> > >>> > > > >> > > > above. I
> > >>> > > > >> > > > > > > don't
> > >>> > > > >> > > > > > > > >> know
> > >>> > > > >> > > > > > > > >> > > if
> > >>> > > > >> > > > > > > > >> > > > I
> > >>> > > > >> > > > > > > > >> > > > > > >> have a
> > >>> > > > >> > > > > > > > >> > > > > > >> > > big
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > preference over used
> > tagged
> > >>> > fields
> > >>> > > > or
> > >>> > > > >> > not.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > 13. I was thinking that
> in
> > >>> the
> > >>> > > case
> > >>> > > > >> where
> > >>> > > > >> > > the
> > >>> > > > >> > > > > > file
> > >>> > > > >> > > > > > > > and
> > >>> > > > >> > > > > > > > >> the
> > >>> > > > >> > > > > > > > >> > > > > request
> > >>> > > > >> > > > > > > > >> > > > > > >> > topic
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > ids don't match, it
> means
> > >>> that
> > >>> > the
> > >>> > > > >> > broker's
> > >>> > > > >> > > > > > > topic/the
> > >>> > > > >> > > > > > > > >> one
> > >>> > > > >> > > > > > > > >> > in
> > >>> > > > >> > > > > > > > >> > > > the
> > >>> > > > >> > > > > > > > >> > > > > > >> file
> > >>> > > > >> > > > > > > > >> > > > > > >> > has
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > been deleted. In that
> > case,
> > >>> we
> > >>> > > would
> > >>> > > > >> need
> > >>> > > > >> > > to
> > >>> > > > >> > > > > > delete
> > >>> > > > >> > > > > > > > the
> > >>> > > > >> > > > > > > > >> > old
> > >>> > > > >> > > > > > > > >> > > > > topic
> > >>> > > > >> > > > > > > > >> > > > > > >> and
> > >>> > > > >> > > > > > > > >> > > > > > >> > > start
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > receiving the new
> version.
> > >>> If
> > >>> > the
> > >>> > > > >> topic
> > >>> > > > >> > > name
> > >>> > > > >> > > > > were
> > >>> > > > >> > > > > > > to
> > >>> > > > >> > > > > > > > >> > change,
> > >>> > > > >> > > > > > > > >> > > > but
> > >>> > > > >> > > > > > > > >> > > > > > the
> > >>> > > > >> > > > > > > > >> > > > > > >> > ids
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > still matched, the file
> > >>> would
> > >>> > also
> > >>> > > > >> need
> > >>> > > > >> > to
> > >>> > > > >> > > > > > update.
> > >>> > > > >> > > > > > > > Am I
> > >>> > > > >> > > > > > > > >> > > > missing
> > >>> > > > >> > > > > > > > >> > > > > a
> > >>> > > > >> > > > > > > > >> > > > > > >> case
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > where the file would be
> > >>> correct
> > >>> > > and
> > >>> > > > >> not
> > >>> > > > >> > the
> > >>> > > > >> > > > > > > request?
> > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > Thanks,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > Justine
> > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > On Thu, Sep 17, 2020 at
> > >>> 3:18 PM
> > >>> > > Jun
> > >>> > > > >> Rao <
> > >>> > > > >> > > > > > > > >> jun@confluent.io
> > >>> > > > >> > > > > > > > >> > >
> > >>> > > > >> > > > > > > > >> > > > > wrote:
> > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Hi, Justine,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Thanks for the reply.
> A
> > >>> couple
> > >>> > > of
> > >>> > > > >> more
> > >>> > > > >> > > > > comments
> > >>> > > > >> > > > > > > > >> below.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > 12. ListOffset and
> > >>> > > OffsetForLeader
> > >>> > > > >> > > > currently
> > >>> > > > >> > > > > > > don't
> > >>> > > > >> > > > > > > > >> > support
> > >>> > > > >> > > > > > > > >> > > > > > >> flexible
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > fields.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > So, we have to bump up
> > the
> > >>> > > version
> > >>> > > > >> > number
> > >>> > > > >> > > > and
> > >>> > > > >> > > > > > use
> > >>> > > > >> > > > > > > > >> IBP at
> > >>> > > > >> > > > > > > > >> > > > least
> > >>> > > > >> > > > > > > > >> > > > > > for
> > >>> > > > >> > > > > > > > >> > > > > > >> > > these
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > two requests. Note
> that
> > it
> > >>> > seems
> > >>> > > > >> 2.7.0
> > >>> > > > >> > > will
> > >>> > > > >> > > > > > > require
> > >>> > > > >> > > > > > > > >> IBP
> > >>> > > > >> > > > > > > > >> > > > anyway
> > >>> > > > >> > > > > > > > >> > > > > > >> > because
> > >>> > > > >> > > > > > > > >> > > > > > >> > > of
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > changes in
> KAFKA-10435.
> > >>> Also,
> > >>> > it
> > >>> > > > >> seems
> > >>> > > > >> > > that
> > >>> > > > >> > > > > the
> > >>> > > > >> > > > > > > > >> version
> > >>> > > > >> > > > > > > > >> > > for
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > LeaderAndIsrRequest
> and
> > >>> > > > StopReplica
> > >>> > > > >> are
> > >>> > > > >> > > > > bumped
> > >>> > > > >> > > > > > > even
> > >>> > > > >> > > > > > > > >> > though
> > >>> > > > >> > > > > > > > >> > > > we
> > >>> > > > >> > > > > > > > >> > > > > > only
> > >>> > > > >> > > > > > > > >> > > > > > >> > > added
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > a
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > tagged field. But
> since
> > >>> IBP is
> > >>> > > > >> needed
> > >>> > > > >> > > > anyway,
> > >>> > > > >> > > > > > we
> > >>> > > > >> > > > > > > > may
> > >>> > > > >> > > > > > > > >> > want
> > >>> > > > >> > > > > > > > >> > > to
> > >>> > > > >> > > > > > > > >> > > > > > >> revisit
> > >>> > > > >> > > > > > > > >> > > > > > >> > > the
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > overall tagged field
> > >>> choice.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > 13. The only downside
> is
> > >>> the
> > >>> > > > >> potential
> > >>> > > > >> > > > > > confusion
> > >>> > > > >> > > > > > > on
> > >>> > > > >> > > > > > > > >> > which
> > >>> > > > >> > > > > > > > >> > > > one
> > >>> > > > >> > > > > > > > >> > > > > is
> > >>> > > > >> > > > > > > > >> > > > > > >> the
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > source
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > of truth if they don't
> > >>> match.
> > >>> > > > >> Another
> > >>> > > > >> > > > option
> > >>> > > > >> > > > > is
> > >>> > > > >> > > > > > > to
> > >>> > > > >> > > > > > > > >> > include
> > >>> > > > >> > > > > > > > >> > > > > those
> > >>> > > > >> > > > > > > > >> > > > > > >> > fields
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > in
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the metadata file when
> > we
> > >>> > > actually
> > >>> > > > >> > change
> > >>> > > > >> > > > the
> > >>> > > > >> > > > > > > > >> directory
> > >>> > > > >> > > > > > > > >> > > > > > structure.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Thanks,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Jun
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > On Thu, Sep 17, 2020
> at
> > >>> 2:01
> > >>> > PM
> > >>> > > > >> Justine
> > >>> > > > >> > > > > Olshan
> > >>> > > > >> > > > > > <
> > >>> > > > >> > > > > > > > >> > > > > > >> jolshan@confluent.io
> > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > wrote:
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > Hello all,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I've thought some
> more
> > >>> about
> > >>> > > > >> removing
> > >>> > > > >> > > the
> > >>> > > > >> > > > > > topic
> > >>> > > > >> > > > > > > > >> name
> > >>> > > > >> > > > > > > > >> > > field
> > >>> > > > >> > > > > > > > >> > > > > > from
> > >>> > > > >> > > > > > > > >> > > > > > >> > some
> > >>> > > > >> > > > > > > > >> > > > > > >> > > of
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > requests. On closer
> > >>> > inspection
> > >>> > > > of
> > >>> > > > >> the
> > >>> > > > >> > > > > > > > >> > > requests/responses,
> > >>> > > > >> > > > > > > > >> > > > it
> > >>> > > > >> > > > > > > > >> > > > > > >> seems
> > >>> > > > >> > > > > > > > >> > > > > > >> > > that
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > internal changes
> would
> > >>> be
> > >>> > much
> > >>> > > > >> larger
> > >>> > > > >> > > > than
> > >>> > > > >> > > > > I
> > >>> > > > >> > > > > > > > >> expected.
> > >>> > > > >> > > > > > > > >> > > > Some
> > >>> > > > >> > > > > > > > >> > > > > > >> > protocols
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > involve clients, so
> > they
> > >>> > would
> > >>> > > > >> > require
> > >>> > > > >> > > > > > changes
> > >>> > > > >> > > > > > > > too.
> > >>> > > > >> > > > > > > > >> > I'm
> > >>> > > > >> > > > > > > > >> > > > > > thinking
> > >>> > > > >> > > > > > > > >> > > > > > >> > that
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > for
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > now, removing the
> > topic
> > >>> name
> > >>> > > > from
> > >>> > > > >> > these
> > >>> > > > >> > > > > > > requests
> > >>> > > > >> > > > > > > > >> and
> > >>> > > > >> > > > > > > > >> > > > > responses
> > >>> > > > >> > > > > > > > >> > > > > > >> are
> > >>> > > > >> > > > > > > > >> > > > > > >> > > out
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > scope.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I have decided to
> just
> > >>> keep
> > >>> > > the
> > >>> > > > >> > change
> > >>> > > > >> > > > > > > > >> > > > LeaderAndIsrResponse
> > >>> > > > >> > > > > > > > >> > > > > to
> > >>> > > > >> > > > > > > > >> > > > > > >> > remove
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > the
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > topic name, and have
> > >>> updated
> > >>> > > the
> > >>> > > > >> KIP
> > >>> > > > >> > to
> > >>> > > > >> > > > > > reflect
> > >>> > > > >> > > > > > > > >> this
> > >>> > > > >> > > > > > > > >> > > > > change. I
> > >>> > > > >> > > > > > > > >> > > > > > >> have
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > also
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > mentioned the other
> > >>> requests
> > >>> > > and
> > >>> > > > >> > > > responses
> > >>> > > > >> > > > > in
> > >>> > > > >> > > > > > > > >> future
> > >>> > > > >> > > > > > > > >> > > work.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I'm hoping to start
> > the
> > >>> > voting
> > >>> > > > >> > process
> > >>> > > > >> > > > > soon,
> > >>> > > > >> > > > > > so
> > >>> > > > >> > > > > > > > >> let me
> > >>> > > > >> > > > > > > > >> > > > know
> > >>> > > > >> > > > > > > > >> > > > > if
> > >>> > > > >> > > > > > > > >> > > > > > >> > there
> > >>> > > > >> > > > > > > > >> > > > > > >> > > is
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > anything else we
> > should
> > >>> > > discuss.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > Thank you,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > Justine
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > On Tue, Sep 15, 2020
> > at
> > >>> 3:57
> > >>> > > PM
> > >>> > > > >> > Justine
> > >>> > > > >> > > > > > Olshan
> > >>> > > > >> > > > > > > <
> > >>> > > > >> > > > > > > > >> > > > > > >> > jolshan@confluent.io
> > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > wrote:
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > Hello again,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > To follow up on
> some
> > >>> of
> > >>> > the
> > >>> > > > >> other
> > >>> > > > >> > > > > comments:
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 10/11) We can
> remove
> > >>> the
> > >>> > > topic
> > >>> > > > >> name
> > >>> > > > >> > > > from
> > >>> > > > >> > > > > > > these
> > >>> > > > >> > > > > > > > >> > > > > > >> > requests/responses,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > and
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > that means we will
> > >>> just
> > >>> > have
> > >>> > > > to
> > >>> > > > >> > make
> > >>> > > > >> > > a
> > >>> > > > >> > > > > few
> > >>> > > > >> > > > > > > > >> internal
> > >>> > > > >> > > > > > > > >> > > > > changes
> > >>> > > > >> > > > > > > > >> > > > > > to
> > >>> > > > >> > > > > > > > >> > > > > > >> > make
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > partitions
> > accessible
> > >>> by
> > >>> > > topic
> > >>> > > > >> id
> > >>> > > > >> > and
> > >>> > > > >> > > > > > > > partition.
> > >>> > > > >> > > > > > > > >> I
> > >>> > > > >> > > > > > > > >> > can
> > >>> > > > >> > > > > > > > >> > > > > > update
> > >>> > > > >> > > > > > > > >> > > > > > >> the
> > >>> > > > >> > > > > > > > >> > > > > > >> > > KIP
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > remove them unless
> > >>> anyone
> > >>> > > > thinks
> > >>> > > > >> > they
> > >>> > > > >> > > > > > should
> > >>> > > > >> > > > > > > > >> stay.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 12) Addressed in
> the
> > >>> > > previous
> > >>> > > > >> > email.
> > >>> > > > >> > > > I've
> > >>> > > > >> > > > > > > > updated
> > >>> > > > >> > > > > > > > >> > the
> > >>> > > > >> > > > > > > > >> > > > KIP
> > >>> > > > >> > > > > > > > >> > > > > to
> > >>> > > > >> > > > > > > > >> > > > > > >> > > include
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > tagged fields for
> > the
> > >>> > > requests
> > >>> > > > >> and
> > >>> > > > >> > > > > > responses.
> > >>> > > > >> > > > > > > > >> (More
> > >>> > > > >> > > > > > > > >> > on
> > >>> > > > >> > > > > > > > >> > > > > that
> > >>> > > > >> > > > > > > > >> > > > > > >> > below)
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 13) I think part
> of
> > >>> the
> > >>> > idea
> > >>> > > > for
> > >>> > > > >> > > > > including
> > >>> > > > >> > > > > > > this
> > >>> > > > >> > > > > > > > >> > > > > information
> > >>> > > > >> > > > > > > > >> > > > > > >> is to
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > prepare
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > for future
> changes.
> > >>> > Perhaps
> > >>> > > > the
> > >>> > > > >> > > > directory
> > >>> > > > >> > > > > > > > >> structure
> > >>> > > > >> > > > > > > > >> > > > might
> > >>> > > > >> > > > > > > > >> > > > > > >> change
> > >>> > > > >> > > > > > > > >> > > > > > >> > > from
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > >>> topicName_partitionNumber
> > >>> > to
> > >>> > > > >> > > something
> > >>> > > > >> > > > > like
> > >>> > > > >> > > > > > > > >> > > > > > >> > > topicID_partitionNumber.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Then
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > it would be useful
> > to
> > >>> have
> > >>> > > the
> > >>> > > > >> > topic
> > >>> > > > >> > > > name
> > >>> > > > >> > > > > > in
> > >>> > > > >> > > > > > > > the
> > >>> > > > >> > > > > > > > >> > file
> > >>> > > > >> > > > > > > > >> > > > > since
> > >>> > > > >> > > > > > > > >> > > > > > it
> > >>> > > > >> > > > > > > > >> > > > > > >> > > would
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > not
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > be
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > in the directory
> > >>> > structure.
> > >>> > > > >> > > Supporting
> > >>> > > > >> > > > > > topic
> > >>> > > > >> > > > > > > > >> renames
> > >>> > > > >> > > > > > > > >> > > > might
> > >>> > > > >> > > > > > > > >> > > > > > be
> > >>> > > > >> > > > > > > > >> > > > > > >> > > easier
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > if
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > the
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > other fields are
> > >>> included.
> > >>> > > > Would
> > >>> > > > >> > > there
> > >>> > > > >> > > > be
> > >>> > > > >> > > > > > any
> > >>> > > > >> > > > > > > > >> > > downsides
> > >>> > > > >> > > > > > > > >> > > > to
> > >>> > > > >> > > > > > > > >> > > > > > >> > > including
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > this
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > information?
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 14)  Yes, we would
> > >>> need to
> > >>> > > > copy
> > >>> > > > >> the
> > >>> > > > >> > > > > > partition
> > >>> > > > >> > > > > > > > >> > metadata
> > >>> > > > >> > > > > > > > >> > > > > file
> > >>> > > > >> > > > > > > > >> > > > > > in
> > >>> > > > >> > > > > > > > >> > > > > > >> > this
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > process. I've
> > updated
> > >>> the
> > >>> > > KIP
> > >>> > > > to
> > >>> > > > >> > > > include
> > >>> > > > >> > > > > > > this.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 15) I believe
> Lucas
> > >>> meant
> > >>> > v1
> > >>> > > > >> and v2
> > >>> > > > >> > > > here.
> > >>> > > > >> > > > > > He
> > >>> > > > >> > > > > > > > was
> > >>> > > > >> > > > > > > > >> > > > referring
> > >>> > > > >> > > > > > > > >> > > > > > to
> > >>> > > > >> > > > > > > > >> > > > > > >> how
> > >>> > > > >> > > > > > > > >> > > > > > >> > > the
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > requests would
> fall
> > >>> under
> > >>> > > > >> different
> > >>> > > > >> > > IBP
> > >>> > > > >> > > > > and
> > >>> > > > >> > > > > > > > meant
> > >>> > > > >> > > > > > > > >> > that
> > >>> > > > >> > > > > > > > >> > > > > older
> > >>> > > > >> > > > > > > > >> > > > > > >> > > brokers
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > would
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > have to use the
> > older
> > >>> > > version
> > >>> > > > of
> > >>> > > > >> > the
> > >>> > > > >> > > > > > request
> > >>> > > > >> > > > > > > > and
> > >>> > > > >> > > > > > > > >> the
> > >>> > > > >> > > > > > > > >> > > > > > existing
> > >>> > > > >> > > > > > > > >> > > > > > >> > topic
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > deletion process.
> At
> > >>> > first,
> > >>> > > it
> > >>> > > > >> > seemed
> > >>> > > > >> > > > > like
> > >>> > > > >> > > > > > > > tagged
> > >>> > > > >> > > > > > > > >> > > fields
> > >>> > > > >> > > > > > > > >> > > > > > would
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > resolve
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > the IBP issue.
> > >>> However, we
> > >>> > > may
> > >>> > > > >> need
> > >>> > > > >> > > IBP
> > >>> > > > >> > > > > for
> > >>> > > > >> > > > > > > > this
> > >>> > > > >> > > > > > > > >> > > request
> > >>> > > > >> > > > > > > > >> > > > > > after
> > >>> > > > >> > > > > > > > >> > > > > > >> > all
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > since
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > the controller
> > >>> handles the
> > >>> > > > topic
> > >>> > > > >> > > > deletion
> > >>> > > > >> > > > > > > > >> > differently
> > >>> > > > >> > > > > > > > >> > > > > > >> depending
> > >>> > > > >> > > > > > > > >> > > > > > >> > on
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > the
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > IBP
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > version. In an
> older
> > >>> > > version,
> > >>> > > > we
> > >>> > > > >> > > can't
> > >>> > > > >> > > > > just
> > >>> > > > >> > > > > > > > send
> > >>> > > > >> > > > > > > > >> a
> > >>> > > > >> > > > > > > > >> > > > > > StopReplica
> > >>> > > > >> > > > > > > > >> > > > > > >> > > delete
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > topic immediately
> > like
> > >>> > we'd
> > >>> > > > >> want to
> > >>> > > > >> > > for
> > >>> > > > >> > > > > > this
> > >>> > > > >> > > > > > > > KIP.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > This makes me
> wonder
> > >>> if we
> > >>> > > > want
> > >>> > > > >> > > tagged
> > >>> > > > >> > > > > > fields
> > >>> > > > >> > > > > > > > on
> > >>> > > > >> > > > > > > > >> all
> > >>> > > > >> > > > > > > > >> > > the
> > >>> > > > >> > > > > > > > >> > > > > > >> requests
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > after
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > all. Let me know
> > your
> > >>> > > > thoughts!
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > Justine
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > On Tue, Sep 15,
> 2020
> > >>> at
> > >>> > 1:03
> > >>> > > > PM
> > >>> > > > >> > > Justine
> > >>> > > > >> > > > > > > Olshan
> > >>> > > > >> > > > > > > > <
> > >>> > > > >> > > > > > > > >> > > > > > >> > > jolshan@confluent.io
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > wrote:
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Hi all,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Jun brought up a
> > good
> > >>> > point
> > >>> > > > in
> > >>> > > > >> his
> > >>> > > > >> > > > last
> > >>> > > > >> > > > > > > email
> > >>> > > > >> > > > > > > > >> about
> > >>> > > > >> > > > > > > > >> > > > > tagged
> > >>> > > > >> > > > > > > > >> > > > > > >> > fields,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > and
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> I've updated the
> > KIP
> > >>> to
> > >>> > > > reflect
> > >>> > > > >> > that
> > >>> > > > >> > > > the
> > >>> > > > >> > > > > > > > >> changes to
> > >>> > > > >> > > > > > > > >> > > > > > requests
> > >>> > > > >> > > > > > > > >> > > > > > >> and
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > responses
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> will be in the
> form
> > >>> of
> > >>> > > tagged
> > >>> > > > >> > fields
> > >>> > > > >> > > > to
> > >>> > > > >> > > > > > > avoid
> > >>> > > > >> > > > > > > > >> > > changing
> > >>> > > > >> > > > > > > > >> > > > > IBP.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Jun: I plan on
> > >>> sending a
> > >>> > > > >> followup
> > >>> > > > >> > > > email
> > >>> > > > >> > > > > to
> > >>> > > > >> > > > > > > > >> address
> > >>> > > > >> > > > > > > > >> > > some
> > >>> > > > >> > > > > > > > >> > > > > of
> > >>> > > > >> > > > > > > > >> > > > > > >> the
> > >>> > > > >> > > > > > > > >> > > > > > >> > > other
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> points.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Thanks,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Justine
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> On Mon, Sep 14,
> > 2020
> > >>> at
> > >>> > > 4:25
> > >>> > > > PM
> > >>> > > > >> > Jun
> > >>> > > > >> > > > Rao
> > >>> > > > >> > > > > <
> > >>> > > > >> > > > > > > > >> > > > > jun@confluent.io>
> > >>> > > > >> > > > > > > > >> > > > > > >> > wrote:
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Hi, Justine,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Thanks for the
> > >>> updated
> > >>> > > KIP.
> > >>> > > > A
> > >>> > > > >> few
> > >>> > > > >> > > > > > comments
> > >>> > > > >> > > > > > > > >> below.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 10. LeaderAndIsr
> > >>> > Response:
> > >>> > > > Do
> > >>> > > > >> we
> > >>> > > > >> > > need
> > >>> > > > >> > > > > the
> > >>> > > > >> > > > > > > > topic
> > >>> > > > >> > > > > > > > >> > > name?
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 11. For the
> > changed
> > >>> > > > >> > > request/response,
> > >>> > > > >> > > > > > other
> > >>> > > > >> > > > > > > > >> than
> > >>> > > > >> > > > > > > > >> > > > > > >> LeaderAndIsr,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> UpdateMetadata,
> > >>> > Metadata,
> > >>> > > do
> > >>> > > > >> we
> > >>> > > > >> > > need
> > >>> > > > >> > > > to
> > >>> > > > >> > > > > > > > include
> > >>> > > > >> > > > > > > > >> > the
> > >>> > > > >> > > > > > > > >> > > > > topic
> > >>> > > > >> > > > > > > > >> > > > > > >> name?
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 12. It seems
> that
> > >>> > upgrades
> > >>> > > > >> don't
> > >>> > > > >> > > > > require
> > >>> > > > >> > > > > > > IBP.
> > >>> > > > >> > > > > > > > >> Does
> > >>> > > > >> > > > > > > > >> > > > that
> > >>> > > > >> > > > > > > > >> > > > > > mean
> > >>> > > > >> > > > > > > > >> > > > > > >> > the
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > new
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> fields
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> in all the
> > >>> > > request/response
> > >>> > > > >> are
> > >>> > > > >> > > added
> > >>> > > > >> > > > > as
> > >>> > > > >> > > > > > > > tagged
> > >>> > > > >> > > > > > > > >> > > fields
> > >>> > > > >> > > > > > > > >> > > > > > >> without
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > bumping
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > up
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> the request
> > >>> version? It
> > >>> > > > would
> > >>> > > > >> be
> > >>> > > > >> > > > useful
> > >>> > > > >> > > > > > to
> > >>> > > > >> > > > > > > > make
> > >>> > > > >> > > > > > > > >> > that
> > >>> > > > >> > > > > > > > >> > > > > > clear.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 13. Partition
> > >>> Metadata
> > >>> > > file:
> > >>> > > > >> Do
> > >>> > > > >> > we
> > >>> > > > >> > > > need
> > >>> > > > >> > > > > > to
> > >>> > > > >> > > > > > > > >> include
> > >>> > > > >> > > > > > > > >> > > the
> > >>> > > > >> > > > > > > > >> > > > > > topic
> > >>> > > > >> > > > > > > > >> > > > > > >> > name
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > and
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > the
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> partition id
> since
> > >>> they
> > >>> > > are
> > >>> > > > >> > implied
> > >>> > > > >> > > > in
> > >>> > > > >> > > > > > the
> > >>> > > > >> > > > > > > > >> > directory
> > >>> > > > >> > > > > > > > >> > > > > name?
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 14. In the JBOD
> > >>> mode, we
> > >>> > > > >> support
> > >>> > > > >> > > > > moving a
> > >>> > > > >> > > > > > > > >> > > partition's
> > >>> > > > >> > > > > > > > >> > > > > data
> > >>> > > > >> > > > > > > > >> > > > > > >> from
> > >>> > > > >> > > > > > > > >> > > > > > >> > > one
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > disk
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> to
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> another. Will
> the
> > >>> new
> > >>> > > > >> partition
> > >>> > > > >> > > > > metadata
> > >>> > > > >> > > > > > > file
> > >>> > > > >> > > > > > > > >> be
> > >>> > > > >> > > > > > > > >> > > > copied
> > >>> > > > >> > > > > > > > >> > > > > > >> during
> > >>> > > > >> > > > > > > > >> > > > > > >> > > that
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> process?
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 15. The KIP says
> > >>> "Remove
> > >>> > > > >> deleted
> > >>> > > > >> > > > topics
> > >>> > > > >> > > > > > > from
> > >>> > > > >> > > > > > > > >> > > replicas
> > >>> > > > >> > > > > > > > >> > > > by
> > >>> > > > >> > > > > > > > >> > > > > > >> > sending
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> StopReplicaRequest
> > >>> V2
> > >>> > for
> > >>> > > > any
> > >>> > > > >> > > topics
> > >>> > > > >> > > > > > which
> > >>> > > > >> > > > > > > do
> > >>> > > > >> > > > > > > > >> not
> > >>> > > > >> > > > > > > > >> > > > > contain
> > >>> > > > >> > > > > > > > >> > > > > > a
> > >>> > > > >> > > > > > > > >> > > > > > >> > topic
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > ID,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > and
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> V3 for any
> topics
> > >>> which
> > >>> > do
> > >>> > > > >> > contain
> > >>> > > > >> > > a
> > >>> > > > >> > > > > > topic
> > >>> > > > >> > > > > > > > >> ID.".
> > >>> > > > >> > > > > > > > >> > > > > However,
> > >>> > > > >> > > > > > > > >> > > > > > it
> > >>> > > > >> > > > > > > > >> > > > > > >> > > seems
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> updated
> controller
> > >>> will
> > >>> > > > create
> > >>> > > > >> > all
> > >>> > > > >> > > > > > missing
> > >>> > > > >> > > > > > > > >> topic
> > >>> > > > >> > > > > > > > >> > IDs
> > >>> > > > >> > > > > > > > >> > > > > first
> > >>> > > > >> > > > > > > > >> > > > > > >> > before
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > doing
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> other actions.
> So,
> > >>> is
> > >>> > > > >> > > > > StopReplicaRequest
> > >>> > > > >> > > > > > V2
> > >>> > > > >> > > > > > > > >> > needed?
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Jun
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> On Fri, Sep 11,
> > >>> 2020 at
> > >>> > > > 10:31
> > >>> > > > >> AM
> > >>> > > > >> > > John
> > >>> > > > >> > > > > > > > Roesler <
> > >>> > > > >> > > > > > > > >> > > > > > >> > > vvcephei@apache.org
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> wrote:
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > Thanks,
> Justine!
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > Your response
> > >>> seems
> > >>> > > > >> compelling
> > >>> > > > >> > to
> > >>> > > > >> > > > me.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > -John
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > On Fri,
> > >>> 2020-09-11 at
> > >>> > > > 10:17
> > >>> > > > >> > > -0700,
> > >>> > > > >> > > > > > > Justine
> > >>> > > > >> > > > > > > > >> > Olshan
> > >>> > > > >> > > > > > > > >> > > > > wrote:
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Hello all,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Thanks for
> > >>> > continuing
> > >>> > > > the
> > >>> > > > >> > > > > > discussion! I
> > >>> > > > >> > > > > > > > >> have a
> > >>> > > > >> > > > > > > > >> > > few
> > >>> > > > >> > > > > > > > >> > > > > > >> > responses
> > >>> > > > >> > > > > > > > >> > > > > > >> > > to
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > your
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > points.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Tom: You are
> > >>> correct
> > >>> > > in
> > >>> > > > >> that
> > >>> > > > >> > > this
> > >>> > > > >> > > > > KIP
> > >>> > > > >> > > > > > > has
> > >>> > > > >> > > > > > > > >> not
> > >>> > > > >> > > > > > > > >> > > > > > mentioned
> > >>> > > > >> > > > > > > > >> > > > > > >> the
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > >>> > DeleteTopicsRequest. I
> > >>> > > > >> think
> > >>> > > > >> > > that
> > >>> > > > >> > > > > > this
> > >>> > > > >> > > > > > > > >> would
> > >>> > > > >> > > > > > > > >> > be
> > >>> > > > >> > > > > > > > >> > > > out
> > >>> > > > >> > > > > > > > >> > > > > of
> > >>> > > > >> > > > > > > > >> > > > > > >> > scope
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > for
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> now, but
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > may be
> > something
> > >>> > worth
> > >>> > > > >> adding
> > >>> > > > >> > > in
> > >>> > > > >> > > > > the
> > >>> > > > >> > > > > > > > >> future.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > John: We did
> > >>> > consider
> > >>> > > > >> > sequence
> > >>> > > > >> > > > ids,
> > >>> > > > >> > > > > > but
> > >>> > > > >> > > > > > > > >> there
> > >>> > > > >> > > > > > > > >> > > are
> > >>> > > > >> > > > > > > > >> > > > a
> > >>> > > > >> > > > > > > > >> > > > > > few
> > >>> > > > >> > > > > > > > >> > > > > > >> > > reasons
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> favor
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > UUIDs. There
> > are
> > >>> > > several
> > >>> > > > >> > cases
> > >>> > > > >> > > > > where
> > >>> > > > >> > > > > > > > topics
> > >>> > > > >> > > > > > > > >> > from
> > >>> > > > >> > > > > > > > >> > > > > > >> different
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > clusters
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> may
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > interact now
> > >>> and in
> > >>> > > the
> > >>> > > > >> > future.
> > >>> > > > >> > > > For
> > >>> > > > >> > > > > > > > >> example,
> > >>> > > > >> > > > > > > > >> > > > Mirror
> > >>> > > > >> > > > > > > > >> > > > > > >> Maker 2
> > >>> > > > >> > > > > > > > >> > > > > > >> > > may
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> benefit
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > from being
> > able
> > >>> to
> > >>> > > > detect
> > >>> > > > >> > when
> > >>> > > > >> > > a
> > >>> > > > >> > > > > > > cluster
> > >>> > > > >> > > > > > > > >> being
> > >>> > > > >> > > > > > > > >> > > > > > mirrored
> > >>> > > > >> > > > > > > > >> > > > > > >> is
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > deleted
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> and
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > recreated
> and
> > >>> > globally
> > >>> > > > >> unique
> > >>> > > > >> > > > > > > identifiers
> > >>> > > > >> > > > > > > > >> > would
> > >>> > > > >> > > > > > > > >> > > > make
> > >>> > > > >> > > > > > > > >> > > > > > >> > > resolving
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > issues
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > easier than
> > >>> sequence
> > >>> > > IDs
> > >>> > > > >> > which
> > >>> > > > >> > > > may
> > >>> > > > >> > > > > > > > collide
> > >>> > > > >> > > > > > > > >> > > between
> > >>> > > > >> > > > > > > > >> > > > > > >> > clusters.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > KIP-405
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > (tiered
> > storage)
> > >>> > will
> > >>> > > > also
> > >>> > > > >> > > > benefit
> > >>> > > > >> > > > > > from
> > >>> > > > >> > > > > > > > >> > globally
> > >>> > > > >> > > > > > > > >> > > > > > unique
> > >>> > > > >> > > > > > > > >> > > > > > >> IDs
> > >>> > > > >> > > > > > > > >> > > > > > >> > > as
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > shared
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > buckets may
> be
> > >>> used
> > >>> > > > >> between
> > >>> > > > >> > > > > clusters.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Globally
> > unique
> > >>> IDs
> > >>> > > > would
> > >>> > > > >> > also
> > >>> > > > >> > > > make
> > >>> > > > >> > > > > > > > >> > > functionality
> > >>> > > > >> > > > > > > > >> > > > > like
> > >>> > > > >> > > > > > > > >> > > > > > >> > moving
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > topics
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > between
> > >>> disparate
> > >>> > > > clusters
> > >>> > > > >> > > easier
> > >>> > > > >> > > > > in
> > >>> > > > >> > > > > > > the
> > >>> > > > >> > > > > > > > >> > future,
> > >>> > > > >> > > > > > > > >> > > > > > >> simplify
> > >>> > > > >> > > > > > > > >> > > > > > >> > any
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > future
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > implementations
> > >>> of
> > >>> > > > backups
> > >>> > > > >> > and
> > >>> > > > >> > > > > > > restores,
> > >>> > > > >> > > > > > > > >> and
> > >>> > > > >> > > > > > > > >> > > more.
> > >>> > > > >> > > > > > > > >> > > > > In
> > >>> > > > >> > > > > > > > >> > > > > > >> > > general,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> unique IDs
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > would ensure
> > >>> that
> > >>> > the
> > >>> > > > >> source
> > >>> > > > >> > > > > cluster
> > >>> > > > >> > > > > > > > >> topics do
> > >>> > > > >> > > > > > > > >> > > not
> > >>> > > > >> > > > > > > > >> > > > > > >> conflict
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > with
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > the
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > destination
> > >>> cluster
> > >>> > > > >> topics.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > If we were
> to
> > >>> use
> > >>> > > > sequence
> > >>> > > > >> > ids,
> > >>> > > > >> > > > we
> > >>> > > > >> > > > > > > would
> > >>> > > > >> > > > > > > > >> need
> > >>> > > > >> > > > > > > > >> > > > > > >> sufficiently
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > large
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> cluster
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > ids to be
> > stored
> > >>> > with
> > >>> > > > the
> > >>> > > > >> > topic
> > >>> > > > >> > > > > > > > >> identifiers or
> > >>> > > > >> > > > > > > > >> > > we
> > >>> > > > >> > > > > > > > >> > > > > run
> > >>> > > > >> > > > > > > > >> > > > > > >> the
> > >>> > > > >> > > > > > > > >> > > > > > >> > > risk
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > collisions.
> > This
> > >>> > will
> > >>> > > > >> give up
> > >>> > > > >> > > any
> > >>> > > > >> > > > > > > > >> advantage in
> > >>> > > > >> > > > > > > > >> > > > > > >> compactness
> > >>> > > > >> > > > > > > > >> > > > > > >> > > that
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> sequence
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > numbers may
> > >>> bring.
> > >>> > > Given
> > >>> > > > >> > these
> > >>> > > > >> > > > > > > > advantages I
> > >>> > > > >> > > > > > > > >> > > think
> > >>> > > > >> > > > > > > > >> > > > it
> > >>> > > > >> > > > > > > > >> > > > > > >> makes
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > sense
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> use
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > UUIDs.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Gokul: This
> is
> > >>> an
> > >>> > > > >> interesting
> > >>> > > > >> > > > idea,
> > >>> > > > >> > > > > > but
> > >>> > > > >> > > > > > > > >> this
> > >>> > > > >> > > > > > > > >> > is
> > >>> > > > >> > > > > > > > >> > > a
> > >>> > > > >> > > > > > > > >> > > > > > >> breaking
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > change.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Out of
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > scope for
> now,
> > >>> but
> > >>> > > maybe
> > >>> > > > >> > worth
> > >>> > > > >> > > > > > > discussing
> > >>> > > > >> > > > > > > > >> in
> > >>> > > > >> > > > > > > > >> > the
> > >>> > > > >> > > > > > > > >> > > > > > future.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Hope this
> > >>> explains
> > >>> > > some
> > >>> > > > of
> > >>> > > > >> > the
> > >>> > > > >> > > > > > > decisions,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Justine
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > On Fri, Sep
> > 11,
> > >>> 2020
> > >>> > > at
> > >>> > > > >> 8:27
> > >>> > > > >> > AM
> > >>> > > > >> > > > > Gokul
> > >>> > > > >> > > > > > > > >> Ramanan
> > >>> > > > >> > > > > > > > >> > > > > > >> Subramanian <
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > >>> > gokul2411s@gmail.com>
> > >>> > > > >> wrote:
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Hi.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Thanks for
> > the
> > >>> > KIP.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Have you
> > >>> thought
> > >>> > > about
> > >>> > > > >> > > whether
> > >>> > > > >> > > > it
> > >>> > > > >> > > > > > > makes
> > >>> > > > >> > > > > > > > >> > sense
> > >>> > > > >> > > > > > > > >> > > to
> > >>> > > > >> > > > > > > > >> > > > > > >> support
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> authorizing a
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > principal
> > for
> > >>> a
> > >>> > > topic
> > >>> > > > ID
> > >>> > > > >> > > rather
> > >>> > > > >> > > > > > than
> > >>> > > > >> > > > > > > a
> > >>> > > > >> > > > > > > > >> topic
> > >>> > > > >> > > > > > > > >> > > > name
> > >>> > > > >> > > > > > > > >> > > > > to
> > >>> > > > >> > > > > > > > >> > > > > > >> > > achieve
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> tighter
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > security?
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Or is the
> > >>> topic ID
> > >>> > > > >> > > > fundamentally
> > >>> > > > >> > > > > an
> > >>> > > > >> > > > > > > > >> internal
> > >>> > > > >> > > > > > > > >> > > > > detail
> > >>> > > > >> > > > > > > > >> > > > > > >> > similar
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > to
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> epochs
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > used
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > in a bunch
> > of
> > >>> > other
> > >>> > > > >> places
> > >>> > > > >> > in
> > >>> > > > >> > > > > > Kafka?
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Thanks.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > On Fri,
> Sep
> > >>> 11,
> > >>> > 2020
> > >>> > > > at
> > >>> > > > >> > 4:06
> > >>> > > > >> > > PM
> > >>> > > > >> > > > > > John
> > >>> > > > >> > > > > > > > >> > Roesler <
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > vvcephei@apache.org
> >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > wrote:
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Hello
> > >>> Justine,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Thanks
> for
> > >>> the
> > >>> > > KIP!
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > I happen
> > to
> > >>> have
> > >>> > > > been
> > >>> > > > >> > > > > confronted
> > >>> > > > >> > > > > > > > >> recently
> > >>> > > > >> > > > > > > > >> > > with
> > >>> > > > >> > > > > > > > >> > > > > the
> > >>> > > > >> > > > > > > > >> > > > > > >> need
> > >>> > > > >> > > > > > > > >> > > > > > >> > > to
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > keep
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > track of
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > a
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > large
> > >>> number of
> > >>> > > > >> topics as
> > >>> > > > >> > > > > > compactly
> > >>> > > > >> > > > > > > > as
> > >>> > > > >> > > > > > > > >> > > > > possible. I
> > >>> > > > >> > > > > > > > >> > > > > > >> was
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > going
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> come
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > up
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > with
> some
> > >>> way to
> > >>> > > > >> > dictionary
> > >>> > > > >> > > > > > encode
> > >>> > > > >> > > > > > > > the
> > >>> > > > >> > > > > > > > >> > topic
> > >>> > > > >> > > > > > > > >> > > > > names
> > >>> > > > >> > > > > > > > >> > > > > > >> as
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > integers,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> but
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > this
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > seems
> much
> > >>> > better!
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> Apologies
> > if
> > >>> > this
> > >>> > > > has
> > >>> > > > >> > been
> > >>> > > > >> > > > > raised
> > >>> > > > >> > > > > > > > >> before,
> > >>> > > > >> > > > > > > > >> > > but
> > >>> > > > >> > > > > > > > >> > > > > I’m
> > >>> > > > >> > > > > > > > >> > > > > > >> > > wondering
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> about the
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > choice
> of
> > >>> UUID
> > >>> > vs
> > >>> > > > >> > sequence
> > >>> > > > >> > > > > number
> > >>> > > > >> > > > > > > for
> > >>> > > > >> > > > > > > > >> the
> > >>> > > > >> > > > > > > > >> > > ids.
> > >>> > > > >> > > > > > > > >> > > > > > >> > Typically,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > I’ve
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> seen
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > UUIDs
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > in two
> > >>> > situations:
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 1. When
> > >>> > processes
> > >>> > > > >> need to
> > >>> > > > >> > > > > > generate
> > >>> > > > >> > > > > > > > >> > > > non-colliding
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > identifiers
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> without
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > >>> coordination.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 2. When
> > the
> > >>> > > > identifier
> > >>> > > > >> > > needs
> > >>> > > > >> > > > to
> > >>> > > > >> > > > > > be
> > >>> > > > >> > > > > > > > >> > > > “universally
> > >>> > > > >> > > > > > > > >> > > > > > >> > unique”;
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > I.e.,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> the
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> identifier
> > >>> needs
> > >>> > > to
> > >>> > > > >> > > > distinguish
> > >>> > > > >> > > > > > the
> > >>> > > > >> > > > > > > > >> entity
> > >>> > > > >> > > > > > > > >> > > > from
> > >>> > > > >> > > > > > > > >> > > > > > all
> > >>> > > > >> > > > > > > > >> > > > > > >> > other
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> entities
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > that
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > could
> ever
> > >>> > exist.
> > >>> > > > >> This is
> > >>> > > > >> > > > > useful
> > >>> > > > >> > > > > > in
> > >>> > > > >> > > > > > > > >> cases
> > >>> > > > >> > > > > > > > >> > > > where
> > >>> > > > >> > > > > > > > >> > > > > > >> > entities
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > from
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > all
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > kinds
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > of
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > systems
> > get
> > >>> > mixed
> > >>> > > > >> > together,
> > >>> > > > >> > > > > such
> > >>> > > > >> > > > > > as
> > >>> > > > >> > > > > > > > >> when
> > >>> > > > >> > > > > > > > >> > > > dumping
> > >>> > > > >> > > > > > > > >> > > > > > >> logs
> > >>> > > > >> > > > > > > > >> > > > > > >> > > from
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > all
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > processes
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > in
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a
> company
> > >>> into a
> > >>> > > > >> common
> > >>> > > > >> > > > system.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Maybe
> I’m
> > >>> being
> > >>> > > > >> > > > short-sighted,
> > >>> > > > >> > > > > > but
> > >>> > > > >> > > > > > > it
> > >>> > > > >> > > > > > > > >> > > doesn’t
> > >>> > > > >> > > > > > > > >> > > > > seem
> > >>> > > > >> > > > > > > > >> > > > > > >> like
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > either
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> really
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > applies
> > >>> here. It
> > >>> > > > seems
> > >>> > > > >> > like
> > >>> > > > >> > > > the
> > >>> > > > >> > > > > > > > brokers
> > >>> > > > >> > > > > > > > >> > > could
> > >>> > > > >> > > > > > > > >> > > > > and
> > >>> > > > >> > > > > > > > >> > > > > > >> would
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > achieve
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > consensus
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > when
> > >>> creating a
> > >>> > > > topic
> > >>> > > > >> > > anyway,
> > >>> > > > >> > > > > > which
> > >>> > > > >> > > > > > > > is
> > >>> > > > >> > > > > > > > >> all
> > >>> > > > >> > > > > > > > >> > > > > that’s
> > >>> > > > >> > > > > > > > >> > > > > > >> > > required
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > to
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > generate
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > >>> non-colliding
> > >>> > > > sequence
> > >>> > > > >> > ids.
> > >>> > > > >> > > > For
> > >>> > > > >> > > > > > the
> > >>> > > > >> > > > > > > > >> > second,
> > >>> > > > >> > > > > > > > >> > > as
> > >>> > > > >> > > > > > > > >> > > > > you
> > >>> > > > >> > > > > > > > >> > > > > > >> > > mention,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > we
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> could
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > assign
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a UUID
> to
> > >>> the
> > >>> > > > cluster
> > >>> > > > >> as
> > >>> > > > >> > a
> > >>> > > > >> > > > > whole,
> > >>> > > > >> > > > > > > > which
> > >>> > > > >> > > > > > > > >> > > would
> > >>> > > > >> > > > > > > > >> > > > > > render
> > >>> > > > >> > > > > > > > >> > > > > > >> > any
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > resource
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > scoped
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > the
> broker
> > >>> > > > universally
> > >>> > > > >> > > unique
> > >>> > > > >> > > > > as
> > >>> > > > >> > > > > > > > well.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > The
> > reason I
> > >>> > > mention
> > >>> > > > >> this
> > >>> > > > >> > > is
> > >>> > > > >> > > > > > that,
> > >>> > > > >> > > > > > > > >> > although
> > >>> > > > >> > > > > > > > >> > > a
> > >>> > > > >> > > > > > > > >> > > > > UUID
> > >>> > > > >> > > > > > > > >> > > > > > >> is
> > >>> > > > >> > > > > > > > >> > > > > > >> > way
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > more
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > compact
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > than
> topic
> > >>> > names,
> > >>> > > > it’s
> > >>> > > > >> > > still
> > >>> > > > >> > > > 16
> > >>> > > > >> > > > > > > > bytes.
> > >>> > > > >> > > > > > > > >> In
> > >>> > > > >> > > > > > > > >> > > > > > contrast,
> > >>> > > > >> > > > > > > > >> > > > > > >> a
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > 4-byte
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> integer
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > sequence
> > id
> > >>> > would
> > >>> > > > give
> > >>> > > > >> > us 4
> > >>> > > > >> > > > > > billion
> > >>> > > > >> > > > > > > > >> unique
> > >>> > > > >> > > > > > > > >> > > > > topics
> > >>> > > > >> > > > > > > > >> > > > > > >> per
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > cluster,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> which
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > seems
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > like
> > enough
> > >>> ;)
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > Considering
> > >>> the
> > >>> > > > >> number of
> > >>> > > > >> > > > > > different
> > >>> > > > >> > > > > > > > >> times
> > >>> > > > >> > > > > > > > >> > > > these
> > >>> > > > >> > > > > > > > >> > > > > > >> topic
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> identifiers are
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > sent
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > over the
> > >>> wire or
> > >>> > > > >> stored
> > >>> > > > >> > in
> > >>> > > > >> > > > > > memory,
> > >>> > > > >> > > > > > > it
> > >>> > > > >> > > > > > > > >> > seems
> > >>> > > > >> > > > > > > > >> > > > like
> > >>> > > > >> > > > > > > > >> > > > > > it
> > >>> > > > >> > > > > > > > >> > > > > > >> > might
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > be
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> worth
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > the
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> additional
> > >>> 4x
> > >>> > > space
> > >>> > > > >> > > savings.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > What do
> > you
> > >>> > think
> > >>> > > > >> about
> > >>> > > > >> > > this?
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Thanks,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > John
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > On Fri,
> > Sep
> > >>> 11,
> > >>> > > > 2020,
> > >>> > > > >> at
> > >>> > > > >> > > > 03:20,
> > >>> > > > >> > > > > > Tom
> > >>> > > > >> > > > > > > > >> > Bentley
> > >>> > > > >> > > > > > > > >> > > > > wrote:
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Hi
> > >>> Justine,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > This
> > looks
> > >>> > like
> > >>> > > a
> > >>> > > > >> very
> > >>> > > > >> > > > > welcome
> > >>> > > > >> > > > > > > > >> > > improvement.
> > >>> > > > >> > > > > > > > >> > > > > > >> Thanks!
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Maybe
> I
> > >>> missed
> > >>> > > it,
> > >>> > > > >> but
> > >>> > > > >> > > the
> > >>> > > > >> > > > > KIP
> > >>> > > > >> > > > > > > > >> doesn't
> > >>> > > > >> > > > > > > > >> > > seem
> > >>> > > > >> > > > > > > > >> > > > to
> > >>> > > > >> > > > > > > > >> > > > > > >> > mention
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > changing
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >>> > > > DeleteTopicsRequest
> > >>> > > > >> to
> > >>> > > > >> > > > > identify
> > >>> > > > >> > > > > > > the
> > >>> > > > >> > > > > > > > >> > topic
> > >>> > > > >> > > > > > > > >> > > > > using
> > >>> > > > >> > > > > > > > >> > > > > > an
> > >>> > > > >> > > > > > > > >> > > > > > >> > id.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Maybe
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > that's out
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > of
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > scope,
> > but
> > >>> > > > >> > > > > DeleteTopicsRequest
> > >>> > > > >> > > > > > is
> > >>> > > > >> > > > > > > > not
> > >>> > > > >> > > > > > > > >> > > listed
> > >>> > > > >> > > > > > > > >> > > > > > among
> > >>> > > > >> > > > > > > > >> > > > > > >> > the
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > Future
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Work
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > APIs
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> either.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Kind
> > >>> regards,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Tom
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > On
> Thu,
> > >>> Sep
> > >>> > 10,
> > >>> > > > >> 2020 at
> > >>> > > > >> > > > 3:59
> > >>> > > > >> > > > > PM
> > >>> > > > >> > > > > > > > >> Satish
> > >>> > > > >> > > > > > > > >> > > > > Duggana <
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > >>> > > > >> satish.duggana@gmail.com
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > wrote:
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> Thanks
> > >>> > > > >> Lucas/Justine
> > >>> > > > >> > > for
> > >>> > > > >> > > > > the
> > >>> > > > >> > > > > > > nice
> > >>> > > > >> > > > > > > > >> KIP.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > It
> has
> > >>> > several
> > >>> > > > >> > benefits
> > >>> > > > >> > > > > which
> > >>> > > > >> > > > > > > > also
> > >>> > > > >> > > > > > > > >> > > include
> > >>> > > > >> > > > > > > > >> > > > > > >> > > simplifying
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> topic
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > deletion
> > >>> > > process
> > >>> > > > >> by
> > >>> > > > >> > > > > > controller
> > >>> > > > >> > > > > > > > and
> > >>> > > > >> > > > > > > > >> > logs
> > >>> > > > >> > > > > > > > >> > > > > > cleanup
> > >>> > > > >> > > > > > > > >> > > > > > >> by
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > brokers
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > in
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > corner
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> cases.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> Best,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > Satish.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > On
> > Wed,
> > >>> Sep
> > >>> > 9,
> > >>> > > > >> 2020
> > >>> > > > >> > at
> > >>> > > > >> > > > > 10:07
> > >>> > > > >> > > > > > PM
> > >>> > > > >> > > > > > > > >> > Justine
> > >>> > > > >> > > > > > > > >> > > > > > Olshan <
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > >>> jolshan@confluent.io
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> wrote:
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > Hello
> > >>> all,
> > >>> > > > it's
> > >>> > > > >> > been
> > >>> > > > >> > > > > > almost a
> > >>> > > > >> > > > > > > > >> year!
> > >>> > > > >> > > > > > > > >> > > I've
> > >>> > > > >> > > > > > > > >> > > > > > made
> > >>> > > > >> > > > > > > > >> > > > > > >> > some
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > changes
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> to
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > this
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > KIP
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > and
> > >>> hope to
> > >>> > > > >> continue
> > >>> > > > >> > > the
> > >>> > > > >> > > > > > > > >> discussion.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> One
> > >>> of the
> > >>> > > > main
> > >>> > > > >> > > changes
> > >>> > > > >> > > > > > I've
> > >>> > > > >> > > > > > > > >> added
> > >>> > > > >> > > > > > > > >> > is
> > >>> > > > >> > > > > > > > >> > > > now
> > >>> > > > >> > > > > > > > >> > > > > > the
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > metadata
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> response
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > will
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > include
> > >>> the
> > >>> > > > topic
> > >>> > > > >> ID
> > >>> > > > >> > > (as
> > >>> > > > >> > > > > > Colin
> > >>> > > > >> > > > > > > > >> > > suggested).
> > >>> > > > >> > > > > > > > >> > > > > > >> Clients
> > >>> > > > >> > > > > > > > >> > > > > > >> > > can
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> obtain the
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > topicID
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > of a
> > >>> given
> > >>> > > topic
> > >>> > > > >> > > through
> > >>> > > > >> > > > a
> > >>> > > > >> > > > > > > > >> > > > TopicDescription.
> > >>> > > > >> > > > > > > > >> > > > > > The
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > topicId
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > will
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > also be
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > included
> > >>> > with
> > >>> > > > the
> > >>> > > > >> > > > > > > UpdateMetadata
> > >>> > > > >> > > > > > > > >> > > request.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> Let
> > me
> > >>> > know
> > >>> > > > what
> > >>> > > > >> > you
> > >>> > > > >> > > > all
> > >>> > > > >> > > > > > > think.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > Thank
> > >>> you,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >>> Justine
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > On
> > >>> > > 2019/09/13
> > >>> > > > >> > > 16:38:26,
> > >>> > > > >> > > > > > > "Colin
> > >>> > > > >> > > > > > > > >> > > McCabe" <
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > cmccabe@apache.org
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > wrote:
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> Hi
> > >>> > Lucas,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >>> Thanks
> > >>> > for
> > >>> > > > >> > tackling
> > >>> > > > >> > > > > this.
> > >>> > > > >> > > > > > > > >> Topic
> > >>> > > > >> > > > > > > > >> > IDs
> > >>> > > > >> > > > > > > > >> > > > > are a
> > >>> > > > >> > > > > > > > >> > > > > > >> > great
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > idea,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> and
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > this
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > is
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> really
> > >>> good
> > >>> > > > >> writeup.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > For
> > >>> > > > >> > > > > > > /brokers/topics/[topic],
> > >>> > > > >> > > > > > > > >> the
> > >>> > > > >> > > > > > > > >> > > > schema
> > >>> > > > >> > > > > > > > >> > > > > > >> version
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > should
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > be
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > bumped
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > version
> > >>> 3,
> > >>> > > > rather
> > >>> > > > >> > than
> > >>> > > > >> > > 2.
> > >>> > > > >> > > > > > > > KIP-455
> > >>> > > > >> > > > > > > > >> > > bumped
> > >>> > > > >> > > > > > > > >> > > > > the
> > >>> > > > >> > > > > > > > >> > > > > > >> > version
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > this
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > znode
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 2
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > already
> > >>> :)
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >>> Given
> > >>> > that
> > >>> > > > >> we're
> > >>> > > > >> > > > going
> > >>> > > > >> > > > > to
> > >>> > > > >> > > > > > > be
> > >>> > > > >> > > > > > > > >> > seeing
> > >>> > > > >> > > > > > > > >> > > > > these
> > >>> > > > >> > > > > > > > >> > > > > > >> > things
> > >>> > > > >> > > > > > > > >> > > > > > >> > > as
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> strings
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > as
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > lot
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > (in
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> logs,
> > in
> > >>> > > > >> ZooKeeper,
> > >>> > > > >> > on
> > >>> > > > >> > > > the
> > >>> > > > >> > > > > > > > >> > command-line,
> > >>> > > > >> > > > > > > > >> > > > > > etc.),
> > >>> > > > >> > > > > > > > >> > > > > > >> > does
> > >>> > > > >> > > > > > > > >> > > > > > >> > > it
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > make
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > sense to
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > use
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> base64
> > >>> when
> > >>> > > > >> > converting
> > >>> > > > >> > > > them
> > >>> > > > >> > > > > > to
> > >>> > > > >> > > > > > > > >> > strings?
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >>> Here is
> > >>> > an
> > >>> > > > >> > example
> > >>> > > > >> > > of
> > >>> > > > >> > > > > the
> > >>> > > > >> > > > > > > hex
> > >>> > > > >> > > > > > > > >> > > > > > >> representation:
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >>> > > > >> > > > > > > > >> > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > And
> > >>> here
> > >>> > > is
> > >>> > > > an
> > >>> > > > >> > > > example
> > >>> > > > >> > > > > in
> > >>> > > > >> > > > > > > > >> base64.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >>> > > > >> > > > b8tRS7h4TJ2Vt43Dp85v2A
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > The
> > >>> > base64
> > >>> > > > >> > version
> > >>> > > > >> > > > > saves
> > >>> > > > >> > > > > > 15
> > >>> > > > >> > > > > > > > >> > letters
> > >>> > > > >> > > > > > > > >> > > > (to
> > >>> > > > >> > > > > > > > >> > > > > be
> > >>> > > > >> > > > > > > > >> > > > > > >> > fair,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > 4
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> those
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > were
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> dashes
> > >>> that
> > >>> > we
> > >>> > > > >> could
> > >>> > > > >> > > have
> > >>> > > > >> > > > > > > elided
> > >>> > > > >> > > > > > > > in
> > >>> > > > >> > > > > > > > >> > the
> > >>> > > > >> > > > > > > > >> > > > hex
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > representation.)
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >>> Another
> > >>> > > > thing
> > >>> > > > >> to
> > >>> > > > >> > > > > consider
> > >>> > > > >> > > > > > > is
> > >>> > > > >> > > > > > > > >> that
> > >>> > > > >> > > > > > > > >> > we
> > >>> > > > >> > > > > > > > >> > > > > > should
> > >>> > > > >> > > > > > > > >> > > > > > >> > > specify
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > that
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> the
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > >>> all-zeroes
> > >>> > > UUID
> > >>> > > > is
> > >>> > > > >> > not
> > >>> > > > >> > > a
> > >>> > > > >> > > > > > valid
> > >>> > > > >> > > > > > > > >> topic
> > >>> > > > >> > > > > > > > >> > > UUID.
> > >>> > > > >> > > > > > > > >> > > > > >  We
> > >>> > > > >> > > > > > > > >> > > > > > >> > can't
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > use
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> null
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > for
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > this
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > because
> > >>> we
> > >>> > > can't
> > >>> > > > >> > pass a
> > >>> > > > >> > > > > null
> > >>> > > > >> > > > > > > UUID
> > >>> > > > >> > > > > > > > >> over
> > >>> > > > >> > > > > > > > >> > > the
> > >>> > > > >> > > > > > > > >> > > > > RPC
> > >>> > > > >> > > > > > > > >> > > > > > >> > > protocol
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> (there
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > is no
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > special
> > >>> > > pattern
> > >>> > > > >> for
> > >>> > > > >> > > null,
> > >>> > > > >> > > > > nor
> > >>> > > > >> > > > > > > do
> > >>> > > > >> > > > > > > > we
> > >>> > > > >> > > > > > > > >> > want
> > >>> > > > >> > > > > > > > >> > > > to
> > >>> > > > >> > > > > > > > >> > > > > > >> waste
> > >>> > > > >> > > > > > > > >> > > > > > >> > > space
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> reserving
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > such
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > >>> pattern.)
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >>> Maybe I
> > >>> > > > missed
> > >>> > > > >> > it,
> > >>> > > > >> > > > but
> > >>> > > > >> > > > > > did
> > >>> > > > >> > > > > > > > you
> > >>> > > > >> > > > > > > > >> > > > describe
> > >>> > > > >> > > > > > > > >> > > > > > >> > > "migration
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > of...
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > existing
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > topic[s]
> > >>> > > without
> > >>> > > > >> > topic
> > >>> > > > >> > > > IDs"
> > >>> > > > >> > > > > > in
> > >>> > > > >> > > > > > > > >> detail
> > >>> > > > >> > > > > > > > >> > in
> > >>> > > > >> > > > > > > > >> > > > any
> > >>> > > > >> > > > > > > > >> > > > > > >> > section?
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > It
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> seems
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > like
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > when
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > the
> > new
> > >>> > > > controller
> > >>> > > > >> > > > becomes
> > >>> > > > >> > > > > > > > active,
> > >>> > > > >> > > > > > > > >> it
> > >>> > > > >> > > > > > > > >> > > > should
> > >>> > > > >> > > > > > > > >> > > > > > >> just
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > generate
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> random
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > UUIDs
> for
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> these,
> > >>> and
> > >>> > > write
> > >>> > > > >> the
> > >>> > > > >> > > > random
> > >>> > > > >> > > > > > > UUIDs
> > >>> > > > >> > > > > > > > >> back
> > >>> > > > >> > > > > > > > >> > > to
> > >>> > > > >> > > > > > > > >> > > > > > >> > ZooKeeper.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > It
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> would be
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > good
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > to
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> spell
> > >>> that
> > >>> > > out.
> > >>> > > > >> We
> > >>> > > > >> > > > should
> > >>> > > > >> > > > > > make
> > >>> > > > >> > > > > > > > it
> > >>> > > > >> > > > > > > > >> > clear
> > >>> > > > >> > > > > > > > >> > > > > that
> > >>> > > > >> > > > > > > > >> > > > > > >> this
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > happens
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > regardless
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > of
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > the
> > >>> > > inter-broker
> > >>> > > > >> > > protocol
> > >>> > > > >> > > > > > > version
> > >>> > > > >> > > > > > > > >> > (it's
> > >>> > > > >> > > > > > > > >> > > a
> > >>> > > > >> > > > > > > > >> > > > > > >> > compatible
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > change).
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >>> > > > >> > > "LeaderAndIsrRequests
> > >>> > > > >> > > > > > > > >> including an
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > is_every_partition
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> flag"
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > seems a
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > bit
> > >>> wordy.
> > >>> > > Can
> > >>> > > > we
> > >>> > > > >> > just
> > >>> > > > >> > > > > call
> > >>> > > > >> > > > > > > > these
> > >>> > > > >> > > > > > > > >> > "full
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > >>> LeaderAndIsrRequests"?
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > Then
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > the
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > RPC
> > >>> field
> > >>> > > could
> > >>> > > > be
> > >>> > > > >> > > named
> > >>> > > > >> > > > > > > "full".
> > >>> > > > >> > > > > > > > >> > Also,
> > >>> > > > >> > > > > > > > >> > > it
> > >>> > > > >> > > > > > > > >> > > > > > would
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > probably
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > be
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > better
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > for the
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > RPC
> > >>> field to
> > >>> > > be
> > >>> > > > an
> > >>> > > > >> > enum
> > >>> > > > >> > > > of
> > >>> > > > >> > > > > {
> > >>> > > > >> > > > > > > > >> > > UNSPECIFIED,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > INCREMENTAL,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > FULL
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> }, so
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > that
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > we
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > can
> > >>> cleanly
> > >>> > > > handle
> > >>> > > > >> > old
> > >>> > > > >> > > > > > versions
> > >>> > > > >> > > > > > > > (by
> > >>> > > > >> > > > > > > > >> > > > treating
> > >>> > > > >> > > > > > > > >> > > > > > >> them
> > >>> > > > >> > > > > > > > >> > > > > > >> > as
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> UNSPECIFIED)
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> In
> > >>> the
> > >>> > > > >> > > > > > LeaderAndIsrRequest
> > >>> > > > >> > > > > > > > >> > section,
> > >>> > > > >> > > > > > > > >> > > > you
> > >>> > > > >> > > > > > > > >> > > > > > >> write
> > >>> > > > >> > > > > > > > >> > > > > > >> > "A
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > final
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > deletion
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > event
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > will
> > be
> > >>> > > > secheduled
> > >>> > > > >> > for
> > >>> > > > >> > > X
> > >>> > > > >> > > > ms
> > >>> > > > >> > > > > > > after
> > >>> > > > >> > > > > > > > >> the
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > LeaderAndIsrRequest
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > was
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > first
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > >>> received..."
> > >>> > > I
> > >>> > > > >> guess
> > >>> > > > >> > > > the X
> > >>> > > > >> > > > > > > was a
> > >>> > > > >> > > > > > > > >> > > > > placeholder
> > >>> > > > >> > > > > > > > >> > > > > > >> that
> > >>> > > > >> > > > > > > > >> > > > > > >> > > you
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> intended
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > replace
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> before
> > >>> > > posting?
> > >>> > > > :)
> > >>> > > > >> > In
> > >>> > > > >> > > > any
> > >>> > > > >> > > > > > > case,
> > >>> > > > >> > > > > > > > >> this
> > >>> > > > >> > > > > > > > >> > > > seems
> > >>> > > > >> > > > > > > > >> > > > > > like
> > >>> > > > >> > > > > > > > >> > > > > > >> > the
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > kind
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > of
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > thing
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > we'd
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> want a
> > >>> > > > >> configuration
> > >>> > > > >> > > for.
> > >>> > > > >> > > > > > > Let's
> > >>> > > > >> > > > > > > > >> > > describe
> > >>> > > > >> > > > > > > > >> > > > > that
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > configuration
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> key
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> somewhere
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > in
> > this
> > >>> KIP,
> > >>> > > > >> > including
> > >>> > > > >> > > > what
> > >>> > > > >> > > > > > its
> > >>> > > > >> > > > > > > > >> > default
> > >>> > > > >> > > > > > > > >> > > > > value
> > >>> > > > >> > > > > > > > >> > > > > > >> is.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> We
> > >>> > should
> > >>> > > > >> > probably
> > >>> > > > >> > > > also
> > >>> > > > >> > > > > > > log a
> > >>> > > > >> > > > > > > > >> > bunch
> > >>> > > > >> > > > > > > > >> > > of
> > >>> > > > >> > > > > > > > >> > > > > > >> messages
> > >>> > > > >> > > > > > > > >> > > > > > >> > > at
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > WARN
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> level
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > when
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > >>> something is
> > >>> > > > >> > scheduled
> > >>> > > > >> > > > for
> > >>> > > > >> > > > > > > > >> deletion,
> > >>> > > > >> > > > > > > > >> > as
> > >>> > > > >> > > > > > > > >> > > > > well.
> > >>> > > > >> > > > > > > > >> > > > > > >> > (Maybe
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > this
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> was
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > assumed,
> > but
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > it
> > >>> would be
> > >>> > > good
> > >>> > > > >> to
> > >>> > > > >> > > > mention
> > >>> > > > >> > > > > > > it).
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> I
> > >>> feel
> > >>> > > like
> > >>> > > > >> there
> > >>> > > > >> > > > are a
> > >>> > > > >> > > > > > few
> > >>> > > > >> > > > > > > > >> > sections
> > >>> > > > >> > > > > > > > >> > > > > that
> > >>> > > > >> > > > > > > > >> > > > > > >> > should
> > >>> > > > >> > > > > > > > >> > > > > > >> > > be
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> moved to
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> "rejected
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > >>> > alternatives."
> > >>> > > > >> For
> > >>> > > > >> > > > > example,
> > >>> > > > >> > > > > > in
> > >>> > > > >> > > > > > > > the
> > >>> > > > >> > > > > > > > >> > > > > > DeleteTopics
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > section,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> since
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > we're
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > not
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> going
> > >>> to do
> > >>> > > > >> option 1
> > >>> > > > >> > or
> > >>> > > > >> > > > 2,
> > >>> > > > >> > > > > > > these
> > >>> > > > >> > > > > > > > >> > should
> > >>> > > > >> > > > > > > > >> > > be
> > >>> > > > >> > > > > > > > >> > > > > > moved
> > >>> > > > >> > > > > > > > >> > > > > > >> > into
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> "rejected
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > >>> > alternatives,"
> > >>> > > > >> > rather
> > >>> > > > >> > > > than
> > >>> > > > >> > > > > > > > >> appearing
> > >>> > > > >> > > > > > > > >> > > > > inline.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > Another
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > case
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> is
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > the
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > "Should
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > we
> > >>> remove
> > >>> > > topic
> > >>> > > > >> name
> > >>> > > > >> > > from
> > >>> > > > >> > > > > the
> > >>> > > > >> > > > > > > > >> protocol
> > >>> > > > >> > > > > > > > >> > > > where
> > >>> > > > >> > > > > > > > >> > > > > > >> > > possible"
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> section.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > This
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > is
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > clearly
> > >>> > > > >> discussing a
> > >>> > > > >> > > > design
> > >>> > > > >> > > > > > > > >> > alternative
> > >>> > > > >> > > > > > > > >> > > > that
> > >>> > > > >> > > > > > > > >> > > > > > >> we're
> > >>> > > > >> > > > > > > > >> > > > > > >> > > not
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> proposing
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > >>> implement:
> > >>> > > > >> removing
> > >>> > > > >> > the
> > >>> > > > >> > > > > topic
> > >>> > > > >> > > > > > > > name
> > >>> > > > >> > > > > > > > >> > from
> > >>> > > > >> > > > > > > > >> > > > > those
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > protocols.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> Is
> > >>> it
> > >>> > > really
> > >>> > > > >> > > > necessary
> > >>> > > > >> > > > > to
> > >>> > > > >> > > > > > > > have
> > >>> > > > >> > > > > > > > >> a
> > >>> > > > >> > > > > > > > >> > new
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > >>> > > /admin/delete_topics_by_id
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > path
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > in
> > >>> > ZooKeeper?
> > >>> > > > It
> > >>> > > > >> > seems
> > >>> > > > >> > > > > like
> > >>> > > > >> > > > > > we
> > >>> > > > >> > > > > > > > >> don't
> > >>> > > > >> > > > > > > > >> > > > really
> > >>> > > > >> > > > > > > > >> > > > > > >> need
> > >>> > > > >> > > > > > > > >> > > > > > >> > > this.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Whenever
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > there is
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > a
> new
> > >>> > > > controller,
> > >>> > > > >> > we'll
> > >>> > > > >> > > > > send
> > >>> > > > >> > > > > > > out
> > >>> > > > >> > > > > > > > >> full
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > LeaderAndIsrRequests
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> which
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > will
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > trigger
> > >>> the
> > >>> > > > stale
> > >>> > > > >> > > topics
> > >>> > > > >> > > > to
> > >>> > > > >> > > > > > be
> > >>> > > > >> > > > > > > > >> cleaned
> > >>> > > > >> > > > > > > > >> > > up.
> > >>> > > > >> > > > > > > > >> > > > > >  The
> > >>> > > > >> > > > > > > > >> > > > > > >> > > active
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > controller
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > will
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > also
> > >>> send
> > >>> > the
> > >>> > > > full
> > >>> > > > >> > > > > > > > >> LeaderAndIsrRequest
> > >>> > > > >> > > > > > > > >> > > to
> > >>> > > > >> > > > > > > > >> > > > > > >> brokers
> > >>> > > > >> > > > > > > > >> > > > > > >> > > that
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > are
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> just
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > starting
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > up.
> > >>> So we
> > >>> > > > don't
> > >>> > > > >> > > really
> > >>> > > > >> > > > > > need
> > >>> > > > >> > > > > > > > this
> > >>> > > > >> > > > > > > > >> > kind
> > >>> > > > >> > > > > > > > >> > > > of
> > >>> > > > >> > > > > > > > >> > > > > > >> > two-phase
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > commit
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > (send
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > out
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > >>> > > > >> StopReplicasRequest,
> > >>> > > > >> > > get
> > >>> > > > >> > > > > ACKs
> > >>> > > > >> > > > > > > > from
> > >>> > > > >> > > > > > > > >> all
> > >>> > > > >> > > > > > > > >> > > > > nodes,
> > >>> > > > >> > > > > > > > >> > > > > > >> > commit
> > >>> > > > >> > > > > > > > >> > > > > > >> > > by
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> removing
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > >>> > > > >> /admin/delete_topics
> > >>> > > > >> > > > node)
> > >>> > > > >> > > > > > any
> > >>> > > > >> > > > > > > > >> more.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > You
> > >>> > > mention
> > >>> > > > >> that
> > >>> > > > >> > > > > > > FetchRequest
> > >>> > > > >> > > > > > > > >> will
> > >>> > > > >> > > > > > > > >> > > now
> > >>> > > > >> > > > > > > > >> > > > > > >> include
> > >>> > > > >> > > > > > > > >> > > > > > >> > > UUID
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> avoid
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > issues
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> where
> > >>> > requests
> > >>> > > > are
> > >>> > > > >> > made
> > >>> > > > >> > > > to
> > >>> > > > >> > > > > > > stale
> > >>> > > > >> > > > > > > > >> > > > partitions.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > However,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> adding a
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > UUID
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > >>> > > MetadataRequest
> > >>> > > > is
> > >>> > > > >> > > listed
> > >>> > > > >> > > > > as
> > >>> > > > >> > > > > > > > future
> > >>> > > > >> > > > > > > > >> > > work,
> > >>> > > > >> > > > > > > > >> > > > > out
> > >>> > > > >> > > > > > > > >> > > > > > of
> > >>> > > > >> > > > > > > > >> > > > > > >> > > scope
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > for
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> this
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > KIP.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > How
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > will
> > the
> > >>> > > client
> > >>> > > > >> learn
> > >>> > > > >> > > > what
> > >>> > > > >> > > > > > the
> > >>> > > > >> > > > > > > > >> topic
> > >>> > > > >> > > > > > > > >> > > UUID
> > >>> > > > >> > > > > > > > >> > > > > is,
> > >>> > > > >> > > > > > > > >> > > > > > if
> > >>> > > > >> > > > > > > > >> > > > > > >> > the
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > metadata
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > response
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > doesn't
> > >>> > > include
> > >>> > > > >> that
> > >>> > > > >> > > > > > > information?
> > >>> > > > >> > > > > > > > >> It
> > >>> > > > >> > > > > > > > >> > > > seems
> > >>> > > > >> > > > > > > > >> > > > > > like
> > >>> > > > >> > > > > > > > >> > > > > > >> > > adding
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> UUID
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > >>> > > MetadataResponse
> > >>> > > > >> > would
> > >>> > > > >> > > be
> > >>> > > > >> > > > > an
> > >>> > > > >> > > > > > > > >> > improvement
> > >>> > > > >> > > > > > > > >> > > > > here
> > >>> > > > >> > > > > > > > >> > > > > > >> that
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > might
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > not
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> be
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > too
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > hard to
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> make.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >>> best,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >>> Colin
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> On
> > >>> Mon,
> > >>> > > Sep
> > >>> > > > 9,
> > >>> > > > >> > > 2019,
> > >>> > > > >> > > > at
> > >>> > > > >> > > > > > > > 17:48,
> > >>> > > > >> > > > > > > > >> > > Ryanne
> > >>> > > > >> > > > > > > > >> > > > > > Dolan
> > >>> > > > >> > > > > > > > >> > > > > > >> > > wrote:
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >
> > >>> Lucas,
> > >>> > > > this
> > >>> > > > >> > would
> > >>> > > > >> > > > be
> > >>> > > > >> > > > > > > great.
> > >>> > > > >> > > > > > > > >> I've
> > >>> > > > >> > > > > > > > >> > > run
> > >>> > > > >> > > > > > > > >> > > > > > into
> > >>> > > > >> > > > > > > > >> > > > > > >> > > issues
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > with
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > topics
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > being
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >
> > >>> > > > resurrected
> > >>> > > > >> > > > > > accidentally,
> > >>> > > > >> > > > > > > > >> since
> > >>> > > > >> > > > > > > > >> > a
> > >>> > > > >> > > > > > > > >> > > > > client
> > >>> > > > >> > > > > > > > >> > > > > > >> > cannot
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > easily
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > distinguish
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > between
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >
> > a
> > >>> > > deleted
> > >>> > > > >> topic
> > >>> > > > >> > > > and a
> > >>> > > > >> > > > > > new
> > >>> > > > >> > > > > > > > >> topic
> > >>> > > > >> > > > > > > > >> > > with
> > >>> > > > >> > > > > > > > >> > > > > the
> > >>> > > > >> > > > > > > > >> > > > > > >> same
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > name.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I'd
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > need
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > the
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > ID
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >
> > >>> > > accessible
> > >>> > > > >> from
> > >>> > > > >> > > the
> > >>> > > > >> > > > > > > client
> > >>> > > > >> > > > > > > > to
> > >>> > > > >> > > > > > > > >> > > solve
> > >>> > > > >> > > > > > > > >> > > > > that
> > >>> > > > >> > > > > > > > >> > > > > > >> > issue,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > but
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> this
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > is a
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > good
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> first
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >
> > >>> step.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >
> > >>> Ryanne
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >
> > On
> > >>> > Wed,
> > >>> > > > Sep
> > >>> > > > >> 4,
> > >>> > > > >> > > 2019
> > >>> > > > >> > > > > at
> > >>> > > > >> > > > > > > 1:41
> > >>> > > > >> > > > > > > > >> PM
> > >>> > > > >> > > > > > > > >> > > Lucas
> > >>> > > > >> > > > > > > > >> > > > > > >> > > Bradstreet <
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > >>> > > lucas@confluent.io>
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> wrote:
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >
> > >
> > >>> Hi
> > >>> > > all,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >
> > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >
> > >
> > >>> I
> > >>> > > would
> > >>> > > > >> like
> > >>> > > > >> > to
> > >>> > > > >> > > > > kick
> > >>> > > > >> > > > > > > off
> > >>> > > > >> > > > > > > > >> > > > discussion
> > >>> > > > >> > > > > > > > >> > > > > of
> > >>> > > > >> > > > > > > > >> > > > > > >> > > KIP-516,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > an
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > >>> implementation
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > of
> > topic
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >
> > >
> > >>> IDs
> > >>> > > for
> > >>> > > > >> > Kafka.
> > >>> > > > >> > > > > Topic
> > >>> > > > >> > > > > > > IDs
> > >>> > > > >> > > > > > > > >> aim
> > >>> > > > >> > > > > > > > >> > to
> > >>> > > > >> > > > > > > > >> > > > > solve
> > >>> > > > >> > > > > > > > >> > > > > > >> topic
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> uniqueness
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > problems
> > in
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> Kafka,
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >
> > >
> > >>> > where
> > >>> > > > >> > referring
> > >>> > > > >> > > > to
> > >>> > > > >> > > > > a
> > >>> > > > >> > > > > > > > topic
> > >>> > > > >> > > > > > > > >> by
> > >>> > > > >> > > > > > > > >> > > name
> > >>> > > > >> > > > > > > > >> > > > > > >> alone is
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> insufficient.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Such
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> cases
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >
> > >
> > >>> > > include
> > >>> > > > >> when
> > >>> > > > >> > a
> > >>> > > > >> > > > > topic
> > >>> > > > >> > > > > > > has
> > >>> > > > >> > > > > > > > >> been
> > >>> > > > >> > > > > > > > >> > > > > deleted
> > >>> > > > >> > > > > > > > >> > > > > > >> and
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > recreated
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> with
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > the
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > same
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> name.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >
> > >
> > >>> > Unique
> > >>> > > > >> > > > identifiers
> > >>> > > > >> > > > > > will
> > >>> > > > >> > > > > > > > >> help
> > >>> > > > >> > > > > > > > >> > > > > simplify
> > >>> > > > >> > > > > > > > >> > > > > > >> and
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > improve
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Kafka's
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > topic
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > deletion
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >
> > >
> > >>> > > process,
> > >>> > > > >> as
> > >>> > > > >> > > well
> > >>> > > > >> > > > as
> > >>> > > > >> > > > > > > > prevent
> > >>> > > > >> > > > > > > > >> > > cases
> > >>> > > > >> > > > > > > > >> > > > > > where
> > >>> > > > >> > > > > > > > >> > > > > > >> > > brokers
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > may
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> incorrectly
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > interact
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >
> > >
> > >>> with
> > >>> > > > stale
> > >>> > > > >> > > > versions
> > >>> > > > >> > > > > > of
> > >>> > > > >> > > > > > > > >> topics.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >
> > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >
> > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >
> > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > >>> > > > >> > > > > > > > >> > > > > > >> >
> > >>> > > > >> > > > > > > > >> > > > > > >>
> > >>> > > > >> > > > > > > > >> > > > > >
> > >>> > > > >> > > > > > > > >> > > > >
> > >>> > > > >> > > > > > > > >> > > >
> > >>> > > > >> > > > > > > > >> > >
> > >>> > > > >> > > > > > > > >> >
> > >>> > > > >> > > > > > > > >>
> > >>> > > > >> > > > > > > >
> > >>> > > > >> > > > > > >
> > >>> > > > >> > > > > >
> > >>> > > > >> > > > >
> > >>> > > > >> > > >
> > >>> > > > >> > >
> > >>> > > > >> >
> > >>> > > > >>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >
> > >
> > >>> > > Looking
> > >>> > > > >> > forward
> > >>> > > > >> > > > to
> > >>> > > > >> > > > > > your
> > >>> > > > >> > > > > > > > >> > > thoughts.
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >
> > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >
> > >
> > >>> > Lucas
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >
> > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > > >
> > >>> > > > >> > > > > > > > >> > > > > > >> > >
> > >>> > > > >> > > > > > > > >> > > > > > >> >
> > >>> > > > >> > > > > > > > >> > > > > > >>
> > >>> > > > >> > > > > > > > >> > > > > > >
> > >>> > > > >> > > > > > > > >> > > > > >
> > >>> > > > >> > > > > > > > >> > > > >
> > >>> > > > >> > > > > > > > >> > > >
> > >>> > > > >> > > > > > > > >> > >
> > >>> > > > >> > > > > > > > >> >
> > >>> > > > >> > > > > > > > >>
> > >>> > > > >> > > > > > > > >
> > >>> > > > >> > > > > > > >
> > >>> > > > >> > > > > > >
> > >>> > > > >> > > > > >
> > >>> > > > >> > > > >
> > >>> > > > >> > > >
> > >>> > > > >> > >
> > >>> > > > >> >
> > >>> > > > >>
> > >>> > > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> > >>
> >
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Jun Rao <ju...@confluent.io>]

Hi, Justine,

Adding a TopicIdPartition in the public interface may be useful. Where
would you expect it to be used?

Related to this, since we are exposing topicId in the tool, it seems that
we may want to expose topicId in at least TopicDescription used by
AdminClient.describeTopics() and TopicMetadataAndConfig used in
AdminClient.createTopics(Collection<NewTopic> newTopics). I am also not
sure if we plan to expose topicId in other AdminClient methods in this KIP
or in some future KIP. For example, it might be useful to support topicId
in the following methods.

    DescribeTopicsResult describeTopics(Collection<String> topicNames,
DescribeTopicsOptions options);
    DeleteTopicsResult deleteTopics(Collection<String> topics,
DeleteTopicsOptions options);

Thanks,

Jun

On Mon, Oct 5, 2020 at 11:03 AM Justine Olshan <jo...@confluent.io> wrote:

> Hi all,
>
> I was thinking about this KIP some more. We are removing the topic name
> from many of these request types, and some of them (eg. Fetch) use
> TopicPartition as keys in a map when handling the request. I was thinking
> it would make sense to make some sort of TopicIdPartition with topic name,
> topic id, and partition fields (name and id would potentially be optional
> types since different request versions will contain only one of the two) I
> was wondering if it made sense to include this as a public facing API
> similar to TopicPartition. If so, I will add this class to the KIP as well.
>
> Thanks,
> Justine
>
> On Mon, Oct 5, 2020 at 8:52 AM Justine Olshan <jo...@confluent.io>
> wrote:
>
> > Hi all,
> > I've made a few minor changes to the UUID class. I've added a few more
> > methods to support UUIDs more generally, by including a constructor and a
> > method to generate random UUIDs. Both of these will be useful for
> testing.
> >
> > Let me know if there are any other areas for discussion.
> >
> > Thank you,
> > Justine
> >
> > On Thu, Oct 1, 2020 at 5:29 PM Justine Olshan <jo...@confluent.io>
> > wrote:
> >
> >> Hi Jun,
> >> Thanks for looking at it again. Here's the new spec. (I fixed the typo
> in
> >> it too.)
> >> {"name": "id", "type": "string", "doc": option[UUID]}
> >>
> >> Justine
> >>
> >>
> >> On Thu, Oct 1, 2020 at 5:03 PM Jun Rao <ju...@confluent.io> wrote:
> >>
> >>> Hi, Justine,
> >>>
> >>> Thanks for the update. The KIP looks good to me now. Just a minor
> comment
> >>> below.
> >>>
> >>> 30. Perhaps "option[UUID]" can be put in the doc.
> >>>
> >>> Jun
> >>>
> >>> On Thu, Oct 1, 2020 at 3:28 PM Justine Olshan <jo...@confluent.io>
> >>> wrote:
> >>>
> >>> > Hi Jun,
> >>> > Thanks for the response!
> >>> >
> >>> > 30. I think I might have changed this in between. The current version
> >>> > says:  {"name":
> >>> > "id", "type": "option[UUID]"}, "doc": topic id}
> >>> > I have switched to the option type to cover the migration case where
> a
> >>> > TopicZNode does not yet have a topic ID.
> >>> > I understand that due to json, this field is written as a string, so
> >>> if I
> >>> > should move the "option[uuid]" to the doc field and the type should
> be
> >>> > "string" please let me know.
> >>> >
> >>> > 40. I've added a definition for UUID.
> >>> > 41,42. Fixed
> >>> >
> >>> > Thank you,
> >>> > Justine
> >>> >
> >>> > On Wed, Sep 30, 2020 at 1:15 PM Jun Rao <ju...@confluent.io> wrote:
> >>> >
> >>> > > Hi, Justine,
> >>> > >
> >>> > > Thanks for the summary. Just a few minor comments blow.
> >>> > >
> >>> > > 30.  {"name": "id", "type": "string", "doc": "version id"}}: The
> doc
> >>> > should
> >>> > > say UUID. The issue seems unfixed.
> >>> > >
> >>> > > 40. Since UUID is public facing, could you include its definition?
> >>> > >
> >>> > > 41. StopReplicaResponse still includes the topic field.
> >>> > >
> >>> > > 42. "It is unnecessary to include the name of the topic in the
> >>> following
> >>> > > Request/Response calls" It would be useful to include all modified
> >>> > requests
> >>> > > (e.g. produce) in the list below.
> >>> > >
> >>> > > Thanks,
> >>> > >
> >>> > > Jun
> >>> > >
> >>> > >
> >>> > > On Wed, Sep 30, 2020 at 10:17 AM Justine Olshan <
> >>> jolshan@confluent.io>
> >>> > > wrote:
> >>> > >
> >>> > > > Hello again,
> >>> > > >
> >>> > > > I've taken some time to discuss some of the remaining points
> >>> brought up
> >>> > > by
> >>> > > > the previous emails offline. Here are some of the conclusions.
> >>> > > >
> >>> > > > 1. Directory Structure:
> >>> > > > There was some talk about whether the directory structure should
> be
> >>> > > changed
> >>> > > > to replace all topic names with topic IDs.
> >>> > > > This will be a useful change, but will prevent downgrades. It
> will
> >>> be
> >>> > > best
> >>> > > > to wait until a major release, likely alongside KIP-500 changes
> >>> that
> >>> > will
> >>> > > > prevent downgrades. I've updated the KIP to include this change
> >>> with
> >>> > the
> >>> > > > note about migration and deprecation.
> >>> > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-log.dirlayout
> >>> > > >
> >>> > > > 2. Partition Metadata file
> >>> > > > There was some disagreement about the inclusion of the partition
> >>> > metadata
> >>> > > > file.
> >>> > > > This file will be needed to persist the topic ID, especially
> while
> >>> we
> >>> > > still
> >>> > > > have the old directory structure. Even after the changes, this
> >>> file can
> >>> > > be
> >>> > > > useful for debugging and recovery.
> >>> > > > Creating a single mapping file from topic names to topic IDs was
> >>> > > > considered, but ultimately scrapped as it would not be as easy to
> >>> > > maintain.
> >>> > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PartitionMetadatafile
> >>> > > >
> >>> > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PersistingTopicIDs
> >>> > > >
> >>> > > > 3. Produce Protocols
> >>> > > > After some further discussion, this replacing the topic name with
> >>> topic
> >>> > > ID
> >>> > > > in these protocols has been added to the KIP.
> >>> > > > This will cut down on the size of the protocol. Since changes to
> >>> fetch
> >>> > > are
> >>> > > > included, it does make sense to update these protocols.
> >>> > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-Produce
> >>> > > >
> >>> > > > 4. KIP-500 Compatibility
> >>> > > > After some discussion with those involved with KIP-500, it seems
> >>> best
> >>> > to
> >>> > > > use a sentinel topic ID for the metadata topic that is used
> before
> >>> the
> >>> > > > first controller election.
> >>> > > > We had an issue where this metadata topic may not be assigned an
> ID
> >>> > > before
> >>> > > > utilizing the Vote and Fetch protocols. It was decided to
> reserve a
> >>> > > unique
> >>> > > > ID for this topic to be used until the controller could give the
> >>> topic
> >>> > a
> >>> > > > unique ID.
> >>> > > > Having the topic keep the sentinel ID (not be reassigned to a
> >>> unique
> >>> > ID)
> >>> > > > was considered, but it seemed like a good idea to leave the
> option
> >>> open
> >>> > > for
> >>> > > > the metadata topic to have a unique ID in cases where it would
> >>> need to
> >>> > be
> >>> > > > differentiated from other clusters' metadata topics. (ex. tiered
> >>> > storage)
> >>> > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-CompatibilitywithKIP-500
> >>> > > >
> >>> > > > I've also split up the KIP into sub-tasks on the JIRA. Hopefully
> >>> this
> >>> > > will
> >>> > > > provide a better idea about what tasks we have, and eventually
> >>> provide
> >>> > a
> >>> > > > place to see what's done and what is left.
> >>> > > > If there is a task I am missing, please let me know!
> >>> > > > https://issues.apache.org/jira/browse/KAFKA-8872
> >>> > > >
> >>> > > > Of course, these decisions are not set in stone, and I would love
> >>> to
> >>> > hear
> >>> > > > any feedback.
> >>> > > >
> >>> > > > Thanks,
> >>> > > > Justine
> >>> > > >
> >>> > > > On Mon, Sep 28, 2020 at 11:38 AM Justine Olshan <
> >>> jolshan@confluent.io>
> >>> > > > wrote:
> >>> > > >
> >>> > > > > Hello all,
> >>> > > > >
> >>> > > > > I just wanted to follow up on this discussion. Did we come to
> an
> >>> > > > > understanding about the directory structure?
> >>> > > > >
> >>> > > > > I think the biggest question here is what is acceptable to
> leave
> >>> out
> >>> > > due
> >>> > > > > to scope vs. what is considered to be too much tech debt.
> >>> > > > > This KIP is already pretty large with the number of changes,
> but
> >>> it
> >>> > > also
> >>> > > > > makes sense to do things correctly, so I'd love to hear
> >>> everyone's
> >>> > > > thoughts.
> >>> > > > >
> >>> > > > > Thanks,
> >>> > > > > Justine
> >>> > > > >
> >>> > > > > On Fri, Sep 25, 2020 at 8:19 AM Lucas Bradstreet <
> >>> lucas@confluent.io
> >>> > >
> >>> > > > > wrote:
> >>> > > > >
> >>> > > > >> Hi Ismael,
> >>> > > > >>
> >>> > > > >> If you do not store it in a metadata file or in the directory
> >>> > > structure
> >>> > > > >> would you then
> >>> > > > >> require the LeaderAndIsrRequest following startup to give you
> >>> some
> >>> > > > notion
> >>> > > > >> of
> >>> > > > >> topic name in memory? We would still need this information for
> >>> the
> >>> > > older
> >>> > > > >> protocols, but
> >>> > > > >> perhaps this is what's meant by tech debt.
> >>> > > > >>
> >>> > > > >> Once we're free of the old non-topicID requests then I think
> you
> >>> > > > wouldn't
> >>> > > > >> need to retain the topic name.
> >>> > > > >> I think the ability to easily look up topic names associated
> >>> with
> >>> > > > >> partition
> >>> > > > >> directories would still be missed
> >>> > > > >> when diagnosing issues, though maybe it wouldn't be a deal
> >>> breaker
> >>> > > with
> >>> > > > >> the
> >>> > > > >> right tooling.
> >>> > > > >>
> >>> > > > >> Thanks,
> >>> > > > >>
> >>> > > > >> Lucas
> >>> > > > >>
> >>> > > > >> On Fri, Sep 25, 2020 at 7:55 AM Ismael Juma <
> ismael@juma.me.uk>
> >>> > > wrote:
> >>> > > > >>
> >>> > > > >> > Hi Lucas,
> >>> > > > >> >
> >>> > > > >> > Why would you include the name and id? I think you'd want to
> >>> > remove
> >>> > > > the
> >>> > > > >> > name from the directory name right? Jason's suggestion was
> >>> that if
> >>> > > you
> >>> > > > >> > remove the name from the directory, then why would you need
> >>> the id
> >>> > > > name
> >>> > > > >> > mapping file?
> >>> > > > >> >
> >>> > > > >> > Ismael
> >>> > > > >> >
> >>> > > > >> > On Thu, Sep 24, 2020 at 4:24 PM Lucas Bradstreet <
> >>> > > lucas@confluent.io>
> >>> > > > >> > wrote:
> >>> > > > >> >
> >>> > > > >> > > > 2. Part of the usage of the file is to have persistent
> >>> storage
> >>> > > of
> >>> > > > >> the
> >>> > > > >> > > topic
> >>> > > > >> > > ID and use it to compare with the ID supplied in the
> >>> > LeaderAndIsr
> >>> > > > >> > Request.
> >>> > > > >> > > There is some discussion in the KIP about changes to the
> >>> > directory
> >>> > > > >> > > structure, but I believe directory changes were considered
> >>> to be
> >>> > > out
> >>> > > > >> of
> >>> > > > >> > > scope when the KIP was written.
> >>> > > > >> > >
> >>> > > > >> > >
> >>> > > > >> > > Yeah, I was hoping to get a better understanding of why it
> >>> was
> >>> > > taken
> >>> > > > >> out
> >>> > > > >> > of
> >>> > > > >> > > scope. Perhaps Lucas Bradstreet might have more insight
> >>> about
> >>> > the
> >>> > > > >> > decision.
> >>> > > > >> > > Basically my point is that we have to create additional
> >>> > > > infrastructure
> >>> > > > >> > here
> >>> > > > >> > > to support the name/id mapping, so I wanted to understand
> >>> if we
> >>> > > just
> >>> > > > >> > > consider this a sort of tech debt. It would be useful to
> >>> cover
> >>> > how
> >>> > > > we
> >>> > > > >> > > handle the case when this file gets corrupted. Seems like
> we
> >>> > just
> >>> > > > >> have to
> >>> > > > >> > > trust that it matches whatever the controller tells us and
> >>> > rewrite
> >>> > > > it?
> >>> > > > >> > >
> >>> > > > >> > >
> >>> > > > >> > > Hi Jason, Justine,
> >>> > > > >> > >
> >>> > > > >> > > My thought process is that we will likely need the
> metadata
> >>> file
> >>> > > > >> whether
> >>> > > > >> > we
> >>> > > > >> > > rename the directories or not.
> >>> > > > >> > > Linux supports filenames of up to 255 bytes and that would
> >>> not
> >>> > be
> >>> > > > >> enough
> >>> > > > >> > to
> >>> > > > >> > > support a directory name
> >>> > > > >> > >  including both the name and topic ID. Given that fact, it
> >>> > seemed
> >>> > > > >> > > reasonable to add the metadata file
> >>> > > > >> > > and leave the directory rename until some time in the
> future
> >>> > > > (possibly
> >>> > > > >> > > never).
> >>> > > > >> > >
> >>> > > > >> > > If the file does get corrupted I think you're right. We
> >>> would
> >>> > > either
> >>> > > > >> have
> >>> > > > >> > > to trust it matches what the controller tells us
> >>> > > > >> > >  or error out and let an administrator resolve it by
> >>> checking
> >>> > > across
> >>> > > > >> > > replicas for consistency.
> >>> > > > >> > >
> >>> > > > >> > > Lucas
> >>> > > > >> > >
> >>> > > > >> > >
> >>> > > > >> > > On Thu, Sep 24, 2020 at 3:41 PM Jason Gustafson <
> >>> > > jason@confluent.io
> >>> > > > >
> >>> > > > >> > > wrote:
> >>> > > > >> > >
> >>> > > > >> > > > Thanks Justine. Responses below:
> >>> > > > >> > > >
> >>> > > > >> > > > > 1. Yes, the directory will still be based on the topic
> >>> > names.
> >>> > > > >> > > > LeaderAndIsrRequest is one of the few requests that will
> >>> still
> >>> > > > >> contain
> >>> > > > >> > > the
> >>> > > > >> > > > topic name. So I think we have this covered. Sorry for
> >>> > > confusion.
> >>> > > > >> > > >
> >>> > > > >> > > > Ah, you're right. My eyes passed right over the field.
> >>> > > > >> > > >
> >>> > > > >> > > > > 2. Part of the usage of the file is to have persistent
> >>> > storage
> >>> > > > of
> >>> > > > >> the
> >>> > > > >> > > > topic
> >>> > > > >> > > > ID and use it to compare with the ID supplied in the
> >>> > > LeaderAndIsr
> >>> > > > >> > > Request.
> >>> > > > >> > > > There is some discussion in the KIP about changes to the
> >>> > > directory
> >>> > > > >> > > > structure, but I believe directory changes were
> >>> considered to
> >>> > be
> >>> > > > >> out of
> >>> > > > >> > > > scope when the KIP was written.
> >>> > > > >> > > >
> >>> > > > >> > > > Yeah, I was hoping to get a better understanding of why
> >>> it was
> >>> > > > taken
> >>> > > > >> > out
> >>> > > > >> > > of
> >>> > > > >> > > > scope. Perhaps Lucas Bradstreet might have more insight
> >>> about
> >>> > > the
> >>> > > > >> > > decision.
> >>> > > > >> > > > Basically my point is that we have to create additional
> >>> > > > >> infrastructure
> >>> > > > >> > > here
> >>> > > > >> > > > to support the name/id mapping, so I wanted to
> understand
> >>> if
> >>> > we
> >>> > > > just
> >>> > > > >> > > > consider this a sort of tech debt. It would be useful to
> >>> cover
> >>> > > how
> >>> > > > >> we
> >>> > > > >> > > > handle the case when this file gets corrupted. Seems
> like
> >>> we
> >>> > > just
> >>> > > > >> have
> >>> > > > >> > to
> >>> > > > >> > > > trust that it matches whatever the controller tells us
> and
> >>> > > rewrite
> >>> > > > >> it?
> >>> > > > >> > > >
> >>> > > > >> > > > > 3. I think this is a good point, but I again I wonder
> >>> about
> >>> > > the
> >>> > > > >> scope
> >>> > > > >> > > of
> >>> > > > >> > > > the KIP. Most of the changes mentioned in the KIP are
> for
> >>> > > > supporting
> >>> > > > >> > > topic
> >>> > > > >> > > > deletion and I believe that is why the produce request
> was
> >>> > > listed
> >>> > > > >> under
> >>> > > > >> > > > future work.
> >>> > > > >> > > >
> >>> > > > >> > > > That's fair. I brought it up since `Fetch` is already
> >>> > included.
> >>> > > If
> >>> > > > >> > we've
> >>> > > > >> > > > got `Metadata` and `Fetch`, seems we may as well do
> >>> `Produce`
> >>> > > and
> >>> > > > >> save
> >>> > > > >> > an
> >>> > > > >> > > > extra kip. No strong objection though if you want to
> >>> leave it
> >>> > > out.
> >>> > > > >> > > >
> >>> > > > >> > > >
> >>> > > > >> > > > -Jason
> >>> > > > >> > > >
> >>> > > > >> > > >
> >>> > > > >> > > > On Thu, Sep 24, 2020 at 3:26 PM Justine Olshan <
> >>> > > > >> jolshan@confluent.io>
> >>> > > > >> > > > wrote:
> >>> > > > >> > > >
> >>> > > > >> > > > > Hi Jason,
> >>> > > > >> > > > >
> >>> > > > >> > > > > Thanks for your comments.
> >>> > > > >> > > > >
> >>> > > > >> > > > > 1. Yes, the directory will still be based on the topic
> >>> > names.
> >>> > > > >> > > > > LeaderAndIsrRequest is one of the few requests that
> will
> >>> > still
> >>> > > > >> > contain
> >>> > > > >> > > > the
> >>> > > > >> > > > > topic name. So I think we have this covered. Sorry for
> >>> > > > confusion.
> >>> > > > >> > > > >
> >>> > > > >> > > > > 2. Part of the usage of the file is to have persistent
> >>> > storage
> >>> > > > of
> >>> > > > >> the
> >>> > > > >> > > > topic
> >>> > > > >> > > > > ID and use it to compare with the ID supplied in the
> >>> > > > LeaderAndIsr
> >>> > > > >> > > > Request.
> >>> > > > >> > > > > There is some discussion in the KIP about changes to
> the
> >>> > > > directory
> >>> > > > >> > > > > structure, but I believe directory changes were
> >>> considered
> >>> > to
> >>> > > be
> >>> > > > >> out
> >>> > > > >> > of
> >>> > > > >> > > > > scope when the KIP was written.
> >>> > > > >> > > > >
> >>> > > > >> > > > > 3. I think this is a good point, but I again I wonder
> >>> about
> >>> > > the
> >>> > > > >> scope
> >>> > > > >> > > of
> >>> > > > >> > > > > the KIP. Most of the changes mentioned in the KIP are
> >>> for
> >>> > > > >> supporting
> >>> > > > >> > > > topic
> >>> > > > >> > > > > deletion and I believe that is why the produce request
> >>> was
> >>> > > > listed
> >>> > > > >> > under
> >>> > > > >> > > > > future work.
> >>> > > > >> > > > >
> >>> > > > >> > > > > 4. This sounds like it might be a good solution, but I
> >>> will
> >>> > > need
> >>> > > > >> to
> >>> > > > >> > > > discuss
> >>> > > > >> > > > > more with KIP-500 folks to get the details right.
> >>> > > > >> > > > >
> >>> > > > >> > > > > Thanks,
> >>> > > > >> > > > > Justine
> >>> > > > >> > > > >
> >>> > > > >> > > > > On Thu, Sep 24, 2020 at 12:30 PM Jason Gustafson <
> >>> > > > >> jason@confluent.io
> >>> > > > >> > >
> >>> > > > >> > > > > wrote:
> >>> > > > >> > > > >
> >>> > > > >> > > > > > Hi Justine,
> >>> > > > >> > > > > >
> >>> > > > >> > > > > > Thanks for picking up this work. I have a few
> >>> > > > >> questions/comments:
> >>> > > > >> > > > > >
> >>> > > > >> > > > > > 1. It sounds like the directory structure is still
> >>> going
> >>> > to
> >>> > > be
> >>> > > > >> > based
> >>> > > > >> > > on
> >>> > > > >> > > > > > topic names. Do I have that right? One complication
> is
> >>> > that
> >>> > > > the
> >>> > > > >> > > > > > LeaderAndIsr request does not include the topic name
> >>> any
> >>> > > > longer.
> >>> > > > >> > This
> >>> > > > >> > > > > means
> >>> > > > >> > > > > > that a replica must wait for the UpdateMetadata
> >>> request to
> >>> > > > >> arrive
> >>> > > > >> > > with
> >>> > > > >> > > > > the
> >>> > > > >> > > > > > topic name mapping before it can create the
> >>> directory. I
> >>> > > > wonder
> >>> > > > >> if
> >>> > > > >> > it
> >>> > > > >> > > > > would
> >>> > > > >> > > > > > be simpler to avoid assumptions on the ordering of
> >>> > > > >> UpdateMetadata
> >>> > > > >> > and
> >>> > > > >> > > > let
> >>> > > > >> > > > > > LeaderAndIsr include the topic name as well. Feels
> >>> like we
> >>> > > are
> >>> > > > >> not
> >>> > > > >> > > > saving
> >>> > > > >> > > > > > that much by excluding it.
> >>> > > > >> > > > > >
> >>> > > > >> > > > > > 2. On a related note, it seems that the reason we
> >>> have the
> >>> > > > >> > partition
> >>> > > > >> > > > > > metadata file is because we are not changing the
> >>> directory
> >>> > > > >> > structure.
> >>> > > > >> > > > We
> >>> > > > >> > > > > > need it so that we remember which directories map to
> >>> which
> >>> > > > topic
> >>> > > > >> > id.
> >>> > > > >> > > I
> >>> > > > >> > > > > > think it would be useful to add some detail about
> why
> >>> > > changing
> >>> > > > >> the
> >>> > > > >> > > > > > directory layout was rejected. Basically trying to
> >>> > > understand
> >>> > > > >> how
> >>> > > > >> > > much
> >>> > > > >> > > > > > effort we are saving by skipping this step if we
> have
> >>> to
> >>> > > > >> implement
> >>> > > > >> > > this
> >>> > > > >> > > > > > additional bookkeeping. One concern I have for
> >>> example is
> >>> > > that
> >>> > > > >> the
> >>> > > > >> > > > > > partition metadata file gets corrupt and then all
> >>> bets are
> >>> > > off
> >>> > > > >> as
> >>> > > > >> > far
> >>> > > > >> > > > as
> >>> > > > >> > > > > > topic consistency.
> >>> > > > >> > > > > >
> >>> > > > >> > > > > > 3. Is it worth adding support now for topic ids in
> the
> >>> > > > `Produce`
> >>> > > > >> > > > request
> >>> > > > >> > > > > > now? Overhead is mentioned as one of the motivations
> >>> and
> >>> > the
> >>> > > > >> best
> >>> > > > >> > > APIs
> >>> > > > >> > > > to
> >>> > > > >> > > > > > get savings from are `Produce` and `Fetch`.
> >>> > > > >> > > > > >
> >>> > > > >> > > > > > 4. When it comes to bootstrapping the metadata topic
> >>> with
> >>> > > > >> respect
> >>> > > > >> > to
> >>> > > > >> > > > > > KIP-500, one suggestion would be to reserve a
> sentinel
> >>> > topic
> >>> > > > ID
> >>> > > > >> > which
> >>> > > > >> > > > can
> >>> > > > >> > > > > > be used initially by new brokers. When the first
> >>> > controller
> >>> > > is
> >>> > > > >> > > elected,
> >>> > > > >> > > > > it
> >>> > > > >> > > > > > can assign a topicId to the metadata topic. This is
> a
> >>> bit
> >>> > > > >> similar
> >>> > > > >> > to
> >>> > > > >> > > > how
> >>> > > > >> > > > > we
> >>> > > > >> > > > > > were planning to generate the clusterId.
> >>> > > > >> > > > > >
> >>> > > > >> > > > > > Thanks,
> >>> > > > >> > > > > > Jason
> >>> > > > >> > > > > >
> >>> > > > >> > > > > >
> >>> > > > >> > > > > > On Thu, Sep 24, 2020 at 11:10 AM Jun Rao <
> >>> > jun@confluent.io>
> >>> > > > >> wrote:
> >>> > > > >> > > > > >
> >>> > > > >> > > > > > > Hi, Justine,
> >>> > > > >> > > > > > >
> >>> > > > >> > > > > > > Thanks for the updated KIP. A few more comments
> >>> below.
> >>> > > > >> > > > > > >
> >>> > > > >> > > > > > > 30.  {"name": "id", "type": "string", "doc":
> >>> "version
> >>> > > id"}}:
> >>> > > > >> The
> >>> > > > >> > > doc
> >>> > > > >> > > > > > should
> >>> > > > >> > > > > > > say UUID.
> >>> > > > >> > > > > > >
> >>> > > > >> > > > > > > 31. LeaderAndIsrResponse v5 and
> StopReplicaResponse
> >>> v4 :
> >>> > > It
> >>> > > > >> seems
> >>> > > > >> > > > there
> >>> > > > >> > > > > > is
> >>> > > > >> > > > > > > no need to add topic_id at partitions level.
> >>> > > > >> > > > > > >
> >>> > > > >> > > > > > > 32. Regarding partition metadata file. Perhaps the
> >>> key
> >>> > can
> >>> > > > be
> >>> > > > >> a
> >>> > > > >> > > > single
> >>> > > > >> > > > > > > word, sth like the following.
> >>> > > > >> > > > > > > version: 0
> >>> > > > >> > > > > > > topic_id: 46bdb63f-9e8d-4a38-bf7b-ee4eb2a794e4
> >>> > > > >> > > > > > >
> >>> > > > >> > > > > > > 33. Another tricky thing that I realized is how to
> >>> > support
> >>> > > > the
> >>> > > > >> > > > metadata
> >>> > > > >> > > > > > > topic introduced in KIP-595. It's a bit tricky to
> >>> > assign a
> >>> > > > >> UUID
> >>> > > > >> > to
> >>> > > > >> > > > the
> >>> > > > >> > > > > > > metadata topic since we have a chicken-and-egg
> >>> problem.
> >>> > > The
> >>> > > > >> > > > controller
> >>> > > > >> > > > > > > needs to persist the UUID in the metadata topic in
> >>> order
> >>> > > to
> >>> > > > >> > assign
> >>> > > > >> > > > one
> >>> > > > >> > > > > > > successfully, but the metadata topic is needed to
> >>> elect
> >>> > a
> >>> > > > >> > > controller
> >>> > > > >> > > > > > > (KIP-631). So, this probably needs a bit more
> >>> thought.
> >>> > > > >> > > > > > >
> >>> > > > >> > > > > > > Jun
> >>> > > > >> > > > > > >
> >>> > > > >> > > > > > > On Thu, Sep 24, 2020 at 3:04 AM Ismael Juma <
> >>> > > > >> ismael@juma.me.uk>
> >>> > > > >> > > > wrote:
> >>> > > > >> > > > > > >
> >>> > > > >> > > > > > > > Also, can we provide more details on how the
> >>> Partition
> >>> > > > >> Metadata
> >>> > > > >> > > > file
> >>> > > > >> > > > > > will
> >>> > > > >> > > > > > > > be used?
> >>> > > > >> > > > > > > >
> >>> > > > >> > > > > > > > Ismael
> >>> > > > >> > > > > > > >
> >>> > > > >> > > > > > > > On Thu, Sep 24, 2020 at 3:01 AM Ismael Juma <
> >>> > > > >> ismael@juma.me.uk
> >>> > > > >> > >
> >>> > > > >> > > > > wrote:
> >>> > > > >> > > > > > > >
> >>> > > > >> > > > > > > > > Hi Justine,
> >>> > > > >> > > > > > > > >
> >>> > > > >> > > > > > > > > I think we need to update the "Rejected
> >>> > Alternatives"
> >>> > > > >> section
> >>> > > > >> > > to
> >>> > > > >> > > > > take
> >>> > > > >> > > > > > > > into
> >>> > > > >> > > > > > > > > account that the proposal now removes the
> topic
> >>> name
> >>> > > > from
> >>> > > > >> the
> >>> > > > >> > > > fetch
> >>> > > > >> > > > > > > > > request. Also, if we are removing it from the
> >>> Fetch
> >>> > > > >> request,
> >>> > > > >> > > does
> >>> > > > >> > > > > it
> >>> > > > >> > > > > > > make
> >>> > > > >> > > > > > > > > sense not to remove it from similar requests
> >>> like
> >>> > > > >> > > > > ListOffsetRequest?
> >>> > > > >> > > > > > > > >
> >>> > > > >> > > > > > > > > Ismael
> >>> > > > >> > > > > > > > >
> >>> > > > >> > > > > > > > > On Thu, Sep 24, 2020 at 2:46 AM David Jacot <
> >>> > > > >> > > djacot@confluent.io
> >>> > > > >> > > > >
> >>> > > > >> > > > > > > wrote:
> >>> > > > >> > > > > > > > >
> >>> > > > >> > > > > > > > >> Hi Justine,
> >>> > > > >> > > > > > > > >>
> >>> > > > >> > > > > > > > >> Thanks for the KIP. I finally had time to
> read
> >>> it
> >>> > :).
> >>> > > > It
> >>> > > > >> is
> >>> > > > >> > a
> >>> > > > >> > > > > great
> >>> > > > >> > > > > > > > >> improvement.
> >>> > > > >> > > > > > > > >>
> >>> > > > >> > > > > > > > >> I have a few comments/questions:
> >>> > > > >> > > > > > > > >>
> >>> > > > >> > > > > > > > >> 1. It seems that the schema of the
> >>> > StopReplicaRequest
> >>> > > > is
> >>> > > > >> > > > slightly
> >>> > > > >> > > > > > > > >> outdated.
> >>> > > > >> > > > > > > > >> We
> >>> > > > >> > > > > > > > >> did some changes as part of KIP-570. V3 is
> >>> already
> >>> > > > >> organized
> >>> > > > >> > > by
> >>> > > > >> > > > > > > topics.
> >>> > > > >> > > > > > > > >>
> >>> > > > >> > > > > > > > >> 2. I just want to make sure that I understand
> >>> the
> >>> > > > >> > > reconciliation
> >>> > > > >> > > > > > > > >> logic correctly. When
> >>> > > > >> > > > > > > > >> an "INCREMENTAL" LeaderAndIsr Request is
> >>> received,
> >>> > > the
> >>> > > > >> > broker
> >>> > > > >> > > > will
> >>> > > > >> > > > > > > also
> >>> > > > >> > > > > > > > >> reconcile
> >>> > > > >> > > > > > > > >> when the local uuid does not match the uuid
> in
> >>> the
> >>> > > > >> request,
> >>> > > > >> > > > right?
> >>> > > > >> > > > > > In
> >>> > > > >> > > > > > > > this
> >>> > > > >> > > > > > > > >> case, the
> >>> > > > >> > > > > > > > >> local log is staged for deletion.
> >>> > > > >> > > > > > > > >>
> >>> > > > >> > > > > > > > >> 3. In the documentation of the `
> >>> > > > >> delete.stale.topic.delay.ms
> >>> > > > >> > `
> >>> > > > >> > > > > > config,
> >>> > > > >> > > > > > > it
> >>> > > > >> > > > > > > > >> says "When a
> >>> > > > >> > > > > > > > >> FULL LeaderAndIsrRequest is received..." but
> I
> >>> > > suppose
> >>> > > > it
> >>> > > > >> > > > applies
> >>> > > > >> > > > > to
> >>> > > > >> > > > > > > > both
> >>> > > > >> > > > > > > > >> types.
> >>> > > > >> > > > > > > > >>
> >>> > > > >> > > > > > > > >> Best,
> >>> > > > >> > > > > > > > >> David
> >>> > > > >> > > > > > > > >>
> >>> > > > >> > > > > > > > >> On Thu, Sep 24, 2020 at 1:40 AM Justine
> Olshan
> >>> <
> >>> > > > >> > > > > > jolshan@confluent.io>
> >>> > > > >> > > > > > > > >> wrote:
> >>> > > > >> > > > > > > > >>
> >>> > > > >> > > > > > > > >> > Hi Jun,
> >>> > > > >> > > > > > > > >> >
> >>> > > > >> > > > > > > > >> > Thanks for the comments. I apologize for
> >>> some of
> >>> > > the
> >>> > > > >> typos
> >>> > > > >> > > and
> >>> > > > >> > > > > > > > >> confusion.
> >>> > > > >> > > > > > > > >> > I’ve updated the KIP to fix some of the
> >>> issues
> >>> > you
> >>> > > > >> > > mentioned.
> >>> > > > >> > > > > > > > >> >
> >>> > > > >> > > > > > > > >> > 20.2 I’ve changed the type to String.
> >>> > > > >> > > > > > > > >> > 20.1/3 I’ve updated the TopicZNode to fix
> >>> > > formatting
> >>> > > > >> and
> >>> > > > >> > > > reflect
> >>> > > > >> > > > > > the
> >>> > > > >> > > > > > > > >> latest
> >>> > > > >> > > > > > > > >> > version before this change.
> >>> > > > >> > > > > > > > >> >
> >>> > > > >> > > > > > > > >> > 21. You are correct and I’ve removed this
> >>> line.
> >>> > > I’ve
> >>> > > > >> also
> >>> > > > >> > > > added
> >>> > > > >> > > > > a
> >>> > > > >> > > > > > > line
> >>> > > > >> > > > > > > > >> > mentioning an IBP bump is necessary for
> >>> migration
> >>> > > > >> > > > > > > > >> >
> >>> > > > >> > > > > > > > >> > 22. I think the wording was unclear but
> your
> >>> > > summary
> >>> > > > is
> >>> > > > >> > what
> >>> > > > >> > > > was
> >>> > > > >> > > > > > > > >> intended
> >>> > > > >> > > > > > > > >> > by this line. I’ve updated to make this
> point
> >>> > more
> >>> > > > >> clear.
> >>> > > > >> > > > > “Remove
> >>> > > > >> > > > > > > > >> deleted
> >>> > > > >> > > > > > > > >> > topics from replicas by sending
> >>> > StopReplicaRequest
> >>> > > V3
> >>> > > > >> > before
> >>> > > > >> > > > the
> >>> > > > >> > > > > > IBP
> >>> > > > >> > > > > > > > >> bump
> >>> > > > >> > > > > > > > >> > using the old logic, and using V4 and the
> new
> >>> > logic
> >>> > > > >> with
> >>> > > > >> > > topic
> >>> > > > >> > > > > IDs
> >>> > > > >> > > > > > > > after
> >>> > > > >> > > > > > > > >> > the IBP bump.”
> >>> > > > >> > > > > > > > >> >
> >>> > > > >> > > > > > > > >> > 23. I’ve removed the unspecified type from
> >>> the
> >>> > KIP
> >>> > > > and
> >>> > > > >> > > mention
> >>> > > > >> > > > > > that
> >>> > > > >> > > > > > > > IBP
> >>> > > > >> > > > > > > > >> > will be used to handle this request. “IBP
> >>> will be
> >>> > > > used
> >>> > > > >> to
> >>> > > > >> > > > > > determine
> >>> > > > >> > > > > > > > >> whether
> >>> > > > >> > > > > > > > >> > this new form of the request will be used.
> >>> For
> >>> > > older
> >>> > > > >> > > requests,
> >>> > > > >> > > > > we
> >>> > > > >> > > > > > > will
> >>> > > > >> > > > > > > > >> > ignore this field and default to previous
> >>> > > behavior.”
> >>> > > > >> > > > > > > > >> >
> >>> > > > >> > > > > > > > >> > 24. I’ve fixed this typo.
> >>> > > > >> > > > > > > > >> >
> >>> > > > >> > > > > > > > >> > 25. I've added a topics at a higher level
> for
> >>> > > > >> > > > > LeaderAndIsrResponse
> >>> > > > >> > > > > > > v5,
> >>> > > > >> > > > > > > > >> > StopReplicaResponse v4. I've also changed
> >>> > > > >> > StopReplicaRequest
> >>> > > > >> > > > v4
> >>> > > > >> > > > > to
> >>> > > > >> > > > > > > > have
> >>> > > > >> > > > > > > > >> > topics at a higher level.
> >>> > > > >> > > > > > > > >> >
> >>> > > > >> > > > > > > > >> > 26. I’ve updated
> >>> forgotten_topics_data--added the
> >>> > > > >> topic ID
> >>> > > > >> > > and
> >>> > > > >> > > > > > > removed
> >>> > > > >> > > > > > > > >> the
> >>> > > > >> > > > > > > > >> > topic name
> >>> > > > >> > > > > > > > >> >
> >>> > > > >> > > > > > > > >> > 27. I’ve decided on plain text, and I’ve
> >>> added an
> >>> > > > >> example.
> >>> > > > >> > > > > > > > >> >
> >>> > > > >> > > > > > > > >> > 28. I’ve added this idea to future work.
> >>> > > > >> > > > > > > > >> >
> >>> > > > >> > > > > > > > >> > Thanks again for taking a look,
> >>> > > > >> > > > > > > > >> >
> >>> > > > >> > > > > > > > >> > Justine
> >>> > > > >> > > > > > > > >> >
> >>> > > > >> > > > > > > > >> > On Wed, Sep 23, 2020 at 10:28 AM Jun Rao <
> >>> > > > >> > jun@confluent.io>
> >>> > > > >> > > > > > wrote:
> >>> > > > >> > > > > > > > >> >
> >>> > > > >> > > > > > > > >> > > Hi, Justine,
> >>> > > > >> > > > > > > > >> > >
> >>> > > > >> > > > > > > > >> > > Thanks for the response. Made another
> >>> pass. A
> >>> > few
> >>> > > > >> more
> >>> > > > >> > > > > comments
> >>> > > > >> > > > > > > > below.
> >>> > > > >> > > > > > > > >> > >
> >>> > > > >> > > > > > > > >> > > 20. znode schema:
> >>> > > > >> > > > > > > > >> > > 20.1 It seems that {"name": "version",
> >>> "type":
> >>> > > > "int",
> >>> > > > >> > > "id":
> >>> > > > >> > > > > > > "UUID",
> >>> > > > >> > > > > > > > >> > "doc":
> >>> > > > >> > > > > > > > >> > > "version id"} should be {"name":
> "version",
> >>> > > "type":
> >>> > > > >> > > "int"},
> >>> > > > >> > > > > > > {"name":
> >>> > > > >> > > > > > > > >> > "id",
> >>> > > > >> > > > > > > > >> > > "type": "UUID", "doc": "version id"}.
> >>> > > > >> > > > > > > > >> > > 20.2 The znode format is JSON which
> doesn't
> >>> > have
> >>> > > > UUID
> >>> > > > >> > > type.
> >>> > > > >> > > > So
> >>> > > > >> > > > > > the
> >>> > > > >> > > > > > > > >> type
> >>> > > > >> > > > > > > > >> > > probably should be string?
> >>> > > > >> > > > > > > > >> > > 20.3 Also, the existing format used seems
> >>> > > outdated.
> >>> > > > >> It
> >>> > > > >> > > > should
> >>> > > > >> > > > > > have
> >>> > > > >> > > > > > > > the
> >>> > > > >> > > > > > > > >> > > following format.
> >>> > > > >> > > > > > > > >> > >     Json.encodeAsBytes(Map(
> >>> > > > >> > > > > > > > >> > >       "version" -> 2,
> >>> > > > >> > > > > > > > >> > >       "partitions" ->
> >>> > > replicaAssignmentJson.asJava,
> >>> > > > >> > > > > > > > >> > >       "adding_replicas" ->
> >>> > > > >> > > > > addingReplicasAssignmentJson.asJava,
> >>> > > > >> > > > > > > > >> > >       "removing_replicas" ->
> >>> > > > >> > > > > > removingReplicasAssignmentJson.asJava
> >>> > > > >> > > > > > > > >> > >     ).asJava)
> >>> > > > >> > > > > > > > >> > >   }
> >>> > > > >> > > > > > > > >> > >
> >>> > > > >> > > > > > > > >> > > 21. Migration: The KIP says "The
> migration
> >>> > > process
> >>> > > > >> can
> >>> > > > >> > > take
> >>> > > > >> > > > > > place
> >>> > > > >> > > > > > > > >> without
> >>> > > > >> > > > > > > > >> > > an inter-broker protocol bump, as the
> >>> format
> >>> > > stored
> >>> > > > >> in
> >>> > > > >> > > > > > > > >> > > /brokers/topics/[topic] will be
> compatible
> >>> with
> >>> > > > older
> >>> > > > >> > > broker
> >>> > > > >> > > > > > > > >> versions."
> >>> > > > >> > > > > > > > >> > > However, since we are bumping up the
> >>> version of
> >>> > > > >> > > inter-broker
> >>> > > > >> > > > > > > > >> requests, it
> >>> > > > >> > > > > > > > >> > > seems that we need to use IBP for
> >>> migration.
> >>> > > > >> > > > > > > > >> > >
> >>> > > > >> > > > > > > > >> > > 22. The KIP says "Remove deleted topics
> >>> from
> >>> > > > >> replicas by
> >>> > > > >> > > > > sending
> >>> > > > >> > > > > > > > >> > > StopReplicaRequest V3 for any topics
> which
> >>> do
> >>> > not
> >>> > > > >> > contain
> >>> > > > >> > > a
> >>> > > > >> > > > > > topic
> >>> > > > >> > > > > > > > ID,
> >>> > > > >> > > > > > > > >> and
> >>> > > > >> > > > > > > > >> > > V4 for any topics which do contain a
> topic
> >>> > ID.".
> >>> > > > >> > However,
> >>> > > > >> > > if
> >>> > > > >> > > > > we
> >>> > > > >> > > > > > > use
> >>> > > > >> > > > > > > > >> IBP,
> >>> > > > >> > > > > > > > >> > it
> >>> > > > >> > > > > > > > >> > > seems that the controller will either
> send
> >>> > > > >> > > > StopReplicaRequest
> >>> > > > >> > > > > V3
> >>> > > > >> > > > > > > > >> > > or StopReplicaRequest V4, but never mixed
> >>> V3
> >>> > and
> >>> > > V4
> >>> > > > >> for
> >>> > > > >> > > > > > different
> >>> > > > >> > > > > > > > >> topics.
> >>> > > > >> > > > > > > > >> > > Basically, before the IBP bump, V3 will
> be
> >>> > used.
> >>> > > > >> After
> >>> > > > >> > the
> >>> > > > >> > > > IBP
> >>> > > > >> > > > > > > bump,
> >>> > > > >> > > > > > > > >> > > topicId will be created and V4 will be
> >>> used.
> >>> > > > >> > > > > > > > >> > >
> >>> > > > >> > > > > > > > >> > > 23. Given that we depend on IBP, do we
> >>> still
> >>> > need
> >>> > > > "0
> >>> > > > >> > > > > > UNSPECIFIED"
> >>> > > > >> > > > > > > > >> > > in LeaderAndIsr?
> >>> > > > >> > > > > > > > >> > >
> >>> > > > >> > > > > > > > >> > > 24. LeaderAndIsrResponse v5 : It still
> has
> >>> the
> >>> > > > topic
> >>> > > > >> > > field.
> >>> > > > >> > > > > > > > >> > >
> >>> > > > >> > > > > > > > >> > > 25. LeaderAndIsrResponse v5,
> >>> > StopReplicaResponse
> >>> > > > v4:
> >>> > > > >> > Could
> >>> > > > >> > > > we
> >>> > > > >> > > > > > use
> >>> > > > >> > > > > > > > this
> >>> > > > >> > > > > > > > >> > > opportunity to organize the response in 2
> >>> > levels,
> >>> > > > >> first
> >>> > > > >> > by
> >>> > > > >> > > > > > topic,
> >>> > > > >> > > > > > > > >> then by
> >>> > > > >> > > > > > > > >> > > partition, as most other
> >>> requests/responses?
> >>> > > > >> > > > > > > > >> > >
> >>> > > > >> > > > > > > > >> > > 26. FetchRequest v13 : Should
> >>> > > forgotten_topics_data
> >>> > > > >> use
> >>> > > > >> > > > > topicId
> >>> > > > >> > > > > > > too?
> >>> > > > >> > > > > > > > >> > >
> >>> > > > >> > > > > > > > >> > > 27. "This file can either be plain text
> >>> > > (key/value
> >>> > > > >> > pairs)
> >>> > > > >> > > or
> >>> > > > >> > > > > > > JSON."
> >>> > > > >> > > > > > > > >> Have
> >>> > > > >> > > > > > > > >> > we
> >>> > > > >> > > > > > > > >> > > decided which one to use? Also, it would
> be
> >>> > > helpful
> >>> > > > >> to
> >>> > > > >> > > > provide
> >>> > > > >> > > > > > an
> >>> > > > >> > > > > > > > >> > example.
> >>> > > > >> > > > > > > > >> > >
> >>> > > > >> > > > > > > > >> > > 28. Future improvement: Another future
> >>> > > improvement
> >>> > > > >> > > > opportunity
> >>> > > > >> > > > > > is
> >>> > > > >> > > > > > > to
> >>> > > > >> > > > > > > > >> use
> >>> > > > >> > > > > > > > >> > > topicId in
> >>> GroupMetadataManager.offsetCommitKey
> >>> > > in
> >>> > > > >> the
> >>> > > > >> > > > > > > offset_commit
> >>> > > > >> > > > > > > > >> > topic.
> >>> > > > >> > > > > > > > >> > > This may save some space.
> >>> > > > >> > > > > > > > >> > >
> >>> > > > >> > > > > > > > >> > > Thanks,
> >>> > > > >> > > > > > > > >> > >
> >>> > > > >> > > > > > > > >> > > Jun
> >>> > > > >> > > > > > > > >> > >
> >>> > > > >> > > > > > > > >> > > On Wed, Sep 23, 2020 at 8:50 AM Justine
> >>> Olshan
> >>> > <
> >>> > > > >> > > > > > > > jolshan@confluent.io>
> >>> > > > >> > > > > > > > >> > > wrote:
> >>> > > > >> > > > > > > > >> > >
> >>> > > > >> > > > > > > > >> > > > Hi Tom,
> >>> > > > >> > > > > > > > >> > > >
> >>> > > > >> > > > > > > > >> > > > Thanks for the comment. I think this
> is a
> >>> > > really
> >>> > > > >> good
> >>> > > > >> > > idea
> >>> > > > >> > > > > and
> >>> > > > >> > > > > > > it
> >>> > > > >> > > > > > > > >> has
> >>> > > > >> > > > > > > > >> > > been
> >>> > > > >> > > > > > > > >> > > > added to the KIP under the newly added
> >>> > tooling
> >>> > > > >> > section.
> >>> > > > >> > > > > > > > >> > > >
> >>> > > > >> > > > > > > > >> > > > Thanks again,
> >>> > > > >> > > > > > > > >> > > > Justine
> >>> > > > >> > > > > > > > >> > > >
> >>> > > > >> > > > > > > > >> > > > On Wed, Sep 23, 2020 at 3:17 AM Tom
> >>> Bentley <
> >>> > > > >> > > > > > > tbentley@redhat.com>
> >>> > > > >> > > > > > > > >> > wrote:
> >>> > > > >> > > > > > > > >> > > >
> >>> > > > >> > > > > > > > >> > > > > Hi Justine,
> >>> > > > >> > > > > > > > >> > > > >
> >>> > > > >> > > > > > > > >> > > > > I know you started the vote thread,
> >>> but on
> >>> > > > >> > re-reading
> >>> > > > >> > > > the
> >>> > > > >> > > > > > KIP
> >>> > > > >> > > > > > > I
> >>> > > > >> > > > > > > > >> > noticed
> >>> > > > >> > > > > > > > >> > > > > that although the topic id is
> included
> >>> in
> >>> > the
> >>> > > > >> > > > > > MetadataResponse
> >>> > > > >> > > > > > > > >> it's
> >>> > > > >> > > > > > > > >> > not
> >>> > > > >> > > > > > > > >> > > > > surfaced in the output from
> >>> > `kafka-topics.sh
> >>> > > > >> > > > --describe`.
> >>> > > > >> > > > > > > Maybe
> >>> > > > >> > > > > > > > >> that
> >>> > > > >> > > > > > > > >> > > was
> >>> > > > >> > > > > > > > >> > > > > intentional because ids are
> >>> intentionally
> >>> > not
> >>> > > > >> really
> >>> > > > >> > > > > > something
> >>> > > > >> > > > > > > > the
> >>> > > > >> > > > > > > > >> > user
> >>> > > > >> > > > > > > > >> > > > > should care deeply about, but it
> would
> >>> also
> >>> > > > make
> >>> > > > >> > life
> >>> > > > >> > > > > harder
> >>> > > > >> > > > > > > for
> >>> > > > >> > > > > > > > >> > anyone
> >>> > > > >> > > > > > > > >> > > > > debugging Kafka and this would likely
> >>> get
> >>> > > worse
> >>> > > > >> the
> >>> > > > >> > > more
> >>> > > > >> > > > > > topic
> >>> > > > >> > > > > > > > ids
> >>> > > > >> > > > > > > > >> > got
> >>> > > > >> > > > > > > > >> > > > > rolled out across the protocols,
> >>> clients
> >>> > etc.
> >>> > > > It
> >>> > > > >> > seems
> >>> > > > >> > > > > > likely
> >>> > > > >> > > > > > > > that
> >>> > > > >> > > > > > > > >> > > > > `kafka-topics.sh` will eventually
> need
> >>> the
> >>> > > > >> ability
> >>> > > > >> > to
> >>> > > > >> > > > show
> >>> > > > >> > > > > > the
> >>> > > > >> > > > > > > > id
> >>> > > > >> > > > > > > > >> of
> >>> > > > >> > > > > > > > >> > a
> >>> > > > >> > > > > > > > >> > > > > topic and perhaps find a topic name
> >>> given
> >>> > an
> >>> > > > id.
> >>> > > > >> Is
> >>> > > > >> > > > there
> >>> > > > >> > > > > > any
> >>> > > > >> > > > > > > > >> reason
> >>> > > > >> > > > > > > > >> > > not
> >>> > > > >> > > > > > > > >> > > > to
> >>> > > > >> > > > > > > > >> > > > > implement that in this KIP?
> >>> > > > >> > > > > > > > >> > > > >
> >>> > > > >> > > > > > > > >> > > > > Many thanks,
> >>> > > > >> > > > > > > > >> > > > >
> >>> > > > >> > > > > > > > >> > > > > Tom
> >>> > > > >> > > > > > > > >> > > > >
> >>> > > > >> > > > > > > > >> > > > > On Mon, Sep 21, 2020 at 9:54 PM
> Justine
> >>> > > Olshan
> >>> > > > <
> >>> > > > >> > > > > > > > >> jolshan@confluent.io
> >>> > > > >> > > > > > > > >> > >
> >>> > > > >> > > > > > > > >> > > > > wrote:
> >>> > > > >> > > > > > > > >> > > > >
> >>> > > > >> > > > > > > > >> > > > > > Hi all,
> >>> > > > >> > > > > > > > >> > > > > >
> >>> > > > >> > > > > > > > >> > > > > > After thinking about it, I've
> >>> decided to
> >>> > > > remove
> >>> > > > >> > the
> >>> > > > >> > > > > topic
> >>> > > > >> > > > > > > name
> >>> > > > >> > > > > > > > >> from
> >>> > > > >> > > > > > > > >> > > the
> >>> > > > >> > > > > > > > >> > > > > > Fetch Request and Response after
> all.
> >>> > Since
> >>> > > > >> there
> >>> > > > >> > > are
> >>> > > > >> > > > so
> >>> > > > >> > > > > > > many
> >>> > > > >> > > > > > > > of
> >>> > > > >> > > > > > > > >> > > these
> >>> > > > >> > > > > > > > >> > > > > > requests per second, it is worth
> >>> removing
> >>> > > the
> >>> > > > >> > extra
> >>> > > > >> > > > > > > > information.
> >>> > > > >> > > > > > > > >> > I've
> >>> > > > >> > > > > > > > >> > > > > > updated the KIP to reflect this
> >>> change.
> >>> > > > >> > > > > > > > >> > > > > >
> >>> > > > >> > > > > > > > >> > > > > > Please let me know if there is
> >>> anything
> >>> > > else
> >>> > > > we
> >>> > > > >> > > should
> >>> > > > >> > > > > > > discuss
> >>> > > > >> > > > > > > > >> > before
> >>> > > > >> > > > > > > > >> > > > > > voting.
> >>> > > > >> > > > > > > > >> > > > > >
> >>> > > > >> > > > > > > > >> > > > > > Thank you,
> >>> > > > >> > > > > > > > >> > > > > > Justine
> >>> > > > >> > > > > > > > >> > > > > >
> >>> > > > >> > > > > > > > >> > > > > > On Fri, Sep 18, 2020 at 9:46 AM
> >>> Justine
> >>> > > > Olshan
> >>> > > > >> <
> >>> > > > >> > > > > > > > >> > jolshan@confluent.io
> >>> > > > >> > > > > > > > >> > > >
> >>> > > > >> > > > > > > > >> > > > > > wrote:
> >>> > > > >> > > > > > > > >> > > > > >
> >>> > > > >> > > > > > > > >> > > > > > > Hi Jun,
> >>> > > > >> > > > > > > > >> > > > > > >
> >>> > > > >> > > > > > > > >> > > > > > > I see what you are saying. For
> now
> >>> we
> >>> > can
> >>> > > > >> remove
> >>> > > > >> > > the
> >>> > > > >> > > > > > extra
> >>> > > > >> > > > > > > > >> > > > information.
> >>> > > > >> > > > > > > > >> > > > > > > I'll leave the option to add more
> >>> > fields
> >>> > > to
> >>> > > > >> the
> >>> > > > >> > > file
> >>> > > > >> > > > > in
> >>> > > > >> > > > > > > the
> >>> > > > >> > > > > > > > >> > future.
> >>> > > > >> > > > > > > > >> > > > The
> >>> > > > >> > > > > > > > >> > > > > > KIP
> >>> > > > >> > > > > > > > >> > > > > > > has been updated to reflect this
> >>> > change.
> >>> > > > >> > > > > > > > >> > > > > > >
> >>> > > > >> > > > > > > > >> > > > > > > Thanks,
> >>> > > > >> > > > > > > > >> > > > > > > Justine
> >>> > > > >> > > > > > > > >> > > > > > >
> >>> > > > >> > > > > > > > >> > > > > > > On Fri, Sep 18, 2020 at 8:46 AM
> Jun
> >>> > Rao <
> >>> > > > >> > > > > > jun@confluent.io
> >>> > > > >> > > > > > > >
> >>> > > > >> > > > > > > > >> > wrote:
> >>> > > > >> > > > > > > > >> > > > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> Hi, Justine,
> >>> > > > >> > > > > > > > >> > > > > > >>
> >>> > > > >> > > > > > > > >> > > > > > >> Thanks for the reply.
> >>> > > > >> > > > > > > > >> > > > > > >>
> >>> > > > >> > > > > > > > >> > > > > > >> 13. If the log directory is the
> >>> source
> >>> > > of
> >>> > > > >> > truth,
> >>> > > > >> > > it
> >>> > > > >> > > > > > means
> >>> > > > >> > > > > > > > >> that
> >>> > > > >> > > > > > > > >> > the
> >>> > > > >> > > > > > > > >> > > > > > >> redundant info in the metadata
> >>> file
> >>> > will
> >>> > > > be
> >>> > > > >> > > > ignored.
> >>> > > > >> > > > > > Then
> >>> > > > >> > > > > > > > the
> >>> > > > >> > > > > > > > >> > > > question
> >>> > > > >> > > > > > > > >> > > > > > is
> >>> > > > >> > > > > > > > >> > > > > > >> why do we need to put the
> >>> redundant
> >>> > info
> >>> > > > in
> >>> > > > >> the
> >>> > > > >> > > > > > metadata
> >>> > > > >> > > > > > > > file
> >>> > > > >> > > > > > > > >> > now?
> >>> > > > >> > > > > > > > >> > > > > > >>
> >>> > > > >> > > > > > > > >> > > > > > >> Thanks,
> >>> > > > >> > > > > > > > >> > > > > > >>
> >>> > > > >> > > > > > > > >> > > > > > >> Jun
> >>> > > > >> > > > > > > > >> > > > > > >>
> >>> > > > >> > > > > > > > >> > > > > > >> On Thu, Sep 17, 2020 at 5:07 PM
> >>> > Justine
> >>> > > > >> Olshan
> >>> > > > >> > <
> >>> > > > >> > > > > > > > >> > > > jolshan@confluent.io>
> >>> > > > >> > > > > > > > >> > > > > > >> wrote:
> >>> > > > >> > > > > > > > >> > > > > > >>
> >>> > > > >> > > > > > > > >> > > > > > >> > Hi Jun,
> >>> > > > >> > > > > > > > >> > > > > > >> > Thanks for the quick response!
> >>> > > > >> > > > > > > > >> > > > > > >> >
> >>> > > > >> > > > > > > > >> > > > > > >> > 12. I've decided to bump up
> the
> >>> > > versions
> >>> > > > >> on
> >>> > > > >> > the
> >>> > > > >> > > > > > > requests
> >>> > > > >> > > > > > > > >> and
> >>> > > > >> > > > > > > > >> > > > updated
> >>> > > > >> > > > > > > > >> > > > > > the
> >>> > > > >> > > > > > > > >> > > > > > >> > KIP. I think it's good we
> >>> thoroughly
> >>> > > > >> > discussed
> >>> > > > >> > > > the
> >>> > > > >> > > > > > > > options
> >>> > > > >> > > > > > > > >> > here,
> >>> > > > >> > > > > > > > >> > > > so
> >>> > > > >> > > > > > > > >> > > > > we
> >>> > > > >> > > > > > > > >> > > > > > >> know
> >>> > > > >> > > > > > > > >> > > > > > >> > we made a good choice. :)
> >>> > > > >> > > > > > > > >> > > > > > >> >
> >>> > > > >> > > > > > > > >> > > > > > >> > 13. This is an interesting
> >>> > situation.
> >>> > > I
> >>> > > > >> think
> >>> > > > >> > > if
> >>> > > > >> > > > > this
> >>> > > > >> > > > > > > > does
> >>> > > > >> > > > > > > > >> > occur
> >>> > > > >> > > > > > > > >> > > > we
> >>> > > > >> > > > > > > > >> > > > > > >> should
> >>> > > > >> > > > > > > > >> > > > > > >> > give a warning. I agree that
> >>> it's
> >>> > hard
> >>> > > > to
> >>> > > > >> > know
> >>> > > > >> > > > the
> >>> > > > >> > > > > > > source
> >>> > > > >> > > > > > > > >> of
> >>> > > > >> > > > > > > > >> > > truth
> >>> > > > >> > > > > > > > >> > > > > for
> >>> > > > >> > > > > > > > >> > > > > > >> sure
> >>> > > > >> > > > > > > > >> > > > > > >> > since the directory or the
> file
> >>> > could
> >>> > > be
> >>> > > > >> > > manually
> >>> > > > >> > > > > > > > >> modified. I
> >>> > > > >> > > > > > > > >> > > > guess
> >>> > > > >> > > > > > > > >> > > > > > the
> >>> > > > >> > > > > > > > >> > > > > > >> > directory could be used as the
> >>> > source
> >>> > > of
> >>> > > > >> > truth.
> >>> > > > >> > > > To
> >>> > > > >> > > > > be
> >>> > > > >> > > > > > > > >> honest,
> >>> > > > >> > > > > > > > >> > > I'm
> >>> > > > >> > > > > > > > >> > > > > not
> >>> > > > >> > > > > > > > >> > > > > > >> > really sure what happens in
> >>> kafka
> >>> > when
> >>> > > > the
> >>> > > > >> > log
> >>> > > > >> > > > > > > directory
> >>> > > > >> > > > > > > > is
> >>> > > > >> > > > > > > > >> > > > renamed
> >>> > > > >> > > > > > > > >> > > > > > >> > manually in such a way. I'm
> also
> >>> > > > >> wondering if
> >>> > > > >> > > the
> >>> > > > >> > > > > > > > >> situation is
> >>> > > > >> > > > > > > > >> > > > > > >> recoverable
> >>> > > > >> > > > > > > > >> > > > > > >> > in this scenario.
> >>> > > > >> > > > > > > > >> > > > > > >> >
> >>> > > > >> > > > > > > > >> > > > > > >> > Thanks,
> >>> > > > >> > > > > > > > >> > > > > > >> > Justine
> >>> > > > >> > > > > > > > >> > > > > > >> >
> >>> > > > >> > > > > > > > >> > > > > > >> > On Thu, Sep 17, 2020 at 4:28
> PM
> >>> Jun
> >>> > > Rao
> >>> > > > <
> >>> > > > >> > > > > > > > jun@confluent.io>
> >>> > > > >> > > > > > > > >> > > wrote:
> >>> > > > >> > > > > > > > >> > > > > > >> >
> >>> > > > >> > > > > > > > >> > > > > > >> > > Hi, Justine,
> >>> > > > >> > > > > > > > >> > > > > > >> > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > Thanks for the reply.
> >>> > > > >> > > > > > > > >> > > > > > >> > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > 12. I don't have a strong
> >>> > preference
> >>> > > > >> > either.
> >>> > > > >> > > > > > However,
> >>> > > > >> > > > > > > > if
> >>> > > > >> > > > > > > > >> we
> >>> > > > >> > > > > > > > >> > > need
> >>> > > > >> > > > > > > > >> > > > > IBP
> >>> > > > >> > > > > > > > >> > > > > > >> > > anyway, maybe it's easier to
> >>> just
> >>> > > bump
> >>> > > > >> up
> >>> > > > >> > the
> >>> > > > >> > > > > > version
> >>> > > > >> > > > > > > > for
> >>> > > > >> > > > > > > > >> > all
> >>> > > > >> > > > > > > > >> > > > > inter
> >>> > > > >> > > > > > > > >> > > > > > >> > broker
> >>> > > > >> > > > > > > > >> > > > > > >> > > requests and add the topic
> id
> >>> > field
> >>> > > > as a
> >>> > > > >> > > > regular
> >>> > > > >> > > > > > > > field. A
> >>> > > > >> > > > > > > > >> > > > regular
> >>> > > > >> > > > > > > > >> > > > > > >> field
> >>> > > > >> > > > > > > > >> > > > > > >> > is
> >>> > > > >> > > > > > > > >> > > > > > >> > > a bit more concise in wire
> >>> > transfer
> >>> > > > >> than a
> >>> > > > >> > > > > flexible
> >>> > > > >> > > > > > > > >> field.
> >>> > > > >> > > > > > > > >> > > > > > >> > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > 13. The confusion that I was
> >>> > > referring
> >>> > > > >> to
> >>> > > > >> > is
> >>> > > > >> > > > > > between
> >>> > > > >> > > > > > > > the
> >>> > > > >> > > > > > > > >> > topic
> >>> > > > >> > > > > > > > >> > > > > name
> >>> > > > >> > > > > > > > >> > > > > > >> and
> >>> > > > >> > > > > > > > >> > > > > > >> > > partition number between the
> >>> log
> >>> > dir
> >>> > > > and
> >>> > > > >> > the
> >>> > > > >> > > > > > metadata
> >>> > > > >> > > > > > > > >> file.
> >>> > > > >> > > > > > > > >> > > For
> >>> > > > >> > > > > > > > >> > > > > > >> example,
> >>> > > > >> > > > > > > > >> > > > > > >> > if
> >>> > > > >> > > > > > > > >> > > > > > >> > > the log dir is topicA-1 and
> >>> the
> >>> > > > metadata
> >>> > > > >> > file
> >>> > > > >> > > > in
> >>> > > > >> > > > > it
> >>> > > > >> > > > > > > has
> >>> > > > >> > > > > > > > >> > topicB
> >>> > > > >> > > > > > > > >> > > > and
> >>> > > > >> > > > > > > > >> > > > > > >> > > partition 0 (say due to a
> bug
> >>> or
> >>> > > > manual
> >>> > > > >> > > > > > > modification),
> >>> > > > >> > > > > > > > >> which
> >>> > > > >> > > > > > > > >> > > one
> >>> > > > >> > > > > > > > >> > > > > do
> >>> > > > >> > > > > > > > >> > > > > > we
> >>> > > > >> > > > > > > > >> > > > > > >> > use
> >>> > > > >> > > > > > > > >> > > > > > >> > > as the source of truth?
> >>> > > > >> > > > > > > > >> > > > > > >> > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > Jun
> >>> > > > >> > > > > > > > >> > > > > > >> > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > On Thu, Sep 17, 2020 at 3:43
> >>> PM
> >>> > > > Justine
> >>> > > > >> > > Olshan
> >>> > > > >> > > > <
> >>> > > > >> > > > > > > > >> > > > > > jolshan@confluent.io>
> >>> > > > >> > > > > > > > >> > > > > > >> > > wrote:
> >>> > > > >> > > > > > > > >> > > > > > >> > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > Hi Jun,
> >>> > > > >> > > > > > > > >> > > > > > >> > > > Thanks for the comments.
> >>> > > > >> > > > > > > > >> > > > > > >> > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > 12. I bumped the
> >>> > > LeaderAndIsrRequest
> >>> > > > >> > > because
> >>> > > > >> > > > I
> >>> > > > >> > > > > > > > removed
> >>> > > > >> > > > > > > > >> the
> >>> > > > >> > > > > > > > >> > > > topic
> >>> > > > >> > > > > > > > >> > > > > > >> name
> >>> > > > >> > > > > > > > >> > > > > > >> > > field
> >>> > > > >> > > > > > > > >> > > > > > >> > > > in the response. It may be
> >>> > > possible
> >>> > > > to
> >>> > > > >> > > avoid
> >>> > > > >> > > > > > > bumping
> >>> > > > >> > > > > > > > >> the
> >>> > > > >> > > > > > > > >> > > > version
> >>> > > > >> > > > > > > > >> > > > > > >> > without
> >>> > > > >> > > > > > > > >> > > > > > >> > > > that change, but I may be
> >>> > missing
> >>> > > > >> > > something.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > I believe StopReplica is
> >>> > actually
> >>> > > on
> >>> > > > >> > > version
> >>> > > > >> > > > 3
> >>> > > > >> > > > > > now,
> >>> > > > >> > > > > > > > but
> >>> > > > >> > > > > > > > >> > > > because
> >>> > > > >> > > > > > > > >> > > > > > >> > version 2
> >>> > > > >> > > > > > > > >> > > > > > >> > > > is flexible, I kept that
> >>> listed
> >>> > as
> >>> > > > >> > version
> >>> > > > >> > > 2
> >>> > > > >> > > > on
> >>> > > > >> > > > > > the
> >>> > > > >> > > > > > > > KIP
> >>> > > > >> > > > > > > > >> > > page.
> >>> > > > >> > > > > > > > >> > > > > > >> However,
> >>> > > > >> > > > > > > > >> > > > > > >> > > you
> >>> > > > >> > > > > > > > >> > > > > > >> > > > may be right in that we
> may
> >>> need
> >>> > > to
> >>> > > > >> bump
> >>> > > > >> > > the
> >>> > > > >> > > > > > > version
> >>> > > > >> > > > > > > > on
> >>> > > > >> > > > > > > > >> > > > > > StopReplica
> >>> > > > >> > > > > > > > >> > > > > > >> to
> >>> > > > >> > > > > > > > >> > > > > > >> > > deal
> >>> > > > >> > > > > > > > >> > > > > > >> > > > with deletion differently
> as
> >>> > > > mentioned
> >>> > > > >> > > > above. I
> >>> > > > >> > > > > > > don't
> >>> > > > >> > > > > > > > >> know
> >>> > > > >> > > > > > > > >> > > if
> >>> > > > >> > > > > > > > >> > > > I
> >>> > > > >> > > > > > > > >> > > > > > >> have a
> >>> > > > >> > > > > > > > >> > > > > > >> > > big
> >>> > > > >> > > > > > > > >> > > > > > >> > > > preference over used
> tagged
> >>> > fields
> >>> > > > or
> >>> > > > >> > not.
> >>> > > > >> > > > > > > > >> > > > > > >> > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > 13. I was thinking that in
> >>> the
> >>> > > case
> >>> > > > >> where
> >>> > > > >> > > the
> >>> > > > >> > > > > > file
> >>> > > > >> > > > > > > > and
> >>> > > > >> > > > > > > > >> the
> >>> > > > >> > > > > > > > >> > > > > request
> >>> > > > >> > > > > > > > >> > > > > > >> > topic
> >>> > > > >> > > > > > > > >> > > > > > >> > > > ids don't match, it means
> >>> that
> >>> > the
> >>> > > > >> > broker's
> >>> > > > >> > > > > > > topic/the
> >>> > > > >> > > > > > > > >> one
> >>> > > > >> > > > > > > > >> > in
> >>> > > > >> > > > > > > > >> > > > the
> >>> > > > >> > > > > > > > >> > > > > > >> file
> >>> > > > >> > > > > > > > >> > > > > > >> > has
> >>> > > > >> > > > > > > > >> > > > > > >> > > > been deleted. In that
> case,
> >>> we
> >>> > > would
> >>> > > > >> need
> >>> > > > >> > > to
> >>> > > > >> > > > > > delete
> >>> > > > >> > > > > > > > the
> >>> > > > >> > > > > > > > >> > old
> >>> > > > >> > > > > > > > >> > > > > topic
> >>> > > > >> > > > > > > > >> > > > > > >> and
> >>> > > > >> > > > > > > > >> > > > > > >> > > start
> >>> > > > >> > > > > > > > >> > > > > > >> > > > receiving the new version.
> >>> If
> >>> > the
> >>> > > > >> topic
> >>> > > > >> > > name
> >>> > > > >> > > > > were
> >>> > > > >> > > > > > > to
> >>> > > > >> > > > > > > > >> > change,
> >>> > > > >> > > > > > > > >> > > > but
> >>> > > > >> > > > > > > > >> > > > > > the
> >>> > > > >> > > > > > > > >> > > > > > >> > ids
> >>> > > > >> > > > > > > > >> > > > > > >> > > > still matched, the file
> >>> would
> >>> > also
> >>> > > > >> need
> >>> > > > >> > to
> >>> > > > >> > > > > > update.
> >>> > > > >> > > > > > > > Am I
> >>> > > > >> > > > > > > > >> > > > missing
> >>> > > > >> > > > > > > > >> > > > > a
> >>> > > > >> > > > > > > > >> > > > > > >> case
> >>> > > > >> > > > > > > > >> > > > > > >> > > > where the file would be
> >>> correct
> >>> > > and
> >>> > > > >> not
> >>> > > > >> > the
> >>> > > > >> > > > > > > request?
> >>> > > > >> > > > > > > > >> > > > > > >> > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > Thanks,
> >>> > > > >> > > > > > > > >> > > > > > >> > > > Justine
> >>> > > > >> > > > > > > > >> > > > > > >> > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > On Thu, Sep 17, 2020 at
> >>> 3:18 PM
> >>> > > Jun
> >>> > > > >> Rao <
> >>> > > > >> > > > > > > > >> jun@confluent.io
> >>> > > > >> > > > > > > > >> > >
> >>> > > > >> > > > > > > > >> > > > > wrote:
> >>> > > > >> > > > > > > > >> > > > > > >> > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > Hi, Justine,
> >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > Thanks for the reply. A
> >>> couple
> >>> > > of
> >>> > > > >> more
> >>> > > > >> > > > > comments
> >>> > > > >> > > > > > > > >> below.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > 12. ListOffset and
> >>> > > OffsetForLeader
> >>> > > > >> > > > currently
> >>> > > > >> > > > > > > don't
> >>> > > > >> > > > > > > > >> > support
> >>> > > > >> > > > > > > > >> > > > > > >> flexible
> >>> > > > >> > > > > > > > >> > > > > > >> > > > fields.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > So, we have to bump up
> the
> >>> > > version
> >>> > > > >> > number
> >>> > > > >> > > > and
> >>> > > > >> > > > > > use
> >>> > > > >> > > > > > > > >> IBP at
> >>> > > > >> > > > > > > > >> > > > least
> >>> > > > >> > > > > > > > >> > > > > > for
> >>> > > > >> > > > > > > > >> > > > > > >> > > these
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > two requests. Note that
> it
> >>> > seems
> >>> > > > >> 2.7.0
> >>> > > > >> > > will
> >>> > > > >> > > > > > > require
> >>> > > > >> > > > > > > > >> IBP
> >>> > > > >> > > > > > > > >> > > > anyway
> >>> > > > >> > > > > > > > >> > > > > > >> > because
> >>> > > > >> > > > > > > > >> > > > > > >> > > of
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > changes in KAFKA-10435.
> >>> Also,
> >>> > it
> >>> > > > >> seems
> >>> > > > >> > > that
> >>> > > > >> > > > > the
> >>> > > > >> > > > > > > > >> version
> >>> > > > >> > > > > > > > >> > > for
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > LeaderAndIsrRequest and
> >>> > > > StopReplica
> >>> > > > >> are
> >>> > > > >> > > > > bumped
> >>> > > > >> > > > > > > even
> >>> > > > >> > > > > > > > >> > though
> >>> > > > >> > > > > > > > >> > > > we
> >>> > > > >> > > > > > > > >> > > > > > only
> >>> > > > >> > > > > > > > >> > > > > > >> > > added
> >>> > > > >> > > > > > > > >> > > > > > >> > > > a
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > tagged field. But since
> >>> IBP is
> >>> > > > >> needed
> >>> > > > >> > > > anyway,
> >>> > > > >> > > > > > we
> >>> > > > >> > > > > > > > may
> >>> > > > >> > > > > > > > >> > want
> >>> > > > >> > > > > > > > >> > > to
> >>> > > > >> > > > > > > > >> > > > > > >> revisit
> >>> > > > >> > > > > > > > >> > > > > > >> > > the
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > overall tagged field
> >>> choice.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > 13. The only downside is
> >>> the
> >>> > > > >> potential
> >>> > > > >> > > > > > confusion
> >>> > > > >> > > > > > > on
> >>> > > > >> > > > > > > > >> > which
> >>> > > > >> > > > > > > > >> > > > one
> >>> > > > >> > > > > > > > >> > > > > is
> >>> > > > >> > > > > > > > >> > > > > > >> the
> >>> > > > >> > > > > > > > >> > > > > > >> > > > source
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > of truth if they don't
> >>> match.
> >>> > > > >> Another
> >>> > > > >> > > > option
> >>> > > > >> > > > > is
> >>> > > > >> > > > > > > to
> >>> > > > >> > > > > > > > >> > include
> >>> > > > >> > > > > > > > >> > > > > those
> >>> > > > >> > > > > > > > >> > > > > > >> > fields
> >>> > > > >> > > > > > > > >> > > > > > >> > > > in
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > the metadata file when
> we
> >>> > > actually
> >>> > > > >> > change
> >>> > > > >> > > > the
> >>> > > > >> > > > > > > > >> directory
> >>> > > > >> > > > > > > > >> > > > > > structure.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > Thanks,
> >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > Jun
> >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > On Thu, Sep 17, 2020 at
> >>> 2:01
> >>> > PM
> >>> > > > >> Justine
> >>> > > > >> > > > > Olshan
> >>> > > > >> > > > > > <
> >>> > > > >> > > > > > > > >> > > > > > >> jolshan@confluent.io
> >>> > > > >> > > > > > > > >> > > > > > >> > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > wrote:
> >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > Hello all,
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I've thought some more
> >>> about
> >>> > > > >> removing
> >>> > > > >> > > the
> >>> > > > >> > > > > > topic
> >>> > > > >> > > > > > > > >> name
> >>> > > > >> > > > > > > > >> > > field
> >>> > > > >> > > > > > > > >> > > > > > from
> >>> > > > >> > > > > > > > >> > > > > > >> > some
> >>> > > > >> > > > > > > > >> > > > > > >> > > of
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > requests. On closer
> >>> > inspection
> >>> > > > of
> >>> > > > >> the
> >>> > > > >> > > > > > > > >> > > requests/responses,
> >>> > > > >> > > > > > > > >> > > > it
> >>> > > > >> > > > > > > > >> > > > > > >> seems
> >>> > > > >> > > > > > > > >> > > > > > >> > > that
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > internal changes would
> >>> be
> >>> > much
> >>> > > > >> larger
> >>> > > > >> > > > than
> >>> > > > >> > > > > I
> >>> > > > >> > > > > > > > >> expected.
> >>> > > > >> > > > > > > > >> > > > Some
> >>> > > > >> > > > > > > > >> > > > > > >> > protocols
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > involve clients, so
> they
> >>> > would
> >>> > > > >> > require
> >>> > > > >> > > > > > changes
> >>> > > > >> > > > > > > > too.
> >>> > > > >> > > > > > > > >> > I'm
> >>> > > > >> > > > > > > > >> > > > > > thinking
> >>> > > > >> > > > > > > > >> > > > > > >> > that
> >>> > > > >> > > > > > > > >> > > > > > >> > > > for
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > now, removing the
> topic
> >>> name
> >>> > > > from
> >>> > > > >> > these
> >>> > > > >> > > > > > > requests
> >>> > > > >> > > > > > > > >> and
> >>> > > > >> > > > > > > > >> > > > > responses
> >>> > > > >> > > > > > > > >> > > > > > >> are
> >>> > > > >> > > > > > > > >> > > > > > >> > > out
> >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > scope.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I have decided to just
> >>> keep
> >>> > > the
> >>> > > > >> > change
> >>> > > > >> > > > > > > > >> > > > LeaderAndIsrResponse
> >>> > > > >> > > > > > > > >> > > > > to
> >>> > > > >> > > > > > > > >> > > > > > >> > remove
> >>> > > > >> > > > > > > > >> > > > > > >> > > > the
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > topic name, and have
> >>> updated
> >>> > > the
> >>> > > > >> KIP
> >>> > > > >> > to
> >>> > > > >> > > > > > reflect
> >>> > > > >> > > > > > > > >> this
> >>> > > > >> > > > > > > > >> > > > > change. I
> >>> > > > >> > > > > > > > >> > > > > > >> have
> >>> > > > >> > > > > > > > >> > > > > > >> > > > also
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > mentioned the other
> >>> requests
> >>> > > and
> >>> > > > >> > > > responses
> >>> > > > >> > > > > in
> >>> > > > >> > > > > > > > >> future
> >>> > > > >> > > > > > > > >> > > work.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I'm hoping to start
> the
> >>> > voting
> >>> > > > >> > process
> >>> > > > >> > > > > soon,
> >>> > > > >> > > > > > so
> >>> > > > >> > > > > > > > >> let me
> >>> > > > >> > > > > > > > >> > > > know
> >>> > > > >> > > > > > > > >> > > > > if
> >>> > > > >> > > > > > > > >> > > > > > >> > there
> >>> > > > >> > > > > > > > >> > > > > > >> > > is
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > anything else we
> should
> >>> > > discuss.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > Thank you,
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > Justine
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > On Tue, Sep 15, 2020
> at
> >>> 3:57
> >>> > > PM
> >>> > > > >> > Justine
> >>> > > > >> > > > > > Olshan
> >>> > > > >> > > > > > > <
> >>> > > > >> > > > > > > > >> > > > > > >> > jolshan@confluent.io
> >>> > > > >> > > > > > > > >> > > > > > >> > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > wrote:
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > Hello again,
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > To follow up on some
> >>> of
> >>> > the
> >>> > > > >> other
> >>> > > > >> > > > > comments:
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 10/11) We can remove
> >>> the
> >>> > > topic
> >>> > > > >> name
> >>> > > > >> > > > from
> >>> > > > >> > > > > > > these
> >>> > > > >> > > > > > > > >> > > > > > >> > requests/responses,
> >>> > > > >> > > > > > > > >> > > > > > >> > > > and
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > that means we will
> >>> just
> >>> > have
> >>> > > > to
> >>> > > > >> > make
> >>> > > > >> > > a
> >>> > > > >> > > > > few
> >>> > > > >> > > > > > > > >> internal
> >>> > > > >> > > > > > > > >> > > > > changes
> >>> > > > >> > > > > > > > >> > > > > > to
> >>> > > > >> > > > > > > > >> > > > > > >> > make
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > partitions
> accessible
> >>> by
> >>> > > topic
> >>> > > > >> id
> >>> > > > >> > and
> >>> > > > >> > > > > > > > partition.
> >>> > > > >> > > > > > > > >> I
> >>> > > > >> > > > > > > > >> > can
> >>> > > > >> > > > > > > > >> > > > > > update
> >>> > > > >> > > > > > > > >> > > > > > >> the
> >>> > > > >> > > > > > > > >> > > > > > >> > > KIP
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > remove them unless
> >>> anyone
> >>> > > > thinks
> >>> > > > >> > they
> >>> > > > >> > > > > > should
> >>> > > > >> > > > > > > > >> stay.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 12) Addressed in the
> >>> > > previous
> >>> > > > >> > email.
> >>> > > > >> > > > I've
> >>> > > > >> > > > > > > > updated
> >>> > > > >> > > > > > > > >> > the
> >>> > > > >> > > > > > > > >> > > > KIP
> >>> > > > >> > > > > > > > >> > > > > to
> >>> > > > >> > > > > > > > >> > > > > > >> > > include
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > tagged fields for
> the
> >>> > > requests
> >>> > > > >> and
> >>> > > > >> > > > > > responses.
> >>> > > > >> > > > > > > > >> (More
> >>> > > > >> > > > > > > > >> > on
> >>> > > > >> > > > > > > > >> > > > > that
> >>> > > > >> > > > > > > > >> > > > > > >> > below)
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 13) I think part of
> >>> the
> >>> > idea
> >>> > > > for
> >>> > > > >> > > > > including
> >>> > > > >> > > > > > > this
> >>> > > > >> > > > > > > > >> > > > > information
> >>> > > > >> > > > > > > > >> > > > > > >> is to
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > prepare
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > for future changes.
> >>> > Perhaps
> >>> > > > the
> >>> > > > >> > > > directory
> >>> > > > >> > > > > > > > >> structure
> >>> > > > >> > > > > > > > >> > > > might
> >>> > > > >> > > > > > > > >> > > > > > >> change
> >>> > > > >> > > > > > > > >> > > > > > >> > > from
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> >>> topicName_partitionNumber
> >>> > to
> >>> > > > >> > > something
> >>> > > > >> > > > > like
> >>> > > > >> > > > > > > > >> > > > > > >> > > topicID_partitionNumber.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > Then
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > it would be useful
> to
> >>> have
> >>> > > the
> >>> > > > >> > topic
> >>> > > > >> > > > name
> >>> > > > >> > > > > > in
> >>> > > > >> > > > > > > > the
> >>> > > > >> > > > > > > > >> > file
> >>> > > > >> > > > > > > > >> > > > > since
> >>> > > > >> > > > > > > > >> > > > > > it
> >>> > > > >> > > > > > > > >> > > > > > >> > > would
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > not
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > be
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > in the directory
> >>> > structure.
> >>> > > > >> > > Supporting
> >>> > > > >> > > > > > topic
> >>> > > > >> > > > > > > > >> renames
> >>> > > > >> > > > > > > > >> > > > might
> >>> > > > >> > > > > > > > >> > > > > > be
> >>> > > > >> > > > > > > > >> > > > > > >> > > easier
> >>> > > > >> > > > > > > > >> > > > > > >> > > > if
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > the
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > other fields are
> >>> included.
> >>> > > > Would
> >>> > > > >> > > there
> >>> > > > >> > > > be
> >>> > > > >> > > > > > any
> >>> > > > >> > > > > > > > >> > > downsides
> >>> > > > >> > > > > > > > >> > > > to
> >>> > > > >> > > > > > > > >> > > > > > >> > > including
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > this
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > information?
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 14)  Yes, we would
> >>> need to
> >>> > > > copy
> >>> > > > >> the
> >>> > > > >> > > > > > partition
> >>> > > > >> > > > > > > > >> > metadata
> >>> > > > >> > > > > > > > >> > > > > file
> >>> > > > >> > > > > > > > >> > > > > > in
> >>> > > > >> > > > > > > > >> > > > > > >> > this
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > process. I've
> updated
> >>> the
> >>> > > KIP
> >>> > > > to
> >>> > > > >> > > > include
> >>> > > > >> > > > > > > this.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 15) I believe Lucas
> >>> meant
> >>> > v1
> >>> > > > >> and v2
> >>> > > > >> > > > here.
> >>> > > > >> > > > > > He
> >>> > > > >> > > > > > > > was
> >>> > > > >> > > > > > > > >> > > > referring
> >>> > > > >> > > > > > > > >> > > > > > to
> >>> > > > >> > > > > > > > >> > > > > > >> how
> >>> > > > >> > > > > > > > >> > > > > > >> > > the
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > requests would fall
> >>> under
> >>> > > > >> different
> >>> > > > >> > > IBP
> >>> > > > >> > > > > and
> >>> > > > >> > > > > > > > meant
> >>> > > > >> > > > > > > > >> > that
> >>> > > > >> > > > > > > > >> > > > > older
> >>> > > > >> > > > > > > > >> > > > > > >> > > brokers
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > would
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > have to use the
> older
> >>> > > version
> >>> > > > of
> >>> > > > >> > the
> >>> > > > >> > > > > > request
> >>> > > > >> > > > > > > > and
> >>> > > > >> > > > > > > > >> the
> >>> > > > >> > > > > > > > >> > > > > > existing
> >>> > > > >> > > > > > > > >> > > > > > >> > topic
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > deletion process. At
> >>> > first,
> >>> > > it
> >>> > > > >> > seemed
> >>> > > > >> > > > > like
> >>> > > > >> > > > > > > > tagged
> >>> > > > >> > > > > > > > >> > > fields
> >>> > > > >> > > > > > > > >> > > > > > would
> >>> > > > >> > > > > > > > >> > > > > > >> > > > resolve
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > the IBP issue.
> >>> However, we
> >>> > > may
> >>> > > > >> need
> >>> > > > >> > > IBP
> >>> > > > >> > > > > for
> >>> > > > >> > > > > > > > this
> >>> > > > >> > > > > > > > >> > > request
> >>> > > > >> > > > > > > > >> > > > > > after
> >>> > > > >> > > > > > > > >> > > > > > >> > all
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > since
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > the controller
> >>> handles the
> >>> > > > topic
> >>> > > > >> > > > deletion
> >>> > > > >> > > > > > > > >> > differently
> >>> > > > >> > > > > > > > >> > > > > > >> depending
> >>> > > > >> > > > > > > > >> > > > > > >> > on
> >>> > > > >> > > > > > > > >> > > > > > >> > > > the
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > IBP
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > version. In an older
> >>> > > version,
> >>> > > > we
> >>> > > > >> > > can't
> >>> > > > >> > > > > just
> >>> > > > >> > > > > > > > send
> >>> > > > >> > > > > > > > >> a
> >>> > > > >> > > > > > > > >> > > > > > StopReplica
> >>> > > > >> > > > > > > > >> > > > > > >> > > delete
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > topic immediately
> like
> >>> > we'd
> >>> > > > >> want to
> >>> > > > >> > > for
> >>> > > > >> > > > > > this
> >>> > > > >> > > > > > > > KIP.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > This makes me wonder
> >>> if we
> >>> > > > want
> >>> > > > >> > > tagged
> >>> > > > >> > > > > > fields
> >>> > > > >> > > > > > > > on
> >>> > > > >> > > > > > > > >> all
> >>> > > > >> > > > > > > > >> > > the
> >>> > > > >> > > > > > > > >> > > > > > >> requests
> >>> > > > >> > > > > > > > >> > > > > > >> > > > after
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > all. Let me know
> your
> >>> > > > thoughts!
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > Justine
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > On Tue, Sep 15, 2020
> >>> at
> >>> > 1:03
> >>> > > > PM
> >>> > > > >> > > Justine
> >>> > > > >> > > > > > > Olshan
> >>> > > > >> > > > > > > > <
> >>> > > > >> > > > > > > > >> > > > > > >> > > jolshan@confluent.io
> >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > > wrote:
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Hi all,
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Jun brought up a
> good
> >>> > point
> >>> > > > in
> >>> > > > >> his
> >>> > > > >> > > > last
> >>> > > > >> > > > > > > email
> >>> > > > >> > > > > > > > >> about
> >>> > > > >> > > > > > > > >> > > > > tagged
> >>> > > > >> > > > > > > > >> > > > > > >> > fields,
> >>> > > > >> > > > > > > > >> > > > > > >> > > > and
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> I've updated the
> KIP
> >>> to
> >>> > > > reflect
> >>> > > > >> > that
> >>> > > > >> > > > the
> >>> > > > >> > > > > > > > >> changes to
> >>> > > > >> > > > > > > > >> > > > > > requests
> >>> > > > >> > > > > > > > >> > > > > > >> and
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > responses
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> will be in the form
> >>> of
> >>> > > tagged
> >>> > > > >> > fields
> >>> > > > >> > > > to
> >>> > > > >> > > > > > > avoid
> >>> > > > >> > > > > > > > >> > > changing
> >>> > > > >> > > > > > > > >> > > > > IBP.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Jun: I plan on
> >>> sending a
> >>> > > > >> followup
> >>> > > > >> > > > email
> >>> > > > >> > > > > to
> >>> > > > >> > > > > > > > >> address
> >>> > > > >> > > > > > > > >> > > some
> >>> > > > >> > > > > > > > >> > > > > of
> >>> > > > >> > > > > > > > >> > > > > > >> the
> >>> > > > >> > > > > > > > >> > > > > > >> > > other
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> points.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Thanks,
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Justine
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> On Mon, Sep 14,
> 2020
> >>> at
> >>> > > 4:25
> >>> > > > PM
> >>> > > > >> > Jun
> >>> > > > >> > > > Rao
> >>> > > > >> > > > > <
> >>> > > > >> > > > > > > > >> > > > > jun@confluent.io>
> >>> > > > >> > > > > > > > >> > > > > > >> > wrote:
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Hi, Justine,
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Thanks for the
> >>> updated
> >>> > > KIP.
> >>> > > > A
> >>> > > > >> few
> >>> > > > >> > > > > > comments
> >>> > > > >> > > > > > > > >> below.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 10. LeaderAndIsr
> >>> > Response:
> >>> > > > Do
> >>> > > > >> we
> >>> > > > >> > > need
> >>> > > > >> > > > > the
> >>> > > > >> > > > > > > > topic
> >>> > > > >> > > > > > > > >> > > name?
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 11. For the
> changed
> >>> > > > >> > > request/response,
> >>> > > > >> > > > > > other
> >>> > > > >> > > > > > > > >> than
> >>> > > > >> > > > > > > > >> > > > > > >> LeaderAndIsr,
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> UpdateMetadata,
> >>> > Metadata,
> >>> > > do
> >>> > > > >> we
> >>> > > > >> > > need
> >>> > > > >> > > > to
> >>> > > > >> > > > > > > > include
> >>> > > > >> > > > > > > > >> > the
> >>> > > > >> > > > > > > > >> > > > > topic
> >>> > > > >> > > > > > > > >> > > > > > >> name?
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 12. It seems that
> >>> > upgrades
> >>> > > > >> don't
> >>> > > > >> > > > > require
> >>> > > > >> > > > > > > IBP.
> >>> > > > >> > > > > > > > >> Does
> >>> > > > >> > > > > > > > >> > > > that
> >>> > > > >> > > > > > > > >> > > > > > mean
> >>> > > > >> > > > > > > > >> > > > > > >> > the
> >>> > > > >> > > > > > > > >> > > > > > >> > > > new
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> fields
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> in all the
> >>> > > request/response
> >>> > > > >> are
> >>> > > > >> > > added
> >>> > > > >> > > > > as
> >>> > > > >> > > > > > > > tagged
> >>> > > > >> > > > > > > > >> > > fields
> >>> > > > >> > > > > > > > >> > > > > > >> without
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > bumping
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > up
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> the request
> >>> version? It
> >>> > > > would
> >>> > > > >> be
> >>> > > > >> > > > useful
> >>> > > > >> > > > > > to
> >>> > > > >> > > > > > > > make
> >>> > > > >> > > > > > > > >> > that
> >>> > > > >> > > > > > > > >> > > > > > clear.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 13. Partition
> >>> Metadata
> >>> > > file:
> >>> > > > >> Do
> >>> > > > >> > we
> >>> > > > >> > > > need
> >>> > > > >> > > > > > to
> >>> > > > >> > > > > > > > >> include
> >>> > > > >> > > > > > > > >> > > the
> >>> > > > >> > > > > > > > >> > > > > > topic
> >>> > > > >> > > > > > > > >> > > > > > >> > name
> >>> > > > >> > > > > > > > >> > > > > > >> > > > and
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > the
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> partition id since
> >>> they
> >>> > > are
> >>> > > > >> > implied
> >>> > > > >> > > > in
> >>> > > > >> > > > > > the
> >>> > > > >> > > > > > > > >> > directory
> >>> > > > >> > > > > > > > >> > > > > name?
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 14. In the JBOD
> >>> mode, we
> >>> > > > >> support
> >>> > > > >> > > > > moving a
> >>> > > > >> > > > > > > > >> > > partition's
> >>> > > > >> > > > > > > > >> > > > > data
> >>> > > > >> > > > > > > > >> > > > > > >> from
> >>> > > > >> > > > > > > > >> > > > > > >> > > one
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > disk
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> to
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> another. Will the
> >>> new
> >>> > > > >> partition
> >>> > > > >> > > > > metadata
> >>> > > > >> > > > > > > file
> >>> > > > >> > > > > > > > >> be
> >>> > > > >> > > > > > > > >> > > > copied
> >>> > > > >> > > > > > > > >> > > > > > >> during
> >>> > > > >> > > > > > > > >> > > > > > >> > > that
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> process?
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 15. The KIP says
> >>> "Remove
> >>> > > > >> deleted
> >>> > > > >> > > > topics
> >>> > > > >> > > > > > > from
> >>> > > > >> > > > > > > > >> > > replicas
> >>> > > > >> > > > > > > > >> > > > by
> >>> > > > >> > > > > > > > >> > > > > > >> > sending
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> StopReplicaRequest
> >>> V2
> >>> > for
> >>> > > > any
> >>> > > > >> > > topics
> >>> > > > >> > > > > > which
> >>> > > > >> > > > > > > do
> >>> > > > >> > > > > > > > >> not
> >>> > > > >> > > > > > > > >> > > > > contain
> >>> > > > >> > > > > > > > >> > > > > > a
> >>> > > > >> > > > > > > > >> > > > > > >> > topic
> >>> > > > >> > > > > > > > >> > > > > > >> > > > ID,
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > and
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> V3 for any topics
> >>> which
> >>> > do
> >>> > > > >> > contain
> >>> > > > >> > > a
> >>> > > > >> > > > > > topic
> >>> > > > >> > > > > > > > >> ID.".
> >>> > > > >> > > > > > > > >> > > > > However,
> >>> > > > >> > > > > > > > >> > > > > > it
> >>> > > > >> > > > > > > > >> > > > > > >> > > seems
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> updated controller
> >>> will
> >>> > > > create
> >>> > > > >> > all
> >>> > > > >> > > > > > missing
> >>> > > > >> > > > > > > > >> topic
> >>> > > > >> > > > > > > > >> > IDs
> >>> > > > >> > > > > > > > >> > > > > first
> >>> > > > >> > > > > > > > >> > > > > > >> > before
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > doing
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> other actions. So,
> >>> is
> >>> > > > >> > > > > StopReplicaRequest
> >>> > > > >> > > > > > V2
> >>> > > > >> > > > > > > > >> > needed?
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Jun
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> On Fri, Sep 11,
> >>> 2020 at
> >>> > > > 10:31
> >>> > > > >> AM
> >>> > > > >> > > John
> >>> > > > >> > > > > > > > Roesler <
> >>> > > > >> > > > > > > > >> > > > > > >> > > vvcephei@apache.org
> >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> wrote:
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > Thanks, Justine!
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > Your response
> >>> seems
> >>> > > > >> compelling
> >>> > > > >> > to
> >>> > > > >> > > > me.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > -John
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > On Fri,
> >>> 2020-09-11 at
> >>> > > > 10:17
> >>> > > > >> > > -0700,
> >>> > > > >> > > > > > > Justine
> >>> > > > >> > > > > > > > >> > Olshan
> >>> > > > >> > > > > > > > >> > > > > wrote:
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Hello all,
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Thanks for
> >>> > continuing
> >>> > > > the
> >>> > > > >> > > > > > discussion! I
> >>> > > > >> > > > > > > > >> have a
> >>> > > > >> > > > > > > > >> > > few
> >>> > > > >> > > > > > > > >> > > > > > >> > responses
> >>> > > > >> > > > > > > > >> > > > > > >> > > to
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > your
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > points.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Tom: You are
> >>> correct
> >>> > > in
> >>> > > > >> that
> >>> > > > >> > > this
> >>> > > > >> > > > > KIP
> >>> > > > >> > > > > > > has
> >>> > > > >> > > > > > > > >> not
> >>> > > > >> > > > > > > > >> > > > > > mentioned
> >>> > > > >> > > > > > > > >> > > > > > >> the
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >>> > DeleteTopicsRequest. I
> >>> > > > >> think
> >>> > > > >> > > that
> >>> > > > >> > > > > > this
> >>> > > > >> > > > > > > > >> would
> >>> > > > >> > > > > > > > >> > be
> >>> > > > >> > > > > > > > >> > > > out
> >>> > > > >> > > > > > > > >> > > > > of
> >>> > > > >> > > > > > > > >> > > > > > >> > scope
> >>> > > > >> > > > > > > > >> > > > > > >> > > > for
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> now, but
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > may be
> something
> >>> > worth
> >>> > > > >> adding
> >>> > > > >> > > in
> >>> > > > >> > > > > the
> >>> > > > >> > > > > > > > >> future.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > John: We did
> >>> > consider
> >>> > > > >> > sequence
> >>> > > > >> > > > ids,
> >>> > > > >> > > > > > but
> >>> > > > >> > > > > > > > >> there
> >>> > > > >> > > > > > > > >> > > are
> >>> > > > >> > > > > > > > >> > > > a
> >>> > > > >> > > > > > > > >> > > > > > few
> >>> > > > >> > > > > > > > >> > > > > > >> > > reasons
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> favor
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > UUIDs. There
> are
> >>> > > several
> >>> > > > >> > cases
> >>> > > > >> > > > > where
> >>> > > > >> > > > > > > > topics
> >>> > > > >> > > > > > > > >> > from
> >>> > > > >> > > > > > > > >> > > > > > >> different
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > clusters
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> may
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > interact now
> >>> and in
> >>> > > the
> >>> > > > >> > future.
> >>> > > > >> > > > For
> >>> > > > >> > > > > > > > >> example,
> >>> > > > >> > > > > > > > >> > > > Mirror
> >>> > > > >> > > > > > > > >> > > > > > >> Maker 2
> >>> > > > >> > > > > > > > >> > > > > > >> > > may
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> benefit
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > from being
> able
> >>> to
> >>> > > > detect
> >>> > > > >> > when
> >>> > > > >> > > a
> >>> > > > >> > > > > > > cluster
> >>> > > > >> > > > > > > > >> being
> >>> > > > >> > > > > > > > >> > > > > > mirrored
> >>> > > > >> > > > > > > > >> > > > > > >> is
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > deleted
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> and
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > recreated and
> >>> > globally
> >>> > > > >> unique
> >>> > > > >> > > > > > > identifiers
> >>> > > > >> > > > > > > > >> > would
> >>> > > > >> > > > > > > > >> > > > make
> >>> > > > >> > > > > > > > >> > > > > > >> > > resolving
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > issues
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > easier than
> >>> sequence
> >>> > > IDs
> >>> > > > >> > which
> >>> > > > >> > > > may
> >>> > > > >> > > > > > > > collide
> >>> > > > >> > > > > > > > >> > > between
> >>> > > > >> > > > > > > > >> > > > > > >> > clusters.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > KIP-405
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > (tiered
> storage)
> >>> > will
> >>> > > > also
> >>> > > > >> > > > benefit
> >>> > > > >> > > > > > from
> >>> > > > >> > > > > > > > >> > globally
> >>> > > > >> > > > > > > > >> > > > > > unique
> >>> > > > >> > > > > > > > >> > > > > > >> IDs
> >>> > > > >> > > > > > > > >> > > > > > >> > > as
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > shared
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > buckets may be
> >>> used
> >>> > > > >> between
> >>> > > > >> > > > > clusters.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Globally
> unique
> >>> IDs
> >>> > > > would
> >>> > > > >> > also
> >>> > > > >> > > > make
> >>> > > > >> > > > > > > > >> > > functionality
> >>> > > > >> > > > > > > > >> > > > > like
> >>> > > > >> > > > > > > > >> > > > > > >> > moving
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > topics
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > between
> >>> disparate
> >>> > > > clusters
> >>> > > > >> > > easier
> >>> > > > >> > > > > in
> >>> > > > >> > > > > > > the
> >>> > > > >> > > > > > > > >> > future,
> >>> > > > >> > > > > > > > >> > > > > > >> simplify
> >>> > > > >> > > > > > > > >> > > > > > >> > any
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > future
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> implementations
> >>> of
> >>> > > > backups
> >>> > > > >> > and
> >>> > > > >> > > > > > > restores,
> >>> > > > >> > > > > > > > >> and
> >>> > > > >> > > > > > > > >> > > more.
> >>> > > > >> > > > > > > > >> > > > > In
> >>> > > > >> > > > > > > > >> > > > > > >> > > general,
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> unique IDs
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > would ensure
> >>> that
> >>> > the
> >>> > > > >> source
> >>> > > > >> > > > > cluster
> >>> > > > >> > > > > > > > >> topics do
> >>> > > > >> > > > > > > > >> > > not
> >>> > > > >> > > > > > > > >> > > > > > >> conflict
> >>> > > > >> > > > > > > > >> > > > > > >> > > > with
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > the
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > destination
> >>> cluster
> >>> > > > >> topics.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > If we were to
> >>> use
> >>> > > > sequence
> >>> > > > >> > ids,
> >>> > > > >> > > > we
> >>> > > > >> > > > > > > would
> >>> > > > >> > > > > > > > >> need
> >>> > > > >> > > > > > > > >> > > > > > >> sufficiently
> >>> > > > >> > > > > > > > >> > > > > > >> > > > large
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> cluster
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > ids to be
> stored
> >>> > with
> >>> > > > the
> >>> > > > >> > topic
> >>> > > > >> > > > > > > > >> identifiers or
> >>> > > > >> > > > > > > > >> > > we
> >>> > > > >> > > > > > > > >> > > > > run
> >>> > > > >> > > > > > > > >> > > > > > >> the
> >>> > > > >> > > > > > > > >> > > > > > >> > > risk
> >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > collisions.
> This
> >>> > will
> >>> > > > >> give up
> >>> > > > >> > > any
> >>> > > > >> > > > > > > > >> advantage in
> >>> > > > >> > > > > > > > >> > > > > > >> compactness
> >>> > > > >> > > > > > > > >> > > > > > >> > > that
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> sequence
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > numbers may
> >>> bring.
> >>> > > Given
> >>> > > > >> > these
> >>> > > > >> > > > > > > > advantages I
> >>> > > > >> > > > > > > > >> > > think
> >>> > > > >> > > > > > > > >> > > > it
> >>> > > > >> > > > > > > > >> > > > > > >> makes
> >>> > > > >> > > > > > > > >> > > > > > >> > > > sense
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> use
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > UUIDs.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Gokul: This is
> >>> an
> >>> > > > >> interesting
> >>> > > > >> > > > idea,
> >>> > > > >> > > > > > but
> >>> > > > >> > > > > > > > >> this
> >>> > > > >> > > > > > > > >> > is
> >>> > > > >> > > > > > > > >> > > a
> >>> > > > >> > > > > > > > >> > > > > > >> breaking
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > change.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Out of
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > scope for now,
> >>> but
> >>> > > maybe
> >>> > > > >> > worth
> >>> > > > >> > > > > > > discussing
> >>> > > > >> > > > > > > > >> in
> >>> > > > >> > > > > > > > >> > the
> >>> > > > >> > > > > > > > >> > > > > > future.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Hope this
> >>> explains
> >>> > > some
> >>> > > > of
> >>> > > > >> > the
> >>> > > > >> > > > > > > decisions,
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Justine
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > On Fri, Sep
> 11,
> >>> 2020
> >>> > > at
> >>> > > > >> 8:27
> >>> > > > >> > AM
> >>> > > > >> > > > > Gokul
> >>> > > > >> > > > > > > > >> Ramanan
> >>> > > > >> > > > > > > > >> > > > > > >> Subramanian <
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >>> > gokul2411s@gmail.com>
> >>> > > > >> wrote:
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Hi.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Thanks for
> the
> >>> > KIP.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Have you
> >>> thought
> >>> > > about
> >>> > > > >> > > whether
> >>> > > > >> > > > it
> >>> > > > >> > > > > > > makes
> >>> > > > >> > > > > > > > >> > sense
> >>> > > > >> > > > > > > > >> > > to
> >>> > > > >> > > > > > > > >> > > > > > >> support
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> authorizing a
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > principal
> for
> >>> a
> >>> > > topic
> >>> > > > ID
> >>> > > > >> > > rather
> >>> > > > >> > > > > > than
> >>> > > > >> > > > > > > a
> >>> > > > >> > > > > > > > >> topic
> >>> > > > >> > > > > > > > >> > > > name
> >>> > > > >> > > > > > > > >> > > > > to
> >>> > > > >> > > > > > > > >> > > > > > >> > > achieve
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> tighter
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > security?
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Or is the
> >>> topic ID
> >>> > > > >> > > > fundamentally
> >>> > > > >> > > > > an
> >>> > > > >> > > > > > > > >> internal
> >>> > > > >> > > > > > > > >> > > > > detail
> >>> > > > >> > > > > > > > >> > > > > > >> > similar
> >>> > > > >> > > > > > > > >> > > > > > >> > > > to
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> epochs
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > used
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > in a bunch
> of
> >>> > other
> >>> > > > >> places
> >>> > > > >> > in
> >>> > > > >> > > > > > Kafka?
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Thanks.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > On Fri, Sep
> >>> 11,
> >>> > 2020
> >>> > > > at
> >>> > > > >> > 4:06
> >>> > > > >> > > PM
> >>> > > > >> > > > > > John
> >>> > > > >> > > > > > > > >> > Roesler <
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > vvcephei@apache.org>
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > wrote:
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Hello
> >>> Justine,
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Thanks for
> >>> the
> >>> > > KIP!
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > I happen
> to
> >>> have
> >>> > > > been
> >>> > > > >> > > > > confronted
> >>> > > > >> > > > > > > > >> recently
> >>> > > > >> > > > > > > > >> > > with
> >>> > > > >> > > > > > > > >> > > > > the
> >>> > > > >> > > > > > > > >> > > > > > >> need
> >>> > > > >> > > > > > > > >> > > > > > >> > > to
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > keep
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > track of
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > a
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > large
> >>> number of
> >>> > > > >> topics as
> >>> > > > >> > > > > > compactly
> >>> > > > >> > > > > > > > as
> >>> > > > >> > > > > > > > >> > > > > possible. I
> >>> > > > >> > > > > > > > >> > > > > > >> was
> >>> > > > >> > > > > > > > >> > > > > > >> > > > going
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> come
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > up
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > with some
> >>> way to
> >>> > > > >> > dictionary
> >>> > > > >> > > > > > encode
> >>> > > > >> > > > > > > > the
> >>> > > > >> > > > > > > > >> > topic
> >>> > > > >> > > > > > > > >> > > > > names
> >>> > > > >> > > > > > > > >> > > > > > >> as
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > integers,
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> but
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > this
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > seems much
> >>> > better!
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Apologies
> if
> >>> > this
> >>> > > > has
> >>> > > > >> > been
> >>> > > > >> > > > > raised
> >>> > > > >> > > > > > > > >> before,
> >>> > > > >> > > > > > > > >> > > but
> >>> > > > >> > > > > > > > >> > > > > I’m
> >>> > > > >> > > > > > > > >> > > > > > >> > > wondering
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> about the
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > choice of
> >>> UUID
> >>> > vs
> >>> > > > >> > sequence
> >>> > > > >> > > > > number
> >>> > > > >> > > > > > > for
> >>> > > > >> > > > > > > > >> the
> >>> > > > >> > > > > > > > >> > > ids.
> >>> > > > >> > > > > > > > >> > > > > > >> > Typically,
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > I’ve
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> seen
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > UUIDs
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > in two
> >>> > situations:
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 1. When
> >>> > processes
> >>> > > > >> need to
> >>> > > > >> > > > > > generate
> >>> > > > >> > > > > > > > >> > > > non-colliding
> >>> > > > >> > > > > > > > >> > > > > > >> > > > identifiers
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> without
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >>> coordination.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 2. When
> the
> >>> > > > identifier
> >>> > > > >> > > needs
> >>> > > > >> > > > to
> >>> > > > >> > > > > > be
> >>> > > > >> > > > > > > > >> > > > “universally
> >>> > > > >> > > > > > > > >> > > > > > >> > unique”;
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > I.e.,
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> the
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > identifier
> >>> needs
> >>> > > to
> >>> > > > >> > > > distinguish
> >>> > > > >> > > > > > the
> >>> > > > >> > > > > > > > >> entity
> >>> > > > >> > > > > > > > >> > > > from
> >>> > > > >> > > > > > > > >> > > > > > all
> >>> > > > >> > > > > > > > >> > > > > > >> > other
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> entities
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > that
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > could ever
> >>> > exist.
> >>> > > > >> This is
> >>> > > > >> > > > > useful
> >>> > > > >> > > > > > in
> >>> > > > >> > > > > > > > >> cases
> >>> > > > >> > > > > > > > >> > > > where
> >>> > > > >> > > > > > > > >> > > > > > >> > entities
> >>> > > > >> > > > > > > > >> > > > > > >> > > > from
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > all
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > kinds
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > of
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > systems
> get
> >>> > mixed
> >>> > > > >> > together,
> >>> > > > >> > > > > such
> >>> > > > >> > > > > > as
> >>> > > > >> > > > > > > > >> when
> >>> > > > >> > > > > > > > >> > > > dumping
> >>> > > > >> > > > > > > > >> > > > > > >> logs
> >>> > > > >> > > > > > > > >> > > > > > >> > > from
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > all
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > processes
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > in
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a company
> >>> into a
> >>> > > > >> common
> >>> > > > >> > > > system.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Maybe I’m
> >>> being
> >>> > > > >> > > > short-sighted,
> >>> > > > >> > > > > > but
> >>> > > > >> > > > > > > it
> >>> > > > >> > > > > > > > >> > > doesn’t
> >>> > > > >> > > > > > > > >> > > > > seem
> >>> > > > >> > > > > > > > >> > > > > > >> like
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > either
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> really
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > applies
> >>> here. It
> >>> > > > seems
> >>> > > > >> > like
> >>> > > > >> > > > the
> >>> > > > >> > > > > > > > brokers
> >>> > > > >> > > > > > > > >> > > could
> >>> > > > >> > > > > > > > >> > > > > and
> >>> > > > >> > > > > > > > >> > > > > > >> would
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > achieve
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > consensus
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > when
> >>> creating a
> >>> > > > topic
> >>> > > > >> > > anyway,
> >>> > > > >> > > > > > which
> >>> > > > >> > > > > > > > is
> >>> > > > >> > > > > > > > >> all
> >>> > > > >> > > > > > > > >> > > > > that’s
> >>> > > > >> > > > > > > > >> > > > > > >> > > required
> >>> > > > >> > > > > > > > >> > > > > > >> > > > to
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > generate
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >>> non-colliding
> >>> > > > sequence
> >>> > > > >> > ids.
> >>> > > > >> > > > For
> >>> > > > >> > > > > > the
> >>> > > > >> > > > > > > > >> > second,
> >>> > > > >> > > > > > > > >> > > as
> >>> > > > >> > > > > > > > >> > > > > you
> >>> > > > >> > > > > > > > >> > > > > > >> > > mention,
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > we
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> could
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > assign
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a UUID to
> >>> the
> >>> > > > cluster
> >>> > > > >> as
> >>> > > > >> > a
> >>> > > > >> > > > > whole,
> >>> > > > >> > > > > > > > which
> >>> > > > >> > > > > > > > >> > > would
> >>> > > > >> > > > > > > > >> > > > > > render
> >>> > > > >> > > > > > > > >> > > > > > >> > any
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > resource
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > scoped
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > the broker
> >>> > > > universally
> >>> > > > >> > > unique
> >>> > > > >> > > > > as
> >>> > > > >> > > > > > > > well.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > The
> reason I
> >>> > > mention
> >>> > > > >> this
> >>> > > > >> > > is
> >>> > > > >> > > > > > that,
> >>> > > > >> > > > > > > > >> > although
> >>> > > > >> > > > > > > > >> > > a
> >>> > > > >> > > > > > > > >> > > > > UUID
> >>> > > > >> > > > > > > > >> > > > > > >> is
> >>> > > > >> > > > > > > > >> > > > > > >> > way
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > more
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > compact
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > than topic
> >>> > names,
> >>> > > > it’s
> >>> > > > >> > > still
> >>> > > > >> > > > 16
> >>> > > > >> > > > > > > > bytes.
> >>> > > > >> > > > > > > > >> In
> >>> > > > >> > > > > > > > >> > > > > > contrast,
> >>> > > > >> > > > > > > > >> > > > > > >> a
> >>> > > > >> > > > > > > > >> > > > > > >> > > > 4-byte
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> integer
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > sequence
> id
> >>> > would
> >>> > > > give
> >>> > > > >> > us 4
> >>> > > > >> > > > > > billion
> >>> > > > >> > > > > > > > >> unique
> >>> > > > >> > > > > > > > >> > > > > topics
> >>> > > > >> > > > > > > > >> > > > > > >> per
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > cluster,
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> which
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > seems
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > like
> enough
> >>> ;)
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> Considering
> >>> the
> >>> > > > >> number of
> >>> > > > >> > > > > > different
> >>> > > > >> > > > > > > > >> times
> >>> > > > >> > > > > > > > >> > > > these
> >>> > > > >> > > > > > > > >> > > > > > >> topic
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> identifiers are
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > sent
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > over the
> >>> wire or
> >>> > > > >> stored
> >>> > > > >> > in
> >>> > > > >> > > > > > memory,
> >>> > > > >> > > > > > > it
> >>> > > > >> > > > > > > > >> > seems
> >>> > > > >> > > > > > > > >> > > > like
> >>> > > > >> > > > > > > > >> > > > > > it
> >>> > > > >> > > > > > > > >> > > > > > >> > might
> >>> > > > >> > > > > > > > >> > > > > > >> > > > be
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> worth
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > the
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > additional
> >>> 4x
> >>> > > space
> >>> > > > >> > > savings.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > What do
> you
> >>> > think
> >>> > > > >> about
> >>> > > > >> > > this?
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Thanks,
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > John
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > On Fri,
> Sep
> >>> 11,
> >>> > > > 2020,
> >>> > > > >> at
> >>> > > > >> > > > 03:20,
> >>> > > > >> > > > > > Tom
> >>> > > > >> > > > > > > > >> > Bentley
> >>> > > > >> > > > > > > > >> > > > > wrote:
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Hi
> >>> Justine,
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > This
> looks
> >>> > like
> >>> > > a
> >>> > > > >> very
> >>> > > > >> > > > > welcome
> >>> > > > >> > > > > > > > >> > > improvement.
> >>> > > > >> > > > > > > > >> > > > > > >> Thanks!
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Maybe I
> >>> missed
> >>> > > it,
> >>> > > > >> but
> >>> > > > >> > > the
> >>> > > > >> > > > > KIP
> >>> > > > >> > > > > > > > >> doesn't
> >>> > > > >> > > > > > > > >> > > seem
> >>> > > > >> > > > > > > > >> > > > to
> >>> > > > >> > > > > > > > >> > > > > > >> > mention
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > changing
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >>> > > > DeleteTopicsRequest
> >>> > > > >> to
> >>> > > > >> > > > > identify
> >>> > > > >> > > > > > > the
> >>> > > > >> > > > > > > > >> > topic
> >>> > > > >> > > > > > > > >> > > > > using
> >>> > > > >> > > > > > > > >> > > > > > an
> >>> > > > >> > > > > > > > >> > > > > > >> > id.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > Maybe
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > that's out
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > of
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > scope,
> but
> >>> > > > >> > > > > DeleteTopicsRequest
> >>> > > > >> > > > > > is
> >>> > > > >> > > > > > > > not
> >>> > > > >> > > > > > > > >> > > listed
> >>> > > > >> > > > > > > > >> > > > > > among
> >>> > > > >> > > > > > > > >> > > > > > >> > the
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > Future
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Work
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > APIs
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > either.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Kind
> >>> regards,
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Tom
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > On Thu,
> >>> Sep
> >>> > 10,
> >>> > > > >> 2020 at
> >>> > > > >> > > > 3:59
> >>> > > > >> > > > > PM
> >>> > > > >> > > > > > > > >> Satish
> >>> > > > >> > > > > > > > >> > > > > Duggana <
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >>> > > > >> satish.duggana@gmail.com
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > wrote:
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Thanks
> >>> > > > >> Lucas/Justine
> >>> > > > >> > > for
> >>> > > > >> > > > > the
> >>> > > > >> > > > > > > nice
> >>> > > > >> > > > > > > > >> KIP.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > It has
> >>> > several
> >>> > > > >> > benefits
> >>> > > > >> > > > > which
> >>> > > > >> > > > > > > > also
> >>> > > > >> > > > > > > > >> > > include
> >>> > > > >> > > > > > > > >> > > > > > >> > > simplifying
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> topic
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> deletion
> >>> > > process
> >>> > > > >> by
> >>> > > > >> > > > > > controller
> >>> > > > >> > > > > > > > and
> >>> > > > >> > > > > > > > >> > logs
> >>> > > > >> > > > > > > > >> > > > > > cleanup
> >>> > > > >> > > > > > > > >> > > > > > >> by
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > brokers
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > in
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > corner
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > cases.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Best,
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> Satish.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > On
> Wed,
> >>> Sep
> >>> > 9,
> >>> > > > >> 2020
> >>> > > > >> > at
> >>> > > > >> > > > > 10:07
> >>> > > > >> > > > > > PM
> >>> > > > >> > > > > > > > >> > Justine
> >>> > > > >> > > > > > > > >> > > > > > Olshan <
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >>> jolshan@confluent.io
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > wrote:
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> Hello
> >>> all,
> >>> > > > it's
> >>> > > > >> > been
> >>> > > > >> > > > > > almost a
> >>> > > > >> > > > > > > > >> year!
> >>> > > > >> > > > > > > > >> > > I've
> >>> > > > >> > > > > > > > >> > > > > > made
> >>> > > > >> > > > > > > > >> > > > > > >> > some
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > changes
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> to
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > this
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > KIP
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > and
> >>> hope to
> >>> > > > >> continue
> >>> > > > >> > > the
> >>> > > > >> > > > > > > > >> discussion.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > One
> >>> of the
> >>> > > > main
> >>> > > > >> > > changes
> >>> > > > >> > > > > > I've
> >>> > > > >> > > > > > > > >> added
> >>> > > > >> > > > > > > > >> > is
> >>> > > > >> > > > > > > > >> > > > now
> >>> > > > >> > > > > > > > >> > > > > > the
> >>> > > > >> > > > > > > > >> > > > > > >> > > > metadata
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> response
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > will
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> include
> >>> the
> >>> > > > topic
> >>> > > > >> ID
> >>> > > > >> > > (as
> >>> > > > >> > > > > > Colin
> >>> > > > >> > > > > > > > >> > > suggested).
> >>> > > > >> > > > > > > > >> > > > > > >> Clients
> >>> > > > >> > > > > > > > >> > > > > > >> > > can
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> obtain the
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > topicID
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > of a
> >>> given
> >>> > > topic
> >>> > > > >> > > through
> >>> > > > >> > > > a
> >>> > > > >> > > > > > > > >> > > > TopicDescription.
> >>> > > > >> > > > > > > > >> > > > > > The
> >>> > > > >> > > > > > > > >> > > > > > >> > > > topicId
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > will
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > also be
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> included
> >>> > with
> >>> > > > the
> >>> > > > >> > > > > > > UpdateMetadata
> >>> > > > >> > > > > > > > >> > > request.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Let
> me
> >>> > know
> >>> > > > what
> >>> > > > >> > you
> >>> > > > >> > > > all
> >>> > > > >> > > > > > > think.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> Thank
> >>> you,
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >>> Justine
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > On
> >>> > > 2019/09/13
> >>> > > > >> > > 16:38:26,
> >>> > > > >> > > > > > > "Colin
> >>> > > > >> > > > > > > > >> > > McCabe" <
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > cmccabe@apache.org
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > wrote:
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Hi
> >>> > Lucas,
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >>> Thanks
> >>> > for
> >>> > > > >> > tackling
> >>> > > > >> > > > > this.
> >>> > > > >> > > > > > > > >> Topic
> >>> > > > >> > > > > > > > >> > IDs
> >>> > > > >> > > > > > > > >> > > > > are a
> >>> > > > >> > > > > > > > >> > > > > > >> > great
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > idea,
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> and
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > this
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > is
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > really
> >>> good
> >>> > > > >> writeup.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> For
> >>> > > > >> > > > > > > /brokers/topics/[topic],
> >>> > > > >> > > > > > > > >> the
> >>> > > > >> > > > > > > > >> > > > schema
> >>> > > > >> > > > > > > > >> > > > > > >> version
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > should
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > be
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > bumped
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> version
> >>> 3,
> >>> > > > rather
> >>> > > > >> > than
> >>> > > > >> > > 2.
> >>> > > > >> > > > > > > > KIP-455
> >>> > > > >> > > > > > > > >> > > bumped
> >>> > > > >> > > > > > > > >> > > > > the
> >>> > > > >> > > > > > > > >> > > > > > >> > version
> >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > this
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > znode
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 2
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> already
> >>> :)
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >>> Given
> >>> > that
> >>> > > > >> we're
> >>> > > > >> > > > going
> >>> > > > >> > > > > to
> >>> > > > >> > > > > > > be
> >>> > > > >> > > > > > > > >> > seeing
> >>> > > > >> > > > > > > > >> > > > > these
> >>> > > > >> > > > > > > > >> > > > > > >> > things
> >>> > > > >> > > > > > > > >> > > > > > >> > > as
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> strings
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > as
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > lot
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > (in
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > logs,
> in
> >>> > > > >> ZooKeeper,
> >>> > > > >> > on
> >>> > > > >> > > > the
> >>> > > > >> > > > > > > > >> > command-line,
> >>> > > > >> > > > > > > > >> > > > > > etc.),
> >>> > > > >> > > > > > > > >> > > > > > >> > does
> >>> > > > >> > > > > > > > >> > > > > > >> > > it
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > make
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > sense to
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > use
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > base64
> >>> when
> >>> > > > >> > converting
> >>> > > > >> > > > them
> >>> > > > >> > > > > > to
> >>> > > > >> > > > > > > > >> > strings?
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >>> Here is
> >>> > an
> >>> > > > >> > example
> >>> > > > >> > > of
> >>> > > > >> > > > > the
> >>> > > > >> > > > > > > hex
> >>> > > > >> > > > > > > > >> > > > > > >> representation:
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >>> > > > >> > > > > > > > >> > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> And
> >>> here
> >>> > > is
> >>> > > > an
> >>> > > > >> > > > example
> >>> > > > >> > > > > in
> >>> > > > >> > > > > > > > >> base64.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >>> > > > >> > > > b8tRS7h4TJ2Vt43Dp85v2A
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> The
> >>> > base64
> >>> > > > >> > version
> >>> > > > >> > > > > saves
> >>> > > > >> > > > > > 15
> >>> > > > >> > > > > > > > >> > letters
> >>> > > > >> > > > > > > > >> > > > (to
> >>> > > > >> > > > > > > > >> > > > > be
> >>> > > > >> > > > > > > > >> > > > > > >> > fair,
> >>> > > > >> > > > > > > > >> > > > > > >> > > 4
> >>> > > > >> > > > > > > > >> > > > > > >> > > > of
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> those
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > were
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > dashes
> >>> that
> >>> > we
> >>> > > > >> could
> >>> > > > >> > > have
> >>> > > > >> > > > > > > elided
> >>> > > > >> > > > > > > > in
> >>> > > > >> > > > > > > > >> > the
> >>> > > > >> > > > > > > > >> > > > hex
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > representation.)
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >>> Another
> >>> > > > thing
> >>> > > > >> to
> >>> > > > >> > > > > consider
> >>> > > > >> > > > > > > is
> >>> > > > >> > > > > > > > >> that
> >>> > > > >> > > > > > > > >> > we
> >>> > > > >> > > > > > > > >> > > > > > should
> >>> > > > >> > > > > > > > >> > > > > > >> > > specify
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > that
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> the
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> >>> all-zeroes
> >>> > > UUID
> >>> > > > is
> >>> > > > >> > not
> >>> > > > >> > > a
> >>> > > > >> > > > > > valid
> >>> > > > >> > > > > > > > >> topic
> >>> > > > >> > > > > > > > >> > > UUID.
> >>> > > > >> > > > > > > > >> > > > > >  We
> >>> > > > >> > > > > > > > >> > > > > > >> > can't
> >>> > > > >> > > > > > > > >> > > > > > >> > > > use
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> null
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > for
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > this
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> because
> >>> we
> >>> > > can't
> >>> > > > >> > pass a
> >>> > > > >> > > > > null
> >>> > > > >> > > > > > > UUID
> >>> > > > >> > > > > > > > >> over
> >>> > > > >> > > > > > > > >> > > the
> >>> > > > >> > > > > > > > >> > > > > RPC
> >>> > > > >> > > > > > > > >> > > > > > >> > > protocol
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> (there
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > is no
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> special
> >>> > > pattern
> >>> > > > >> for
> >>> > > > >> > > null,
> >>> > > > >> > > > > nor
> >>> > > > >> > > > > > > do
> >>> > > > >> > > > > > > > we
> >>> > > > >> > > > > > > > >> > want
> >>> > > > >> > > > > > > > >> > > > to
> >>> > > > >> > > > > > > > >> > > > > > >> waste
> >>> > > > >> > > > > > > > >> > > > > > >> > > space
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> reserving
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > such
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> >>> pattern.)
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >>> Maybe I
> >>> > > > missed
> >>> > > > >> > it,
> >>> > > > >> > > > but
> >>> > > > >> > > > > > did
> >>> > > > >> > > > > > > > you
> >>> > > > >> > > > > > > > >> > > > describe
> >>> > > > >> > > > > > > > >> > > > > > >> > > "migration
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > of...
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > existing
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> topic[s]
> >>> > > without
> >>> > > > >> > topic
> >>> > > > >> > > > IDs"
> >>> > > > >> > > > > > in
> >>> > > > >> > > > > > > > >> detail
> >>> > > > >> > > > > > > > >> > in
> >>> > > > >> > > > > > > > >> > > > any
> >>> > > > >> > > > > > > > >> > > > > > >> > section?
> >>> > > > >> > > > > > > > >> > > > > > >> > > > It
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> seems
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > like
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > when
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > the
> new
> >>> > > > controller
> >>> > > > >> > > > becomes
> >>> > > > >> > > > > > > > active,
> >>> > > > >> > > > > > > > >> it
> >>> > > > >> > > > > > > > >> > > > should
> >>> > > > >> > > > > > > > >> > > > > > >> just
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > generate
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> random
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > UUIDs for
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > these,
> >>> and
> >>> > > write
> >>> > > > >> the
> >>> > > > >> > > > random
> >>> > > > >> > > > > > > UUIDs
> >>> > > > >> > > > > > > > >> back
> >>> > > > >> > > > > > > > >> > > to
> >>> > > > >> > > > > > > > >> > > > > > >> > ZooKeeper.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > It
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> would be
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > good
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > to
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > spell
> >>> that
> >>> > > out.
> >>> > > > >> We
> >>> > > > >> > > > should
> >>> > > > >> > > > > > make
> >>> > > > >> > > > > > > > it
> >>> > > > >> > > > > > > > >> > clear
> >>> > > > >> > > > > > > > >> > > > > that
> >>> > > > >> > > > > > > > >> > > > > > >> this
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > happens
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > regardless
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > of
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > the
> >>> > > inter-broker
> >>> > > > >> > > protocol
> >>> > > > >> > > > > > > version
> >>> > > > >> > > > > > > > >> > (it's
> >>> > > > >> > > > > > > > >> > > a
> >>> > > > >> > > > > > > > >> > > > > > >> > compatible
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > change).
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >>> > > > >> > > "LeaderAndIsrRequests
> >>> > > > >> > > > > > > > >> including an
> >>> > > > >> > > > > > > > >> > > > > > >> > > > is_every_partition
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> flag"
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > seems a
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > bit
> >>> wordy.
> >>> > > Can
> >>> > > > we
> >>> > > > >> > just
> >>> > > > >> > > > > call
> >>> > > > >> > > > > > > > these
> >>> > > > >> > > > > > > > >> > "full
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >>> LeaderAndIsrRequests"?
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > Then
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > the
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > RPC
> >>> field
> >>> > > could
> >>> > > > be
> >>> > > > >> > > named
> >>> > > > >> > > > > > > "full".
> >>> > > > >> > > > > > > > >> > Also,
> >>> > > > >> > > > > > > > >> > > it
> >>> > > > >> > > > > > > > >> > > > > > would
> >>> > > > >> > > > > > > > >> > > > > > >> > > > probably
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > be
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > better
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > for the
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > RPC
> >>> field to
> >>> > > be
> >>> > > > an
> >>> > > > >> > enum
> >>> > > > >> > > > of
> >>> > > > >> > > > > {
> >>> > > > >> > > > > > > > >> > > UNSPECIFIED,
> >>> > > > >> > > > > > > > >> > > > > > >> > > INCREMENTAL,
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > FULL
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> }, so
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > that
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > we
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > can
> >>> cleanly
> >>> > > > handle
> >>> > > > >> > old
> >>> > > > >> > > > > > versions
> >>> > > > >> > > > > > > > (by
> >>> > > > >> > > > > > > > >> > > > treating
> >>> > > > >> > > > > > > > >> > > > > > >> them
> >>> > > > >> > > > > > > > >> > > > > > >> > as
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> UNSPECIFIED)
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > In
> >>> the
> >>> > > > >> > > > > > LeaderAndIsrRequest
> >>> > > > >> > > > > > > > >> > section,
> >>> > > > >> > > > > > > > >> > > > you
> >>> > > > >> > > > > > > > >> > > > > > >> write
> >>> > > > >> > > > > > > > >> > > > > > >> > "A
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > final
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > deletion
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > event
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > will
> be
> >>> > > > secheduled
> >>> > > > >> > for
> >>> > > > >> > > X
> >>> > > > >> > > > ms
> >>> > > > >> > > > > > > after
> >>> > > > >> > > > > > > > >> the
> >>> > > > >> > > > > > > > >> > > > > > >> > > > LeaderAndIsrRequest
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > was
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > first
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> >>> received..."
> >>> > > I
> >>> > > > >> guess
> >>> > > > >> > > > the X
> >>> > > > >> > > > > > > was a
> >>> > > > >> > > > > > > > >> > > > > placeholder
> >>> > > > >> > > > > > > > >> > > > > > >> that
> >>> > > > >> > > > > > > > >> > > > > > >> > > you
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> intended
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > replace
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > before
> >>> > > posting?
> >>> > > > :)
> >>> > > > >> > In
> >>> > > > >> > > > any
> >>> > > > >> > > > > > > case,
> >>> > > > >> > > > > > > > >> this
> >>> > > > >> > > > > > > > >> > > > seems
> >>> > > > >> > > > > > > > >> > > > > > like
> >>> > > > >> > > > > > > > >> > > > > > >> > the
> >>> > > > >> > > > > > > > >> > > > > > >> > > > kind
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > of
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > thing
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > we'd
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > want a
> >>> > > > >> configuration
> >>> > > > >> > > for.
> >>> > > > >> > > > > > > Let's
> >>> > > > >> > > > > > > > >> > > describe
> >>> > > > >> > > > > > > > >> > > > > that
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > configuration
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> key
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > somewhere
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > in
> this
> >>> KIP,
> >>> > > > >> > including
> >>> > > > >> > > > what
> >>> > > > >> > > > > > its
> >>> > > > >> > > > > > > > >> > default
> >>> > > > >> > > > > > > > >> > > > > value
> >>> > > > >> > > > > > > > >> > > > > > >> is.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > We
> >>> > should
> >>> > > > >> > probably
> >>> > > > >> > > > also
> >>> > > > >> > > > > > > log a
> >>> > > > >> > > > > > > > >> > bunch
> >>> > > > >> > > > > > > > >> > > of
> >>> > > > >> > > > > > > > >> > > > > > >> messages
> >>> > > > >> > > > > > > > >> > > > > > >> > > at
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > WARN
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> level
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > when
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> >>> something is
> >>> > > > >> > scheduled
> >>> > > > >> > > > for
> >>> > > > >> > > > > > > > >> deletion,
> >>> > > > >> > > > > > > > >> > as
> >>> > > > >> > > > > > > > >> > > > > well.
> >>> > > > >> > > > > > > > >> > > > > > >> > (Maybe
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > this
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> was
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > assumed,
> but
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > it
> >>> would be
> >>> > > good
> >>> > > > >> to
> >>> > > > >> > > > mention
> >>> > > > >> > > > > > > it).
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > I
> >>> feel
> >>> > > like
> >>> > > > >> there
> >>> > > > >> > > > are a
> >>> > > > >> > > > > > few
> >>> > > > >> > > > > > > > >> > sections
> >>> > > > >> > > > > > > > >> > > > > that
> >>> > > > >> > > > > > > > >> > > > > > >> > should
> >>> > > > >> > > > > > > > >> > > > > > >> > > be
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> moved to
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > "rejected
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> >>> > alternatives."
> >>> > > > >> For
> >>> > > > >> > > > > example,
> >>> > > > >> > > > > > in
> >>> > > > >> > > > > > > > the
> >>> > > > >> > > > > > > > >> > > > > > DeleteTopics
> >>> > > > >> > > > > > > > >> > > > > > >> > > > section,
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> since
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > we're
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > not
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > going
> >>> to do
> >>> > > > >> option 1
> >>> > > > >> > or
> >>> > > > >> > > > 2,
> >>> > > > >> > > > > > > these
> >>> > > > >> > > > > > > > >> > should
> >>> > > > >> > > > > > > > >> > > be
> >>> > > > >> > > > > > > > >> > > > > > moved
> >>> > > > >> > > > > > > > >> > > > > > >> > into
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> "rejected
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> >>> > alternatives,"
> >>> > > > >> > rather
> >>> > > > >> > > > than
> >>> > > > >> > > > > > > > >> appearing
> >>> > > > >> > > > > > > > >> > > > > inline.
> >>> > > > >> > > > > > > > >> > > > > > >> > > Another
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > case
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> is
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > the
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > "Should
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > we
> >>> remove
> >>> > > topic
> >>> > > > >> name
> >>> > > > >> > > from
> >>> > > > >> > > > > the
> >>> > > > >> > > > > > > > >> protocol
> >>> > > > >> > > > > > > > >> > > > where
> >>> > > > >> > > > > > > > >> > > > > > >> > > possible"
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> section.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > This
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > is
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> clearly
> >>> > > > >> discussing a
> >>> > > > >> > > > design
> >>> > > > >> > > > > > > > >> > alternative
> >>> > > > >> > > > > > > > >> > > > that
> >>> > > > >> > > > > > > > >> > > > > > >> we're
> >>> > > > >> > > > > > > > >> > > > > > >> > > not
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> proposing
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> >>> implement:
> >>> > > > >> removing
> >>> > > > >> > the
> >>> > > > >> > > > > topic
> >>> > > > >> > > > > > > > name
> >>> > > > >> > > > > > > > >> > from
> >>> > > > >> > > > > > > > >> > > > > those
> >>> > > > >> > > > > > > > >> > > > > > >> > > > protocols.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Is
> >>> it
> >>> > > really
> >>> > > > >> > > > necessary
> >>> > > > >> > > > > to
> >>> > > > >> > > > > > > > have
> >>> > > > >> > > > > > > > >> a
> >>> > > > >> > > > > > > > >> > new
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >>> > > /admin/delete_topics_by_id
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > path
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > in
> >>> > ZooKeeper?
> >>> > > > It
> >>> > > > >> > seems
> >>> > > > >> > > > > like
> >>> > > > >> > > > > > we
> >>> > > > >> > > > > > > > >> don't
> >>> > > > >> > > > > > > > >> > > > really
> >>> > > > >> > > > > > > > >> > > > > > >> need
> >>> > > > >> > > > > > > > >> > > > > > >> > > this.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Whenever
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > there is
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > a new
> >>> > > > controller,
> >>> > > > >> > we'll
> >>> > > > >> > > > > send
> >>> > > > >> > > > > > > out
> >>> > > > >> > > > > > > > >> full
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > LeaderAndIsrRequests
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> which
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > will
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> trigger
> >>> the
> >>> > > > stale
> >>> > > > >> > > topics
> >>> > > > >> > > > to
> >>> > > > >> > > > > > be
> >>> > > > >> > > > > > > > >> cleaned
> >>> > > > >> > > > > > > > >> > > up.
> >>> > > > >> > > > > > > > >> > > > > >  The
> >>> > > > >> > > > > > > > >> > > > > > >> > > active
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > controller
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > will
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > also
> >>> send
> >>> > the
> >>> > > > full
> >>> > > > >> > > > > > > > >> LeaderAndIsrRequest
> >>> > > > >> > > > > > > > >> > > to
> >>> > > > >> > > > > > > > >> > > > > > >> brokers
> >>> > > > >> > > > > > > > >> > > > > > >> > > that
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > are
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> just
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > starting
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > up.
> >>> So we
> >>> > > > don't
> >>> > > > >> > > really
> >>> > > > >> > > > > > need
> >>> > > > >> > > > > > > > this
> >>> > > > >> > > > > > > > >> > kind
> >>> > > > >> > > > > > > > >> > > > of
> >>> > > > >> > > > > > > > >> > > > > > >> > two-phase
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > commit
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > (send
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > out
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> >>> > > > >> StopReplicasRequest,
> >>> > > > >> > > get
> >>> > > > >> > > > > ACKs
> >>> > > > >> > > > > > > > from
> >>> > > > >> > > > > > > > >> all
> >>> > > > >> > > > > > > > >> > > > > nodes,
> >>> > > > >> > > > > > > > >> > > > > > >> > commit
> >>> > > > >> > > > > > > > >> > > > > > >> > > by
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> removing
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> >>> > > > >> /admin/delete_topics
> >>> > > > >> > > > node)
> >>> > > > >> > > > > > any
> >>> > > > >> > > > > > > > >> more.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> You
> >>> > > mention
> >>> > > > >> that
> >>> > > > >> > > > > > > FetchRequest
> >>> > > > >> > > > > > > > >> will
> >>> > > > >> > > > > > > > >> > > now
> >>> > > > >> > > > > > > > >> > > > > > >> include
> >>> > > > >> > > > > > > > >> > > > > > >> > > UUID
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > to
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> avoid
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > issues
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > where
> >>> > requests
> >>> > > > are
> >>> > > > >> > made
> >>> > > > >> > > > to
> >>> > > > >> > > > > > > stale
> >>> > > > >> > > > > > > > >> > > > partitions.
> >>> > > > >> > > > > > > > >> > > > > > >> > > However,
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> adding a
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > UUID
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> >>> > > MetadataRequest
> >>> > > > is
> >>> > > > >> > > listed
> >>> > > > >> > > > > as
> >>> > > > >> > > > > > > > future
> >>> > > > >> > > > > > > > >> > > work,
> >>> > > > >> > > > > > > > >> > > > > out
> >>> > > > >> > > > > > > > >> > > > > > of
> >>> > > > >> > > > > > > > >> > > > > > >> > > scope
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > for
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> this
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > KIP.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > How
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > will
> the
> >>> > > client
> >>> > > > >> learn
> >>> > > > >> > > > what
> >>> > > > >> > > > > > the
> >>> > > > >> > > > > > > > >> topic
> >>> > > > >> > > > > > > > >> > > UUID
> >>> > > > >> > > > > > > > >> > > > > is,
> >>> > > > >> > > > > > > > >> > > > > > if
> >>> > > > >> > > > > > > > >> > > > > > >> > the
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > metadata
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > response
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> doesn't
> >>> > > include
> >>> > > > >> that
> >>> > > > >> > > > > > > information?
> >>> > > > >> > > > > > > > >> It
> >>> > > > >> > > > > > > > >> > > > seems
> >>> > > > >> > > > > > > > >> > > > > > like
> >>> > > > >> > > > > > > > >> > > > > > >> > > adding
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > the
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> UUID
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> >>> > > MetadataResponse
> >>> > > > >> > would
> >>> > > > >> > > be
> >>> > > > >> > > > > an
> >>> > > > >> > > > > > > > >> > improvement
> >>> > > > >> > > > > > > > >> > > > > here
> >>> > > > >> > > > > > > > >> > > > > > >> that
> >>> > > > >> > > > > > > > >> > > > > > >> > > > might
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > not
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> be
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > too
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > hard to
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > make.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >>> best,
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >>> Colin
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > On
> >>> Mon,
> >>> > > Sep
> >>> > > > 9,
> >>> > > > >> > > 2019,
> >>> > > > >> > > > at
> >>> > > > >> > > > > > > > 17:48,
> >>> > > > >> > > > > > > > >> > > Ryanne
> >>> > > > >> > > > > > > > >> > > > > > Dolan
> >>> > > > >> > > > > > > > >> > > > > > >> > > wrote:
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> >>> Lucas,
> >>> > > > this
> >>> > > > >> > would
> >>> > > > >> > > > be
> >>> > > > >> > > > > > > great.
> >>> > > > >> > > > > > > > >> I've
> >>> > > > >> > > > > > > > >> > > run
> >>> > > > >> > > > > > > > >> > > > > > into
> >>> > > > >> > > > > > > > >> > > > > > >> > > issues
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > with
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > topics
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > being
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> >>> > > > resurrected
> >>> > > > >> > > > > > accidentally,
> >>> > > > >> > > > > > > > >> since
> >>> > > > >> > > > > > > > >> > a
> >>> > > > >> > > > > > > > >> > > > > client
> >>> > > > >> > > > > > > > >> > > > > > >> > cannot
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > easily
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> distinguish
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> between
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> a
> >>> > > deleted
> >>> > > > >> topic
> >>> > > > >> > > > and a
> >>> > > > >> > > > > > new
> >>> > > > >> > > > > > > > >> topic
> >>> > > > >> > > > > > > > >> > > with
> >>> > > > >> > > > > > > > >> > > > > the
> >>> > > > >> > > > > > > > >> > > > > > >> same
> >>> > > > >> > > > > > > > >> > > > > > >> > > > name.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > I'd
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > need
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > the
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > ID
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> >>> > > accessible
> >>> > > > >> from
> >>> > > > >> > > the
> >>> > > > >> > > > > > > client
> >>> > > > >> > > > > > > > to
> >>> > > > >> > > > > > > > >> > > solve
> >>> > > > >> > > > > > > > >> > > > > that
> >>> > > > >> > > > > > > > >> > > > > > >> > issue,
> >>> > > > >> > > > > > > > >> > > > > > >> > > > but
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> this
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > is a
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > good
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > first
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> >>> step.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> >>> Ryanne
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> On
> >>> > Wed,
> >>> > > > Sep
> >>> > > > >> 4,
> >>> > > > >> > > 2019
> >>> > > > >> > > > > at
> >>> > > > >> > > > > > > 1:41
> >>> > > > >> > > > > > > > >> PM
> >>> > > > >> > > > > > > > >> > > Lucas
> >>> > > > >> > > > > > > > >> > > > > > >> > > Bradstreet <
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >>> > > lucas@confluent.io>
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > wrote:
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> >
> >>> Hi
> >>> > > all,
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> >
> >>> I
> >>> > > would
> >>> > > > >> like
> >>> > > > >> > to
> >>> > > > >> > > > > kick
> >>> > > > >> > > > > > > off
> >>> > > > >> > > > > > > > >> > > > discussion
> >>> > > > >> > > > > > > > >> > > > > of
> >>> > > > >> > > > > > > > >> > > > > > >> > > KIP-516,
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > an
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >>> implementation
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > of
> topic
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> >
> >>> IDs
> >>> > > for
> >>> > > > >> > Kafka.
> >>> > > > >> > > > > Topic
> >>> > > > >> > > > > > > IDs
> >>> > > > >> > > > > > > > >> aim
> >>> > > > >> > > > > > > > >> > to
> >>> > > > >> > > > > > > > >> > > > > solve
> >>> > > > >> > > > > > > > >> > > > > > >> topic
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> uniqueness
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > problems
> in
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Kafka,
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> >
> >>> > where
> >>> > > > >> > referring
> >>> > > > >> > > > to
> >>> > > > >> > > > > a
> >>> > > > >> > > > > > > > topic
> >>> > > > >> > > > > > > > >> by
> >>> > > > >> > > > > > > > >> > > name
> >>> > > > >> > > > > > > > >> > > > > > >> alone is
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> insufficient.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Such
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > cases
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> >
> >>> > > include
> >>> > > > >> when
> >>> > > > >> > a
> >>> > > > >> > > > > topic
> >>> > > > >> > > > > > > has
> >>> > > > >> > > > > > > > >> been
> >>> > > > >> > > > > > > > >> > > > > deleted
> >>> > > > >> > > > > > > > >> > > > > > >> and
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > recreated
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> with
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > the
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > same
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > name.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> >
> >>> > Unique
> >>> > > > >> > > > identifiers
> >>> > > > >> > > > > > will
> >>> > > > >> > > > > > > > >> help
> >>> > > > >> > > > > > > > >> > > > > simplify
> >>> > > > >> > > > > > > > >> > > > > > >> and
> >>> > > > >> > > > > > > > >> > > > > > >> > > > improve
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Kafka's
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > topic
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> deletion
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> >
> >>> > > process,
> >>> > > > >> as
> >>> > > > >> > > well
> >>> > > > >> > > > as
> >>> > > > >> > > > > > > > prevent
> >>> > > > >> > > > > > > > >> > > cases
> >>> > > > >> > > > > > > > >> > > > > > where
> >>> > > > >> > > > > > > > >> > > > > > >> > > brokers
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > may
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > incorrectly
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> interact
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> >
> >>> with
> >>> > > > stale
> >>> > > > >> > > > versions
> >>> > > > >> > > > > > of
> >>> > > > >> > > > > > > > >> topics.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > >
> >>> > > > >> > > > > > > > >> > > > > > >> >
> >>> > > > >> > > > > > > > >> > > > > > >>
> >>> > > > >> > > > > > > > >> > > > > >
> >>> > > > >> > > > > > > > >> > > > >
> >>> > > > >> > > > > > > > >> > > >
> >>> > > > >> > > > > > > > >> > >
> >>> > > > >> > > > > > > > >> >
> >>> > > > >> > > > > > > > >>
> >>> > > > >> > > > > > > >
> >>> > > > >> > > > > > >
> >>> > > > >> > > > > >
> >>> > > > >> > > > >
> >>> > > > >> > > >
> >>> > > > >> > >
> >>> > > > >> >
> >>> > > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> >
> >>> > > Looking
> >>> > > > >> > forward
> >>> > > > >> > > > to
> >>> > > > >> > > > > > your
> >>> > > > >> > > > > > > > >> > > thoughts.
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> >
> >>> > Lucas
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> >>> > > > >> > > > > > > > >> > > > > > >> > > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > > >
> >>> > > > >> > > > > > > > >> > > > > > >> > >
> >>> > > > >> > > > > > > > >> > > > > > >> >
> >>> > > > >> > > > > > > > >> > > > > > >>
> >>> > > > >> > > > > > > > >> > > > > > >
> >>> > > > >> > > > > > > > >> > > > > >
> >>> > > > >> > > > > > > > >> > > > >
> >>> > > > >> > > > > > > > >> > > >
> >>> > > > >> > > > > > > > >> > >
> >>> > > > >> > > > > > > > >> >
> >>> > > > >> > > > > > > > >>
> >>> > > > >> > > > > > > > >
> >>> > > > >> > > > > > > >
> >>> > > > >> > > > > > >
> >>> > > > >> > > > > >
> >>> > > > >> > > > >
> >>> > > > >> > > >
> >>> > > > >> > >
> >>> > > > >> >
> >>> > > > >>
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> >>
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Justine Olshan <jo...@confluent.io>]

Hi all,

I was thinking about this KIP some more. We are removing the topic name
from many of these request types, and some of them (eg. Fetch) use
TopicPartition as keys in a map when handling the request. I was thinking
it would make sense to make some sort of TopicIdPartition with topic name,
topic id, and partition fields (name and id would potentially be optional
types since different request versions will contain only one of the two) I
was wondering if it made sense to include this as a public facing API
similar to TopicPartition. If so, I will add this class to the KIP as well.

Thanks,
Justine

On Mon, Oct 5, 2020 at 8:52 AM Justine Olshan <jo...@confluent.io> wrote:

> Hi all,
> I've made a few minor changes to the UUID class. I've added a few more
> methods to support UUIDs more generally, by including a constructor and a
> method to generate random UUIDs. Both of these will be useful for testing.
>
> Let me know if there are any other areas for discussion.
>
> Thank you,
> Justine
>
> On Thu, Oct 1, 2020 at 5:29 PM Justine Olshan <jo...@confluent.io>
> wrote:
>
>> Hi Jun,
>> Thanks for looking at it again. Here's the new spec. (I fixed the typo in
>> it too.)
>> {"name": "id", "type": "string", "doc": option[UUID]}
>>
>> Justine
>>
>>
>> On Thu, Oct 1, 2020 at 5:03 PM Jun Rao <ju...@confluent.io> wrote:
>>
>>> Hi, Justine,
>>>
>>> Thanks for the update. The KIP looks good to me now. Just a minor comment
>>> below.
>>>
>>> 30. Perhaps "option[UUID]" can be put in the doc.
>>>
>>> Jun
>>>
>>> On Thu, Oct 1, 2020 at 3:28 PM Justine Olshan <jo...@confluent.io>
>>> wrote:
>>>
>>> > Hi Jun,
>>> > Thanks for the response!
>>> >
>>> > 30. I think I might have changed this in between. The current version
>>> > says:  {"name":
>>> > "id", "type": "option[UUID]"}, "doc": topic id}
>>> > I have switched to the option type to cover the migration case where a
>>> > TopicZNode does not yet have a topic ID.
>>> > I understand that due to json, this field is written as a string, so
>>> if I
>>> > should move the "option[uuid]" to the doc field and the type should be
>>> > "string" please let me know.
>>> >
>>> > 40. I've added a definition for UUID.
>>> > 41,42. Fixed
>>> >
>>> > Thank you,
>>> > Justine
>>> >
>>> > On Wed, Sep 30, 2020 at 1:15 PM Jun Rao <ju...@confluent.io> wrote:
>>> >
>>> > > Hi, Justine,
>>> > >
>>> > > Thanks for the summary. Just a few minor comments blow.
>>> > >
>>> > > 30.  {"name": "id", "type": "string", "doc": "version id"}}: The doc
>>> > should
>>> > > say UUID. The issue seems unfixed.
>>> > >
>>> > > 40. Since UUID is public facing, could you include its definition?
>>> > >
>>> > > 41. StopReplicaResponse still includes the topic field.
>>> > >
>>> > > 42. "It is unnecessary to include the name of the topic in the
>>> following
>>> > > Request/Response calls" It would be useful to include all modified
>>> > requests
>>> > > (e.g. produce) in the list below.
>>> > >
>>> > > Thanks,
>>> > >
>>> > > Jun
>>> > >
>>> > >
>>> > > On Wed, Sep 30, 2020 at 10:17 AM Justine Olshan <
>>> jolshan@confluent.io>
>>> > > wrote:
>>> > >
>>> > > > Hello again,
>>> > > >
>>> > > > I've taken some time to discuss some of the remaining points
>>> brought up
>>> > > by
>>> > > > the previous emails offline. Here are some of the conclusions.
>>> > > >
>>> > > > 1. Directory Structure:
>>> > > > There was some talk about whether the directory structure should be
>>> > > changed
>>> > > > to replace all topic names with topic IDs.
>>> > > > This will be a useful change, but will prevent downgrades. It will
>>> be
>>> > > best
>>> > > > to wait until a major release, likely alongside KIP-500 changes
>>> that
>>> > will
>>> > > > prevent downgrades. I've updated the KIP to include this change
>>> with
>>> > the
>>> > > > note about migration and deprecation.
>>> > > >
>>> > > >
>>> > >
>>> >
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-log.dirlayout
>>> > > >
>>> > > > 2. Partition Metadata file
>>> > > > There was some disagreement about the inclusion of the partition
>>> > metadata
>>> > > > file.
>>> > > > This file will be needed to persist the topic ID, especially while
>>> we
>>> > > still
>>> > > > have the old directory structure. Even after the changes, this
>>> file can
>>> > > be
>>> > > > useful for debugging and recovery.
>>> > > > Creating a single mapping file from topic names to topic IDs was
>>> > > > considered, but ultimately scrapped as it would not be as easy to
>>> > > maintain.
>>> > > >
>>> > > >
>>> > >
>>> >
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PartitionMetadatafile
>>> > > >
>>> > > >
>>> > > >
>>> > >
>>> >
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PersistingTopicIDs
>>> > > >
>>> > > > 3. Produce Protocols
>>> > > > After some further discussion, this replacing the topic name with
>>> topic
>>> > > ID
>>> > > > in these protocols has been added to the KIP.
>>> > > > This will cut down on the size of the protocol. Since changes to
>>> fetch
>>> > > are
>>> > > > included, it does make sense to update these protocols.
>>> > > >
>>> > > >
>>> > >
>>> >
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-Produce
>>> > > >
>>> > > > 4. KIP-500 Compatibility
>>> > > > After some discussion with those involved with KIP-500, it seems
>>> best
>>> > to
>>> > > > use a sentinel topic ID for the metadata topic that is used before
>>> the
>>> > > > first controller election.
>>> > > > We had an issue where this metadata topic may not be assigned an ID
>>> > > before
>>> > > > utilizing the Vote and Fetch protocols. It was decided to reserve a
>>> > > unique
>>> > > > ID for this topic to be used until the controller could give the
>>> topic
>>> > a
>>> > > > unique ID.
>>> > > > Having the topic keep the sentinel ID (not be reassigned to a
>>> unique
>>> > ID)
>>> > > > was considered, but it seemed like a good idea to leave the option
>>> open
>>> > > for
>>> > > > the metadata topic to have a unique ID in cases where it would
>>> need to
>>> > be
>>> > > > differentiated from other clusters' metadata topics. (ex. tiered
>>> > storage)
>>> > > >
>>> > > >
>>> > >
>>> >
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-CompatibilitywithKIP-500
>>> > > >
>>> > > > I've also split up the KIP into sub-tasks on the JIRA. Hopefully
>>> this
>>> > > will
>>> > > > provide a better idea about what tasks we have, and eventually
>>> provide
>>> > a
>>> > > > place to see what's done and what is left.
>>> > > > If there is a task I am missing, please let me know!
>>> > > > https://issues.apache.org/jira/browse/KAFKA-8872
>>> > > >
>>> > > > Of course, these decisions are not set in stone, and I would love
>>> to
>>> > hear
>>> > > > any feedback.
>>> > > >
>>> > > > Thanks,
>>> > > > Justine
>>> > > >
>>> > > > On Mon, Sep 28, 2020 at 11:38 AM Justine Olshan <
>>> jolshan@confluent.io>
>>> > > > wrote:
>>> > > >
>>> > > > > Hello all,
>>> > > > >
>>> > > > > I just wanted to follow up on this discussion. Did we come to an
>>> > > > > understanding about the directory structure?
>>> > > > >
>>> > > > > I think the biggest question here is what is acceptable to leave
>>> out
>>> > > due
>>> > > > > to scope vs. what is considered to be too much tech debt.
>>> > > > > This KIP is already pretty large with the number of changes, but
>>> it
>>> > > also
>>> > > > > makes sense to do things correctly, so I'd love to hear
>>> everyone's
>>> > > > thoughts.
>>> > > > >
>>> > > > > Thanks,
>>> > > > > Justine
>>> > > > >
>>> > > > > On Fri, Sep 25, 2020 at 8:19 AM Lucas Bradstreet <
>>> lucas@confluent.io
>>> > >
>>> > > > > wrote:
>>> > > > >
>>> > > > >> Hi Ismael,
>>> > > > >>
>>> > > > >> If you do not store it in a metadata file or in the directory
>>> > > structure
>>> > > > >> would you then
>>> > > > >> require the LeaderAndIsrRequest following startup to give you
>>> some
>>> > > > notion
>>> > > > >> of
>>> > > > >> topic name in memory? We would still need this information for
>>> the
>>> > > older
>>> > > > >> protocols, but
>>> > > > >> perhaps this is what's meant by tech debt.
>>> > > > >>
>>> > > > >> Once we're free of the old non-topicID requests then I think you
>>> > > > wouldn't
>>> > > > >> need to retain the topic name.
>>> > > > >> I think the ability to easily look up topic names associated
>>> with
>>> > > > >> partition
>>> > > > >> directories would still be missed
>>> > > > >> when diagnosing issues, though maybe it wouldn't be a deal
>>> breaker
>>> > > with
>>> > > > >> the
>>> > > > >> right tooling.
>>> > > > >>
>>> > > > >> Thanks,
>>> > > > >>
>>> > > > >> Lucas
>>> > > > >>
>>> > > > >> On Fri, Sep 25, 2020 at 7:55 AM Ismael Juma <is...@juma.me.uk>
>>> > > wrote:
>>> > > > >>
>>> > > > >> > Hi Lucas,
>>> > > > >> >
>>> > > > >> > Why would you include the name and id? I think you'd want to
>>> > remove
>>> > > > the
>>> > > > >> > name from the directory name right? Jason's suggestion was
>>> that if
>>> > > you
>>> > > > >> > remove the name from the directory, then why would you need
>>> the id
>>> > > > name
>>> > > > >> > mapping file?
>>> > > > >> >
>>> > > > >> > Ismael
>>> > > > >> >
>>> > > > >> > On Thu, Sep 24, 2020 at 4:24 PM Lucas Bradstreet <
>>> > > lucas@confluent.io>
>>> > > > >> > wrote:
>>> > > > >> >
>>> > > > >> > > > 2. Part of the usage of the file is to have persistent
>>> storage
>>> > > of
>>> > > > >> the
>>> > > > >> > > topic
>>> > > > >> > > ID and use it to compare with the ID supplied in the
>>> > LeaderAndIsr
>>> > > > >> > Request.
>>> > > > >> > > There is some discussion in the KIP about changes to the
>>> > directory
>>> > > > >> > > structure, but I believe directory changes were considered
>>> to be
>>> > > out
>>> > > > >> of
>>> > > > >> > > scope when the KIP was written.
>>> > > > >> > >
>>> > > > >> > >
>>> > > > >> > > Yeah, I was hoping to get a better understanding of why it
>>> was
>>> > > taken
>>> > > > >> out
>>> > > > >> > of
>>> > > > >> > > scope. Perhaps Lucas Bradstreet might have more insight
>>> about
>>> > the
>>> > > > >> > decision.
>>> > > > >> > > Basically my point is that we have to create additional
>>> > > > infrastructure
>>> > > > >> > here
>>> > > > >> > > to support the name/id mapping, so I wanted to understand
>>> if we
>>> > > just
>>> > > > >> > > consider this a sort of tech debt. It would be useful to
>>> cover
>>> > how
>>> > > > we
>>> > > > >> > > handle the case when this file gets corrupted. Seems like we
>>> > just
>>> > > > >> have to
>>> > > > >> > > trust that it matches whatever the controller tells us and
>>> > rewrite
>>> > > > it?
>>> > > > >> > >
>>> > > > >> > >
>>> > > > >> > > Hi Jason, Justine,
>>> > > > >> > >
>>> > > > >> > > My thought process is that we will likely need the metadata
>>> file
>>> > > > >> whether
>>> > > > >> > we
>>> > > > >> > > rename the directories or not.
>>> > > > >> > > Linux supports filenames of up to 255 bytes and that would
>>> not
>>> > be
>>> > > > >> enough
>>> > > > >> > to
>>> > > > >> > > support a directory name
>>> > > > >> > >  including both the name and topic ID. Given that fact, it
>>> > seemed
>>> > > > >> > > reasonable to add the metadata file
>>> > > > >> > > and leave the directory rename until some time in the future
>>> > > > (possibly
>>> > > > >> > > never).
>>> > > > >> > >
>>> > > > >> > > If the file does get corrupted I think you're right. We
>>> would
>>> > > either
>>> > > > >> have
>>> > > > >> > > to trust it matches what the controller tells us
>>> > > > >> > >  or error out and let an administrator resolve it by
>>> checking
>>> > > across
>>> > > > >> > > replicas for consistency.
>>> > > > >> > >
>>> > > > >> > > Lucas
>>> > > > >> > >
>>> > > > >> > >
>>> > > > >> > > On Thu, Sep 24, 2020 at 3:41 PM Jason Gustafson <
>>> > > jason@confluent.io
>>> > > > >
>>> > > > >> > > wrote:
>>> > > > >> > >
>>> > > > >> > > > Thanks Justine. Responses below:
>>> > > > >> > > >
>>> > > > >> > > > > 1. Yes, the directory will still be based on the topic
>>> > names.
>>> > > > >> > > > LeaderAndIsrRequest is one of the few requests that will
>>> still
>>> > > > >> contain
>>> > > > >> > > the
>>> > > > >> > > > topic name. So I think we have this covered. Sorry for
>>> > > confusion.
>>> > > > >> > > >
>>> > > > >> > > > Ah, you're right. My eyes passed right over the field.
>>> > > > >> > > >
>>> > > > >> > > > > 2. Part of the usage of the file is to have persistent
>>> > storage
>>> > > > of
>>> > > > >> the
>>> > > > >> > > > topic
>>> > > > >> > > > ID and use it to compare with the ID supplied in the
>>> > > LeaderAndIsr
>>> > > > >> > > Request.
>>> > > > >> > > > There is some discussion in the KIP about changes to the
>>> > > directory
>>> > > > >> > > > structure, but I believe directory changes were
>>> considered to
>>> > be
>>> > > > >> out of
>>> > > > >> > > > scope when the KIP was written.
>>> > > > >> > > >
>>> > > > >> > > > Yeah, I was hoping to get a better understanding of why
>>> it was
>>> > > > taken
>>> > > > >> > out
>>> > > > >> > > of
>>> > > > >> > > > scope. Perhaps Lucas Bradstreet might have more insight
>>> about
>>> > > the
>>> > > > >> > > decision.
>>> > > > >> > > > Basically my point is that we have to create additional
>>> > > > >> infrastructure
>>> > > > >> > > here
>>> > > > >> > > > to support the name/id mapping, so I wanted to understand
>>> if
>>> > we
>>> > > > just
>>> > > > >> > > > consider this a sort of tech debt. It would be useful to
>>> cover
>>> > > how
>>> > > > >> we
>>> > > > >> > > > handle the case when this file gets corrupted. Seems like
>>> we
>>> > > just
>>> > > > >> have
>>> > > > >> > to
>>> > > > >> > > > trust that it matches whatever the controller tells us and
>>> > > rewrite
>>> > > > >> it?
>>> > > > >> > > >
>>> > > > >> > > > > 3. I think this is a good point, but I again I wonder
>>> about
>>> > > the
>>> > > > >> scope
>>> > > > >> > > of
>>> > > > >> > > > the KIP. Most of the changes mentioned in the KIP are for
>>> > > > supporting
>>> > > > >> > > topic
>>> > > > >> > > > deletion and I believe that is why the produce request was
>>> > > listed
>>> > > > >> under
>>> > > > >> > > > future work.
>>> > > > >> > > >
>>> > > > >> > > > That's fair. I brought it up since `Fetch` is already
>>> > included.
>>> > > If
>>> > > > >> > we've
>>> > > > >> > > > got `Metadata` and `Fetch`, seems we may as well do
>>> `Produce`
>>> > > and
>>> > > > >> save
>>> > > > >> > an
>>> > > > >> > > > extra kip. No strong objection though if you want to
>>> leave it
>>> > > out.
>>> > > > >> > > >
>>> > > > >> > > >
>>> > > > >> > > > -Jason
>>> > > > >> > > >
>>> > > > >> > > >
>>> > > > >> > > > On Thu, Sep 24, 2020 at 3:26 PM Justine Olshan <
>>> > > > >> jolshan@confluent.io>
>>> > > > >> > > > wrote:
>>> > > > >> > > >
>>> > > > >> > > > > Hi Jason,
>>> > > > >> > > > >
>>> > > > >> > > > > Thanks for your comments.
>>> > > > >> > > > >
>>> > > > >> > > > > 1. Yes, the directory will still be based on the topic
>>> > names.
>>> > > > >> > > > > LeaderAndIsrRequest is one of the few requests that will
>>> > still
>>> > > > >> > contain
>>> > > > >> > > > the
>>> > > > >> > > > > topic name. So I think we have this covered. Sorry for
>>> > > > confusion.
>>> > > > >> > > > >
>>> > > > >> > > > > 2. Part of the usage of the file is to have persistent
>>> > storage
>>> > > > of
>>> > > > >> the
>>> > > > >> > > > topic
>>> > > > >> > > > > ID and use it to compare with the ID supplied in the
>>> > > > LeaderAndIsr
>>> > > > >> > > > Request.
>>> > > > >> > > > > There is some discussion in the KIP about changes to the
>>> > > > directory
>>> > > > >> > > > > structure, but I believe directory changes were
>>> considered
>>> > to
>>> > > be
>>> > > > >> out
>>> > > > >> > of
>>> > > > >> > > > > scope when the KIP was written.
>>> > > > >> > > > >
>>> > > > >> > > > > 3. I think this is a good point, but I again I wonder
>>> about
>>> > > the
>>> > > > >> scope
>>> > > > >> > > of
>>> > > > >> > > > > the KIP. Most of the changes mentioned in the KIP are
>>> for
>>> > > > >> supporting
>>> > > > >> > > > topic
>>> > > > >> > > > > deletion and I believe that is why the produce request
>>> was
>>> > > > listed
>>> > > > >> > under
>>> > > > >> > > > > future work.
>>> > > > >> > > > >
>>> > > > >> > > > > 4. This sounds like it might be a good solution, but I
>>> will
>>> > > need
>>> > > > >> to
>>> > > > >> > > > discuss
>>> > > > >> > > > > more with KIP-500 folks to get the details right.
>>> > > > >> > > > >
>>> > > > >> > > > > Thanks,
>>> > > > >> > > > > Justine
>>> > > > >> > > > >
>>> > > > >> > > > > On Thu, Sep 24, 2020 at 12:30 PM Jason Gustafson <
>>> > > > >> jason@confluent.io
>>> > > > >> > >
>>> > > > >> > > > > wrote:
>>> > > > >> > > > >
>>> > > > >> > > > > > Hi Justine,
>>> > > > >> > > > > >
>>> > > > >> > > > > > Thanks for picking up this work. I have a few
>>> > > > >> questions/comments:
>>> > > > >> > > > > >
>>> > > > >> > > > > > 1. It sounds like the directory structure is still
>>> going
>>> > to
>>> > > be
>>> > > > >> > based
>>> > > > >> > > on
>>> > > > >> > > > > > topic names. Do I have that right? One complication is
>>> > that
>>> > > > the
>>> > > > >> > > > > > LeaderAndIsr request does not include the topic name
>>> any
>>> > > > longer.
>>> > > > >> > This
>>> > > > >> > > > > means
>>> > > > >> > > > > > that a replica must wait for the UpdateMetadata
>>> request to
>>> > > > >> arrive
>>> > > > >> > > with
>>> > > > >> > > > > the
>>> > > > >> > > > > > topic name mapping before it can create the
>>> directory. I
>>> > > > wonder
>>> > > > >> if
>>> > > > >> > it
>>> > > > >> > > > > would
>>> > > > >> > > > > > be simpler to avoid assumptions on the ordering of
>>> > > > >> UpdateMetadata
>>> > > > >> > and
>>> > > > >> > > > let
>>> > > > >> > > > > > LeaderAndIsr include the topic name as well. Feels
>>> like we
>>> > > are
>>> > > > >> not
>>> > > > >> > > > saving
>>> > > > >> > > > > > that much by excluding it.
>>> > > > >> > > > > >
>>> > > > >> > > > > > 2. On a related note, it seems that the reason we
>>> have the
>>> > > > >> > partition
>>> > > > >> > > > > > metadata file is because we are not changing the
>>> directory
>>> > > > >> > structure.
>>> > > > >> > > > We
>>> > > > >> > > > > > need it so that we remember which directories map to
>>> which
>>> > > > topic
>>> > > > >> > id.
>>> > > > >> > > I
>>> > > > >> > > > > > think it would be useful to add some detail about why
>>> > > changing
>>> > > > >> the
>>> > > > >> > > > > > directory layout was rejected. Basically trying to
>>> > > understand
>>> > > > >> how
>>> > > > >> > > much
>>> > > > >> > > > > > effort we are saving by skipping this step if we have
>>> to
>>> > > > >> implement
>>> > > > >> > > this
>>> > > > >> > > > > > additional bookkeeping. One concern I have for
>>> example is
>>> > > that
>>> > > > >> the
>>> > > > >> > > > > > partition metadata file gets corrupt and then all
>>> bets are
>>> > > off
>>> > > > >> as
>>> > > > >> > far
>>> > > > >> > > > as
>>> > > > >> > > > > > topic consistency.
>>> > > > >> > > > > >
>>> > > > >> > > > > > 3. Is it worth adding support now for topic ids in the
>>> > > > `Produce`
>>> > > > >> > > > request
>>> > > > >> > > > > > now? Overhead is mentioned as one of the motivations
>>> and
>>> > the
>>> > > > >> best
>>> > > > >> > > APIs
>>> > > > >> > > > to
>>> > > > >> > > > > > get savings from are `Produce` and `Fetch`.
>>> > > > >> > > > > >
>>> > > > >> > > > > > 4. When it comes to bootstrapping the metadata topic
>>> with
>>> > > > >> respect
>>> > > > >> > to
>>> > > > >> > > > > > KIP-500, one suggestion would be to reserve a sentinel
>>> > topic
>>> > > > ID
>>> > > > >> > which
>>> > > > >> > > > can
>>> > > > >> > > > > > be used initially by new brokers. When the first
>>> > controller
>>> > > is
>>> > > > >> > > elected,
>>> > > > >> > > > > it
>>> > > > >> > > > > > can assign a topicId to the metadata topic. This is a
>>> bit
>>> > > > >> similar
>>> > > > >> > to
>>> > > > >> > > > how
>>> > > > >> > > > > we
>>> > > > >> > > > > > were planning to generate the clusterId.
>>> > > > >> > > > > >
>>> > > > >> > > > > > Thanks,
>>> > > > >> > > > > > Jason
>>> > > > >> > > > > >
>>> > > > >> > > > > >
>>> > > > >> > > > > > On Thu, Sep 24, 2020 at 11:10 AM Jun Rao <
>>> > jun@confluent.io>
>>> > > > >> wrote:
>>> > > > >> > > > > >
>>> > > > >> > > > > > > Hi, Justine,
>>> > > > >> > > > > > >
>>> > > > >> > > > > > > Thanks for the updated KIP. A few more comments
>>> below.
>>> > > > >> > > > > > >
>>> > > > >> > > > > > > 30.  {"name": "id", "type": "string", "doc":
>>> "version
>>> > > id"}}:
>>> > > > >> The
>>> > > > >> > > doc
>>> > > > >> > > > > > should
>>> > > > >> > > > > > > say UUID.
>>> > > > >> > > > > > >
>>> > > > >> > > > > > > 31. LeaderAndIsrResponse v5 and StopReplicaResponse
>>> v4 :
>>> > > It
>>> > > > >> seems
>>> > > > >> > > > there
>>> > > > >> > > > > > is
>>> > > > >> > > > > > > no need to add topic_id at partitions level.
>>> > > > >> > > > > > >
>>> > > > >> > > > > > > 32. Regarding partition metadata file. Perhaps the
>>> key
>>> > can
>>> > > > be
>>> > > > >> a
>>> > > > >> > > > single
>>> > > > >> > > > > > > word, sth like the following.
>>> > > > >> > > > > > > version: 0
>>> > > > >> > > > > > > topic_id: 46bdb63f-9e8d-4a38-bf7b-ee4eb2a794e4
>>> > > > >> > > > > > >
>>> > > > >> > > > > > > 33. Another tricky thing that I realized is how to
>>> > support
>>> > > > the
>>> > > > >> > > > metadata
>>> > > > >> > > > > > > topic introduced in KIP-595. It's a bit tricky to
>>> > assign a
>>> > > > >> UUID
>>> > > > >> > to
>>> > > > >> > > > the
>>> > > > >> > > > > > > metadata topic since we have a chicken-and-egg
>>> problem.
>>> > > The
>>> > > > >> > > > controller
>>> > > > >> > > > > > > needs to persist the UUID in the metadata topic in
>>> order
>>> > > to
>>> > > > >> > assign
>>> > > > >> > > > one
>>> > > > >> > > > > > > successfully, but the metadata topic is needed to
>>> elect
>>> > a
>>> > > > >> > > controller
>>> > > > >> > > > > > > (KIP-631). So, this probably needs a bit more
>>> thought.
>>> > > > >> > > > > > >
>>> > > > >> > > > > > > Jun
>>> > > > >> > > > > > >
>>> > > > >> > > > > > > On Thu, Sep 24, 2020 at 3:04 AM Ismael Juma <
>>> > > > >> ismael@juma.me.uk>
>>> > > > >> > > > wrote:
>>> > > > >> > > > > > >
>>> > > > >> > > > > > > > Also, can we provide more details on how the
>>> Partition
>>> > > > >> Metadata
>>> > > > >> > > > file
>>> > > > >> > > > > > will
>>> > > > >> > > > > > > > be used?
>>> > > > >> > > > > > > >
>>> > > > >> > > > > > > > Ismael
>>> > > > >> > > > > > > >
>>> > > > >> > > > > > > > On Thu, Sep 24, 2020 at 3:01 AM Ismael Juma <
>>> > > > >> ismael@juma.me.uk
>>> > > > >> > >
>>> > > > >> > > > > wrote:
>>> > > > >> > > > > > > >
>>> > > > >> > > > > > > > > Hi Justine,
>>> > > > >> > > > > > > > >
>>> > > > >> > > > > > > > > I think we need to update the "Rejected
>>> > Alternatives"
>>> > > > >> section
>>> > > > >> > > to
>>> > > > >> > > > > take
>>> > > > >> > > > > > > > into
>>> > > > >> > > > > > > > > account that the proposal now removes the topic
>>> name
>>> > > > from
>>> > > > >> the
>>> > > > >> > > > fetch
>>> > > > >> > > > > > > > > request. Also, if we are removing it from the
>>> Fetch
>>> > > > >> request,
>>> > > > >> > > does
>>> > > > >> > > > > it
>>> > > > >> > > > > > > make
>>> > > > >> > > > > > > > > sense not to remove it from similar requests
>>> like
>>> > > > >> > > > > ListOffsetRequest?
>>> > > > >> > > > > > > > >
>>> > > > >> > > > > > > > > Ismael
>>> > > > >> > > > > > > > >
>>> > > > >> > > > > > > > > On Thu, Sep 24, 2020 at 2:46 AM David Jacot <
>>> > > > >> > > djacot@confluent.io
>>> > > > >> > > > >
>>> > > > >> > > > > > > wrote:
>>> > > > >> > > > > > > > >
>>> > > > >> > > > > > > > >> Hi Justine,
>>> > > > >> > > > > > > > >>
>>> > > > >> > > > > > > > >> Thanks for the KIP. I finally had time to read
>>> it
>>> > :).
>>> > > > It
>>> > > > >> is
>>> > > > >> > a
>>> > > > >> > > > > great
>>> > > > >> > > > > > > > >> improvement.
>>> > > > >> > > > > > > > >>
>>> > > > >> > > > > > > > >> I have a few comments/questions:
>>> > > > >> > > > > > > > >>
>>> > > > >> > > > > > > > >> 1. It seems that the schema of the
>>> > StopReplicaRequest
>>> > > > is
>>> > > > >> > > > slightly
>>> > > > >> > > > > > > > >> outdated.
>>> > > > >> > > > > > > > >> We
>>> > > > >> > > > > > > > >> did some changes as part of KIP-570. V3 is
>>> already
>>> > > > >> organized
>>> > > > >> > > by
>>> > > > >> > > > > > > topics.
>>> > > > >> > > > > > > > >>
>>> > > > >> > > > > > > > >> 2. I just want to make sure that I understand
>>> the
>>> > > > >> > > reconciliation
>>> > > > >> > > > > > > > >> logic correctly. When
>>> > > > >> > > > > > > > >> an "INCREMENTAL" LeaderAndIsr Request is
>>> received,
>>> > > the
>>> > > > >> > broker
>>> > > > >> > > > will
>>> > > > >> > > > > > > also
>>> > > > >> > > > > > > > >> reconcile
>>> > > > >> > > > > > > > >> when the local uuid does not match the uuid in
>>> the
>>> > > > >> request,
>>> > > > >> > > > right?
>>> > > > >> > > > > > In
>>> > > > >> > > > > > > > this
>>> > > > >> > > > > > > > >> case, the
>>> > > > >> > > > > > > > >> local log is staged for deletion.
>>> > > > >> > > > > > > > >>
>>> > > > >> > > > > > > > >> 3. In the documentation of the `
>>> > > > >> delete.stale.topic.delay.ms
>>> > > > >> > `
>>> > > > >> > > > > > config,
>>> > > > >> > > > > > > it
>>> > > > >> > > > > > > > >> says "When a
>>> > > > >> > > > > > > > >> FULL LeaderAndIsrRequest is received..." but I
>>> > > suppose
>>> > > > it
>>> > > > >> > > > applies
>>> > > > >> > > > > to
>>> > > > >> > > > > > > > both
>>> > > > >> > > > > > > > >> types.
>>> > > > >> > > > > > > > >>
>>> > > > >> > > > > > > > >> Best,
>>> > > > >> > > > > > > > >> David
>>> > > > >> > > > > > > > >>
>>> > > > >> > > > > > > > >> On Thu, Sep 24, 2020 at 1:40 AM Justine Olshan
>>> <
>>> > > > >> > > > > > jolshan@confluent.io>
>>> > > > >> > > > > > > > >> wrote:
>>> > > > >> > > > > > > > >>
>>> > > > >> > > > > > > > >> > Hi Jun,
>>> > > > >> > > > > > > > >> >
>>> > > > >> > > > > > > > >> > Thanks for the comments. I apologize for
>>> some of
>>> > > the
>>> > > > >> typos
>>> > > > >> > > and
>>> > > > >> > > > > > > > >> confusion.
>>> > > > >> > > > > > > > >> > I’ve updated the KIP to fix some of the
>>> issues
>>> > you
>>> > > > >> > > mentioned.
>>> > > > >> > > > > > > > >> >
>>> > > > >> > > > > > > > >> > 20.2 I’ve changed the type to String.
>>> > > > >> > > > > > > > >> > 20.1/3 I’ve updated the TopicZNode to fix
>>> > > formatting
>>> > > > >> and
>>> > > > >> > > > reflect
>>> > > > >> > > > > > the
>>> > > > >> > > > > > > > >> latest
>>> > > > >> > > > > > > > >> > version before this change.
>>> > > > >> > > > > > > > >> >
>>> > > > >> > > > > > > > >> > 21. You are correct and I’ve removed this
>>> line.
>>> > > I’ve
>>> > > > >> also
>>> > > > >> > > > added
>>> > > > >> > > > > a
>>> > > > >> > > > > > > line
>>> > > > >> > > > > > > > >> > mentioning an IBP bump is necessary for
>>> migration
>>> > > > >> > > > > > > > >> >
>>> > > > >> > > > > > > > >> > 22. I think the wording was unclear but your
>>> > > summary
>>> > > > is
>>> > > > >> > what
>>> > > > >> > > > was
>>> > > > >> > > > > > > > >> intended
>>> > > > >> > > > > > > > >> > by this line. I’ve updated to make this point
>>> > more
>>> > > > >> clear.
>>> > > > >> > > > > “Remove
>>> > > > >> > > > > > > > >> deleted
>>> > > > >> > > > > > > > >> > topics from replicas by sending
>>> > StopReplicaRequest
>>> > > V3
>>> > > > >> > before
>>> > > > >> > > > the
>>> > > > >> > > > > > IBP
>>> > > > >> > > > > > > > >> bump
>>> > > > >> > > > > > > > >> > using the old logic, and using V4 and the new
>>> > logic
>>> > > > >> with
>>> > > > >> > > topic
>>> > > > >> > > > > IDs
>>> > > > >> > > > > > > > after
>>> > > > >> > > > > > > > >> > the IBP bump.”
>>> > > > >> > > > > > > > >> >
>>> > > > >> > > > > > > > >> > 23. I’ve removed the unspecified type from
>>> the
>>> > KIP
>>> > > > and
>>> > > > >> > > mention
>>> > > > >> > > > > > that
>>> > > > >> > > > > > > > IBP
>>> > > > >> > > > > > > > >> > will be used to handle this request. “IBP
>>> will be
>>> > > > used
>>> > > > >> to
>>> > > > >> > > > > > determine
>>> > > > >> > > > > > > > >> whether
>>> > > > >> > > > > > > > >> > this new form of the request will be used.
>>> For
>>> > > older
>>> > > > >> > > requests,
>>> > > > >> > > > > we
>>> > > > >> > > > > > > will
>>> > > > >> > > > > > > > >> > ignore this field and default to previous
>>> > > behavior.”
>>> > > > >> > > > > > > > >> >
>>> > > > >> > > > > > > > >> > 24. I’ve fixed this typo.
>>> > > > >> > > > > > > > >> >
>>> > > > >> > > > > > > > >> > 25. I've added a topics at a higher level for
>>> > > > >> > > > > LeaderAndIsrResponse
>>> > > > >> > > > > > > v5,
>>> > > > >> > > > > > > > >> > StopReplicaResponse v4. I've also changed
>>> > > > >> > StopReplicaRequest
>>> > > > >> > > > v4
>>> > > > >> > > > > to
>>> > > > >> > > > > > > > have
>>> > > > >> > > > > > > > >> > topics at a higher level.
>>> > > > >> > > > > > > > >> >
>>> > > > >> > > > > > > > >> > 26. I’ve updated
>>> forgotten_topics_data--added the
>>> > > > >> topic ID
>>> > > > >> > > and
>>> > > > >> > > > > > > removed
>>> > > > >> > > > > > > > >> the
>>> > > > >> > > > > > > > >> > topic name
>>> > > > >> > > > > > > > >> >
>>> > > > >> > > > > > > > >> > 27. I’ve decided on plain text, and I’ve
>>> added an
>>> > > > >> example.
>>> > > > >> > > > > > > > >> >
>>> > > > >> > > > > > > > >> > 28. I’ve added this idea to future work.
>>> > > > >> > > > > > > > >> >
>>> > > > >> > > > > > > > >> > Thanks again for taking a look,
>>> > > > >> > > > > > > > >> >
>>> > > > >> > > > > > > > >> > Justine
>>> > > > >> > > > > > > > >> >
>>> > > > >> > > > > > > > >> > On Wed, Sep 23, 2020 at 10:28 AM Jun Rao <
>>> > > > >> > jun@confluent.io>
>>> > > > >> > > > > > wrote:
>>> > > > >> > > > > > > > >> >
>>> > > > >> > > > > > > > >> > > Hi, Justine,
>>> > > > >> > > > > > > > >> > >
>>> > > > >> > > > > > > > >> > > Thanks for the response. Made another
>>> pass. A
>>> > few
>>> > > > >> more
>>> > > > >> > > > > comments
>>> > > > >> > > > > > > > below.
>>> > > > >> > > > > > > > >> > >
>>> > > > >> > > > > > > > >> > > 20. znode schema:
>>> > > > >> > > > > > > > >> > > 20.1 It seems that {"name": "version",
>>> "type":
>>> > > > "int",
>>> > > > >> > > "id":
>>> > > > >> > > > > > > "UUID",
>>> > > > >> > > > > > > > >> > "doc":
>>> > > > >> > > > > > > > >> > > "version id"} should be {"name": "version",
>>> > > "type":
>>> > > > >> > > "int"},
>>> > > > >> > > > > > > {"name":
>>> > > > >> > > > > > > > >> > "id",
>>> > > > >> > > > > > > > >> > > "type": "UUID", "doc": "version id"}.
>>> > > > >> > > > > > > > >> > > 20.2 The znode format is JSON which doesn't
>>> > have
>>> > > > UUID
>>> > > > >> > > type.
>>> > > > >> > > > So
>>> > > > >> > > > > > the
>>> > > > >> > > > > > > > >> type
>>> > > > >> > > > > > > > >> > > probably should be string?
>>> > > > >> > > > > > > > >> > > 20.3 Also, the existing format used seems
>>> > > outdated.
>>> > > > >> It
>>> > > > >> > > > should
>>> > > > >> > > > > > have
>>> > > > >> > > > > > > > the
>>> > > > >> > > > > > > > >> > > following format.
>>> > > > >> > > > > > > > >> > >     Json.encodeAsBytes(Map(
>>> > > > >> > > > > > > > >> > >       "version" -> 2,
>>> > > > >> > > > > > > > >> > >       "partitions" ->
>>> > > replicaAssignmentJson.asJava,
>>> > > > >> > > > > > > > >> > >       "adding_replicas" ->
>>> > > > >> > > > > addingReplicasAssignmentJson.asJava,
>>> > > > >> > > > > > > > >> > >       "removing_replicas" ->
>>> > > > >> > > > > > removingReplicasAssignmentJson.asJava
>>> > > > >> > > > > > > > >> > >     ).asJava)
>>> > > > >> > > > > > > > >> > >   }
>>> > > > >> > > > > > > > >> > >
>>> > > > >> > > > > > > > >> > > 21. Migration: The KIP says "The migration
>>> > > process
>>> > > > >> can
>>> > > > >> > > take
>>> > > > >> > > > > > place
>>> > > > >> > > > > > > > >> without
>>> > > > >> > > > > > > > >> > > an inter-broker protocol bump, as the
>>> format
>>> > > stored
>>> > > > >> in
>>> > > > >> > > > > > > > >> > > /brokers/topics/[topic] will be compatible
>>> with
>>> > > > older
>>> > > > >> > > broker
>>> > > > >> > > > > > > > >> versions."
>>> > > > >> > > > > > > > >> > > However, since we are bumping up the
>>> version of
>>> > > > >> > > inter-broker
>>> > > > >> > > > > > > > >> requests, it
>>> > > > >> > > > > > > > >> > > seems that we need to use IBP for
>>> migration.
>>> > > > >> > > > > > > > >> > >
>>> > > > >> > > > > > > > >> > > 22. The KIP says "Remove deleted topics
>>> from
>>> > > > >> replicas by
>>> > > > >> > > > > sending
>>> > > > >> > > > > > > > >> > > StopReplicaRequest V3 for any topics which
>>> do
>>> > not
>>> > > > >> > contain
>>> > > > >> > > a
>>> > > > >> > > > > > topic
>>> > > > >> > > > > > > > ID,
>>> > > > >> > > > > > > > >> and
>>> > > > >> > > > > > > > >> > > V4 for any topics which do contain a topic
>>> > ID.".
>>> > > > >> > However,
>>> > > > >> > > if
>>> > > > >> > > > > we
>>> > > > >> > > > > > > use
>>> > > > >> > > > > > > > >> IBP,
>>> > > > >> > > > > > > > >> > it
>>> > > > >> > > > > > > > >> > > seems that the controller will either send
>>> > > > >> > > > StopReplicaRequest
>>> > > > >> > > > > V3
>>> > > > >> > > > > > > > >> > > or StopReplicaRequest V4, but never mixed
>>> V3
>>> > and
>>> > > V4
>>> > > > >> for
>>> > > > >> > > > > > different
>>> > > > >> > > > > > > > >> topics.
>>> > > > >> > > > > > > > >> > > Basically, before the IBP bump, V3 will be
>>> > used.
>>> > > > >> After
>>> > > > >> > the
>>> > > > >> > > > IBP
>>> > > > >> > > > > > > bump,
>>> > > > >> > > > > > > > >> > > topicId will be created and V4 will be
>>> used.
>>> > > > >> > > > > > > > >> > >
>>> > > > >> > > > > > > > >> > > 23. Given that we depend on IBP, do we
>>> still
>>> > need
>>> > > > "0
>>> > > > >> > > > > > UNSPECIFIED"
>>> > > > >> > > > > > > > >> > > in LeaderAndIsr?
>>> > > > >> > > > > > > > >> > >
>>> > > > >> > > > > > > > >> > > 24. LeaderAndIsrResponse v5 : It still has
>>> the
>>> > > > topic
>>> > > > >> > > field.
>>> > > > >> > > > > > > > >> > >
>>> > > > >> > > > > > > > >> > > 25. LeaderAndIsrResponse v5,
>>> > StopReplicaResponse
>>> > > > v4:
>>> > > > >> > Could
>>> > > > >> > > > we
>>> > > > >> > > > > > use
>>> > > > >> > > > > > > > this
>>> > > > >> > > > > > > > >> > > opportunity to organize the response in 2
>>> > levels,
>>> > > > >> first
>>> > > > >> > by
>>> > > > >> > > > > > topic,
>>> > > > >> > > > > > > > >> then by
>>> > > > >> > > > > > > > >> > > partition, as most other
>>> requests/responses?
>>> > > > >> > > > > > > > >> > >
>>> > > > >> > > > > > > > >> > > 26. FetchRequest v13 : Should
>>> > > forgotten_topics_data
>>> > > > >> use
>>> > > > >> > > > > topicId
>>> > > > >> > > > > > > too?
>>> > > > >> > > > > > > > >> > >
>>> > > > >> > > > > > > > >> > > 27. "This file can either be plain text
>>> > > (key/value
>>> > > > >> > pairs)
>>> > > > >> > > or
>>> > > > >> > > > > > > JSON."
>>> > > > >> > > > > > > > >> Have
>>> > > > >> > > > > > > > >> > we
>>> > > > >> > > > > > > > >> > > decided which one to use? Also, it would be
>>> > > helpful
>>> > > > >> to
>>> > > > >> > > > provide
>>> > > > >> > > > > > an
>>> > > > >> > > > > > > > >> > example.
>>> > > > >> > > > > > > > >> > >
>>> > > > >> > > > > > > > >> > > 28. Future improvement: Another future
>>> > > improvement
>>> > > > >> > > > opportunity
>>> > > > >> > > > > > is
>>> > > > >> > > > > > > to
>>> > > > >> > > > > > > > >> use
>>> > > > >> > > > > > > > >> > > topicId in
>>> GroupMetadataManager.offsetCommitKey
>>> > > in
>>> > > > >> the
>>> > > > >> > > > > > > offset_commit
>>> > > > >> > > > > > > > >> > topic.
>>> > > > >> > > > > > > > >> > > This may save some space.
>>> > > > >> > > > > > > > >> > >
>>> > > > >> > > > > > > > >> > > Thanks,
>>> > > > >> > > > > > > > >> > >
>>> > > > >> > > > > > > > >> > > Jun
>>> > > > >> > > > > > > > >> > >
>>> > > > >> > > > > > > > >> > > On Wed, Sep 23, 2020 at 8:50 AM Justine
>>> Olshan
>>> > <
>>> > > > >> > > > > > > > jolshan@confluent.io>
>>> > > > >> > > > > > > > >> > > wrote:
>>> > > > >> > > > > > > > >> > >
>>> > > > >> > > > > > > > >> > > > Hi Tom,
>>> > > > >> > > > > > > > >> > > >
>>> > > > >> > > > > > > > >> > > > Thanks for the comment. I think this is a
>>> > > really
>>> > > > >> good
>>> > > > >> > > idea
>>> > > > >> > > > > and
>>> > > > >> > > > > > > it
>>> > > > >> > > > > > > > >> has
>>> > > > >> > > > > > > > >> > > been
>>> > > > >> > > > > > > > >> > > > added to the KIP under the newly added
>>> > tooling
>>> > > > >> > section.
>>> > > > >> > > > > > > > >> > > >
>>> > > > >> > > > > > > > >> > > > Thanks again,
>>> > > > >> > > > > > > > >> > > > Justine
>>> > > > >> > > > > > > > >> > > >
>>> > > > >> > > > > > > > >> > > > On Wed, Sep 23, 2020 at 3:17 AM Tom
>>> Bentley <
>>> > > > >> > > > > > > tbentley@redhat.com>
>>> > > > >> > > > > > > > >> > wrote:
>>> > > > >> > > > > > > > >> > > >
>>> > > > >> > > > > > > > >> > > > > Hi Justine,
>>> > > > >> > > > > > > > >> > > > >
>>> > > > >> > > > > > > > >> > > > > I know you started the vote thread,
>>> but on
>>> > > > >> > re-reading
>>> > > > >> > > > the
>>> > > > >> > > > > > KIP
>>> > > > >> > > > > > > I
>>> > > > >> > > > > > > > >> > noticed
>>> > > > >> > > > > > > > >> > > > > that although the topic id is included
>>> in
>>> > the
>>> > > > >> > > > > > MetadataResponse
>>> > > > >> > > > > > > > >> it's
>>> > > > >> > > > > > > > >> > not
>>> > > > >> > > > > > > > >> > > > > surfaced in the output from
>>> > `kafka-topics.sh
>>> > > > >> > > > --describe`.
>>> > > > >> > > > > > > Maybe
>>> > > > >> > > > > > > > >> that
>>> > > > >> > > > > > > > >> > > was
>>> > > > >> > > > > > > > >> > > > > intentional because ids are
>>> intentionally
>>> > not
>>> > > > >> really
>>> > > > >> > > > > > something
>>> > > > >> > > > > > > > the
>>> > > > >> > > > > > > > >> > user
>>> > > > >> > > > > > > > >> > > > > should care deeply about, but it would
>>> also
>>> > > > make
>>> > > > >> > life
>>> > > > >> > > > > harder
>>> > > > >> > > > > > > for
>>> > > > >> > > > > > > > >> > anyone
>>> > > > >> > > > > > > > >> > > > > debugging Kafka and this would likely
>>> get
>>> > > worse
>>> > > > >> the
>>> > > > >> > > more
>>> > > > >> > > > > > topic
>>> > > > >> > > > > > > > ids
>>> > > > >> > > > > > > > >> > got
>>> > > > >> > > > > > > > >> > > > > rolled out across the protocols,
>>> clients
>>> > etc.
>>> > > > It
>>> > > > >> > seems
>>> > > > >> > > > > > likely
>>> > > > >> > > > > > > > that
>>> > > > >> > > > > > > > >> > > > > `kafka-topics.sh` will eventually need
>>> the
>>> > > > >> ability
>>> > > > >> > to
>>> > > > >> > > > show
>>> > > > >> > > > > > the
>>> > > > >> > > > > > > > id
>>> > > > >> > > > > > > > >> of
>>> > > > >> > > > > > > > >> > a
>>> > > > >> > > > > > > > >> > > > > topic and perhaps find a topic name
>>> given
>>> > an
>>> > > > id.
>>> > > > >> Is
>>> > > > >> > > > there
>>> > > > >> > > > > > any
>>> > > > >> > > > > > > > >> reason
>>> > > > >> > > > > > > > >> > > not
>>> > > > >> > > > > > > > >> > > > to
>>> > > > >> > > > > > > > >> > > > > implement that in this KIP?
>>> > > > >> > > > > > > > >> > > > >
>>> > > > >> > > > > > > > >> > > > > Many thanks,
>>> > > > >> > > > > > > > >> > > > >
>>> > > > >> > > > > > > > >> > > > > Tom
>>> > > > >> > > > > > > > >> > > > >
>>> > > > >> > > > > > > > >> > > > > On Mon, Sep 21, 2020 at 9:54 PM Justine
>>> > > Olshan
>>> > > > <
>>> > > > >> > > > > > > > >> jolshan@confluent.io
>>> > > > >> > > > > > > > >> > >
>>> > > > >> > > > > > > > >> > > > > wrote:
>>> > > > >> > > > > > > > >> > > > >
>>> > > > >> > > > > > > > >> > > > > > Hi all,
>>> > > > >> > > > > > > > >> > > > > >
>>> > > > >> > > > > > > > >> > > > > > After thinking about it, I've
>>> decided to
>>> > > > remove
>>> > > > >> > the
>>> > > > >> > > > > topic
>>> > > > >> > > > > > > name
>>> > > > >> > > > > > > > >> from
>>> > > > >> > > > > > > > >> > > the
>>> > > > >> > > > > > > > >> > > > > > Fetch Request and Response after all.
>>> > Since
>>> > > > >> there
>>> > > > >> > > are
>>> > > > >> > > > so
>>> > > > >> > > > > > > many
>>> > > > >> > > > > > > > of
>>> > > > >> > > > > > > > >> > > these
>>> > > > >> > > > > > > > >> > > > > > requests per second, it is worth
>>> removing
>>> > > the
>>> > > > >> > extra
>>> > > > >> > > > > > > > information.
>>> > > > >> > > > > > > > >> > I've
>>> > > > >> > > > > > > > >> > > > > > updated the KIP to reflect this
>>> change.
>>> > > > >> > > > > > > > >> > > > > >
>>> > > > >> > > > > > > > >> > > > > > Please let me know if there is
>>> anything
>>> > > else
>>> > > > we
>>> > > > >> > > should
>>> > > > >> > > > > > > discuss
>>> > > > >> > > > > > > > >> > before
>>> > > > >> > > > > > > > >> > > > > > voting.
>>> > > > >> > > > > > > > >> > > > > >
>>> > > > >> > > > > > > > >> > > > > > Thank you,
>>> > > > >> > > > > > > > >> > > > > > Justine
>>> > > > >> > > > > > > > >> > > > > >
>>> > > > >> > > > > > > > >> > > > > > On Fri, Sep 18, 2020 at 9:46 AM
>>> Justine
>>> > > > Olshan
>>> > > > >> <
>>> > > > >> > > > > > > > >> > jolshan@confluent.io
>>> > > > >> > > > > > > > >> > > >
>>> > > > >> > > > > > > > >> > > > > > wrote:
>>> > > > >> > > > > > > > >> > > > > >
>>> > > > >> > > > > > > > >> > > > > > > Hi Jun,
>>> > > > >> > > > > > > > >> > > > > > >
>>> > > > >> > > > > > > > >> > > > > > > I see what you are saying. For now
>>> we
>>> > can
>>> > > > >> remove
>>> > > > >> > > the
>>> > > > >> > > > > > extra
>>> > > > >> > > > > > > > >> > > > information.
>>> > > > >> > > > > > > > >> > > > > > > I'll leave the option to add more
>>> > fields
>>> > > to
>>> > > > >> the
>>> > > > >> > > file
>>> > > > >> > > > > in
>>> > > > >> > > > > > > the
>>> > > > >> > > > > > > > >> > future.
>>> > > > >> > > > > > > > >> > > > The
>>> > > > >> > > > > > > > >> > > > > > KIP
>>> > > > >> > > > > > > > >> > > > > > > has been updated to reflect this
>>> > change.
>>> > > > >> > > > > > > > >> > > > > > >
>>> > > > >> > > > > > > > >> > > > > > > Thanks,
>>> > > > >> > > > > > > > >> > > > > > > Justine
>>> > > > >> > > > > > > > >> > > > > > >
>>> > > > >> > > > > > > > >> > > > > > > On Fri, Sep 18, 2020 at 8:46 AM Jun
>>> > Rao <
>>> > > > >> > > > > > jun@confluent.io
>>> > > > >> > > > > > > >
>>> > > > >> > > > > > > > >> > wrote:
>>> > > > >> > > > > > > > >> > > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> Hi, Justine,
>>> > > > >> > > > > > > > >> > > > > > >>
>>> > > > >> > > > > > > > >> > > > > > >> Thanks for the reply.
>>> > > > >> > > > > > > > >> > > > > > >>
>>> > > > >> > > > > > > > >> > > > > > >> 13. If the log directory is the
>>> source
>>> > > of
>>> > > > >> > truth,
>>> > > > >> > > it
>>> > > > >> > > > > > means
>>> > > > >> > > > > > > > >> that
>>> > > > >> > > > > > > > >> > the
>>> > > > >> > > > > > > > >> > > > > > >> redundant info in the metadata
>>> file
>>> > will
>>> > > > be
>>> > > > >> > > > ignored.
>>> > > > >> > > > > > Then
>>> > > > >> > > > > > > > the
>>> > > > >> > > > > > > > >> > > > question
>>> > > > >> > > > > > > > >> > > > > > is
>>> > > > >> > > > > > > > >> > > > > > >> why do we need to put the
>>> redundant
>>> > info
>>> > > > in
>>> > > > >> the
>>> > > > >> > > > > > metadata
>>> > > > >> > > > > > > > file
>>> > > > >> > > > > > > > >> > now?
>>> > > > >> > > > > > > > >> > > > > > >>
>>> > > > >> > > > > > > > >> > > > > > >> Thanks,
>>> > > > >> > > > > > > > >> > > > > > >>
>>> > > > >> > > > > > > > >> > > > > > >> Jun
>>> > > > >> > > > > > > > >> > > > > > >>
>>> > > > >> > > > > > > > >> > > > > > >> On Thu, Sep 17, 2020 at 5:07 PM
>>> > Justine
>>> > > > >> Olshan
>>> > > > >> > <
>>> > > > >> > > > > > > > >> > > > jolshan@confluent.io>
>>> > > > >> > > > > > > > >> > > > > > >> wrote:
>>> > > > >> > > > > > > > >> > > > > > >>
>>> > > > >> > > > > > > > >> > > > > > >> > Hi Jun,
>>> > > > >> > > > > > > > >> > > > > > >> > Thanks for the quick response!
>>> > > > >> > > > > > > > >> > > > > > >> >
>>> > > > >> > > > > > > > >> > > > > > >> > 12. I've decided to bump up the
>>> > > versions
>>> > > > >> on
>>> > > > >> > the
>>> > > > >> > > > > > > requests
>>> > > > >> > > > > > > > >> and
>>> > > > >> > > > > > > > >> > > > updated
>>> > > > >> > > > > > > > >> > > > > > the
>>> > > > >> > > > > > > > >> > > > > > >> > KIP. I think it's good we
>>> thoroughly
>>> > > > >> > discussed
>>> > > > >> > > > the
>>> > > > >> > > > > > > > options
>>> > > > >> > > > > > > > >> > here,
>>> > > > >> > > > > > > > >> > > > so
>>> > > > >> > > > > > > > >> > > > > we
>>> > > > >> > > > > > > > >> > > > > > >> know
>>> > > > >> > > > > > > > >> > > > > > >> > we made a good choice. :)
>>> > > > >> > > > > > > > >> > > > > > >> >
>>> > > > >> > > > > > > > >> > > > > > >> > 13. This is an interesting
>>> > situation.
>>> > > I
>>> > > > >> think
>>> > > > >> > > if
>>> > > > >> > > > > this
>>> > > > >> > > > > > > > does
>>> > > > >> > > > > > > > >> > occur
>>> > > > >> > > > > > > > >> > > > we
>>> > > > >> > > > > > > > >> > > > > > >> should
>>> > > > >> > > > > > > > >> > > > > > >> > give a warning. I agree that
>>> it's
>>> > hard
>>> > > > to
>>> > > > >> > know
>>> > > > >> > > > the
>>> > > > >> > > > > > > source
>>> > > > >> > > > > > > > >> of
>>> > > > >> > > > > > > > >> > > truth
>>> > > > >> > > > > > > > >> > > > > for
>>> > > > >> > > > > > > > >> > > > > > >> sure
>>> > > > >> > > > > > > > >> > > > > > >> > since the directory or the file
>>> > could
>>> > > be
>>> > > > >> > > manually
>>> > > > >> > > > > > > > >> modified. I
>>> > > > >> > > > > > > > >> > > > guess
>>> > > > >> > > > > > > > >> > > > > > the
>>> > > > >> > > > > > > > >> > > > > > >> > directory could be used as the
>>> > source
>>> > > of
>>> > > > >> > truth.
>>> > > > >> > > > To
>>> > > > >> > > > > be
>>> > > > >> > > > > > > > >> honest,
>>> > > > >> > > > > > > > >> > > I'm
>>> > > > >> > > > > > > > >> > > > > not
>>> > > > >> > > > > > > > >> > > > > > >> > really sure what happens in
>>> kafka
>>> > when
>>> > > > the
>>> > > > >> > log
>>> > > > >> > > > > > > directory
>>> > > > >> > > > > > > > is
>>> > > > >> > > > > > > > >> > > > renamed
>>> > > > >> > > > > > > > >> > > > > > >> > manually in such a way. I'm also
>>> > > > >> wondering if
>>> > > > >> > > the
>>> > > > >> > > > > > > > >> situation is
>>> > > > >> > > > > > > > >> > > > > > >> recoverable
>>> > > > >> > > > > > > > >> > > > > > >> > in this scenario.
>>> > > > >> > > > > > > > >> > > > > > >> >
>>> > > > >> > > > > > > > >> > > > > > >> > Thanks,
>>> > > > >> > > > > > > > >> > > > > > >> > Justine
>>> > > > >> > > > > > > > >> > > > > > >> >
>>> > > > >> > > > > > > > >> > > > > > >> > On Thu, Sep 17, 2020 at 4:28 PM
>>> Jun
>>> > > Rao
>>> > > > <
>>> > > > >> > > > > > > > jun@confluent.io>
>>> > > > >> > > > > > > > >> > > wrote:
>>> > > > >> > > > > > > > >> > > > > > >> >
>>> > > > >> > > > > > > > >> > > > > > >> > > Hi, Justine,
>>> > > > >> > > > > > > > >> > > > > > >> > >
>>> > > > >> > > > > > > > >> > > > > > >> > > Thanks for the reply.
>>> > > > >> > > > > > > > >> > > > > > >> > >
>>> > > > >> > > > > > > > >> > > > > > >> > > 12. I don't have a strong
>>> > preference
>>> > > > >> > either.
>>> > > > >> > > > > > However,
>>> > > > >> > > > > > > > if
>>> > > > >> > > > > > > > >> we
>>> > > > >> > > > > > > > >> > > need
>>> > > > >> > > > > > > > >> > > > > IBP
>>> > > > >> > > > > > > > >> > > > > > >> > > anyway, maybe it's easier to
>>> just
>>> > > bump
>>> > > > >> up
>>> > > > >> > the
>>> > > > >> > > > > > version
>>> > > > >> > > > > > > > for
>>> > > > >> > > > > > > > >> > all
>>> > > > >> > > > > > > > >> > > > > inter
>>> > > > >> > > > > > > > >> > > > > > >> > broker
>>> > > > >> > > > > > > > >> > > > > > >> > > requests and add the topic id
>>> > field
>>> > > > as a
>>> > > > >> > > > regular
>>> > > > >> > > > > > > > field. A
>>> > > > >> > > > > > > > >> > > > regular
>>> > > > >> > > > > > > > >> > > > > > >> field
>>> > > > >> > > > > > > > >> > > > > > >> > is
>>> > > > >> > > > > > > > >> > > > > > >> > > a bit more concise in wire
>>> > transfer
>>> > > > >> than a
>>> > > > >> > > > > flexible
>>> > > > >> > > > > > > > >> field.
>>> > > > >> > > > > > > > >> > > > > > >> > >
>>> > > > >> > > > > > > > >> > > > > > >> > > 13. The confusion that I was
>>> > > referring
>>> > > > >> to
>>> > > > >> > is
>>> > > > >> > > > > > between
>>> > > > >> > > > > > > > the
>>> > > > >> > > > > > > > >> > topic
>>> > > > >> > > > > > > > >> > > > > name
>>> > > > >> > > > > > > > >> > > > > > >> and
>>> > > > >> > > > > > > > >> > > > > > >> > > partition number between the
>>> log
>>> > dir
>>> > > > and
>>> > > > >> > the
>>> > > > >> > > > > > metadata
>>> > > > >> > > > > > > > >> file.
>>> > > > >> > > > > > > > >> > > For
>>> > > > >> > > > > > > > >> > > > > > >> example,
>>> > > > >> > > > > > > > >> > > > > > >> > if
>>> > > > >> > > > > > > > >> > > > > > >> > > the log dir is topicA-1 and
>>> the
>>> > > > metadata
>>> > > > >> > file
>>> > > > >> > > > in
>>> > > > >> > > > > it
>>> > > > >> > > > > > > has
>>> > > > >> > > > > > > > >> > topicB
>>> > > > >> > > > > > > > >> > > > and
>>> > > > >> > > > > > > > >> > > > > > >> > > partition 0 (say due to a bug
>>> or
>>> > > > manual
>>> > > > >> > > > > > > modification),
>>> > > > >> > > > > > > > >> which
>>> > > > >> > > > > > > > >> > > one
>>> > > > >> > > > > > > > >> > > > > do
>>> > > > >> > > > > > > > >> > > > > > we
>>> > > > >> > > > > > > > >> > > > > > >> > use
>>> > > > >> > > > > > > > >> > > > > > >> > > as the source of truth?
>>> > > > >> > > > > > > > >> > > > > > >> > >
>>> > > > >> > > > > > > > >> > > > > > >> > > Jun
>>> > > > >> > > > > > > > >> > > > > > >> > >
>>> > > > >> > > > > > > > >> > > > > > >> > > On Thu, Sep 17, 2020 at 3:43
>>> PM
>>> > > > Justine
>>> > > > >> > > Olshan
>>> > > > >> > > > <
>>> > > > >> > > > > > > > >> > > > > > jolshan@confluent.io>
>>> > > > >> > > > > > > > >> > > > > > >> > > wrote:
>>> > > > >> > > > > > > > >> > > > > > >> > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > Hi Jun,
>>> > > > >> > > > > > > > >> > > > > > >> > > > Thanks for the comments.
>>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > 12. I bumped the
>>> > > LeaderAndIsrRequest
>>> > > > >> > > because
>>> > > > >> > > > I
>>> > > > >> > > > > > > > removed
>>> > > > >> > > > > > > > >> the
>>> > > > >> > > > > > > > >> > > > topic
>>> > > > >> > > > > > > > >> > > > > > >> name
>>> > > > >> > > > > > > > >> > > > > > >> > > field
>>> > > > >> > > > > > > > >> > > > > > >> > > > in the response. It may be
>>> > > possible
>>> > > > to
>>> > > > >> > > avoid
>>> > > > >> > > > > > > bumping
>>> > > > >> > > > > > > > >> the
>>> > > > >> > > > > > > > >> > > > version
>>> > > > >> > > > > > > > >> > > > > > >> > without
>>> > > > >> > > > > > > > >> > > > > > >> > > > that change, but I may be
>>> > missing
>>> > > > >> > > something.
>>> > > > >> > > > > > > > >> > > > > > >> > > > I believe StopReplica is
>>> > actually
>>> > > on
>>> > > > >> > > version
>>> > > > >> > > > 3
>>> > > > >> > > > > > now,
>>> > > > >> > > > > > > > but
>>> > > > >> > > > > > > > >> > > > because
>>> > > > >> > > > > > > > >> > > > > > >> > version 2
>>> > > > >> > > > > > > > >> > > > > > >> > > > is flexible, I kept that
>>> listed
>>> > as
>>> > > > >> > version
>>> > > > >> > > 2
>>> > > > >> > > > on
>>> > > > >> > > > > > the
>>> > > > >> > > > > > > > KIP
>>> > > > >> > > > > > > > >> > > page.
>>> > > > >> > > > > > > > >> > > > > > >> However,
>>> > > > >> > > > > > > > >> > > > > > >> > > you
>>> > > > >> > > > > > > > >> > > > > > >> > > > may be right in that we may
>>> need
>>> > > to
>>> > > > >> bump
>>> > > > >> > > the
>>> > > > >> > > > > > > version
>>> > > > >> > > > > > > > on
>>> > > > >> > > > > > > > >> > > > > > StopReplica
>>> > > > >> > > > > > > > >> > > > > > >> to
>>> > > > >> > > > > > > > >> > > > > > >> > > deal
>>> > > > >> > > > > > > > >> > > > > > >> > > > with deletion differently as
>>> > > > mentioned
>>> > > > >> > > > above. I
>>> > > > >> > > > > > > don't
>>> > > > >> > > > > > > > >> know
>>> > > > >> > > > > > > > >> > > if
>>> > > > >> > > > > > > > >> > > > I
>>> > > > >> > > > > > > > >> > > > > > >> have a
>>> > > > >> > > > > > > > >> > > > > > >> > > big
>>> > > > >> > > > > > > > >> > > > > > >> > > > preference over used tagged
>>> > fields
>>> > > > or
>>> > > > >> > not.
>>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > 13. I was thinking that in
>>> the
>>> > > case
>>> > > > >> where
>>> > > > >> > > the
>>> > > > >> > > > > > file
>>> > > > >> > > > > > > > and
>>> > > > >> > > > > > > > >> the
>>> > > > >> > > > > > > > >> > > > > request
>>> > > > >> > > > > > > > >> > > > > > >> > topic
>>> > > > >> > > > > > > > >> > > > > > >> > > > ids don't match, it means
>>> that
>>> > the
>>> > > > >> > broker's
>>> > > > >> > > > > > > topic/the
>>> > > > >> > > > > > > > >> one
>>> > > > >> > > > > > > > >> > in
>>> > > > >> > > > > > > > >> > > > the
>>> > > > >> > > > > > > > >> > > > > > >> file
>>> > > > >> > > > > > > > >> > > > > > >> > has
>>> > > > >> > > > > > > > >> > > > > > >> > > > been deleted. In that case,
>>> we
>>> > > would
>>> > > > >> need
>>> > > > >> > > to
>>> > > > >> > > > > > delete
>>> > > > >> > > > > > > > the
>>> > > > >> > > > > > > > >> > old
>>> > > > >> > > > > > > > >> > > > > topic
>>> > > > >> > > > > > > > >> > > > > > >> and
>>> > > > >> > > > > > > > >> > > > > > >> > > start
>>> > > > >> > > > > > > > >> > > > > > >> > > > receiving the new version.
>>> If
>>> > the
>>> > > > >> topic
>>> > > > >> > > name
>>> > > > >> > > > > were
>>> > > > >> > > > > > > to
>>> > > > >> > > > > > > > >> > change,
>>> > > > >> > > > > > > > >> > > > but
>>> > > > >> > > > > > > > >> > > > > > the
>>> > > > >> > > > > > > > >> > > > > > >> > ids
>>> > > > >> > > > > > > > >> > > > > > >> > > > still matched, the file
>>> would
>>> > also
>>> > > > >> need
>>> > > > >> > to
>>> > > > >> > > > > > update.
>>> > > > >> > > > > > > > Am I
>>> > > > >> > > > > > > > >> > > > missing
>>> > > > >> > > > > > > > >> > > > > a
>>> > > > >> > > > > > > > >> > > > > > >> case
>>> > > > >> > > > > > > > >> > > > > > >> > > > where the file would be
>>> correct
>>> > > and
>>> > > > >> not
>>> > > > >> > the
>>> > > > >> > > > > > > request?
>>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > Thanks,
>>> > > > >> > > > > > > > >> > > > > > >> > > > Justine
>>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > On Thu, Sep 17, 2020 at
>>> 3:18 PM
>>> > > Jun
>>> > > > >> Rao <
>>> > > > >> > > > > > > > >> jun@confluent.io
>>> > > > >> > > > > > > > >> > >
>>> > > > >> > > > > > > > >> > > > > wrote:
>>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > Hi, Justine,
>>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > Thanks for the reply. A
>>> couple
>>> > > of
>>> > > > >> more
>>> > > > >> > > > > comments
>>> > > > >> > > > > > > > >> below.
>>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > 12. ListOffset and
>>> > > OffsetForLeader
>>> > > > >> > > > currently
>>> > > > >> > > > > > > don't
>>> > > > >> > > > > > > > >> > support
>>> > > > >> > > > > > > > >> > > > > > >> flexible
>>> > > > >> > > > > > > > >> > > > > > >> > > > fields.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > So, we have to bump up the
>>> > > version
>>> > > > >> > number
>>> > > > >> > > > and
>>> > > > >> > > > > > use
>>> > > > >> > > > > > > > >> IBP at
>>> > > > >> > > > > > > > >> > > > least
>>> > > > >> > > > > > > > >> > > > > > for
>>> > > > >> > > > > > > > >> > > > > > >> > > these
>>> > > > >> > > > > > > > >> > > > > > >> > > > > two requests. Note that it
>>> > seems
>>> > > > >> 2.7.0
>>> > > > >> > > will
>>> > > > >> > > > > > > require
>>> > > > >> > > > > > > > >> IBP
>>> > > > >> > > > > > > > >> > > > anyway
>>> > > > >> > > > > > > > >> > > > > > >> > because
>>> > > > >> > > > > > > > >> > > > > > >> > > of
>>> > > > >> > > > > > > > >> > > > > > >> > > > > changes in KAFKA-10435.
>>> Also,
>>> > it
>>> > > > >> seems
>>> > > > >> > > that
>>> > > > >> > > > > the
>>> > > > >> > > > > > > > >> version
>>> > > > >> > > > > > > > >> > > for
>>> > > > >> > > > > > > > >> > > > > > >> > > > > LeaderAndIsrRequest and
>>> > > > StopReplica
>>> > > > >> are
>>> > > > >> > > > > bumped
>>> > > > >> > > > > > > even
>>> > > > >> > > > > > > > >> > though
>>> > > > >> > > > > > > > >> > > > we
>>> > > > >> > > > > > > > >> > > > > > only
>>> > > > >> > > > > > > > >> > > > > > >> > > added
>>> > > > >> > > > > > > > >> > > > > > >> > > > a
>>> > > > >> > > > > > > > >> > > > > > >> > > > > tagged field. But since
>>> IBP is
>>> > > > >> needed
>>> > > > >> > > > anyway,
>>> > > > >> > > > > > we
>>> > > > >> > > > > > > > may
>>> > > > >> > > > > > > > >> > want
>>> > > > >> > > > > > > > >> > > to
>>> > > > >> > > > > > > > >> > > > > > >> revisit
>>> > > > >> > > > > > > > >> > > > > > >> > > the
>>> > > > >> > > > > > > > >> > > > > > >> > > > > overall tagged field
>>> choice.
>>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > 13. The only downside is
>>> the
>>> > > > >> potential
>>> > > > >> > > > > > confusion
>>> > > > >> > > > > > > on
>>> > > > >> > > > > > > > >> > which
>>> > > > >> > > > > > > > >> > > > one
>>> > > > >> > > > > > > > >> > > > > is
>>> > > > >> > > > > > > > >> > > > > > >> the
>>> > > > >> > > > > > > > >> > > > > > >> > > > source
>>> > > > >> > > > > > > > >> > > > > > >> > > > > of truth if they don't
>>> match.
>>> > > > >> Another
>>> > > > >> > > > option
>>> > > > >> > > > > is
>>> > > > >> > > > > > > to
>>> > > > >> > > > > > > > >> > include
>>> > > > >> > > > > > > > >> > > > > those
>>> > > > >> > > > > > > > >> > > > > > >> > fields
>>> > > > >> > > > > > > > >> > > > > > >> > > > in
>>> > > > >> > > > > > > > >> > > > > > >> > > > > the metadata file when we
>>> > > actually
>>> > > > >> > change
>>> > > > >> > > > the
>>> > > > >> > > > > > > > >> directory
>>> > > > >> > > > > > > > >> > > > > > structure.
>>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > Thanks,
>>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > Jun
>>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > On Thu, Sep 17, 2020 at
>>> 2:01
>>> > PM
>>> > > > >> Justine
>>> > > > >> > > > > Olshan
>>> > > > >> > > > > > <
>>> > > > >> > > > > > > > >> > > > > > >> jolshan@confluent.io
>>> > > > >> > > > > > > > >> > > > > > >> > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > wrote:
>>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > Hello all,
>>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > I've thought some more
>>> about
>>> > > > >> removing
>>> > > > >> > > the
>>> > > > >> > > > > > topic
>>> > > > >> > > > > > > > >> name
>>> > > > >> > > > > > > > >> > > field
>>> > > > >> > > > > > > > >> > > > > > from
>>> > > > >> > > > > > > > >> > > > > > >> > some
>>> > > > >> > > > > > > > >> > > > > > >> > > of
>>> > > > >> > > > > > > > >> > > > > > >> > > > > the
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > requests. On closer
>>> > inspection
>>> > > > of
>>> > > > >> the
>>> > > > >> > > > > > > > >> > > requests/responses,
>>> > > > >> > > > > > > > >> > > > it
>>> > > > >> > > > > > > > >> > > > > > >> seems
>>> > > > >> > > > > > > > >> > > > > > >> > > that
>>> > > > >> > > > > > > > >> > > > > > >> > > > > the
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > internal changes would
>>> be
>>> > much
>>> > > > >> larger
>>> > > > >> > > > than
>>> > > > >> > > > > I
>>> > > > >> > > > > > > > >> expected.
>>> > > > >> > > > > > > > >> > > > Some
>>> > > > >> > > > > > > > >> > > > > > >> > protocols
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > involve clients, so they
>>> > would
>>> > > > >> > require
>>> > > > >> > > > > > changes
>>> > > > >> > > > > > > > too.
>>> > > > >> > > > > > > > >> > I'm
>>> > > > >> > > > > > > > >> > > > > > thinking
>>> > > > >> > > > > > > > >> > > > > > >> > that
>>> > > > >> > > > > > > > >> > > > > > >> > > > for
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > now, removing the topic
>>> name
>>> > > > from
>>> > > > >> > these
>>> > > > >> > > > > > > requests
>>> > > > >> > > > > > > > >> and
>>> > > > >> > > > > > > > >> > > > > responses
>>> > > > >> > > > > > > > >> > > > > > >> are
>>> > > > >> > > > > > > > >> > > > > > >> > > out
>>> > > > >> > > > > > > > >> > > > > > >> > > > of
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > scope.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > I have decided to just
>>> keep
>>> > > the
>>> > > > >> > change
>>> > > > >> > > > > > > > >> > > > LeaderAndIsrResponse
>>> > > > >> > > > > > > > >> > > > > to
>>> > > > >> > > > > > > > >> > > > > > >> > remove
>>> > > > >> > > > > > > > >> > > > > > >> > > > the
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > topic name, and have
>>> updated
>>> > > the
>>> > > > >> KIP
>>> > > > >> > to
>>> > > > >> > > > > > reflect
>>> > > > >> > > > > > > > >> this
>>> > > > >> > > > > > > > >> > > > > change. I
>>> > > > >> > > > > > > > >> > > > > > >> have
>>> > > > >> > > > > > > > >> > > > > > >> > > > also
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > mentioned the other
>>> requests
>>> > > and
>>> > > > >> > > > responses
>>> > > > >> > > > > in
>>> > > > >> > > > > > > > >> future
>>> > > > >> > > > > > > > >> > > work.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > I'm hoping to start the
>>> > voting
>>> > > > >> > process
>>> > > > >> > > > > soon,
>>> > > > >> > > > > > so
>>> > > > >> > > > > > > > >> let me
>>> > > > >> > > > > > > > >> > > > know
>>> > > > >> > > > > > > > >> > > > > if
>>> > > > >> > > > > > > > >> > > > > > >> > there
>>> > > > >> > > > > > > > >> > > > > > >> > > is
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > anything else we should
>>> > > discuss.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > Thank you,
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > Justine
>>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > On Tue, Sep 15, 2020 at
>>> 3:57
>>> > > PM
>>> > > > >> > Justine
>>> > > > >> > > > > > Olshan
>>> > > > >> > > > > > > <
>>> > > > >> > > > > > > > >> > > > > > >> > jolshan@confluent.io
>>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > wrote:
>>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > > Hello again,
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > > To follow up on some
>>> of
>>> > the
>>> > > > >> other
>>> > > > >> > > > > comments:
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 10/11) We can remove
>>> the
>>> > > topic
>>> > > > >> name
>>> > > > >> > > > from
>>> > > > >> > > > > > > these
>>> > > > >> > > > > > > > >> > > > > > >> > requests/responses,
>>> > > > >> > > > > > > > >> > > > > > >> > > > and
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > > that means we will
>>> just
>>> > have
>>> > > > to
>>> > > > >> > make
>>> > > > >> > > a
>>> > > > >> > > > > few
>>> > > > >> > > > > > > > >> internal
>>> > > > >> > > > > > > > >> > > > > changes
>>> > > > >> > > > > > > > >> > > > > > to
>>> > > > >> > > > > > > > >> > > > > > >> > make
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > > partitions accessible
>>> by
>>> > > topic
>>> > > > >> id
>>> > > > >> > and
>>> > > > >> > > > > > > > partition.
>>> > > > >> > > > > > > > >> I
>>> > > > >> > > > > > > > >> > can
>>> > > > >> > > > > > > > >> > > > > > update
>>> > > > >> > > > > > > > >> > > > > > >> the
>>> > > > >> > > > > > > > >> > > > > > >> > > KIP
>>> > > > >> > > > > > > > >> > > > > > >> > > > > to
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > > remove them unless
>>> anyone
>>> > > > thinks
>>> > > > >> > they
>>> > > > >> > > > > > should
>>> > > > >> > > > > > > > >> stay.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 12) Addressed in the
>>> > > previous
>>> > > > >> > email.
>>> > > > >> > > > I've
>>> > > > >> > > > > > > > updated
>>> > > > >> > > > > > > > >> > the
>>> > > > >> > > > > > > > >> > > > KIP
>>> > > > >> > > > > > > > >> > > > > to
>>> > > > >> > > > > > > > >> > > > > > >> > > include
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > > tagged fields for the
>>> > > requests
>>> > > > >> and
>>> > > > >> > > > > > responses.
>>> > > > >> > > > > > > > >> (More
>>> > > > >> > > > > > > > >> > on
>>> > > > >> > > > > > > > >> > > > > that
>>> > > > >> > > > > > > > >> > > > > > >> > below)
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 13) I think part of
>>> the
>>> > idea
>>> > > > for
>>> > > > >> > > > > including
>>> > > > >> > > > > > > this
>>> > > > >> > > > > > > > >> > > > > information
>>> > > > >> > > > > > > > >> > > > > > >> is to
>>> > > > >> > > > > > > > >> > > > > > >> > > > > prepare
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > > for future changes.
>>> > Perhaps
>>> > > > the
>>> > > > >> > > > directory
>>> > > > >> > > > > > > > >> structure
>>> > > > >> > > > > > > > >> > > > might
>>> > > > >> > > > > > > > >> > > > > > >> change
>>> > > > >> > > > > > > > >> > > > > > >> > > from
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>>> topicName_partitionNumber
>>> > to
>>> > > > >> > > something
>>> > > > >> > > > > like
>>> > > > >> > > > > > > > >> > > > > > >> > > topicID_partitionNumber.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > Then
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > > it would be useful to
>>> have
>>> > > the
>>> > > > >> > topic
>>> > > > >> > > > name
>>> > > > >> > > > > > in
>>> > > > >> > > > > > > > the
>>> > > > >> > > > > > > > >> > file
>>> > > > >> > > > > > > > >> > > > > since
>>> > > > >> > > > > > > > >> > > > > > it
>>> > > > >> > > > > > > > >> > > > > > >> > > would
>>> > > > >> > > > > > > > >> > > > > > >> > > > > not
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > be
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > > in the directory
>>> > structure.
>>> > > > >> > > Supporting
>>> > > > >> > > > > > topic
>>> > > > >> > > > > > > > >> renames
>>> > > > >> > > > > > > > >> > > > might
>>> > > > >> > > > > > > > >> > > > > > be
>>> > > > >> > > > > > > > >> > > > > > >> > > easier
>>> > > > >> > > > > > > > >> > > > > > >> > > > if
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > the
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > > other fields are
>>> included.
>>> > > > Would
>>> > > > >> > > there
>>> > > > >> > > > be
>>> > > > >> > > > > > any
>>> > > > >> > > > > > > > >> > > downsides
>>> > > > >> > > > > > > > >> > > > to
>>> > > > >> > > > > > > > >> > > > > > >> > > including
>>> > > > >> > > > > > > > >> > > > > > >> > > > > this
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > > information?
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 14)  Yes, we would
>>> need to
>>> > > > copy
>>> > > > >> the
>>> > > > >> > > > > > partition
>>> > > > >> > > > > > > > >> > metadata
>>> > > > >> > > > > > > > >> > > > > file
>>> > > > >> > > > > > > > >> > > > > > in
>>> > > > >> > > > > > > > >> > > > > > >> > this
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > > process. I've updated
>>> the
>>> > > KIP
>>> > > > to
>>> > > > >> > > > include
>>> > > > >> > > > > > > this.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 15) I believe Lucas
>>> meant
>>> > v1
>>> > > > >> and v2
>>> > > > >> > > > here.
>>> > > > >> > > > > > He
>>> > > > >> > > > > > > > was
>>> > > > >> > > > > > > > >> > > > referring
>>> > > > >> > > > > > > > >> > > > > > to
>>> > > > >> > > > > > > > >> > > > > > >> how
>>> > > > >> > > > > > > > >> > > > > > >> > > the
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > > requests would fall
>>> under
>>> > > > >> different
>>> > > > >> > > IBP
>>> > > > >> > > > > and
>>> > > > >> > > > > > > > meant
>>> > > > >> > > > > > > > >> > that
>>> > > > >> > > > > > > > >> > > > > older
>>> > > > >> > > > > > > > >> > > > > > >> > > brokers
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > would
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > > have to use the older
>>> > > version
>>> > > > of
>>> > > > >> > the
>>> > > > >> > > > > > request
>>> > > > >> > > > > > > > and
>>> > > > >> > > > > > > > >> the
>>> > > > >> > > > > > > > >> > > > > > existing
>>> > > > >> > > > > > > > >> > > > > > >> > topic
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > > deletion process. At
>>> > first,
>>> > > it
>>> > > > >> > seemed
>>> > > > >> > > > > like
>>> > > > >> > > > > > > > tagged
>>> > > > >> > > > > > > > >> > > fields
>>> > > > >> > > > > > > > >> > > > > > would
>>> > > > >> > > > > > > > >> > > > > > >> > > > resolve
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > > the IBP issue.
>>> However, we
>>> > > may
>>> > > > >> need
>>> > > > >> > > IBP
>>> > > > >> > > > > for
>>> > > > >> > > > > > > > this
>>> > > > >> > > > > > > > >> > > request
>>> > > > >> > > > > > > > >> > > > > > after
>>> > > > >> > > > > > > > >> > > > > > >> > all
>>> > > > >> > > > > > > > >> > > > > > >> > > > > since
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > > the controller
>>> handles the
>>> > > > topic
>>> > > > >> > > > deletion
>>> > > > >> > > > > > > > >> > differently
>>> > > > >> > > > > > > > >> > > > > > >> depending
>>> > > > >> > > > > > > > >> > > > > > >> > on
>>> > > > >> > > > > > > > >> > > > > > >> > > > the
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > IBP
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > > version. In an older
>>> > > version,
>>> > > > we
>>> > > > >> > > can't
>>> > > > >> > > > > just
>>> > > > >> > > > > > > > send
>>> > > > >> > > > > > > > >> a
>>> > > > >> > > > > > > > >> > > > > > StopReplica
>>> > > > >> > > > > > > > >> > > > > > >> > > delete
>>> > > > >> > > > > > > > >> > > > > > >> > > > > the
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > > topic immediately like
>>> > we'd
>>> > > > >> want to
>>> > > > >> > > for
>>> > > > >> > > > > > this
>>> > > > >> > > > > > > > KIP.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > > This makes me wonder
>>> if we
>>> > > > want
>>> > > > >> > > tagged
>>> > > > >> > > > > > fields
>>> > > > >> > > > > > > > on
>>> > > > >> > > > > > > > >> all
>>> > > > >> > > > > > > > >> > > the
>>> > > > >> > > > > > > > >> > > > > > >> requests
>>> > > > >> > > > > > > > >> > > > > > >> > > > after
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > > all. Let me know your
>>> > > > thoughts!
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > > Justine
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > > On Tue, Sep 15, 2020
>>> at
>>> > 1:03
>>> > > > PM
>>> > > > >> > > Justine
>>> > > > >> > > > > > > Olshan
>>> > > > >> > > > > > > > <
>>> > > > >> > > > > > > > >> > > > > > >> > > jolshan@confluent.io
>>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > > wrote:
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Hi all,
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Jun brought up a good
>>> > point
>>> > > > in
>>> > > > >> his
>>> > > > >> > > > last
>>> > > > >> > > > > > > email
>>> > > > >> > > > > > > > >> about
>>> > > > >> > > > > > > > >> > > > > tagged
>>> > > > >> > > > > > > > >> > > > > > >> > fields,
>>> > > > >> > > > > > > > >> > > > > > >> > > > and
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> I've updated the KIP
>>> to
>>> > > > reflect
>>> > > > >> > that
>>> > > > >> > > > the
>>> > > > >> > > > > > > > >> changes to
>>> > > > >> > > > > > > > >> > > > > > requests
>>> > > > >> > > > > > > > >> > > > > > >> and
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > responses
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> will be in the form
>>> of
>>> > > tagged
>>> > > > >> > fields
>>> > > > >> > > > to
>>> > > > >> > > > > > > avoid
>>> > > > >> > > > > > > > >> > > changing
>>> > > > >> > > > > > > > >> > > > > IBP.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Jun: I plan on
>>> sending a
>>> > > > >> followup
>>> > > > >> > > > email
>>> > > > >> > > > > to
>>> > > > >> > > > > > > > >> address
>>> > > > >> > > > > > > > >> > > some
>>> > > > >> > > > > > > > >> > > > > of
>>> > > > >> > > > > > > > >> > > > > > >> the
>>> > > > >> > > > > > > > >> > > > > > >> > > other
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> points.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Thanks,
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Justine
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> On Mon, Sep 14, 2020
>>> at
>>> > > 4:25
>>> > > > PM
>>> > > > >> > Jun
>>> > > > >> > > > Rao
>>> > > > >> > > > > <
>>> > > > >> > > > > > > > >> > > > > jun@confluent.io>
>>> > > > >> > > > > > > > >> > > > > > >> > wrote:
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Hi, Justine,
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Thanks for the
>>> updated
>>> > > KIP.
>>> > > > A
>>> > > > >> few
>>> > > > >> > > > > > comments
>>> > > > >> > > > > > > > >> below.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 10. LeaderAndIsr
>>> > Response:
>>> > > > Do
>>> > > > >> we
>>> > > > >> > > need
>>> > > > >> > > > > the
>>> > > > >> > > > > > > > topic
>>> > > > >> > > > > > > > >> > > name?
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 11. For the changed
>>> > > > >> > > request/response,
>>> > > > >> > > > > > other
>>> > > > >> > > > > > > > >> than
>>> > > > >> > > > > > > > >> > > > > > >> LeaderAndIsr,
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> UpdateMetadata,
>>> > Metadata,
>>> > > do
>>> > > > >> we
>>> > > > >> > > need
>>> > > > >> > > > to
>>> > > > >> > > > > > > > include
>>> > > > >> > > > > > > > >> > the
>>> > > > >> > > > > > > > >> > > > > topic
>>> > > > >> > > > > > > > >> > > > > > >> name?
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 12. It seems that
>>> > upgrades
>>> > > > >> don't
>>> > > > >> > > > > require
>>> > > > >> > > > > > > IBP.
>>> > > > >> > > > > > > > >> Does
>>> > > > >> > > > > > > > >> > > > that
>>> > > > >> > > > > > > > >> > > > > > mean
>>> > > > >> > > > > > > > >> > > > > > >> > the
>>> > > > >> > > > > > > > >> > > > > > >> > > > new
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> fields
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> in all the
>>> > > request/response
>>> > > > >> are
>>> > > > >> > > added
>>> > > > >> > > > > as
>>> > > > >> > > > > > > > tagged
>>> > > > >> > > > > > > > >> > > fields
>>> > > > >> > > > > > > > >> > > > > > >> without
>>> > > > >> > > > > > > > >> > > > > > >> > > > > bumping
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > up
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> the request
>>> version? It
>>> > > > would
>>> > > > >> be
>>> > > > >> > > > useful
>>> > > > >> > > > > > to
>>> > > > >> > > > > > > > make
>>> > > > >> > > > > > > > >> > that
>>> > > > >> > > > > > > > >> > > > > > clear.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 13. Partition
>>> Metadata
>>> > > file:
>>> > > > >> Do
>>> > > > >> > we
>>> > > > >> > > > need
>>> > > > >> > > > > > to
>>> > > > >> > > > > > > > >> include
>>> > > > >> > > > > > > > >> > > the
>>> > > > >> > > > > > > > >> > > > > > topic
>>> > > > >> > > > > > > > >> > > > > > >> > name
>>> > > > >> > > > > > > > >> > > > > > >> > > > and
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > the
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> partition id since
>>> they
>>> > > are
>>> > > > >> > implied
>>> > > > >> > > > in
>>> > > > >> > > > > > the
>>> > > > >> > > > > > > > >> > directory
>>> > > > >> > > > > > > > >> > > > > name?
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 14. In the JBOD
>>> mode, we
>>> > > > >> support
>>> > > > >> > > > > moving a
>>> > > > >> > > > > > > > >> > > partition's
>>> > > > >> > > > > > > > >> > > > > data
>>> > > > >> > > > > > > > >> > > > > > >> from
>>> > > > >> > > > > > > > >> > > > > > >> > > one
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > disk
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> to
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> another. Will the
>>> new
>>> > > > >> partition
>>> > > > >> > > > > metadata
>>> > > > >> > > > > > > file
>>> > > > >> > > > > > > > >> be
>>> > > > >> > > > > > > > >> > > > copied
>>> > > > >> > > > > > > > >> > > > > > >> during
>>> > > > >> > > > > > > > >> > > > > > >> > > that
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> process?
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 15. The KIP says
>>> "Remove
>>> > > > >> deleted
>>> > > > >> > > > topics
>>> > > > >> > > > > > > from
>>> > > > >> > > > > > > > >> > > replicas
>>> > > > >> > > > > > > > >> > > > by
>>> > > > >> > > > > > > > >> > > > > > >> > sending
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> StopReplicaRequest
>>> V2
>>> > for
>>> > > > any
>>> > > > >> > > topics
>>> > > > >> > > > > > which
>>> > > > >> > > > > > > do
>>> > > > >> > > > > > > > >> not
>>> > > > >> > > > > > > > >> > > > > contain
>>> > > > >> > > > > > > > >> > > > > > a
>>> > > > >> > > > > > > > >> > > > > > >> > topic
>>> > > > >> > > > > > > > >> > > > > > >> > > > ID,
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > and
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> V3 for any topics
>>> which
>>> > do
>>> > > > >> > contain
>>> > > > >> > > a
>>> > > > >> > > > > > topic
>>> > > > >> > > > > > > > >> ID.".
>>> > > > >> > > > > > > > >> > > > > However,
>>> > > > >> > > > > > > > >> > > > > > it
>>> > > > >> > > > > > > > >> > > > > > >> > > seems
>>> > > > >> > > > > > > > >> > > > > > >> > > > > the
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> updated controller
>>> will
>>> > > > create
>>> > > > >> > all
>>> > > > >> > > > > > missing
>>> > > > >> > > > > > > > >> topic
>>> > > > >> > > > > > > > >> > IDs
>>> > > > >> > > > > > > > >> > > > > first
>>> > > > >> > > > > > > > >> > > > > > >> > before
>>> > > > >> > > > > > > > >> > > > > > >> > > > > doing
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> other actions. So,
>>> is
>>> > > > >> > > > > StopReplicaRequest
>>> > > > >> > > > > > V2
>>> > > > >> > > > > > > > >> > needed?
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Jun
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> On Fri, Sep 11,
>>> 2020 at
>>> > > > 10:31
>>> > > > >> AM
>>> > > > >> > > John
>>> > > > >> > > > > > > > Roesler <
>>> > > > >> > > > > > > > >> > > > > > >> > > vvcephei@apache.org
>>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> wrote:
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > Thanks, Justine!
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > Your response
>>> seems
>>> > > > >> compelling
>>> > > > >> > to
>>> > > > >> > > > me.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > -John
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > On Fri,
>>> 2020-09-11 at
>>> > > > 10:17
>>> > > > >> > > -0700,
>>> > > > >> > > > > > > Justine
>>> > > > >> > > > > > > > >> > Olshan
>>> > > > >> > > > > > > > >> > > > > wrote:
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Hello all,
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Thanks for
>>> > continuing
>>> > > > the
>>> > > > >> > > > > > discussion! I
>>> > > > >> > > > > > > > >> have a
>>> > > > >> > > > > > > > >> > > few
>>> > > > >> > > > > > > > >> > > > > > >> > responses
>>> > > > >> > > > > > > > >> > > > > > >> > > to
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > your
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > points.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Tom: You are
>>> correct
>>> > > in
>>> > > > >> that
>>> > > > >> > > this
>>> > > > >> > > > > KIP
>>> > > > >> > > > > > > has
>>> > > > >> > > > > > > > >> not
>>> > > > >> > > > > > > > >> > > > > > mentioned
>>> > > > >> > > > > > > > >> > > > > > >> the
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
>>> > DeleteTopicsRequest. I
>>> > > > >> think
>>> > > > >> > > that
>>> > > > >> > > > > > this
>>> > > > >> > > > > > > > >> would
>>> > > > >> > > > > > > > >> > be
>>> > > > >> > > > > > > > >> > > > out
>>> > > > >> > > > > > > > >> > > > > of
>>> > > > >> > > > > > > > >> > > > > > >> > scope
>>> > > > >> > > > > > > > >> > > > > > >> > > > for
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> now, but
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > may be something
>>> > worth
>>> > > > >> adding
>>> > > > >> > > in
>>> > > > >> > > > > the
>>> > > > >> > > > > > > > >> future.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > John: We did
>>> > consider
>>> > > > >> > sequence
>>> > > > >> > > > ids,
>>> > > > >> > > > > > but
>>> > > > >> > > > > > > > >> there
>>> > > > >> > > > > > > > >> > > are
>>> > > > >> > > > > > > > >> > > > a
>>> > > > >> > > > > > > > >> > > > > > few
>>> > > > >> > > > > > > > >> > > > > > >> > > reasons
>>> > > > >> > > > > > > > >> > > > > > >> > > > > to
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> favor
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > UUIDs. There are
>>> > > several
>>> > > > >> > cases
>>> > > > >> > > > > where
>>> > > > >> > > > > > > > topics
>>> > > > >> > > > > > > > >> > from
>>> > > > >> > > > > > > > >> > > > > > >> different
>>> > > > >> > > > > > > > >> > > > > > >> > > > > clusters
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> may
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > interact now
>>> and in
>>> > > the
>>> > > > >> > future.
>>> > > > >> > > > For
>>> > > > >> > > > > > > > >> example,
>>> > > > >> > > > > > > > >> > > > Mirror
>>> > > > >> > > > > > > > >> > > > > > >> Maker 2
>>> > > > >> > > > > > > > >> > > > > > >> > > may
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> benefit
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > from being able
>>> to
>>> > > > detect
>>> > > > >> > when
>>> > > > >> > > a
>>> > > > >> > > > > > > cluster
>>> > > > >> > > > > > > > >> being
>>> > > > >> > > > > > > > >> > > > > > mirrored
>>> > > > >> > > > > > > > >> > > > > > >> is
>>> > > > >> > > > > > > > >> > > > > > >> > > > > deleted
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> and
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > recreated and
>>> > globally
>>> > > > >> unique
>>> > > > >> > > > > > > identifiers
>>> > > > >> > > > > > > > >> > would
>>> > > > >> > > > > > > > >> > > > make
>>> > > > >> > > > > > > > >> > > > > > >> > > resolving
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > issues
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > easier than
>>> sequence
>>> > > IDs
>>> > > > >> > which
>>> > > > >> > > > may
>>> > > > >> > > > > > > > collide
>>> > > > >> > > > > > > > >> > > between
>>> > > > >> > > > > > > > >> > > > > > >> > clusters.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > KIP-405
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > (tiered storage)
>>> > will
>>> > > > also
>>> > > > >> > > > benefit
>>> > > > >> > > > > > from
>>> > > > >> > > > > > > > >> > globally
>>> > > > >> > > > > > > > >> > > > > > unique
>>> > > > >> > > > > > > > >> > > > > > >> IDs
>>> > > > >> > > > > > > > >> > > > > > >> > > as
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > shared
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > buckets may be
>>> used
>>> > > > >> between
>>> > > > >> > > > > clusters.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Globally unique
>>> IDs
>>> > > > would
>>> > > > >> > also
>>> > > > >> > > > make
>>> > > > >> > > > > > > > >> > > functionality
>>> > > > >> > > > > > > > >> > > > > like
>>> > > > >> > > > > > > > >> > > > > > >> > moving
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > topics
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > between
>>> disparate
>>> > > > clusters
>>> > > > >> > > easier
>>> > > > >> > > > > in
>>> > > > >> > > > > > > the
>>> > > > >> > > > > > > > >> > future,
>>> > > > >> > > > > > > > >> > > > > > >> simplify
>>> > > > >> > > > > > > > >> > > > > > >> > any
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > future
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > implementations
>>> of
>>> > > > backups
>>> > > > >> > and
>>> > > > >> > > > > > > restores,
>>> > > > >> > > > > > > > >> and
>>> > > > >> > > > > > > > >> > > more.
>>> > > > >> > > > > > > > >> > > > > In
>>> > > > >> > > > > > > > >> > > > > > >> > > general,
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> unique IDs
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > would ensure
>>> that
>>> > the
>>> > > > >> source
>>> > > > >> > > > > cluster
>>> > > > >> > > > > > > > >> topics do
>>> > > > >> > > > > > > > >> > > not
>>> > > > >> > > > > > > > >> > > > > > >> conflict
>>> > > > >> > > > > > > > >> > > > > > >> > > > with
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > the
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > destination
>>> cluster
>>> > > > >> topics.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > If we were to
>>> use
>>> > > > sequence
>>> > > > >> > ids,
>>> > > > >> > > > we
>>> > > > >> > > > > > > would
>>> > > > >> > > > > > > > >> need
>>> > > > >> > > > > > > > >> > > > > > >> sufficiently
>>> > > > >> > > > > > > > >> > > > > > >> > > > large
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> cluster
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > ids to be stored
>>> > with
>>> > > > the
>>> > > > >> > topic
>>> > > > >> > > > > > > > >> identifiers or
>>> > > > >> > > > > > > > >> > > we
>>> > > > >> > > > > > > > >> > > > > run
>>> > > > >> > > > > > > > >> > > > > > >> the
>>> > > > >> > > > > > > > >> > > > > > >> > > risk
>>> > > > >> > > > > > > > >> > > > > > >> > > > of
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > collisions. This
>>> > will
>>> > > > >> give up
>>> > > > >> > > any
>>> > > > >> > > > > > > > >> advantage in
>>> > > > >> > > > > > > > >> > > > > > >> compactness
>>> > > > >> > > > > > > > >> > > > > > >> > > that
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> sequence
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > numbers may
>>> bring.
>>> > > Given
>>> > > > >> > these
>>> > > > >> > > > > > > > advantages I
>>> > > > >> > > > > > > > >> > > think
>>> > > > >> > > > > > > > >> > > > it
>>> > > > >> > > > > > > > >> > > > > > >> makes
>>> > > > >> > > > > > > > >> > > > > > >> > > > sense
>>> > > > >> > > > > > > > >> > > > > > >> > > > > to
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> use
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > UUIDs.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Gokul: This is
>>> an
>>> > > > >> interesting
>>> > > > >> > > > idea,
>>> > > > >> > > > > > but
>>> > > > >> > > > > > > > >> this
>>> > > > >> > > > > > > > >> > is
>>> > > > >> > > > > > > > >> > > a
>>> > > > >> > > > > > > > >> > > > > > >> breaking
>>> > > > >> > > > > > > > >> > > > > > >> > > > > change.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Out of
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > scope for now,
>>> but
>>> > > maybe
>>> > > > >> > worth
>>> > > > >> > > > > > > discussing
>>> > > > >> > > > > > > > >> in
>>> > > > >> > > > > > > > >> > the
>>> > > > >> > > > > > > > >> > > > > > future.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Hope this
>>> explains
>>> > > some
>>> > > > of
>>> > > > >> > the
>>> > > > >> > > > > > > decisions,
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Justine
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > On Fri, Sep 11,
>>> 2020
>>> > > at
>>> > > > >> 8:27
>>> > > > >> > AM
>>> > > > >> > > > > Gokul
>>> > > > >> > > > > > > > >> Ramanan
>>> > > > >> > > > > > > > >> > > > > > >> Subramanian <
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
>>> > gokul2411s@gmail.com>
>>> > > > >> wrote:
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Hi.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Thanks for the
>>> > KIP.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Have you
>>> thought
>>> > > about
>>> > > > >> > > whether
>>> > > > >> > > > it
>>> > > > >> > > > > > > makes
>>> > > > >> > > > > > > > >> > sense
>>> > > > >> > > > > > > > >> > > to
>>> > > > >> > > > > > > > >> > > > > > >> support
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> authorizing a
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > principal for
>>> a
>>> > > topic
>>> > > > ID
>>> > > > >> > > rather
>>> > > > >> > > > > > than
>>> > > > >> > > > > > > a
>>> > > > >> > > > > > > > >> topic
>>> > > > >> > > > > > > > >> > > > name
>>> > > > >> > > > > > > > >> > > > > to
>>> > > > >> > > > > > > > >> > > > > > >> > > achieve
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> tighter
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > security?
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Or is the
>>> topic ID
>>> > > > >> > > > fundamentally
>>> > > > >> > > > > an
>>> > > > >> > > > > > > > >> internal
>>> > > > >> > > > > > > > >> > > > > detail
>>> > > > >> > > > > > > > >> > > > > > >> > similar
>>> > > > >> > > > > > > > >> > > > > > >> > > > to
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> epochs
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > used
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > in a bunch of
>>> > other
>>> > > > >> places
>>> > > > >> > in
>>> > > > >> > > > > > Kafka?
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Thanks.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > On Fri, Sep
>>> 11,
>>> > 2020
>>> > > > at
>>> > > > >> > 4:06
>>> > > > >> > > PM
>>> > > > >> > > > > > John
>>> > > > >> > > > > > > > >> > Roesler <
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > vvcephei@apache.org>
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > wrote:
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Hello
>>> Justine,
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Thanks for
>>> the
>>> > > KIP!
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > I happen to
>>> have
>>> > > > been
>>> > > > >> > > > > confronted
>>> > > > >> > > > > > > > >> recently
>>> > > > >> > > > > > > > >> > > with
>>> > > > >> > > > > > > > >> > > > > the
>>> > > > >> > > > > > > > >> > > > > > >> need
>>> > > > >> > > > > > > > >> > > > > > >> > > to
>>> > > > >> > > > > > > > >> > > > > > >> > > > > keep
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > track of
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > a
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > large
>>> number of
>>> > > > >> topics as
>>> > > > >> > > > > > compactly
>>> > > > >> > > > > > > > as
>>> > > > >> > > > > > > > >> > > > > possible. I
>>> > > > >> > > > > > > > >> > > > > > >> was
>>> > > > >> > > > > > > > >> > > > > > >> > > > going
>>> > > > >> > > > > > > > >> > > > > > >> > > > > to
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> come
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > up
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > with some
>>> way to
>>> > > > >> > dictionary
>>> > > > >> > > > > > encode
>>> > > > >> > > > > > > > the
>>> > > > >> > > > > > > > >> > topic
>>> > > > >> > > > > > > > >> > > > > names
>>> > > > >> > > > > > > > >> > > > > > >> as
>>> > > > >> > > > > > > > >> > > > > > >> > > > > integers,
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> but
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > this
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > seems much
>>> > better!
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Apologies if
>>> > this
>>> > > > has
>>> > > > >> > been
>>> > > > >> > > > > raised
>>> > > > >> > > > > > > > >> before,
>>> > > > >> > > > > > > > >> > > but
>>> > > > >> > > > > > > > >> > > > > I’m
>>> > > > >> > > > > > > > >> > > > > > >> > > wondering
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> about the
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > choice of
>>> UUID
>>> > vs
>>> > > > >> > sequence
>>> > > > >> > > > > number
>>> > > > >> > > > > > > for
>>> > > > >> > > > > > > > >> the
>>> > > > >> > > > > > > > >> > > ids.
>>> > > > >> > > > > > > > >> > > > > > >> > Typically,
>>> > > > >> > > > > > > > >> > > > > > >> > > > > I’ve
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> seen
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > UUIDs
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > in two
>>> > situations:
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 1. When
>>> > processes
>>> > > > >> need to
>>> > > > >> > > > > > generate
>>> > > > >> > > > > > > > >> > > > non-colliding
>>> > > > >> > > > > > > > >> > > > > > >> > > > identifiers
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> without
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
>>> coordination.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 2. When the
>>> > > > identifier
>>> > > > >> > > needs
>>> > > > >> > > > to
>>> > > > >> > > > > > be
>>> > > > >> > > > > > > > >> > > > “universally
>>> > > > >> > > > > > > > >> > > > > > >> > unique”;
>>> > > > >> > > > > > > > >> > > > > > >> > > > > I.e.,
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> the
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > identifier
>>> needs
>>> > > to
>>> > > > >> > > > distinguish
>>> > > > >> > > > > > the
>>> > > > >> > > > > > > > >> entity
>>> > > > >> > > > > > > > >> > > > from
>>> > > > >> > > > > > > > >> > > > > > all
>>> > > > >> > > > > > > > >> > > > > > >> > other
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> entities
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > that
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > could ever
>>> > exist.
>>> > > > >> This is
>>> > > > >> > > > > useful
>>> > > > >> > > > > > in
>>> > > > >> > > > > > > > >> cases
>>> > > > >> > > > > > > > >> > > > where
>>> > > > >> > > > > > > > >> > > > > > >> > entities
>>> > > > >> > > > > > > > >> > > > > > >> > > > from
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > all
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > kinds
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > of
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > systems get
>>> > mixed
>>> > > > >> > together,
>>> > > > >> > > > > such
>>> > > > >> > > > > > as
>>> > > > >> > > > > > > > >> when
>>> > > > >> > > > > > > > >> > > > dumping
>>> > > > >> > > > > > > > >> > > > > > >> logs
>>> > > > >> > > > > > > > >> > > > > > >> > > from
>>> > > > >> > > > > > > > >> > > > > > >> > > > > all
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > processes
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > in
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a company
>>> into a
>>> > > > >> common
>>> > > > >> > > > system.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Maybe I’m
>>> being
>>> > > > >> > > > short-sighted,
>>> > > > >> > > > > > but
>>> > > > >> > > > > > > it
>>> > > > >> > > > > > > > >> > > doesn’t
>>> > > > >> > > > > > > > >> > > > > seem
>>> > > > >> > > > > > > > >> > > > > > >> like
>>> > > > >> > > > > > > > >> > > > > > >> > > > > either
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> really
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > applies
>>> here. It
>>> > > > seems
>>> > > > >> > like
>>> > > > >> > > > the
>>> > > > >> > > > > > > > brokers
>>> > > > >> > > > > > > > >> > > could
>>> > > > >> > > > > > > > >> > > > > and
>>> > > > >> > > > > > > > >> > > > > > >> would
>>> > > > >> > > > > > > > >> > > > > > >> > > > > achieve
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > consensus
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > when
>>> creating a
>>> > > > topic
>>> > > > >> > > anyway,
>>> > > > >> > > > > > which
>>> > > > >> > > > > > > > is
>>> > > > >> > > > > > > > >> all
>>> > > > >> > > > > > > > >> > > > > that’s
>>> > > > >> > > > > > > > >> > > > > > >> > > required
>>> > > > >> > > > > > > > >> > > > > > >> > > > to
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > generate
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
>>> non-colliding
>>> > > > sequence
>>> > > > >> > ids.
>>> > > > >> > > > For
>>> > > > >> > > > > > the
>>> > > > >> > > > > > > > >> > second,
>>> > > > >> > > > > > > > >> > > as
>>> > > > >> > > > > > > > >> > > > > you
>>> > > > >> > > > > > > > >> > > > > > >> > > mention,
>>> > > > >> > > > > > > > >> > > > > > >> > > > > we
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> could
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > assign
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a UUID to
>>> the
>>> > > > cluster
>>> > > > >> as
>>> > > > >> > a
>>> > > > >> > > > > whole,
>>> > > > >> > > > > > > > which
>>> > > > >> > > > > > > > >> > > would
>>> > > > >> > > > > > > > >> > > > > > render
>>> > > > >> > > > > > > > >> > > > > > >> > any
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > resource
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > scoped
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > the broker
>>> > > > universally
>>> > > > >> > > unique
>>> > > > >> > > > > as
>>> > > > >> > > > > > > > well.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > The reason I
>>> > > mention
>>> > > > >> this
>>> > > > >> > > is
>>> > > > >> > > > > > that,
>>> > > > >> > > > > > > > >> > although
>>> > > > >> > > > > > > > >> > > a
>>> > > > >> > > > > > > > >> > > > > UUID
>>> > > > >> > > > > > > > >> > > > > > >> is
>>> > > > >> > > > > > > > >> > > > > > >> > way
>>> > > > >> > > > > > > > >> > > > > > >> > > > > more
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > compact
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > than topic
>>> > names,
>>> > > > it’s
>>> > > > >> > > still
>>> > > > >> > > > 16
>>> > > > >> > > > > > > > bytes.
>>> > > > >> > > > > > > > >> In
>>> > > > >> > > > > > > > >> > > > > > contrast,
>>> > > > >> > > > > > > > >> > > > > > >> a
>>> > > > >> > > > > > > > >> > > > > > >> > > > 4-byte
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> integer
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > sequence id
>>> > would
>>> > > > give
>>> > > > >> > us 4
>>> > > > >> > > > > > billion
>>> > > > >> > > > > > > > >> unique
>>> > > > >> > > > > > > > >> > > > > topics
>>> > > > >> > > > > > > > >> > > > > > >> per
>>> > > > >> > > > > > > > >> > > > > > >> > > > > cluster,
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> which
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > seems
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > like enough
>>> ;)
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Considering
>>> the
>>> > > > >> number of
>>> > > > >> > > > > > different
>>> > > > >> > > > > > > > >> times
>>> > > > >> > > > > > > > >> > > > these
>>> > > > >> > > > > > > > >> > > > > > >> topic
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> identifiers are
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > sent
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > over the
>>> wire or
>>> > > > >> stored
>>> > > > >> > in
>>> > > > >> > > > > > memory,
>>> > > > >> > > > > > > it
>>> > > > >> > > > > > > > >> > seems
>>> > > > >> > > > > > > > >> > > > like
>>> > > > >> > > > > > > > >> > > > > > it
>>> > > > >> > > > > > > > >> > > > > > >> > might
>>> > > > >> > > > > > > > >> > > > > > >> > > > be
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> worth
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > the
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > additional
>>> 4x
>>> > > space
>>> > > > >> > > savings.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > What do you
>>> > think
>>> > > > >> about
>>> > > > >> > > this?
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Thanks,
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > John
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > On Fri, Sep
>>> 11,
>>> > > > 2020,
>>> > > > >> at
>>> > > > >> > > > 03:20,
>>> > > > >> > > > > > Tom
>>> > > > >> > > > > > > > >> > Bentley
>>> > > > >> > > > > > > > >> > > > > wrote:
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Hi
>>> Justine,
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > This looks
>>> > like
>>> > > a
>>> > > > >> very
>>> > > > >> > > > > welcome
>>> > > > >> > > > > > > > >> > > improvement.
>>> > > > >> > > > > > > > >> > > > > > >> Thanks!
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Maybe I
>>> missed
>>> > > it,
>>> > > > >> but
>>> > > > >> > > the
>>> > > > >> > > > > KIP
>>> > > > >> > > > > > > > >> doesn't
>>> > > > >> > > > > > > > >> > > seem
>>> > > > >> > > > > > > > >> > > > to
>>> > > > >> > > > > > > > >> > > > > > >> > mention
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > changing
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
>>> > > > DeleteTopicsRequest
>>> > > > >> to
>>> > > > >> > > > > identify
>>> > > > >> > > > > > > the
>>> > > > >> > > > > > > > >> > topic
>>> > > > >> > > > > > > > >> > > > > using
>>> > > > >> > > > > > > > >> > > > > > an
>>> > > > >> > > > > > > > >> > > > > > >> > id.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > Maybe
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > that's out
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > of
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > scope, but
>>> > > > >> > > > > DeleteTopicsRequest
>>> > > > >> > > > > > is
>>> > > > >> > > > > > > > not
>>> > > > >> > > > > > > > >> > > listed
>>> > > > >> > > > > > > > >> > > > > > among
>>> > > > >> > > > > > > > >> > > > > > >> > the
>>> > > > >> > > > > > > > >> > > > > > >> > > > > Future
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Work
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > APIs
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > either.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Kind
>>> regards,
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Tom
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > On Thu,
>>> Sep
>>> > 10,
>>> > > > >> 2020 at
>>> > > > >> > > > 3:59
>>> > > > >> > > > > PM
>>> > > > >> > > > > > > > >> Satish
>>> > > > >> > > > > > > > >> > > > > Duggana <
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
>>> > > > >> satish.duggana@gmail.com
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > wrote:
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Thanks
>>> > > > >> Lucas/Justine
>>> > > > >> > > for
>>> > > > >> > > > > the
>>> > > > >> > > > > > > nice
>>> > > > >> > > > > > > > >> KIP.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > It has
>>> > several
>>> > > > >> > benefits
>>> > > > >> > > > > which
>>> > > > >> > > > > > > > also
>>> > > > >> > > > > > > > >> > > include
>>> > > > >> > > > > > > > >> > > > > > >> > > simplifying
>>> > > > >> > > > > > > > >> > > > > > >> > > > > the
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> topic
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > deletion
>>> > > process
>>> > > > >> by
>>> > > > >> > > > > > controller
>>> > > > >> > > > > > > > and
>>> > > > >> > > > > > > > >> > logs
>>> > > > >> > > > > > > > >> > > > > > cleanup
>>> > > > >> > > > > > > > >> > > > > > >> by
>>> > > > >> > > > > > > > >> > > > > > >> > > > > brokers
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > in
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > corner
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > cases.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Best,
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Satish.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > On Wed,
>>> Sep
>>> > 9,
>>> > > > >> 2020
>>> > > > >> > at
>>> > > > >> > > > > 10:07
>>> > > > >> > > > > > PM
>>> > > > >> > > > > > > > >> > Justine
>>> > > > >> > > > > > > > >> > > > > > Olshan <
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
>>> jolshan@confluent.io
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > wrote:
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Hello
>>> all,
>>> > > > it's
>>> > > > >> > been
>>> > > > >> > > > > > almost a
>>> > > > >> > > > > > > > >> year!
>>> > > > >> > > > > > > > >> > > I've
>>> > > > >> > > > > > > > >> > > > > > made
>>> > > > >> > > > > > > > >> > > > > > >> > some
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > changes
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> to
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > this
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > KIP
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > and
>>> hope to
>>> > > > >> continue
>>> > > > >> > > the
>>> > > > >> > > > > > > > >> discussion.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > One
>>> of the
>>> > > > main
>>> > > > >> > > changes
>>> > > > >> > > > > > I've
>>> > > > >> > > > > > > > >> added
>>> > > > >> > > > > > > > >> > is
>>> > > > >> > > > > > > > >> > > > now
>>> > > > >> > > > > > > > >> > > > > > the
>>> > > > >> > > > > > > > >> > > > > > >> > > > metadata
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> response
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > will
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > include
>>> the
>>> > > > topic
>>> > > > >> ID
>>> > > > >> > > (as
>>> > > > >> > > > > > Colin
>>> > > > >> > > > > > > > >> > > suggested).
>>> > > > >> > > > > > > > >> > > > > > >> Clients
>>> > > > >> > > > > > > > >> > > > > > >> > > can
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> obtain the
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > topicID
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > of a
>>> given
>>> > > topic
>>> > > > >> > > through
>>> > > > >> > > > a
>>> > > > >> > > > > > > > >> > > > TopicDescription.
>>> > > > >> > > > > > > > >> > > > > > The
>>> > > > >> > > > > > > > >> > > > > > >> > > > topicId
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > will
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > also be
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > included
>>> > with
>>> > > > the
>>> > > > >> > > > > > > UpdateMetadata
>>> > > > >> > > > > > > > >> > > request.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Let me
>>> > know
>>> > > > what
>>> > > > >> > you
>>> > > > >> > > > all
>>> > > > >> > > > > > > think.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Thank
>>> you,
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
>>> Justine
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > On
>>> > > 2019/09/13
>>> > > > >> > > 16:38:26,
>>> > > > >> > > > > > > "Colin
>>> > > > >> > > > > > > > >> > > McCabe" <
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > cmccabe@apache.org
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > wrote:
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Hi
>>> > Lucas,
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
>>> Thanks
>>> > for
>>> > > > >> > tackling
>>> > > > >> > > > > this.
>>> > > > >> > > > > > > > >> Topic
>>> > > > >> > > > > > > > >> > IDs
>>> > > > >> > > > > > > > >> > > > > are a
>>> > > > >> > > > > > > > >> > > > > > >> > great
>>> > > > >> > > > > > > > >> > > > > > >> > > > > idea,
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> and
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > this
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > is
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > really
>>> good
>>> > > > >> writeup.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > For
>>> > > > >> > > > > > > /brokers/topics/[topic],
>>> > > > >> > > > > > > > >> the
>>> > > > >> > > > > > > > >> > > > schema
>>> > > > >> > > > > > > > >> > > > > > >> version
>>> > > > >> > > > > > > > >> > > > > > >> > > > > should
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > be
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > bumped
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > version
>>> 3,
>>> > > > rather
>>> > > > >> > than
>>> > > > >> > > 2.
>>> > > > >> > > > > > > > KIP-455
>>> > > > >> > > > > > > > >> > > bumped
>>> > > > >> > > > > > > > >> > > > > the
>>> > > > >> > > > > > > > >> > > > > > >> > version
>>> > > > >> > > > > > > > >> > > > > > >> > > > of
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > this
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > znode
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 2
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > already
>>> :)
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
>>> Given
>>> > that
>>> > > > >> we're
>>> > > > >> > > > going
>>> > > > >> > > > > to
>>> > > > >> > > > > > > be
>>> > > > >> > > > > > > > >> > seeing
>>> > > > >> > > > > > > > >> > > > > these
>>> > > > >> > > > > > > > >> > > > > > >> > things
>>> > > > >> > > > > > > > >> > > > > > >> > > as
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> strings
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > as
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > lot
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > (in
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > logs, in
>>> > > > >> ZooKeeper,
>>> > > > >> > on
>>> > > > >> > > > the
>>> > > > >> > > > > > > > >> > command-line,
>>> > > > >> > > > > > > > >> > > > > > etc.),
>>> > > > >> > > > > > > > >> > > > > > >> > does
>>> > > > >> > > > > > > > >> > > > > > >> > > it
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > make
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > sense to
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > use
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > base64
>>> when
>>> > > > >> > converting
>>> > > > >> > > > them
>>> > > > >> > > > > > to
>>> > > > >> > > > > > > > >> > strings?
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
>>> Here is
>>> > an
>>> > > > >> > example
>>> > > > >> > > of
>>> > > > >> > > > > the
>>> > > > >> > > > > > > hex
>>> > > > >> > > > > > > > >> > > > > > >> representation:
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
>>> > > > >> > > > > > > > >> > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > And
>>> here
>>> > > is
>>> > > > an
>>> > > > >> > > > example
>>> > > > >> > > > > in
>>> > > > >> > > > > > > > >> base64.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
>>> > > > >> > > > b8tRS7h4TJ2Vt43Dp85v2A
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > The
>>> > base64
>>> > > > >> > version
>>> > > > >> > > > > saves
>>> > > > >> > > > > > 15
>>> > > > >> > > > > > > > >> > letters
>>> > > > >> > > > > > > > >> > > > (to
>>> > > > >> > > > > > > > >> > > > > be
>>> > > > >> > > > > > > > >> > > > > > >> > fair,
>>> > > > >> > > > > > > > >> > > > > > >> > > 4
>>> > > > >> > > > > > > > >> > > > > > >> > > > of
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> those
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > were
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > dashes
>>> that
>>> > we
>>> > > > >> could
>>> > > > >> > > have
>>> > > > >> > > > > > > elided
>>> > > > >> > > > > > > > in
>>> > > > >> > > > > > > > >> > the
>>> > > > >> > > > > > > > >> > > > hex
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > representation.)
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
>>> Another
>>> > > > thing
>>> > > > >> to
>>> > > > >> > > > > consider
>>> > > > >> > > > > > > is
>>> > > > >> > > > > > > > >> that
>>> > > > >> > > > > > > > >> > we
>>> > > > >> > > > > > > > >> > > > > > should
>>> > > > >> > > > > > > > >> > > > > > >> > > specify
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > that
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> the
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
>>> all-zeroes
>>> > > UUID
>>> > > > is
>>> > > > >> > not
>>> > > > >> > > a
>>> > > > >> > > > > > valid
>>> > > > >> > > > > > > > >> topic
>>> > > > >> > > > > > > > >> > > UUID.
>>> > > > >> > > > > > > > >> > > > > >  We
>>> > > > >> > > > > > > > >> > > > > > >> > can't
>>> > > > >> > > > > > > > >> > > > > > >> > > > use
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> null
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > for
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > this
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > because
>>> we
>>> > > can't
>>> > > > >> > pass a
>>> > > > >> > > > > null
>>> > > > >> > > > > > > UUID
>>> > > > >> > > > > > > > >> over
>>> > > > >> > > > > > > > >> > > the
>>> > > > >> > > > > > > > >> > > > > RPC
>>> > > > >> > > > > > > > >> > > > > > >> > > protocol
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> (there
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > is no
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > special
>>> > > pattern
>>> > > > >> for
>>> > > > >> > > null,
>>> > > > >> > > > > nor
>>> > > > >> > > > > > > do
>>> > > > >> > > > > > > > we
>>> > > > >> > > > > > > > >> > want
>>> > > > >> > > > > > > > >> > > > to
>>> > > > >> > > > > > > > >> > > > > > >> waste
>>> > > > >> > > > > > > > >> > > > > > >> > > space
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> reserving
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > such
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
>>> pattern.)
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
>>> Maybe I
>>> > > > missed
>>> > > > >> > it,
>>> > > > >> > > > but
>>> > > > >> > > > > > did
>>> > > > >> > > > > > > > you
>>> > > > >> > > > > > > > >> > > > describe
>>> > > > >> > > > > > > > >> > > > > > >> > > "migration
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > of...
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > existing
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > topic[s]
>>> > > without
>>> > > > >> > topic
>>> > > > >> > > > IDs"
>>> > > > >> > > > > > in
>>> > > > >> > > > > > > > >> detail
>>> > > > >> > > > > > > > >> > in
>>> > > > >> > > > > > > > >> > > > any
>>> > > > >> > > > > > > > >> > > > > > >> > section?
>>> > > > >> > > > > > > > >> > > > > > >> > > > It
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> seems
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > like
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > when
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > the new
>>> > > > controller
>>> > > > >> > > > becomes
>>> > > > >> > > > > > > > active,
>>> > > > >> > > > > > > > >> it
>>> > > > >> > > > > > > > >> > > > should
>>> > > > >> > > > > > > > >> > > > > > >> just
>>> > > > >> > > > > > > > >> > > > > > >> > > > > generate
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> random
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > UUIDs for
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > these,
>>> and
>>> > > write
>>> > > > >> the
>>> > > > >> > > > random
>>> > > > >> > > > > > > UUIDs
>>> > > > >> > > > > > > > >> back
>>> > > > >> > > > > > > > >> > > to
>>> > > > >> > > > > > > > >> > > > > > >> > ZooKeeper.
>>> > > > >> > > > > > > > >> > > > > > >> > > > It
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> would be
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > good
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > to
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > spell
>>> that
>>> > > out.
>>> > > > >> We
>>> > > > >> > > > should
>>> > > > >> > > > > > make
>>> > > > >> > > > > > > > it
>>> > > > >> > > > > > > > >> > clear
>>> > > > >> > > > > > > > >> > > > > that
>>> > > > >> > > > > > > > >> > > > > > >> this
>>> > > > >> > > > > > > > >> > > > > > >> > > > > happens
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > regardless
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > of
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > the
>>> > > inter-broker
>>> > > > >> > > protocol
>>> > > > >> > > > > > > version
>>> > > > >> > > > > > > > >> > (it's
>>> > > > >> > > > > > > > >> > > a
>>> > > > >> > > > > > > > >> > > > > > >> > compatible
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > change).
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
>>> > > > >> > > "LeaderAndIsrRequests
>>> > > > >> > > > > > > > >> including an
>>> > > > >> > > > > > > > >> > > > > > >> > > > is_every_partition
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> flag"
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > seems a
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > bit
>>> wordy.
>>> > > Can
>>> > > > we
>>> > > > >> > just
>>> > > > >> > > > > call
>>> > > > >> > > > > > > > these
>>> > > > >> > > > > > > > >> > "full
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
>>> LeaderAndIsrRequests"?
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > Then
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > the
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > RPC
>>> field
>>> > > could
>>> > > > be
>>> > > > >> > > named
>>> > > > >> > > > > > > "full".
>>> > > > >> > > > > > > > >> > Also,
>>> > > > >> > > > > > > > >> > > it
>>> > > > >> > > > > > > > >> > > > > > would
>>> > > > >> > > > > > > > >> > > > > > >> > > > probably
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > be
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > better
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > for the
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > RPC
>>> field to
>>> > > be
>>> > > > an
>>> > > > >> > enum
>>> > > > >> > > > of
>>> > > > >> > > > > {
>>> > > > >> > > > > > > > >> > > UNSPECIFIED,
>>> > > > >> > > > > > > > >> > > > > > >> > > INCREMENTAL,
>>> > > > >> > > > > > > > >> > > > > > >> > > > > FULL
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> }, so
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > that
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > we
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > can
>>> cleanly
>>> > > > handle
>>> > > > >> > old
>>> > > > >> > > > > > versions
>>> > > > >> > > > > > > > (by
>>> > > > >> > > > > > > > >> > > > treating
>>> > > > >> > > > > > > > >> > > > > > >> them
>>> > > > >> > > > > > > > >> > > > > > >> > as
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> UNSPECIFIED)
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > In
>>> the
>>> > > > >> > > > > > LeaderAndIsrRequest
>>> > > > >> > > > > > > > >> > section,
>>> > > > >> > > > > > > > >> > > > you
>>> > > > >> > > > > > > > >> > > > > > >> write
>>> > > > >> > > > > > > > >> > > > > > >> > "A
>>> > > > >> > > > > > > > >> > > > > > >> > > > > final
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > deletion
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > event
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > will be
>>> > > > secheduled
>>> > > > >> > for
>>> > > > >> > > X
>>> > > > >> > > > ms
>>> > > > >> > > > > > > after
>>> > > > >> > > > > > > > >> the
>>> > > > >> > > > > > > > >> > > > > > >> > > > LeaderAndIsrRequest
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > was
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > first
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
>>> received..."
>>> > > I
>>> > > > >> guess
>>> > > > >> > > > the X
>>> > > > >> > > > > > > was a
>>> > > > >> > > > > > > > >> > > > > placeholder
>>> > > > >> > > > > > > > >> > > > > > >> that
>>> > > > >> > > > > > > > >> > > > > > >> > > you
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> intended
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > replace
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > before
>>> > > posting?
>>> > > > :)
>>> > > > >> > In
>>> > > > >> > > > any
>>> > > > >> > > > > > > case,
>>> > > > >> > > > > > > > >> this
>>> > > > >> > > > > > > > >> > > > seems
>>> > > > >> > > > > > > > >> > > > > > like
>>> > > > >> > > > > > > > >> > > > > > >> > the
>>> > > > >> > > > > > > > >> > > > > > >> > > > kind
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > of
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > thing
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > we'd
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > want a
>>> > > > >> configuration
>>> > > > >> > > for.
>>> > > > >> > > > > > > Let's
>>> > > > >> > > > > > > > >> > > describe
>>> > > > >> > > > > > > > >> > > > > that
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > configuration
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> key
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > somewhere
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > in this
>>> KIP,
>>> > > > >> > including
>>> > > > >> > > > what
>>> > > > >> > > > > > its
>>> > > > >> > > > > > > > >> > default
>>> > > > >> > > > > > > > >> > > > > value
>>> > > > >> > > > > > > > >> > > > > > >> is.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > We
>>> > should
>>> > > > >> > probably
>>> > > > >> > > > also
>>> > > > >> > > > > > > log a
>>> > > > >> > > > > > > > >> > bunch
>>> > > > >> > > > > > > > >> > > of
>>> > > > >> > > > > > > > >> > > > > > >> messages
>>> > > > >> > > > > > > > >> > > > > > >> > > at
>>> > > > >> > > > > > > > >> > > > > > >> > > > > WARN
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> level
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > when
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
>>> something is
>>> > > > >> > scheduled
>>> > > > >> > > > for
>>> > > > >> > > > > > > > >> deletion,
>>> > > > >> > > > > > > > >> > as
>>> > > > >> > > > > > > > >> > > > > well.
>>> > > > >> > > > > > > > >> > > > > > >> > (Maybe
>>> > > > >> > > > > > > > >> > > > > > >> > > > > this
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> was
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > assumed, but
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > it
>>> would be
>>> > > good
>>> > > > >> to
>>> > > > >> > > > mention
>>> > > > >> > > > > > > it).
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > I
>>> feel
>>> > > like
>>> > > > >> there
>>> > > > >> > > > are a
>>> > > > >> > > > > > few
>>> > > > >> > > > > > > > >> > sections
>>> > > > >> > > > > > > > >> > > > > that
>>> > > > >> > > > > > > > >> > > > > > >> > should
>>> > > > >> > > > > > > > >> > > > > > >> > > be
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> moved to
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > "rejected
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
>>> > alternatives."
>>> > > > >> For
>>> > > > >> > > > > example,
>>> > > > >> > > > > > in
>>> > > > >> > > > > > > > the
>>> > > > >> > > > > > > > >> > > > > > DeleteTopics
>>> > > > >> > > > > > > > >> > > > > > >> > > > section,
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> since
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > we're
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > not
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > going
>>> to do
>>> > > > >> option 1
>>> > > > >> > or
>>> > > > >> > > > 2,
>>> > > > >> > > > > > > these
>>> > > > >> > > > > > > > >> > should
>>> > > > >> > > > > > > > >> > > be
>>> > > > >> > > > > > > > >> > > > > > moved
>>> > > > >> > > > > > > > >> > > > > > >> > into
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> "rejected
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
>>> > alternatives,"
>>> > > > >> > rather
>>> > > > >> > > > than
>>> > > > >> > > > > > > > >> appearing
>>> > > > >> > > > > > > > >> > > > > inline.
>>> > > > >> > > > > > > > >> > > > > > >> > > Another
>>> > > > >> > > > > > > > >> > > > > > >> > > > > case
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> is
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > the
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > "Should
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > we
>>> remove
>>> > > topic
>>> > > > >> name
>>> > > > >> > > from
>>> > > > >> > > > > the
>>> > > > >> > > > > > > > >> protocol
>>> > > > >> > > > > > > > >> > > > where
>>> > > > >> > > > > > > > >> > > > > > >> > > possible"
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> section.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > This
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > is
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > clearly
>>> > > > >> discussing a
>>> > > > >> > > > design
>>> > > > >> > > > > > > > >> > alternative
>>> > > > >> > > > > > > > >> > > > that
>>> > > > >> > > > > > > > >> > > > > > >> we're
>>> > > > >> > > > > > > > >> > > > > > >> > > not
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> proposing
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
>>> implement:
>>> > > > >> removing
>>> > > > >> > the
>>> > > > >> > > > > topic
>>> > > > >> > > > > > > > name
>>> > > > >> > > > > > > > >> > from
>>> > > > >> > > > > > > > >> > > > > those
>>> > > > >> > > > > > > > >> > > > > > >> > > > protocols.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Is
>>> it
>>> > > really
>>> > > > >> > > > necessary
>>> > > > >> > > > > to
>>> > > > >> > > > > > > > have
>>> > > > >> > > > > > > > >> a
>>> > > > >> > > > > > > > >> > new
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
>>> > > /admin/delete_topics_by_id
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > path
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > in
>>> > ZooKeeper?
>>> > > > It
>>> > > > >> > seems
>>> > > > >> > > > > like
>>> > > > >> > > > > > we
>>> > > > >> > > > > > > > >> don't
>>> > > > >> > > > > > > > >> > > > really
>>> > > > >> > > > > > > > >> > > > > > >> need
>>> > > > >> > > > > > > > >> > > > > > >> > > this.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Whenever
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > there is
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > a new
>>> > > > controller,
>>> > > > >> > we'll
>>> > > > >> > > > > send
>>> > > > >> > > > > > > out
>>> > > > >> > > > > > > > >> full
>>> > > > >> > > > > > > > >> > > > > > >> > > > > LeaderAndIsrRequests
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> which
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > will
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > trigger
>>> the
>>> > > > stale
>>> > > > >> > > topics
>>> > > > >> > > > to
>>> > > > >> > > > > > be
>>> > > > >> > > > > > > > >> cleaned
>>> > > > >> > > > > > > > >> > > up.
>>> > > > >> > > > > > > > >> > > > > >  The
>>> > > > >> > > > > > > > >> > > > > > >> > > active
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > controller
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > will
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > also
>>> send
>>> > the
>>> > > > full
>>> > > > >> > > > > > > > >> LeaderAndIsrRequest
>>> > > > >> > > > > > > > >> > > to
>>> > > > >> > > > > > > > >> > > > > > >> brokers
>>> > > > >> > > > > > > > >> > > > > > >> > > that
>>> > > > >> > > > > > > > >> > > > > > >> > > > > are
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> just
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > starting
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > up.
>>> So we
>>> > > > don't
>>> > > > >> > > really
>>> > > > >> > > > > > need
>>> > > > >> > > > > > > > this
>>> > > > >> > > > > > > > >> > kind
>>> > > > >> > > > > > > > >> > > > of
>>> > > > >> > > > > > > > >> > > > > > >> > two-phase
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > commit
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > (send
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > out
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
>>> > > > >> StopReplicasRequest,
>>> > > > >> > > get
>>> > > > >> > > > > ACKs
>>> > > > >> > > > > > > > from
>>> > > > >> > > > > > > > >> all
>>> > > > >> > > > > > > > >> > > > > nodes,
>>> > > > >> > > > > > > > >> > > > > > >> > commit
>>> > > > >> > > > > > > > >> > > > > > >> > > by
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> removing
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
>>> > > > >> /admin/delete_topics
>>> > > > >> > > > node)
>>> > > > >> > > > > > any
>>> > > > >> > > > > > > > >> more.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > You
>>> > > mention
>>> > > > >> that
>>> > > > >> > > > > > > FetchRequest
>>> > > > >> > > > > > > > >> will
>>> > > > >> > > > > > > > >> > > now
>>> > > > >> > > > > > > > >> > > > > > >> include
>>> > > > >> > > > > > > > >> > > > > > >> > > UUID
>>> > > > >> > > > > > > > >> > > > > > >> > > > > to
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> avoid
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > issues
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > where
>>> > requests
>>> > > > are
>>> > > > >> > made
>>> > > > >> > > > to
>>> > > > >> > > > > > > stale
>>> > > > >> > > > > > > > >> > > > partitions.
>>> > > > >> > > > > > > > >> > > > > > >> > > However,
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> adding a
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > UUID
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
>>> > > MetadataRequest
>>> > > > is
>>> > > > >> > > listed
>>> > > > >> > > > > as
>>> > > > >> > > > > > > > future
>>> > > > >> > > > > > > > >> > > work,
>>> > > > >> > > > > > > > >> > > > > out
>>> > > > >> > > > > > > > >> > > > > > of
>>> > > > >> > > > > > > > >> > > > > > >> > > scope
>>> > > > >> > > > > > > > >> > > > > > >> > > > > for
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> this
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > KIP.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > How
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > will the
>>> > > client
>>> > > > >> learn
>>> > > > >> > > > what
>>> > > > >> > > > > > the
>>> > > > >> > > > > > > > >> topic
>>> > > > >> > > > > > > > >> > > UUID
>>> > > > >> > > > > > > > >> > > > > is,
>>> > > > >> > > > > > > > >> > > > > > if
>>> > > > >> > > > > > > > >> > > > > > >> > the
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > metadata
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > response
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > doesn't
>>> > > include
>>> > > > >> that
>>> > > > >> > > > > > > information?
>>> > > > >> > > > > > > > >> It
>>> > > > >> > > > > > > > >> > > > seems
>>> > > > >> > > > > > > > >> > > > > > like
>>> > > > >> > > > > > > > >> > > > > > >> > > adding
>>> > > > >> > > > > > > > >> > > > > > >> > > > > the
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> UUID
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
>>> > > MetadataResponse
>>> > > > >> > would
>>> > > > >> > > be
>>> > > > >> > > > > an
>>> > > > >> > > > > > > > >> > improvement
>>> > > > >> > > > > > > > >> > > > > here
>>> > > > >> > > > > > > > >> > > > > > >> that
>>> > > > >> > > > > > > > >> > > > > > >> > > > might
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > not
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> be
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > too
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > hard to
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > make.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
>>> best,
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
>>> Colin
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > On
>>> Mon,
>>> > > Sep
>>> > > > 9,
>>> > > > >> > > 2019,
>>> > > > >> > > > at
>>> > > > >> > > > > > > > 17:48,
>>> > > > >> > > > > > > > >> > > Ryanne
>>> > > > >> > > > > > > > >> > > > > > Dolan
>>> > > > >> > > > > > > > >> > > > > > >> > > wrote:
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
>>> Lucas,
>>> > > > this
>>> > > > >> > would
>>> > > > >> > > > be
>>> > > > >> > > > > > > great.
>>> > > > >> > > > > > > > >> I've
>>> > > > >> > > > > > > > >> > > run
>>> > > > >> > > > > > > > >> > > > > > into
>>> > > > >> > > > > > > > >> > > > > > >> > > issues
>>> > > > >> > > > > > > > >> > > > > > >> > > > > with
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > topics
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > being
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
>>> > > > resurrected
>>> > > > >> > > > > > accidentally,
>>> > > > >> > > > > > > > >> since
>>> > > > >> > > > > > > > >> > a
>>> > > > >> > > > > > > > >> > > > > client
>>> > > > >> > > > > > > > >> > > > > > >> > cannot
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > easily
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > distinguish
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > between
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > a
>>> > > deleted
>>> > > > >> topic
>>> > > > >> > > > and a
>>> > > > >> > > > > > new
>>> > > > >> > > > > > > > >> topic
>>> > > > >> > > > > > > > >> > > with
>>> > > > >> > > > > > > > >> > > > > the
>>> > > > >> > > > > > > > >> > > > > > >> same
>>> > > > >> > > > > > > > >> > > > > > >> > > > name.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > I'd
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > need
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > the
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > ID
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
>>> > > accessible
>>> > > > >> from
>>> > > > >> > > the
>>> > > > >> > > > > > > client
>>> > > > >> > > > > > > > to
>>> > > > >> > > > > > > > >> > > solve
>>> > > > >> > > > > > > > >> > > > > that
>>> > > > >> > > > > > > > >> > > > > > >> > issue,
>>> > > > >> > > > > > > > >> > > > > > >> > > > but
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> this
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > is a
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > good
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > first
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
>>> step.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
>>> Ryanne
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > On
>>> > Wed,
>>> > > > Sep
>>> > > > >> 4,
>>> > > > >> > > 2019
>>> > > > >> > > > > at
>>> > > > >> > > > > > > 1:41
>>> > > > >> > > > > > > > >> PM
>>> > > > >> > > > > > > > >> > > Lucas
>>> > > > >> > > > > > > > >> > > > > > >> > > Bradstreet <
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
>>> > > lucas@confluent.io>
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > wrote:
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
>>> Hi
>>> > > all,
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
>>> I
>>> > > would
>>> > > > >> like
>>> > > > >> > to
>>> > > > >> > > > > kick
>>> > > > >> > > > > > > off
>>> > > > >> > > > > > > > >> > > > discussion
>>> > > > >> > > > > > > > >> > > > > of
>>> > > > >> > > > > > > > >> > > > > > >> > > KIP-516,
>>> > > > >> > > > > > > > >> > > > > > >> > > > > an
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
>>> implementation
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > of topic
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
>>> IDs
>>> > > for
>>> > > > >> > Kafka.
>>> > > > >> > > > > Topic
>>> > > > >> > > > > > > IDs
>>> > > > >> > > > > > > > >> aim
>>> > > > >> > > > > > > > >> > to
>>> > > > >> > > > > > > > >> > > > > solve
>>> > > > >> > > > > > > > >> > > > > > >> topic
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> uniqueness
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > problems in
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Kafka,
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
>>> > where
>>> > > > >> > referring
>>> > > > >> > > > to
>>> > > > >> > > > > a
>>> > > > >> > > > > > > > topic
>>> > > > >> > > > > > > > >> by
>>> > > > >> > > > > > > > >> > > name
>>> > > > >> > > > > > > > >> > > > > > >> alone is
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> insufficient.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Such
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > cases
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
>>> > > include
>>> > > > >> when
>>> > > > >> > a
>>> > > > >> > > > > topic
>>> > > > >> > > > > > > has
>>> > > > >> > > > > > > > >> been
>>> > > > >> > > > > > > > >> > > > > deleted
>>> > > > >> > > > > > > > >> > > > > > >> and
>>> > > > >> > > > > > > > >> > > > > > >> > > > > recreated
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> with
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > the
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > same
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > name.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
>>> > Unique
>>> > > > >> > > > identifiers
>>> > > > >> > > > > > will
>>> > > > >> > > > > > > > >> help
>>> > > > >> > > > > > > > >> > > > > simplify
>>> > > > >> > > > > > > > >> > > > > > >> and
>>> > > > >> > > > > > > > >> > > > > > >> > > > improve
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Kafka's
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > topic
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > deletion
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
>>> > > process,
>>> > > > >> as
>>> > > > >> > > well
>>> > > > >> > > > as
>>> > > > >> > > > > > > > prevent
>>> > > > >> > > > > > > > >> > > cases
>>> > > > >> > > > > > > > >> > > > > > where
>>> > > > >> > > > > > > > >> > > > > > >> > > brokers
>>> > > > >> > > > > > > > >> > > > > > >> > > > > may
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > incorrectly
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > interact
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
>>> with
>>> > > > stale
>>> > > > >> > > > versions
>>> > > > >> > > > > > of
>>> > > > >> > > > > > > > >> topics.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
>>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> > > > >> > > > > > > > >> > > > > > >> > >
>>> > > > >> > > > > > > > >> > > > > > >> >
>>> > > > >> > > > > > > > >> > > > > > >>
>>> > > > >> > > > > > > > >> > > > > >
>>> > > > >> > > > > > > > >> > > > >
>>> > > > >> > > > > > > > >> > > >
>>> > > > >> > > > > > > > >> > >
>>> > > > >> > > > > > > > >> >
>>> > > > >> > > > > > > > >>
>>> > > > >> > > > > > > >
>>> > > > >> > > > > > >
>>> > > > >> > > > > >
>>> > > > >> > > > >
>>> > > > >> > > >
>>> > > > >> > >
>>> > > > >> >
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
>>> > > Looking
>>> > > > >> > forward
>>> > > > >> > > > to
>>> > > > >> > > > > > your
>>> > > > >> > > > > > > > >> > > thoughts.
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
>>> > Lucas
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
>>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
>>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > > >
>>> > > > >> > > > > > > > >> > > > > > >> > > >
>>> > > > >> > > > > > > > >> > > > > > >> > >
>>> > > > >> > > > > > > > >> > > > > > >> >
>>> > > > >> > > > > > > > >> > > > > > >>
>>> > > > >> > > > > > > > >> > > > > > >
>>> > > > >> > > > > > > > >> > > > > >
>>> > > > >> > > > > > > > >> > > > >
>>> > > > >> > > > > > > > >> > > >
>>> > > > >> > > > > > > > >> > >
>>> > > > >> > > > > > > > >> >
>>> > > > >> > > > > > > > >>
>>> > > > >> > > > > > > > >
>>> > > > >> > > > > > > >
>>> > > > >> > > > > > >
>>> > > > >> > > > > >
>>> > > > >> > > > >
>>> > > > >> > > >
>>> > > > >> > >
>>> > > > >> >
>>> > > > >>
>>> > > > >
>>> > > >
>>> > >
>>> >
>>>
>>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Justine Olshan <jo...@confluent.io>]

Hi all,
I've made a few minor changes to the UUID class. I've added a few more
methods to support UUIDs more generally, by including a constructor and a
method to generate random UUIDs. Both of these will be useful for testing.

Let me know if there are any other areas for discussion.

Thank you,
Justine

On Thu, Oct 1, 2020 at 5:29 PM Justine Olshan <jo...@confluent.io> wrote:

> Hi Jun,
> Thanks for looking at it again. Here's the new spec. (I fixed the typo in
> it too.)
> {"name": "id", "type": "string", "doc": option[UUID]}
>
> Justine
>
>
> On Thu, Oct 1, 2020 at 5:03 PM Jun Rao <ju...@confluent.io> wrote:
>
>> Hi, Justine,
>>
>> Thanks for the update. The KIP looks good to me now. Just a minor comment
>> below.
>>
>> 30. Perhaps "option[UUID]" can be put in the doc.
>>
>> Jun
>>
>> On Thu, Oct 1, 2020 at 3:28 PM Justine Olshan <jo...@confluent.io>
>> wrote:
>>
>> > Hi Jun,
>> > Thanks for the response!
>> >
>> > 30. I think I might have changed this in between. The current version
>> > says:  {"name":
>> > "id", "type": "option[UUID]"}, "doc": topic id}
>> > I have switched to the option type to cover the migration case where a
>> > TopicZNode does not yet have a topic ID.
>> > I understand that due to json, this field is written as a string, so if
>> I
>> > should move the "option[uuid]" to the doc field and the type should be
>> > "string" please let me know.
>> >
>> > 40. I've added a definition for UUID.
>> > 41,42. Fixed
>> >
>> > Thank you,
>> > Justine
>> >
>> > On Wed, Sep 30, 2020 at 1:15 PM Jun Rao <ju...@confluent.io> wrote:
>> >
>> > > Hi, Justine,
>> > >
>> > > Thanks for the summary. Just a few minor comments blow.
>> > >
>> > > 30.  {"name": "id", "type": "string", "doc": "version id"}}: The doc
>> > should
>> > > say UUID. The issue seems unfixed.
>> > >
>> > > 40. Since UUID is public facing, could you include its definition?
>> > >
>> > > 41. StopReplicaResponse still includes the topic field.
>> > >
>> > > 42. "It is unnecessary to include the name of the topic in the
>> following
>> > > Request/Response calls" It would be useful to include all modified
>> > requests
>> > > (e.g. produce) in the list below.
>> > >
>> > > Thanks,
>> > >
>> > > Jun
>> > >
>> > >
>> > > On Wed, Sep 30, 2020 at 10:17 AM Justine Olshan <jolshan@confluent.io
>> >
>> > > wrote:
>> > >
>> > > > Hello again,
>> > > >
>> > > > I've taken some time to discuss some of the remaining points
>> brought up
>> > > by
>> > > > the previous emails offline. Here are some of the conclusions.
>> > > >
>> > > > 1. Directory Structure:
>> > > > There was some talk about whether the directory structure should be
>> > > changed
>> > > > to replace all topic names with topic IDs.
>> > > > This will be a useful change, but will prevent downgrades. It will
>> be
>> > > best
>> > > > to wait until a major release, likely alongside KIP-500 changes that
>> > will
>> > > > prevent downgrades. I've updated the KIP to include this change with
>> > the
>> > > > note about migration and deprecation.
>> > > >
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-log.dirlayout
>> > > >
>> > > > 2. Partition Metadata file
>> > > > There was some disagreement about the inclusion of the partition
>> > metadata
>> > > > file.
>> > > > This file will be needed to persist the topic ID, especially while
>> we
>> > > still
>> > > > have the old directory structure. Even after the changes, this file
>> can
>> > > be
>> > > > useful for debugging and recovery.
>> > > > Creating a single mapping file from topic names to topic IDs was
>> > > > considered, but ultimately scrapped as it would not be as easy to
>> > > maintain.
>> > > >
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PartitionMetadatafile
>> > > >
>> > > >
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PersistingTopicIDs
>> > > >
>> > > > 3. Produce Protocols
>> > > > After some further discussion, this replacing the topic name with
>> topic
>> > > ID
>> > > > in these protocols has been added to the KIP.
>> > > > This will cut down on the size of the protocol. Since changes to
>> fetch
>> > > are
>> > > > included, it does make sense to update these protocols.
>> > > >
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-Produce
>> > > >
>> > > > 4. KIP-500 Compatibility
>> > > > After some discussion with those involved with KIP-500, it seems
>> best
>> > to
>> > > > use a sentinel topic ID for the metadata topic that is used before
>> the
>> > > > first controller election.
>> > > > We had an issue where this metadata topic may not be assigned an ID
>> > > before
>> > > > utilizing the Vote and Fetch protocols. It was decided to reserve a
>> > > unique
>> > > > ID for this topic to be used until the controller could give the
>> topic
>> > a
>> > > > unique ID.
>> > > > Having the topic keep the sentinel ID (not be reassigned to a unique
>> > ID)
>> > > > was considered, but it seemed like a good idea to leave the option
>> open
>> > > for
>> > > > the metadata topic to have a unique ID in cases where it would need
>> to
>> > be
>> > > > differentiated from other clusters' metadata topics. (ex. tiered
>> > storage)
>> > > >
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-CompatibilitywithKIP-500
>> > > >
>> > > > I've also split up the KIP into sub-tasks on the JIRA. Hopefully
>> this
>> > > will
>> > > > provide a better idea about what tasks we have, and eventually
>> provide
>> > a
>> > > > place to see what's done and what is left.
>> > > > If there is a task I am missing, please let me know!
>> > > > https://issues.apache.org/jira/browse/KAFKA-8872
>> > > >
>> > > > Of course, these decisions are not set in stone, and I would love to
>> > hear
>> > > > any feedback.
>> > > >
>> > > > Thanks,
>> > > > Justine
>> > > >
>> > > > On Mon, Sep 28, 2020 at 11:38 AM Justine Olshan <
>> jolshan@confluent.io>
>> > > > wrote:
>> > > >
>> > > > > Hello all,
>> > > > >
>> > > > > I just wanted to follow up on this discussion. Did we come to an
>> > > > > understanding about the directory structure?
>> > > > >
>> > > > > I think the biggest question here is what is acceptable to leave
>> out
>> > > due
>> > > > > to scope vs. what is considered to be too much tech debt.
>> > > > > This KIP is already pretty large with the number of changes, but
>> it
>> > > also
>> > > > > makes sense to do things correctly, so I'd love to hear everyone's
>> > > > thoughts.
>> > > > >
>> > > > > Thanks,
>> > > > > Justine
>> > > > >
>> > > > > On Fri, Sep 25, 2020 at 8:19 AM Lucas Bradstreet <
>> lucas@confluent.io
>> > >
>> > > > > wrote:
>> > > > >
>> > > > >> Hi Ismael,
>> > > > >>
>> > > > >> If you do not store it in a metadata file or in the directory
>> > > structure
>> > > > >> would you then
>> > > > >> require the LeaderAndIsrRequest following startup to give you
>> some
>> > > > notion
>> > > > >> of
>> > > > >> topic name in memory? We would still need this information for
>> the
>> > > older
>> > > > >> protocols, but
>> > > > >> perhaps this is what's meant by tech debt.
>> > > > >>
>> > > > >> Once we're free of the old non-topicID requests then I think you
>> > > > wouldn't
>> > > > >> need to retain the topic name.
>> > > > >> I think the ability to easily look up topic names associated with
>> > > > >> partition
>> > > > >> directories would still be missed
>> > > > >> when diagnosing issues, though maybe it wouldn't be a deal
>> breaker
>> > > with
>> > > > >> the
>> > > > >> right tooling.
>> > > > >>
>> > > > >> Thanks,
>> > > > >>
>> > > > >> Lucas
>> > > > >>
>> > > > >> On Fri, Sep 25, 2020 at 7:55 AM Ismael Juma <is...@juma.me.uk>
>> > > wrote:
>> > > > >>
>> > > > >> > Hi Lucas,
>> > > > >> >
>> > > > >> > Why would you include the name and id? I think you'd want to
>> > remove
>> > > > the
>> > > > >> > name from the directory name right? Jason's suggestion was
>> that if
>> > > you
>> > > > >> > remove the name from the directory, then why would you need
>> the id
>> > > > name
>> > > > >> > mapping file?
>> > > > >> >
>> > > > >> > Ismael
>> > > > >> >
>> > > > >> > On Thu, Sep 24, 2020 at 4:24 PM Lucas Bradstreet <
>> > > lucas@confluent.io>
>> > > > >> > wrote:
>> > > > >> >
>> > > > >> > > > 2. Part of the usage of the file is to have persistent
>> storage
>> > > of
>> > > > >> the
>> > > > >> > > topic
>> > > > >> > > ID and use it to compare with the ID supplied in the
>> > LeaderAndIsr
>> > > > >> > Request.
>> > > > >> > > There is some discussion in the KIP about changes to the
>> > directory
>> > > > >> > > structure, but I believe directory changes were considered
>> to be
>> > > out
>> > > > >> of
>> > > > >> > > scope when the KIP was written.
>> > > > >> > >
>> > > > >> > >
>> > > > >> > > Yeah, I was hoping to get a better understanding of why it
>> was
>> > > taken
>> > > > >> out
>> > > > >> > of
>> > > > >> > > scope. Perhaps Lucas Bradstreet might have more insight about
>> > the
>> > > > >> > decision.
>> > > > >> > > Basically my point is that we have to create additional
>> > > > infrastructure
>> > > > >> > here
>> > > > >> > > to support the name/id mapping, so I wanted to understand if
>> we
>> > > just
>> > > > >> > > consider this a sort of tech debt. It would be useful to
>> cover
>> > how
>> > > > we
>> > > > >> > > handle the case when this file gets corrupted. Seems like we
>> > just
>> > > > >> have to
>> > > > >> > > trust that it matches whatever the controller tells us and
>> > rewrite
>> > > > it?
>> > > > >> > >
>> > > > >> > >
>> > > > >> > > Hi Jason, Justine,
>> > > > >> > >
>> > > > >> > > My thought process is that we will likely need the metadata
>> file
>> > > > >> whether
>> > > > >> > we
>> > > > >> > > rename the directories or not.
>> > > > >> > > Linux supports filenames of up to 255 bytes and that would
>> not
>> > be
>> > > > >> enough
>> > > > >> > to
>> > > > >> > > support a directory name
>> > > > >> > >  including both the name and topic ID. Given that fact, it
>> > seemed
>> > > > >> > > reasonable to add the metadata file
>> > > > >> > > and leave the directory rename until some time in the future
>> > > > (possibly
>> > > > >> > > never).
>> > > > >> > >
>> > > > >> > > If the file does get corrupted I think you're right. We would
>> > > either
>> > > > >> have
>> > > > >> > > to trust it matches what the controller tells us
>> > > > >> > >  or error out and let an administrator resolve it by checking
>> > > across
>> > > > >> > > replicas for consistency.
>> > > > >> > >
>> > > > >> > > Lucas
>> > > > >> > >
>> > > > >> > >
>> > > > >> > > On Thu, Sep 24, 2020 at 3:41 PM Jason Gustafson <
>> > > jason@confluent.io
>> > > > >
>> > > > >> > > wrote:
>> > > > >> > >
>> > > > >> > > > Thanks Justine. Responses below:
>> > > > >> > > >
>> > > > >> > > > > 1. Yes, the directory will still be based on the topic
>> > names.
>> > > > >> > > > LeaderAndIsrRequest is one of the few requests that will
>> still
>> > > > >> contain
>> > > > >> > > the
>> > > > >> > > > topic name. So I think we have this covered. Sorry for
>> > > confusion.
>> > > > >> > > >
>> > > > >> > > > Ah, you're right. My eyes passed right over the field.
>> > > > >> > > >
>> > > > >> > > > > 2. Part of the usage of the file is to have persistent
>> > storage
>> > > > of
>> > > > >> the
>> > > > >> > > > topic
>> > > > >> > > > ID and use it to compare with the ID supplied in the
>> > > LeaderAndIsr
>> > > > >> > > Request.
>> > > > >> > > > There is some discussion in the KIP about changes to the
>> > > directory
>> > > > >> > > > structure, but I believe directory changes were considered
>> to
>> > be
>> > > > >> out of
>> > > > >> > > > scope when the KIP was written.
>> > > > >> > > >
>> > > > >> > > > Yeah, I was hoping to get a better understanding of why it
>> was
>> > > > taken
>> > > > >> > out
>> > > > >> > > of
>> > > > >> > > > scope. Perhaps Lucas Bradstreet might have more insight
>> about
>> > > the
>> > > > >> > > decision.
>> > > > >> > > > Basically my point is that we have to create additional
>> > > > >> infrastructure
>> > > > >> > > here
>> > > > >> > > > to support the name/id mapping, so I wanted to understand
>> if
>> > we
>> > > > just
>> > > > >> > > > consider this a sort of tech debt. It would be useful to
>> cover
>> > > how
>> > > > >> we
>> > > > >> > > > handle the case when this file gets corrupted. Seems like
>> we
>> > > just
>> > > > >> have
>> > > > >> > to
>> > > > >> > > > trust that it matches whatever the controller tells us and
>> > > rewrite
>> > > > >> it?
>> > > > >> > > >
>> > > > >> > > > > 3. I think this is a good point, but I again I wonder
>> about
>> > > the
>> > > > >> scope
>> > > > >> > > of
>> > > > >> > > > the KIP. Most of the changes mentioned in the KIP are for
>> > > > supporting
>> > > > >> > > topic
>> > > > >> > > > deletion and I believe that is why the produce request was
>> > > listed
>> > > > >> under
>> > > > >> > > > future work.
>> > > > >> > > >
>> > > > >> > > > That's fair. I brought it up since `Fetch` is already
>> > included.
>> > > If
>> > > > >> > we've
>> > > > >> > > > got `Metadata` and `Fetch`, seems we may as well do
>> `Produce`
>> > > and
>> > > > >> save
>> > > > >> > an
>> > > > >> > > > extra kip. No strong objection though if you want to leave
>> it
>> > > out.
>> > > > >> > > >
>> > > > >> > > >
>> > > > >> > > > -Jason
>> > > > >> > > >
>> > > > >> > > >
>> > > > >> > > > On Thu, Sep 24, 2020 at 3:26 PM Justine Olshan <
>> > > > >> jolshan@confluent.io>
>> > > > >> > > > wrote:
>> > > > >> > > >
>> > > > >> > > > > Hi Jason,
>> > > > >> > > > >
>> > > > >> > > > > Thanks for your comments.
>> > > > >> > > > >
>> > > > >> > > > > 1. Yes, the directory will still be based on the topic
>> > names.
>> > > > >> > > > > LeaderAndIsrRequest is one of the few requests that will
>> > still
>> > > > >> > contain
>> > > > >> > > > the
>> > > > >> > > > > topic name. So I think we have this covered. Sorry for
>> > > > confusion.
>> > > > >> > > > >
>> > > > >> > > > > 2. Part of the usage of the file is to have persistent
>> > storage
>> > > > of
>> > > > >> the
>> > > > >> > > > topic
>> > > > >> > > > > ID and use it to compare with the ID supplied in the
>> > > > LeaderAndIsr
>> > > > >> > > > Request.
>> > > > >> > > > > There is some discussion in the KIP about changes to the
>> > > > directory
>> > > > >> > > > > structure, but I believe directory changes were
>> considered
>> > to
>> > > be
>> > > > >> out
>> > > > >> > of
>> > > > >> > > > > scope when the KIP was written.
>> > > > >> > > > >
>> > > > >> > > > > 3. I think this is a good point, but I again I wonder
>> about
>> > > the
>> > > > >> scope
>> > > > >> > > of
>> > > > >> > > > > the KIP. Most of the changes mentioned in the KIP are for
>> > > > >> supporting
>> > > > >> > > > topic
>> > > > >> > > > > deletion and I believe that is why the produce request
>> was
>> > > > listed
>> > > > >> > under
>> > > > >> > > > > future work.
>> > > > >> > > > >
>> > > > >> > > > > 4. This sounds like it might be a good solution, but I
>> will
>> > > need
>> > > > >> to
>> > > > >> > > > discuss
>> > > > >> > > > > more with KIP-500 folks to get the details right.
>> > > > >> > > > >
>> > > > >> > > > > Thanks,
>> > > > >> > > > > Justine
>> > > > >> > > > >
>> > > > >> > > > > On Thu, Sep 24, 2020 at 12:30 PM Jason Gustafson <
>> > > > >> jason@confluent.io
>> > > > >> > >
>> > > > >> > > > > wrote:
>> > > > >> > > > >
>> > > > >> > > > > > Hi Justine,
>> > > > >> > > > > >
>> > > > >> > > > > > Thanks for picking up this work. I have a few
>> > > > >> questions/comments:
>> > > > >> > > > > >
>> > > > >> > > > > > 1. It sounds like the directory structure is still
>> going
>> > to
>> > > be
>> > > > >> > based
>> > > > >> > > on
>> > > > >> > > > > > topic names. Do I have that right? One complication is
>> > that
>> > > > the
>> > > > >> > > > > > LeaderAndIsr request does not include the topic name
>> any
>> > > > longer.
>> > > > >> > This
>> > > > >> > > > > means
>> > > > >> > > > > > that a replica must wait for the UpdateMetadata
>> request to
>> > > > >> arrive
>> > > > >> > > with
>> > > > >> > > > > the
>> > > > >> > > > > > topic name mapping before it can create the directory.
>> I
>> > > > wonder
>> > > > >> if
>> > > > >> > it
>> > > > >> > > > > would
>> > > > >> > > > > > be simpler to avoid assumptions on the ordering of
>> > > > >> UpdateMetadata
>> > > > >> > and
>> > > > >> > > > let
>> > > > >> > > > > > LeaderAndIsr include the topic name as well. Feels
>> like we
>> > > are
>> > > > >> not
>> > > > >> > > > saving
>> > > > >> > > > > > that much by excluding it.
>> > > > >> > > > > >
>> > > > >> > > > > > 2. On a related note, it seems that the reason we have
>> the
>> > > > >> > partition
>> > > > >> > > > > > metadata file is because we are not changing the
>> directory
>> > > > >> > structure.
>> > > > >> > > > We
>> > > > >> > > > > > need it so that we remember which directories map to
>> which
>> > > > topic
>> > > > >> > id.
>> > > > >> > > I
>> > > > >> > > > > > think it would be useful to add some detail about why
>> > > changing
>> > > > >> the
>> > > > >> > > > > > directory layout was rejected. Basically trying to
>> > > understand
>> > > > >> how
>> > > > >> > > much
>> > > > >> > > > > > effort we are saving by skipping this step if we have
>> to
>> > > > >> implement
>> > > > >> > > this
>> > > > >> > > > > > additional bookkeeping. One concern I have for example
>> is
>> > > that
>> > > > >> the
>> > > > >> > > > > > partition metadata file gets corrupt and then all bets
>> are
>> > > off
>> > > > >> as
>> > > > >> > far
>> > > > >> > > > as
>> > > > >> > > > > > topic consistency.
>> > > > >> > > > > >
>> > > > >> > > > > > 3. Is it worth adding support now for topic ids in the
>> > > > `Produce`
>> > > > >> > > > request
>> > > > >> > > > > > now? Overhead is mentioned as one of the motivations
>> and
>> > the
>> > > > >> best
>> > > > >> > > APIs
>> > > > >> > > > to
>> > > > >> > > > > > get savings from are `Produce` and `Fetch`.
>> > > > >> > > > > >
>> > > > >> > > > > > 4. When it comes to bootstrapping the metadata topic
>> with
>> > > > >> respect
>> > > > >> > to
>> > > > >> > > > > > KIP-500, one suggestion would be to reserve a sentinel
>> > topic
>> > > > ID
>> > > > >> > which
>> > > > >> > > > can
>> > > > >> > > > > > be used initially by new brokers. When the first
>> > controller
>> > > is
>> > > > >> > > elected,
>> > > > >> > > > > it
>> > > > >> > > > > > can assign a topicId to the metadata topic. This is a
>> bit
>> > > > >> similar
>> > > > >> > to
>> > > > >> > > > how
>> > > > >> > > > > we
>> > > > >> > > > > > were planning to generate the clusterId.
>> > > > >> > > > > >
>> > > > >> > > > > > Thanks,
>> > > > >> > > > > > Jason
>> > > > >> > > > > >
>> > > > >> > > > > >
>> > > > >> > > > > > On Thu, Sep 24, 2020 at 11:10 AM Jun Rao <
>> > jun@confluent.io>
>> > > > >> wrote:
>> > > > >> > > > > >
>> > > > >> > > > > > > Hi, Justine,
>> > > > >> > > > > > >
>> > > > >> > > > > > > Thanks for the updated KIP. A few more comments
>> below.
>> > > > >> > > > > > >
>> > > > >> > > > > > > 30.  {"name": "id", "type": "string", "doc": "version
>> > > id"}}:
>> > > > >> The
>> > > > >> > > doc
>> > > > >> > > > > > should
>> > > > >> > > > > > > say UUID.
>> > > > >> > > > > > >
>> > > > >> > > > > > > 31. LeaderAndIsrResponse v5 and StopReplicaResponse
>> v4 :
>> > > It
>> > > > >> seems
>> > > > >> > > > there
>> > > > >> > > > > > is
>> > > > >> > > > > > > no need to add topic_id at partitions level.
>> > > > >> > > > > > >
>> > > > >> > > > > > > 32. Regarding partition metadata file. Perhaps the
>> key
>> > can
>> > > > be
>> > > > >> a
>> > > > >> > > > single
>> > > > >> > > > > > > word, sth like the following.
>> > > > >> > > > > > > version: 0
>> > > > >> > > > > > > topic_id: 46bdb63f-9e8d-4a38-bf7b-ee4eb2a794e4
>> > > > >> > > > > > >
>> > > > >> > > > > > > 33. Another tricky thing that I realized is how to
>> > support
>> > > > the
>> > > > >> > > > metadata
>> > > > >> > > > > > > topic introduced in KIP-595. It's a bit tricky to
>> > assign a
>> > > > >> UUID
>> > > > >> > to
>> > > > >> > > > the
>> > > > >> > > > > > > metadata topic since we have a chicken-and-egg
>> problem.
>> > > The
>> > > > >> > > > controller
>> > > > >> > > > > > > needs to persist the UUID in the metadata topic in
>> order
>> > > to
>> > > > >> > assign
>> > > > >> > > > one
>> > > > >> > > > > > > successfully, but the metadata topic is needed to
>> elect
>> > a
>> > > > >> > > controller
>> > > > >> > > > > > > (KIP-631). So, this probably needs a bit more
>> thought.
>> > > > >> > > > > > >
>> > > > >> > > > > > > Jun
>> > > > >> > > > > > >
>> > > > >> > > > > > > On Thu, Sep 24, 2020 at 3:04 AM Ismael Juma <
>> > > > >> ismael@juma.me.uk>
>> > > > >> > > > wrote:
>> > > > >> > > > > > >
>> > > > >> > > > > > > > Also, can we provide more details on how the
>> Partition
>> > > > >> Metadata
>> > > > >> > > > file
>> > > > >> > > > > > will
>> > > > >> > > > > > > > be used?
>> > > > >> > > > > > > >
>> > > > >> > > > > > > > Ismael
>> > > > >> > > > > > > >
>> > > > >> > > > > > > > On Thu, Sep 24, 2020 at 3:01 AM Ismael Juma <
>> > > > >> ismael@juma.me.uk
>> > > > >> > >
>> > > > >> > > > > wrote:
>> > > > >> > > > > > > >
>> > > > >> > > > > > > > > Hi Justine,
>> > > > >> > > > > > > > >
>> > > > >> > > > > > > > > I think we need to update the "Rejected
>> > Alternatives"
>> > > > >> section
>> > > > >> > > to
>> > > > >> > > > > take
>> > > > >> > > > > > > > into
>> > > > >> > > > > > > > > account that the proposal now removes the topic
>> name
>> > > > from
>> > > > >> the
>> > > > >> > > > fetch
>> > > > >> > > > > > > > > request. Also, if we are removing it from the
>> Fetch
>> > > > >> request,
>> > > > >> > > does
>> > > > >> > > > > it
>> > > > >> > > > > > > make
>> > > > >> > > > > > > > > sense not to remove it from similar requests like
>> > > > >> > > > > ListOffsetRequest?
>> > > > >> > > > > > > > >
>> > > > >> > > > > > > > > Ismael
>> > > > >> > > > > > > > >
>> > > > >> > > > > > > > > On Thu, Sep 24, 2020 at 2:46 AM David Jacot <
>> > > > >> > > djacot@confluent.io
>> > > > >> > > > >
>> > > > >> > > > > > > wrote:
>> > > > >> > > > > > > > >
>> > > > >> > > > > > > > >> Hi Justine,
>> > > > >> > > > > > > > >>
>> > > > >> > > > > > > > >> Thanks for the KIP. I finally had time to read
>> it
>> > :).
>> > > > It
>> > > > >> is
>> > > > >> > a
>> > > > >> > > > > great
>> > > > >> > > > > > > > >> improvement.
>> > > > >> > > > > > > > >>
>> > > > >> > > > > > > > >> I have a few comments/questions:
>> > > > >> > > > > > > > >>
>> > > > >> > > > > > > > >> 1. It seems that the schema of the
>> > StopReplicaRequest
>> > > > is
>> > > > >> > > > slightly
>> > > > >> > > > > > > > >> outdated.
>> > > > >> > > > > > > > >> We
>> > > > >> > > > > > > > >> did some changes as part of KIP-570. V3 is
>> already
>> > > > >> organized
>> > > > >> > > by
>> > > > >> > > > > > > topics.
>> > > > >> > > > > > > > >>
>> > > > >> > > > > > > > >> 2. I just want to make sure that I understand
>> the
>> > > > >> > > reconciliation
>> > > > >> > > > > > > > >> logic correctly. When
>> > > > >> > > > > > > > >> an "INCREMENTAL" LeaderAndIsr Request is
>> received,
>> > > the
>> > > > >> > broker
>> > > > >> > > > will
>> > > > >> > > > > > > also
>> > > > >> > > > > > > > >> reconcile
>> > > > >> > > > > > > > >> when the local uuid does not match the uuid in
>> the
>> > > > >> request,
>> > > > >> > > > right?
>> > > > >> > > > > > In
>> > > > >> > > > > > > > this
>> > > > >> > > > > > > > >> case, the
>> > > > >> > > > > > > > >> local log is staged for deletion.
>> > > > >> > > > > > > > >>
>> > > > >> > > > > > > > >> 3. In the documentation of the `
>> > > > >> delete.stale.topic.delay.ms
>> > > > >> > `
>> > > > >> > > > > > config,
>> > > > >> > > > > > > it
>> > > > >> > > > > > > > >> says "When a
>> > > > >> > > > > > > > >> FULL LeaderAndIsrRequest is received..." but I
>> > > suppose
>> > > > it
>> > > > >> > > > applies
>> > > > >> > > > > to
>> > > > >> > > > > > > > both
>> > > > >> > > > > > > > >> types.
>> > > > >> > > > > > > > >>
>> > > > >> > > > > > > > >> Best,
>> > > > >> > > > > > > > >> David
>> > > > >> > > > > > > > >>
>> > > > >> > > > > > > > >> On Thu, Sep 24, 2020 at 1:40 AM Justine Olshan <
>> > > > >> > > > > > jolshan@confluent.io>
>> > > > >> > > > > > > > >> wrote:
>> > > > >> > > > > > > > >>
>> > > > >> > > > > > > > >> > Hi Jun,
>> > > > >> > > > > > > > >> >
>> > > > >> > > > > > > > >> > Thanks for the comments. I apologize for some
>> of
>> > > the
>> > > > >> typos
>> > > > >> > > and
>> > > > >> > > > > > > > >> confusion.
>> > > > >> > > > > > > > >> > I’ve updated the KIP to fix some of the issues
>> > you
>> > > > >> > > mentioned.
>> > > > >> > > > > > > > >> >
>> > > > >> > > > > > > > >> > 20.2 I’ve changed the type to String.
>> > > > >> > > > > > > > >> > 20.1/3 I’ve updated the TopicZNode to fix
>> > > formatting
>> > > > >> and
>> > > > >> > > > reflect
>> > > > >> > > > > > the
>> > > > >> > > > > > > > >> latest
>> > > > >> > > > > > > > >> > version before this change.
>> > > > >> > > > > > > > >> >
>> > > > >> > > > > > > > >> > 21. You are correct and I’ve removed this
>> line.
>> > > I’ve
>> > > > >> also
>> > > > >> > > > added
>> > > > >> > > > > a
>> > > > >> > > > > > > line
>> > > > >> > > > > > > > >> > mentioning an IBP bump is necessary for
>> migration
>> > > > >> > > > > > > > >> >
>> > > > >> > > > > > > > >> > 22. I think the wording was unclear but your
>> > > summary
>> > > > is
>> > > > >> > what
>> > > > >> > > > was
>> > > > >> > > > > > > > >> intended
>> > > > >> > > > > > > > >> > by this line. I’ve updated to make this point
>> > more
>> > > > >> clear.
>> > > > >> > > > > “Remove
>> > > > >> > > > > > > > >> deleted
>> > > > >> > > > > > > > >> > topics from replicas by sending
>> > StopReplicaRequest
>> > > V3
>> > > > >> > before
>> > > > >> > > > the
>> > > > >> > > > > > IBP
>> > > > >> > > > > > > > >> bump
>> > > > >> > > > > > > > >> > using the old logic, and using V4 and the new
>> > logic
>> > > > >> with
>> > > > >> > > topic
>> > > > >> > > > > IDs
>> > > > >> > > > > > > > after
>> > > > >> > > > > > > > >> > the IBP bump.”
>> > > > >> > > > > > > > >> >
>> > > > >> > > > > > > > >> > 23. I’ve removed the unspecified type from the
>> > KIP
>> > > > and
>> > > > >> > > mention
>> > > > >> > > > > > that
>> > > > >> > > > > > > > IBP
>> > > > >> > > > > > > > >> > will be used to handle this request. “IBP
>> will be
>> > > > used
>> > > > >> to
>> > > > >> > > > > > determine
>> > > > >> > > > > > > > >> whether
>> > > > >> > > > > > > > >> > this new form of the request will be used. For
>> > > older
>> > > > >> > > requests,
>> > > > >> > > > > we
>> > > > >> > > > > > > will
>> > > > >> > > > > > > > >> > ignore this field and default to previous
>> > > behavior.”
>> > > > >> > > > > > > > >> >
>> > > > >> > > > > > > > >> > 24. I’ve fixed this typo.
>> > > > >> > > > > > > > >> >
>> > > > >> > > > > > > > >> > 25. I've added a topics at a higher level for
>> > > > >> > > > > LeaderAndIsrResponse
>> > > > >> > > > > > > v5,
>> > > > >> > > > > > > > >> > StopReplicaResponse v4. I've also changed
>> > > > >> > StopReplicaRequest
>> > > > >> > > > v4
>> > > > >> > > > > to
>> > > > >> > > > > > > > have
>> > > > >> > > > > > > > >> > topics at a higher level.
>> > > > >> > > > > > > > >> >
>> > > > >> > > > > > > > >> > 26. I’ve updated forgotten_topics_data--added
>> the
>> > > > >> topic ID
>> > > > >> > > and
>> > > > >> > > > > > > removed
>> > > > >> > > > > > > > >> the
>> > > > >> > > > > > > > >> > topic name
>> > > > >> > > > > > > > >> >
>> > > > >> > > > > > > > >> > 27. I’ve decided on plain text, and I’ve
>> added an
>> > > > >> example.
>> > > > >> > > > > > > > >> >
>> > > > >> > > > > > > > >> > 28. I’ve added this idea to future work.
>> > > > >> > > > > > > > >> >
>> > > > >> > > > > > > > >> > Thanks again for taking a look,
>> > > > >> > > > > > > > >> >
>> > > > >> > > > > > > > >> > Justine
>> > > > >> > > > > > > > >> >
>> > > > >> > > > > > > > >> > On Wed, Sep 23, 2020 at 10:28 AM Jun Rao <
>> > > > >> > jun@confluent.io>
>> > > > >> > > > > > wrote:
>> > > > >> > > > > > > > >> >
>> > > > >> > > > > > > > >> > > Hi, Justine,
>> > > > >> > > > > > > > >> > >
>> > > > >> > > > > > > > >> > > Thanks for the response. Made another pass.
>> A
>> > few
>> > > > >> more
>> > > > >> > > > > comments
>> > > > >> > > > > > > > below.
>> > > > >> > > > > > > > >> > >
>> > > > >> > > > > > > > >> > > 20. znode schema:
>> > > > >> > > > > > > > >> > > 20.1 It seems that {"name": "version",
>> "type":
>> > > > "int",
>> > > > >> > > "id":
>> > > > >> > > > > > > "UUID",
>> > > > >> > > > > > > > >> > "doc":
>> > > > >> > > > > > > > >> > > "version id"} should be {"name": "version",
>> > > "type":
>> > > > >> > > "int"},
>> > > > >> > > > > > > {"name":
>> > > > >> > > > > > > > >> > "id",
>> > > > >> > > > > > > > >> > > "type": "UUID", "doc": "version id"}.
>> > > > >> > > > > > > > >> > > 20.2 The znode format is JSON which doesn't
>> > have
>> > > > UUID
>> > > > >> > > type.
>> > > > >> > > > So
>> > > > >> > > > > > the
>> > > > >> > > > > > > > >> type
>> > > > >> > > > > > > > >> > > probably should be string?
>> > > > >> > > > > > > > >> > > 20.3 Also, the existing format used seems
>> > > outdated.
>> > > > >> It
>> > > > >> > > > should
>> > > > >> > > > > > have
>> > > > >> > > > > > > > the
>> > > > >> > > > > > > > >> > > following format.
>> > > > >> > > > > > > > >> > >     Json.encodeAsBytes(Map(
>> > > > >> > > > > > > > >> > >       "version" -> 2,
>> > > > >> > > > > > > > >> > >       "partitions" ->
>> > > replicaAssignmentJson.asJava,
>> > > > >> > > > > > > > >> > >       "adding_replicas" ->
>> > > > >> > > > > addingReplicasAssignmentJson.asJava,
>> > > > >> > > > > > > > >> > >       "removing_replicas" ->
>> > > > >> > > > > > removingReplicasAssignmentJson.asJava
>> > > > >> > > > > > > > >> > >     ).asJava)
>> > > > >> > > > > > > > >> > >   }
>> > > > >> > > > > > > > >> > >
>> > > > >> > > > > > > > >> > > 21. Migration: The KIP says "The migration
>> > > process
>> > > > >> can
>> > > > >> > > take
>> > > > >> > > > > > place
>> > > > >> > > > > > > > >> without
>> > > > >> > > > > > > > >> > > an inter-broker protocol bump, as the format
>> > > stored
>> > > > >> in
>> > > > >> > > > > > > > >> > > /brokers/topics/[topic] will be compatible
>> with
>> > > > older
>> > > > >> > > broker
>> > > > >> > > > > > > > >> versions."
>> > > > >> > > > > > > > >> > > However, since we are bumping up the
>> version of
>> > > > >> > > inter-broker
>> > > > >> > > > > > > > >> requests, it
>> > > > >> > > > > > > > >> > > seems that we need to use IBP for migration.
>> > > > >> > > > > > > > >> > >
>> > > > >> > > > > > > > >> > > 22. The KIP says "Remove deleted topics from
>> > > > >> replicas by
>> > > > >> > > > > sending
>> > > > >> > > > > > > > >> > > StopReplicaRequest V3 for any topics which
>> do
>> > not
>> > > > >> > contain
>> > > > >> > > a
>> > > > >> > > > > > topic
>> > > > >> > > > > > > > ID,
>> > > > >> > > > > > > > >> and
>> > > > >> > > > > > > > >> > > V4 for any topics which do contain a topic
>> > ID.".
>> > > > >> > However,
>> > > > >> > > if
>> > > > >> > > > > we
>> > > > >> > > > > > > use
>> > > > >> > > > > > > > >> IBP,
>> > > > >> > > > > > > > >> > it
>> > > > >> > > > > > > > >> > > seems that the controller will either send
>> > > > >> > > > StopReplicaRequest
>> > > > >> > > > > V3
>> > > > >> > > > > > > > >> > > or StopReplicaRequest V4, but never mixed V3
>> > and
>> > > V4
>> > > > >> for
>> > > > >> > > > > > different
>> > > > >> > > > > > > > >> topics.
>> > > > >> > > > > > > > >> > > Basically, before the IBP bump, V3 will be
>> > used.
>> > > > >> After
>> > > > >> > the
>> > > > >> > > > IBP
>> > > > >> > > > > > > bump,
>> > > > >> > > > > > > > >> > > topicId will be created and V4 will be used.
>> > > > >> > > > > > > > >> > >
>> > > > >> > > > > > > > >> > > 23. Given that we depend on IBP, do we still
>> > need
>> > > > "0
>> > > > >> > > > > > UNSPECIFIED"
>> > > > >> > > > > > > > >> > > in LeaderAndIsr?
>> > > > >> > > > > > > > >> > >
>> > > > >> > > > > > > > >> > > 24. LeaderAndIsrResponse v5 : It still has
>> the
>> > > > topic
>> > > > >> > > field.
>> > > > >> > > > > > > > >> > >
>> > > > >> > > > > > > > >> > > 25. LeaderAndIsrResponse v5,
>> > StopReplicaResponse
>> > > > v4:
>> > > > >> > Could
>> > > > >> > > > we
>> > > > >> > > > > > use
>> > > > >> > > > > > > > this
>> > > > >> > > > > > > > >> > > opportunity to organize the response in 2
>> > levels,
>> > > > >> first
>> > > > >> > by
>> > > > >> > > > > > topic,
>> > > > >> > > > > > > > >> then by
>> > > > >> > > > > > > > >> > > partition, as most other requests/responses?
>> > > > >> > > > > > > > >> > >
>> > > > >> > > > > > > > >> > > 26. FetchRequest v13 : Should
>> > > forgotten_topics_data
>> > > > >> use
>> > > > >> > > > > topicId
>> > > > >> > > > > > > too?
>> > > > >> > > > > > > > >> > >
>> > > > >> > > > > > > > >> > > 27. "This file can either be plain text
>> > > (key/value
>> > > > >> > pairs)
>> > > > >> > > or
>> > > > >> > > > > > > JSON."
>> > > > >> > > > > > > > >> Have
>> > > > >> > > > > > > > >> > we
>> > > > >> > > > > > > > >> > > decided which one to use? Also, it would be
>> > > helpful
>> > > > >> to
>> > > > >> > > > provide
>> > > > >> > > > > > an
>> > > > >> > > > > > > > >> > example.
>> > > > >> > > > > > > > >> > >
>> > > > >> > > > > > > > >> > > 28. Future improvement: Another future
>> > > improvement
>> > > > >> > > > opportunity
>> > > > >> > > > > > is
>> > > > >> > > > > > > to
>> > > > >> > > > > > > > >> use
>> > > > >> > > > > > > > >> > > topicId in
>> GroupMetadataManager.offsetCommitKey
>> > > in
>> > > > >> the
>> > > > >> > > > > > > offset_commit
>> > > > >> > > > > > > > >> > topic.
>> > > > >> > > > > > > > >> > > This may save some space.
>> > > > >> > > > > > > > >> > >
>> > > > >> > > > > > > > >> > > Thanks,
>> > > > >> > > > > > > > >> > >
>> > > > >> > > > > > > > >> > > Jun
>> > > > >> > > > > > > > >> > >
>> > > > >> > > > > > > > >> > > On Wed, Sep 23, 2020 at 8:50 AM Justine
>> Olshan
>> > <
>> > > > >> > > > > > > > jolshan@confluent.io>
>> > > > >> > > > > > > > >> > > wrote:
>> > > > >> > > > > > > > >> > >
>> > > > >> > > > > > > > >> > > > Hi Tom,
>> > > > >> > > > > > > > >> > > >
>> > > > >> > > > > > > > >> > > > Thanks for the comment. I think this is a
>> > > really
>> > > > >> good
>> > > > >> > > idea
>> > > > >> > > > > and
>> > > > >> > > > > > > it
>> > > > >> > > > > > > > >> has
>> > > > >> > > > > > > > >> > > been
>> > > > >> > > > > > > > >> > > > added to the KIP under the newly added
>> > tooling
>> > > > >> > section.
>> > > > >> > > > > > > > >> > > >
>> > > > >> > > > > > > > >> > > > Thanks again,
>> > > > >> > > > > > > > >> > > > Justine
>> > > > >> > > > > > > > >> > > >
>> > > > >> > > > > > > > >> > > > On Wed, Sep 23, 2020 at 3:17 AM Tom
>> Bentley <
>> > > > >> > > > > > > tbentley@redhat.com>
>> > > > >> > > > > > > > >> > wrote:
>> > > > >> > > > > > > > >> > > >
>> > > > >> > > > > > > > >> > > > > Hi Justine,
>> > > > >> > > > > > > > >> > > > >
>> > > > >> > > > > > > > >> > > > > I know you started the vote thread, but
>> on
>> > > > >> > re-reading
>> > > > >> > > > the
>> > > > >> > > > > > KIP
>> > > > >> > > > > > > I
>> > > > >> > > > > > > > >> > noticed
>> > > > >> > > > > > > > >> > > > > that although the topic id is included
>> in
>> > the
>> > > > >> > > > > > MetadataResponse
>> > > > >> > > > > > > > >> it's
>> > > > >> > > > > > > > >> > not
>> > > > >> > > > > > > > >> > > > > surfaced in the output from
>> > `kafka-topics.sh
>> > > > >> > > > --describe`.
>> > > > >> > > > > > > Maybe
>> > > > >> > > > > > > > >> that
>> > > > >> > > > > > > > >> > > was
>> > > > >> > > > > > > > >> > > > > intentional because ids are
>> intentionally
>> > not
>> > > > >> really
>> > > > >> > > > > > something
>> > > > >> > > > > > > > the
>> > > > >> > > > > > > > >> > user
>> > > > >> > > > > > > > >> > > > > should care deeply about, but it would
>> also
>> > > > make
>> > > > >> > life
>> > > > >> > > > > harder
>> > > > >> > > > > > > for
>> > > > >> > > > > > > > >> > anyone
>> > > > >> > > > > > > > >> > > > > debugging Kafka and this would likely
>> get
>> > > worse
>> > > > >> the
>> > > > >> > > more
>> > > > >> > > > > > topic
>> > > > >> > > > > > > > ids
>> > > > >> > > > > > > > >> > got
>> > > > >> > > > > > > > >> > > > > rolled out across the protocols, clients
>> > etc.
>> > > > It
>> > > > >> > seems
>> > > > >> > > > > > likely
>> > > > >> > > > > > > > that
>> > > > >> > > > > > > > >> > > > > `kafka-topics.sh` will eventually need
>> the
>> > > > >> ability
>> > > > >> > to
>> > > > >> > > > show
>> > > > >> > > > > > the
>> > > > >> > > > > > > > id
>> > > > >> > > > > > > > >> of
>> > > > >> > > > > > > > >> > a
>> > > > >> > > > > > > > >> > > > > topic and perhaps find a topic name
>> given
>> > an
>> > > > id.
>> > > > >> Is
>> > > > >> > > > there
>> > > > >> > > > > > any
>> > > > >> > > > > > > > >> reason
>> > > > >> > > > > > > > >> > > not
>> > > > >> > > > > > > > >> > > > to
>> > > > >> > > > > > > > >> > > > > implement that in this KIP?
>> > > > >> > > > > > > > >> > > > >
>> > > > >> > > > > > > > >> > > > > Many thanks,
>> > > > >> > > > > > > > >> > > > >
>> > > > >> > > > > > > > >> > > > > Tom
>> > > > >> > > > > > > > >> > > > >
>> > > > >> > > > > > > > >> > > > > On Mon, Sep 21, 2020 at 9:54 PM Justine
>> > > Olshan
>> > > > <
>> > > > >> > > > > > > > >> jolshan@confluent.io
>> > > > >> > > > > > > > >> > >
>> > > > >> > > > > > > > >> > > > > wrote:
>> > > > >> > > > > > > > >> > > > >
>> > > > >> > > > > > > > >> > > > > > Hi all,
>> > > > >> > > > > > > > >> > > > > >
>> > > > >> > > > > > > > >> > > > > > After thinking about it, I've decided
>> to
>> > > > remove
>> > > > >> > the
>> > > > >> > > > > topic
>> > > > >> > > > > > > name
>> > > > >> > > > > > > > >> from
>> > > > >> > > > > > > > >> > > the
>> > > > >> > > > > > > > >> > > > > > Fetch Request and Response after all.
>> > Since
>> > > > >> there
>> > > > >> > > are
>> > > > >> > > > so
>> > > > >> > > > > > > many
>> > > > >> > > > > > > > of
>> > > > >> > > > > > > > >> > > these
>> > > > >> > > > > > > > >> > > > > > requests per second, it is worth
>> removing
>> > > the
>> > > > >> > extra
>> > > > >> > > > > > > > information.
>> > > > >> > > > > > > > >> > I've
>> > > > >> > > > > > > > >> > > > > > updated the KIP to reflect this
>> change.
>> > > > >> > > > > > > > >> > > > > >
>> > > > >> > > > > > > > >> > > > > > Please let me know if there is
>> anything
>> > > else
>> > > > we
>> > > > >> > > should
>> > > > >> > > > > > > discuss
>> > > > >> > > > > > > > >> > before
>> > > > >> > > > > > > > >> > > > > > voting.
>> > > > >> > > > > > > > >> > > > > >
>> > > > >> > > > > > > > >> > > > > > Thank you,
>> > > > >> > > > > > > > >> > > > > > Justine
>> > > > >> > > > > > > > >> > > > > >
>> > > > >> > > > > > > > >> > > > > > On Fri, Sep 18, 2020 at 9:46 AM
>> Justine
>> > > > Olshan
>> > > > >> <
>> > > > >> > > > > > > > >> > jolshan@confluent.io
>> > > > >> > > > > > > > >> > > >
>> > > > >> > > > > > > > >> > > > > > wrote:
>> > > > >> > > > > > > > >> > > > > >
>> > > > >> > > > > > > > >> > > > > > > Hi Jun,
>> > > > >> > > > > > > > >> > > > > > >
>> > > > >> > > > > > > > >> > > > > > > I see what you are saying. For now
>> we
>> > can
>> > > > >> remove
>> > > > >> > > the
>> > > > >> > > > > > extra
>> > > > >> > > > > > > > >> > > > information.
>> > > > >> > > > > > > > >> > > > > > > I'll leave the option to add more
>> > fields
>> > > to
>> > > > >> the
>> > > > >> > > file
>> > > > >> > > > > in
>> > > > >> > > > > > > the
>> > > > >> > > > > > > > >> > future.
>> > > > >> > > > > > > > >> > > > The
>> > > > >> > > > > > > > >> > > > > > KIP
>> > > > >> > > > > > > > >> > > > > > > has been updated to reflect this
>> > change.
>> > > > >> > > > > > > > >> > > > > > >
>> > > > >> > > > > > > > >> > > > > > > Thanks,
>> > > > >> > > > > > > > >> > > > > > > Justine
>> > > > >> > > > > > > > >> > > > > > >
>> > > > >> > > > > > > > >> > > > > > > On Fri, Sep 18, 2020 at 8:46 AM Jun
>> > Rao <
>> > > > >> > > > > > jun@confluent.io
>> > > > >> > > > > > > >
>> > > > >> > > > > > > > >> > wrote:
>> > > > >> > > > > > > > >> > > > > > >
>> > > > >> > > > > > > > >> > > > > > >> Hi, Justine,
>> > > > >> > > > > > > > >> > > > > > >>
>> > > > >> > > > > > > > >> > > > > > >> Thanks for the reply.
>> > > > >> > > > > > > > >> > > > > > >>
>> > > > >> > > > > > > > >> > > > > > >> 13. If the log directory is the
>> source
>> > > of
>> > > > >> > truth,
>> > > > >> > > it
>> > > > >> > > > > > means
>> > > > >> > > > > > > > >> that
>> > > > >> > > > > > > > >> > the
>> > > > >> > > > > > > > >> > > > > > >> redundant info in the metadata file
>> > will
>> > > > be
>> > > > >> > > > ignored.
>> > > > >> > > > > > Then
>> > > > >> > > > > > > > the
>> > > > >> > > > > > > > >> > > > question
>> > > > >> > > > > > > > >> > > > > > is
>> > > > >> > > > > > > > >> > > > > > >> why do we need to put the redundant
>> > info
>> > > > in
>> > > > >> the
>> > > > >> > > > > > metadata
>> > > > >> > > > > > > > file
>> > > > >> > > > > > > > >> > now?
>> > > > >> > > > > > > > >> > > > > > >>
>> > > > >> > > > > > > > >> > > > > > >> Thanks,
>> > > > >> > > > > > > > >> > > > > > >>
>> > > > >> > > > > > > > >> > > > > > >> Jun
>> > > > >> > > > > > > > >> > > > > > >>
>> > > > >> > > > > > > > >> > > > > > >> On Thu, Sep 17, 2020 at 5:07 PM
>> > Justine
>> > > > >> Olshan
>> > > > >> > <
>> > > > >> > > > > > > > >> > > > jolshan@confluent.io>
>> > > > >> > > > > > > > >> > > > > > >> wrote:
>> > > > >> > > > > > > > >> > > > > > >>
>> > > > >> > > > > > > > >> > > > > > >> > Hi Jun,
>> > > > >> > > > > > > > >> > > > > > >> > Thanks for the quick response!
>> > > > >> > > > > > > > >> > > > > > >> >
>> > > > >> > > > > > > > >> > > > > > >> > 12. I've decided to bump up the
>> > > versions
>> > > > >> on
>> > > > >> > the
>> > > > >> > > > > > > requests
>> > > > >> > > > > > > > >> and
>> > > > >> > > > > > > > >> > > > updated
>> > > > >> > > > > > > > >> > > > > > the
>> > > > >> > > > > > > > >> > > > > > >> > KIP. I think it's good we
>> thoroughly
>> > > > >> > discussed
>> > > > >> > > > the
>> > > > >> > > > > > > > options
>> > > > >> > > > > > > > >> > here,
>> > > > >> > > > > > > > >> > > > so
>> > > > >> > > > > > > > >> > > > > we
>> > > > >> > > > > > > > >> > > > > > >> know
>> > > > >> > > > > > > > >> > > > > > >> > we made a good choice. :)
>> > > > >> > > > > > > > >> > > > > > >> >
>> > > > >> > > > > > > > >> > > > > > >> > 13. This is an interesting
>> > situation.
>> > > I
>> > > > >> think
>> > > > >> > > if
>> > > > >> > > > > this
>> > > > >> > > > > > > > does
>> > > > >> > > > > > > > >> > occur
>> > > > >> > > > > > > > >> > > > we
>> > > > >> > > > > > > > >> > > > > > >> should
>> > > > >> > > > > > > > >> > > > > > >> > give a warning. I agree that it's
>> > hard
>> > > > to
>> > > > >> > know
>> > > > >> > > > the
>> > > > >> > > > > > > source
>> > > > >> > > > > > > > >> of
>> > > > >> > > > > > > > >> > > truth
>> > > > >> > > > > > > > >> > > > > for
>> > > > >> > > > > > > > >> > > > > > >> sure
>> > > > >> > > > > > > > >> > > > > > >> > since the directory or the file
>> > could
>> > > be
>> > > > >> > > manually
>> > > > >> > > > > > > > >> modified. I
>> > > > >> > > > > > > > >> > > > guess
>> > > > >> > > > > > > > >> > > > > > the
>> > > > >> > > > > > > > >> > > > > > >> > directory could be used as the
>> > source
>> > > of
>> > > > >> > truth.
>> > > > >> > > > To
>> > > > >> > > > > be
>> > > > >> > > > > > > > >> honest,
>> > > > >> > > > > > > > >> > > I'm
>> > > > >> > > > > > > > >> > > > > not
>> > > > >> > > > > > > > >> > > > > > >> > really sure what happens in kafka
>> > when
>> > > > the
>> > > > >> > log
>> > > > >> > > > > > > directory
>> > > > >> > > > > > > > is
>> > > > >> > > > > > > > >> > > > renamed
>> > > > >> > > > > > > > >> > > > > > >> > manually in such a way. I'm also
>> > > > >> wondering if
>> > > > >> > > the
>> > > > >> > > > > > > > >> situation is
>> > > > >> > > > > > > > >> > > > > > >> recoverable
>> > > > >> > > > > > > > >> > > > > > >> > in this scenario.
>> > > > >> > > > > > > > >> > > > > > >> >
>> > > > >> > > > > > > > >> > > > > > >> > Thanks,
>> > > > >> > > > > > > > >> > > > > > >> > Justine
>> > > > >> > > > > > > > >> > > > > > >> >
>> > > > >> > > > > > > > >> > > > > > >> > On Thu, Sep 17, 2020 at 4:28 PM
>> Jun
>> > > Rao
>> > > > <
>> > > > >> > > > > > > > jun@confluent.io>
>> > > > >> > > > > > > > >> > > wrote:
>> > > > >> > > > > > > > >> > > > > > >> >
>> > > > >> > > > > > > > >> > > > > > >> > > Hi, Justine,
>> > > > >> > > > > > > > >> > > > > > >> > >
>> > > > >> > > > > > > > >> > > > > > >> > > Thanks for the reply.
>> > > > >> > > > > > > > >> > > > > > >> > >
>> > > > >> > > > > > > > >> > > > > > >> > > 12. I don't have a strong
>> > preference
>> > > > >> > either.
>> > > > >> > > > > > However,
>> > > > >> > > > > > > > if
>> > > > >> > > > > > > > >> we
>> > > > >> > > > > > > > >> > > need
>> > > > >> > > > > > > > >> > > > > IBP
>> > > > >> > > > > > > > >> > > > > > >> > > anyway, maybe it's easier to
>> just
>> > > bump
>> > > > >> up
>> > > > >> > the
>> > > > >> > > > > > version
>> > > > >> > > > > > > > for
>> > > > >> > > > > > > > >> > all
>> > > > >> > > > > > > > >> > > > > inter
>> > > > >> > > > > > > > >> > > > > > >> > broker
>> > > > >> > > > > > > > >> > > > > > >> > > requests and add the topic id
>> > field
>> > > > as a
>> > > > >> > > > regular
>> > > > >> > > > > > > > field. A
>> > > > >> > > > > > > > >> > > > regular
>> > > > >> > > > > > > > >> > > > > > >> field
>> > > > >> > > > > > > > >> > > > > > >> > is
>> > > > >> > > > > > > > >> > > > > > >> > > a bit more concise in wire
>> > transfer
>> > > > >> than a
>> > > > >> > > > > flexible
>> > > > >> > > > > > > > >> field.
>> > > > >> > > > > > > > >> > > > > > >> > >
>> > > > >> > > > > > > > >> > > > > > >> > > 13. The confusion that I was
>> > > referring
>> > > > >> to
>> > > > >> > is
>> > > > >> > > > > > between
>> > > > >> > > > > > > > the
>> > > > >> > > > > > > > >> > topic
>> > > > >> > > > > > > > >> > > > > name
>> > > > >> > > > > > > > >> > > > > > >> and
>> > > > >> > > > > > > > >> > > > > > >> > > partition number between the
>> log
>> > dir
>> > > > and
>> > > > >> > the
>> > > > >> > > > > > metadata
>> > > > >> > > > > > > > >> file.
>> > > > >> > > > > > > > >> > > For
>> > > > >> > > > > > > > >> > > > > > >> example,
>> > > > >> > > > > > > > >> > > > > > >> > if
>> > > > >> > > > > > > > >> > > > > > >> > > the log dir is topicA-1 and the
>> > > > metadata
>> > > > >> > file
>> > > > >> > > > in
>> > > > >> > > > > it
>> > > > >> > > > > > > has
>> > > > >> > > > > > > > >> > topicB
>> > > > >> > > > > > > > >> > > > and
>> > > > >> > > > > > > > >> > > > > > >> > > partition 0 (say due to a bug
>> or
>> > > > manual
>> > > > >> > > > > > > modification),
>> > > > >> > > > > > > > >> which
>> > > > >> > > > > > > > >> > > one
>> > > > >> > > > > > > > >> > > > > do
>> > > > >> > > > > > > > >> > > > > > we
>> > > > >> > > > > > > > >> > > > > > >> > use
>> > > > >> > > > > > > > >> > > > > > >> > > as the source of truth?
>> > > > >> > > > > > > > >> > > > > > >> > >
>> > > > >> > > > > > > > >> > > > > > >> > > Jun
>> > > > >> > > > > > > > >> > > > > > >> > >
>> > > > >> > > > > > > > >> > > > > > >> > > On Thu, Sep 17, 2020 at 3:43 PM
>> > > > Justine
>> > > > >> > > Olshan
>> > > > >> > > > <
>> > > > >> > > > > > > > >> > > > > > jolshan@confluent.io>
>> > > > >> > > > > > > > >> > > > > > >> > > wrote:
>> > > > >> > > > > > > > >> > > > > > >> > >
>> > > > >> > > > > > > > >> > > > > > >> > > > Hi Jun,
>> > > > >> > > > > > > > >> > > > > > >> > > > Thanks for the comments.
>> > > > >> > > > > > > > >> > > > > > >> > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > 12. I bumped the
>> > > LeaderAndIsrRequest
>> > > > >> > > because
>> > > > >> > > > I
>> > > > >> > > > > > > > removed
>> > > > >> > > > > > > > >> the
>> > > > >> > > > > > > > >> > > > topic
>> > > > >> > > > > > > > >> > > > > > >> name
>> > > > >> > > > > > > > >> > > > > > >> > > field
>> > > > >> > > > > > > > >> > > > > > >> > > > in the response. It may be
>> > > possible
>> > > > to
>> > > > >> > > avoid
>> > > > >> > > > > > > bumping
>> > > > >> > > > > > > > >> the
>> > > > >> > > > > > > > >> > > > version
>> > > > >> > > > > > > > >> > > > > > >> > without
>> > > > >> > > > > > > > >> > > > > > >> > > > that change, but I may be
>> > missing
>> > > > >> > > something.
>> > > > >> > > > > > > > >> > > > > > >> > > > I believe StopReplica is
>> > actually
>> > > on
>> > > > >> > > version
>> > > > >> > > > 3
>> > > > >> > > > > > now,
>> > > > >> > > > > > > > but
>> > > > >> > > > > > > > >> > > > because
>> > > > >> > > > > > > > >> > > > > > >> > version 2
>> > > > >> > > > > > > > >> > > > > > >> > > > is flexible, I kept that
>> listed
>> > as
>> > > > >> > version
>> > > > >> > > 2
>> > > > >> > > > on
>> > > > >> > > > > > the
>> > > > >> > > > > > > > KIP
>> > > > >> > > > > > > > >> > > page.
>> > > > >> > > > > > > > >> > > > > > >> However,
>> > > > >> > > > > > > > >> > > > > > >> > > you
>> > > > >> > > > > > > > >> > > > > > >> > > > may be right in that we may
>> need
>> > > to
>> > > > >> bump
>> > > > >> > > the
>> > > > >> > > > > > > version
>> > > > >> > > > > > > > on
>> > > > >> > > > > > > > >> > > > > > StopReplica
>> > > > >> > > > > > > > >> > > > > > >> to
>> > > > >> > > > > > > > >> > > > > > >> > > deal
>> > > > >> > > > > > > > >> > > > > > >> > > > with deletion differently as
>> > > > mentioned
>> > > > >> > > > above. I
>> > > > >> > > > > > > don't
>> > > > >> > > > > > > > >> know
>> > > > >> > > > > > > > >> > > if
>> > > > >> > > > > > > > >> > > > I
>> > > > >> > > > > > > > >> > > > > > >> have a
>> > > > >> > > > > > > > >> > > > > > >> > > big
>> > > > >> > > > > > > > >> > > > > > >> > > > preference over used tagged
>> > fields
>> > > > or
>> > > > >> > not.
>> > > > >> > > > > > > > >> > > > > > >> > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > 13. I was thinking that in
>> the
>> > > case
>> > > > >> where
>> > > > >> > > the
>> > > > >> > > > > > file
>> > > > >> > > > > > > > and
>> > > > >> > > > > > > > >> the
>> > > > >> > > > > > > > >> > > > > request
>> > > > >> > > > > > > > >> > > > > > >> > topic
>> > > > >> > > > > > > > >> > > > > > >> > > > ids don't match, it means
>> that
>> > the
>> > > > >> > broker's
>> > > > >> > > > > > > topic/the
>> > > > >> > > > > > > > >> one
>> > > > >> > > > > > > > >> > in
>> > > > >> > > > > > > > >> > > > the
>> > > > >> > > > > > > > >> > > > > > >> file
>> > > > >> > > > > > > > >> > > > > > >> > has
>> > > > >> > > > > > > > >> > > > > > >> > > > been deleted. In that case,
>> we
>> > > would
>> > > > >> need
>> > > > >> > > to
>> > > > >> > > > > > delete
>> > > > >> > > > > > > > the
>> > > > >> > > > > > > > >> > old
>> > > > >> > > > > > > > >> > > > > topic
>> > > > >> > > > > > > > >> > > > > > >> and
>> > > > >> > > > > > > > >> > > > > > >> > > start
>> > > > >> > > > > > > > >> > > > > > >> > > > receiving the new version. If
>> > the
>> > > > >> topic
>> > > > >> > > name
>> > > > >> > > > > were
>> > > > >> > > > > > > to
>> > > > >> > > > > > > > >> > change,
>> > > > >> > > > > > > > >> > > > but
>> > > > >> > > > > > > > >> > > > > > the
>> > > > >> > > > > > > > >> > > > > > >> > ids
>> > > > >> > > > > > > > >> > > > > > >> > > > still matched, the file would
>> > also
>> > > > >> need
>> > > > >> > to
>> > > > >> > > > > > update.
>> > > > >> > > > > > > > Am I
>> > > > >> > > > > > > > >> > > > missing
>> > > > >> > > > > > > > >> > > > > a
>> > > > >> > > > > > > > >> > > > > > >> case
>> > > > >> > > > > > > > >> > > > > > >> > > > where the file would be
>> correct
>> > > and
>> > > > >> not
>> > > > >> > the
>> > > > >> > > > > > > request?
>> > > > >> > > > > > > > >> > > > > > >> > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > Thanks,
>> > > > >> > > > > > > > >> > > > > > >> > > > Justine
>> > > > >> > > > > > > > >> > > > > > >> > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > On Thu, Sep 17, 2020 at 3:18
>> PM
>> > > Jun
>> > > > >> Rao <
>> > > > >> > > > > > > > >> jun@confluent.io
>> > > > >> > > > > > > > >> > >
>> > > > >> > > > > > > > >> > > > > wrote:
>> > > > >> > > > > > > > >> > > > > > >> > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > Hi, Justine,
>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > Thanks for the reply. A
>> couple
>> > > of
>> > > > >> more
>> > > > >> > > > > comments
>> > > > >> > > > > > > > >> below.
>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > 12. ListOffset and
>> > > OffsetForLeader
>> > > > >> > > > currently
>> > > > >> > > > > > > don't
>> > > > >> > > > > > > > >> > support
>> > > > >> > > > > > > > >> > > > > > >> flexible
>> > > > >> > > > > > > > >> > > > > > >> > > > fields.
>> > > > >> > > > > > > > >> > > > > > >> > > > > So, we have to bump up the
>> > > version
>> > > > >> > number
>> > > > >> > > > and
>> > > > >> > > > > > use
>> > > > >> > > > > > > > >> IBP at
>> > > > >> > > > > > > > >> > > > least
>> > > > >> > > > > > > > >> > > > > > for
>> > > > >> > > > > > > > >> > > > > > >> > > these
>> > > > >> > > > > > > > >> > > > > > >> > > > > two requests. Note that it
>> > seems
>> > > > >> 2.7.0
>> > > > >> > > will
>> > > > >> > > > > > > require
>> > > > >> > > > > > > > >> IBP
>> > > > >> > > > > > > > >> > > > anyway
>> > > > >> > > > > > > > >> > > > > > >> > because
>> > > > >> > > > > > > > >> > > > > > >> > > of
>> > > > >> > > > > > > > >> > > > > > >> > > > > changes in KAFKA-10435.
>> Also,
>> > it
>> > > > >> seems
>> > > > >> > > that
>> > > > >> > > > > the
>> > > > >> > > > > > > > >> version
>> > > > >> > > > > > > > >> > > for
>> > > > >> > > > > > > > >> > > > > > >> > > > > LeaderAndIsrRequest and
>> > > > StopReplica
>> > > > >> are
>> > > > >> > > > > bumped
>> > > > >> > > > > > > even
>> > > > >> > > > > > > > >> > though
>> > > > >> > > > > > > > >> > > > we
>> > > > >> > > > > > > > >> > > > > > only
>> > > > >> > > > > > > > >> > > > > > >> > > added
>> > > > >> > > > > > > > >> > > > > > >> > > > a
>> > > > >> > > > > > > > >> > > > > > >> > > > > tagged field. But since
>> IBP is
>> > > > >> needed
>> > > > >> > > > anyway,
>> > > > >> > > > > > we
>> > > > >> > > > > > > > may
>> > > > >> > > > > > > > >> > want
>> > > > >> > > > > > > > >> > > to
>> > > > >> > > > > > > > >> > > > > > >> revisit
>> > > > >> > > > > > > > >> > > > > > >> > > the
>> > > > >> > > > > > > > >> > > > > > >> > > > > overall tagged field
>> choice.
>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > 13. The only downside is
>> the
>> > > > >> potential
>> > > > >> > > > > > confusion
>> > > > >> > > > > > > on
>> > > > >> > > > > > > > >> > which
>> > > > >> > > > > > > > >> > > > one
>> > > > >> > > > > > > > >> > > > > is
>> > > > >> > > > > > > > >> > > > > > >> the
>> > > > >> > > > > > > > >> > > > > > >> > > > source
>> > > > >> > > > > > > > >> > > > > > >> > > > > of truth if they don't
>> match.
>> > > > >> Another
>> > > > >> > > > option
>> > > > >> > > > > is
>> > > > >> > > > > > > to
>> > > > >> > > > > > > > >> > include
>> > > > >> > > > > > > > >> > > > > those
>> > > > >> > > > > > > > >> > > > > > >> > fields
>> > > > >> > > > > > > > >> > > > > > >> > > > in
>> > > > >> > > > > > > > >> > > > > > >> > > > > the metadata file when we
>> > > actually
>> > > > >> > change
>> > > > >> > > > the
>> > > > >> > > > > > > > >> directory
>> > > > >> > > > > > > > >> > > > > > structure.
>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > Thanks,
>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > Jun
>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > On Thu, Sep 17, 2020 at
>> 2:01
>> > PM
>> > > > >> Justine
>> > > > >> > > > > Olshan
>> > > > >> > > > > > <
>> > > > >> > > > > > > > >> > > > > > >> jolshan@confluent.io
>> > > > >> > > > > > > > >> > > > > > >> > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > wrote:
>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > Hello all,
>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > I've thought some more
>> about
>> > > > >> removing
>> > > > >> > > the
>> > > > >> > > > > > topic
>> > > > >> > > > > > > > >> name
>> > > > >> > > > > > > > >> > > field
>> > > > >> > > > > > > > >> > > > > > from
>> > > > >> > > > > > > > >> > > > > > >> > some
>> > > > >> > > > > > > > >> > > > > > >> > > of
>> > > > >> > > > > > > > >> > > > > > >> > > > > the
>> > > > >> > > > > > > > >> > > > > > >> > > > > > requests. On closer
>> > inspection
>> > > > of
>> > > > >> the
>> > > > >> > > > > > > > >> > > requests/responses,
>> > > > >> > > > > > > > >> > > > it
>> > > > >> > > > > > > > >> > > > > > >> seems
>> > > > >> > > > > > > > >> > > > > > >> > > that
>> > > > >> > > > > > > > >> > > > > > >> > > > > the
>> > > > >> > > > > > > > >> > > > > > >> > > > > > internal changes would be
>> > much
>> > > > >> larger
>> > > > >> > > > than
>> > > > >> > > > > I
>> > > > >> > > > > > > > >> expected.
>> > > > >> > > > > > > > >> > > > Some
>> > > > >> > > > > > > > >> > > > > > >> > protocols
>> > > > >> > > > > > > > >> > > > > > >> > > > > > involve clients, so they
>> > would
>> > > > >> > require
>> > > > >> > > > > > changes
>> > > > >> > > > > > > > too.
>> > > > >> > > > > > > > >> > I'm
>> > > > >> > > > > > > > >> > > > > > thinking
>> > > > >> > > > > > > > >> > > > > > >> > that
>> > > > >> > > > > > > > >> > > > > > >> > > > for
>> > > > >> > > > > > > > >> > > > > > >> > > > > > now, removing the topic
>> name
>> > > > from
>> > > > >> > these
>> > > > >> > > > > > > requests
>> > > > >> > > > > > > > >> and
>> > > > >> > > > > > > > >> > > > > responses
>> > > > >> > > > > > > > >> > > > > > >> are
>> > > > >> > > > > > > > >> > > > > > >> > > out
>> > > > >> > > > > > > > >> > > > > > >> > > > of
>> > > > >> > > > > > > > >> > > > > > >> > > > > > scope.
>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > I have decided to just
>> keep
>> > > the
>> > > > >> > change
>> > > > >> > > > > > > > >> > > > LeaderAndIsrResponse
>> > > > >> > > > > > > > >> > > > > to
>> > > > >> > > > > > > > >> > > > > > >> > remove
>> > > > >> > > > > > > > >> > > > > > >> > > > the
>> > > > >> > > > > > > > >> > > > > > >> > > > > > topic name, and have
>> updated
>> > > the
>> > > > >> KIP
>> > > > >> > to
>> > > > >> > > > > > reflect
>> > > > >> > > > > > > > >> this
>> > > > >> > > > > > > > >> > > > > change. I
>> > > > >> > > > > > > > >> > > > > > >> have
>> > > > >> > > > > > > > >> > > > > > >> > > > also
>> > > > >> > > > > > > > >> > > > > > >> > > > > > mentioned the other
>> requests
>> > > and
>> > > > >> > > > responses
>> > > > >> > > > > in
>> > > > >> > > > > > > > >> future
>> > > > >> > > > > > > > >> > > work.
>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > I'm hoping to start the
>> > voting
>> > > > >> > process
>> > > > >> > > > > soon,
>> > > > >> > > > > > so
>> > > > >> > > > > > > > >> let me
>> > > > >> > > > > > > > >> > > > know
>> > > > >> > > > > > > > >> > > > > if
>> > > > >> > > > > > > > >> > > > > > >> > there
>> > > > >> > > > > > > > >> > > > > > >> > > is
>> > > > >> > > > > > > > >> > > > > > >> > > > > > anything else we should
>> > > discuss.
>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > Thank you,
>> > > > >> > > > > > > > >> > > > > > >> > > > > > Justine
>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > On Tue, Sep 15, 2020 at
>> 3:57
>> > > PM
>> > > > >> > Justine
>> > > > >> > > > > > Olshan
>> > > > >> > > > > > > <
>> > > > >> > > > > > > > >> > > > > > >> > jolshan@confluent.io
>> > > > >> > > > > > > > >> > > > > > >> > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > wrote:
>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > > Hello again,
>> > > > >> > > > > > > > >> > > > > > >> > > > > > > To follow up on some of
>> > the
>> > > > >> other
>> > > > >> > > > > comments:
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 10/11) We can remove
>> the
>> > > topic
>> > > > >> name
>> > > > >> > > > from
>> > > > >> > > > > > > these
>> > > > >> > > > > > > > >> > > > > > >> > requests/responses,
>> > > > >> > > > > > > > >> > > > > > >> > > > and
>> > > > >> > > > > > > > >> > > > > > >> > > > > > > that means we will just
>> > have
>> > > > to
>> > > > >> > make
>> > > > >> > > a
>> > > > >> > > > > few
>> > > > >> > > > > > > > >> internal
>> > > > >> > > > > > > > >> > > > > changes
>> > > > >> > > > > > > > >> > > > > > to
>> > > > >> > > > > > > > >> > > > > > >> > make
>> > > > >> > > > > > > > >> > > > > > >> > > > > > > partitions accessible
>> by
>> > > topic
>> > > > >> id
>> > > > >> > and
>> > > > >> > > > > > > > partition.
>> > > > >> > > > > > > > >> I
>> > > > >> > > > > > > > >> > can
>> > > > >> > > > > > > > >> > > > > > update
>> > > > >> > > > > > > > >> > > > > > >> the
>> > > > >> > > > > > > > >> > > > > > >> > > KIP
>> > > > >> > > > > > > > >> > > > > > >> > > > > to
>> > > > >> > > > > > > > >> > > > > > >> > > > > > > remove them unless
>> anyone
>> > > > thinks
>> > > > >> > they
>> > > > >> > > > > > should
>> > > > >> > > > > > > > >> stay.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 12) Addressed in the
>> > > previous
>> > > > >> > email.
>> > > > >> > > > I've
>> > > > >> > > > > > > > updated
>> > > > >> > > > > > > > >> > the
>> > > > >> > > > > > > > >> > > > KIP
>> > > > >> > > > > > > > >> > > > > to
>> > > > >> > > > > > > > >> > > > > > >> > > include
>> > > > >> > > > > > > > >> > > > > > >> > > > > > > tagged fields for the
>> > > requests
>> > > > >> and
>> > > > >> > > > > > responses.
>> > > > >> > > > > > > > >> (More
>> > > > >> > > > > > > > >> > on
>> > > > >> > > > > > > > >> > > > > that
>> > > > >> > > > > > > > >> > > > > > >> > below)
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 13) I think part of the
>> > idea
>> > > > for
>> > > > >> > > > > including
>> > > > >> > > > > > > this
>> > > > >> > > > > > > > >> > > > > information
>> > > > >> > > > > > > > >> > > > > > >> is to
>> > > > >> > > > > > > > >> > > > > > >> > > > > prepare
>> > > > >> > > > > > > > >> > > > > > >> > > > > > > for future changes.
>> > Perhaps
>> > > > the
>> > > > >> > > > directory
>> > > > >> > > > > > > > >> structure
>> > > > >> > > > > > > > >> > > > might
>> > > > >> > > > > > > > >> > > > > > >> change
>> > > > >> > > > > > > > >> > > > > > >> > > from
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> topicName_partitionNumber
>> > to
>> > > > >> > > something
>> > > > >> > > > > like
>> > > > >> > > > > > > > >> > > > > > >> > > topicID_partitionNumber.
>> > > > >> > > > > > > > >> > > > > > >> > > > > Then
>> > > > >> > > > > > > > >> > > > > > >> > > > > > > it would be useful to
>> have
>> > > the
>> > > > >> > topic
>> > > > >> > > > name
>> > > > >> > > > > > in
>> > > > >> > > > > > > > the
>> > > > >> > > > > > > > >> > file
>> > > > >> > > > > > > > >> > > > > since
>> > > > >> > > > > > > > >> > > > > > it
>> > > > >> > > > > > > > >> > > > > > >> > > would
>> > > > >> > > > > > > > >> > > > > > >> > > > > not
>> > > > >> > > > > > > > >> > > > > > >> > > > > > be
>> > > > >> > > > > > > > >> > > > > > >> > > > > > > in the directory
>> > structure.
>> > > > >> > > Supporting
>> > > > >> > > > > > topic
>> > > > >> > > > > > > > >> renames
>> > > > >> > > > > > > > >> > > > might
>> > > > >> > > > > > > > >> > > > > > be
>> > > > >> > > > > > > > >> > > > > > >> > > easier
>> > > > >> > > > > > > > >> > > > > > >> > > > if
>> > > > >> > > > > > > > >> > > > > > >> > > > > > the
>> > > > >> > > > > > > > >> > > > > > >> > > > > > > other fields are
>> included.
>> > > > Would
>> > > > >> > > there
>> > > > >> > > > be
>> > > > >> > > > > > any
>> > > > >> > > > > > > > >> > > downsides
>> > > > >> > > > > > > > >> > > > to
>> > > > >> > > > > > > > >> > > > > > >> > > including
>> > > > >> > > > > > > > >> > > > > > >> > > > > this
>> > > > >> > > > > > > > >> > > > > > >> > > > > > > information?
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 14)  Yes, we would
>> need to
>> > > > copy
>> > > > >> the
>> > > > >> > > > > > partition
>> > > > >> > > > > > > > >> > metadata
>> > > > >> > > > > > > > >> > > > > file
>> > > > >> > > > > > > > >> > > > > > in
>> > > > >> > > > > > > > >> > > > > > >> > this
>> > > > >> > > > > > > > >> > > > > > >> > > > > > > process. I've updated
>> the
>> > > KIP
>> > > > to
>> > > > >> > > > include
>> > > > >> > > > > > > this.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > > 15) I believe Lucas
>> meant
>> > v1
>> > > > >> and v2
>> > > > >> > > > here.
>> > > > >> > > > > > He
>> > > > >> > > > > > > > was
>> > > > >> > > > > > > > >> > > > referring
>> > > > >> > > > > > > > >> > > > > > to
>> > > > >> > > > > > > > >> > > > > > >> how
>> > > > >> > > > > > > > >> > > > > > >> > > the
>> > > > >> > > > > > > > >> > > > > > >> > > > > > > requests would fall
>> under
>> > > > >> different
>> > > > >> > > IBP
>> > > > >> > > > > and
>> > > > >> > > > > > > > meant
>> > > > >> > > > > > > > >> > that
>> > > > >> > > > > > > > >> > > > > older
>> > > > >> > > > > > > > >> > > > > > >> > > brokers
>> > > > >> > > > > > > > >> > > > > > >> > > > > > would
>> > > > >> > > > > > > > >> > > > > > >> > > > > > > have to use the older
>> > > version
>> > > > of
>> > > > >> > the
>> > > > >> > > > > > request
>> > > > >> > > > > > > > and
>> > > > >> > > > > > > > >> the
>> > > > >> > > > > > > > >> > > > > > existing
>> > > > >> > > > > > > > >> > > > > > >> > topic
>> > > > >> > > > > > > > >> > > > > > >> > > > > > > deletion process. At
>> > first,
>> > > it
>> > > > >> > seemed
>> > > > >> > > > > like
>> > > > >> > > > > > > > tagged
>> > > > >> > > > > > > > >> > > fields
>> > > > >> > > > > > > > >> > > > > > would
>> > > > >> > > > > > > > >> > > > > > >> > > > resolve
>> > > > >> > > > > > > > >> > > > > > >> > > > > > > the IBP issue.
>> However, we
>> > > may
>> > > > >> need
>> > > > >> > > IBP
>> > > > >> > > > > for
>> > > > >> > > > > > > > this
>> > > > >> > > > > > > > >> > > request
>> > > > >> > > > > > > > >> > > > > > after
>> > > > >> > > > > > > > >> > > > > > >> > all
>> > > > >> > > > > > > > >> > > > > > >> > > > > since
>> > > > >> > > > > > > > >> > > > > > >> > > > > > > the controller handles
>> the
>> > > > topic
>> > > > >> > > > deletion
>> > > > >> > > > > > > > >> > differently
>> > > > >> > > > > > > > >> > > > > > >> depending
>> > > > >> > > > > > > > >> > > > > > >> > on
>> > > > >> > > > > > > > >> > > > > > >> > > > the
>> > > > >> > > > > > > > >> > > > > > >> > > > > > IBP
>> > > > >> > > > > > > > >> > > > > > >> > > > > > > version. In an older
>> > > version,
>> > > > we
>> > > > >> > > can't
>> > > > >> > > > > just
>> > > > >> > > > > > > > send
>> > > > >> > > > > > > > >> a
>> > > > >> > > > > > > > >> > > > > > StopReplica
>> > > > >> > > > > > > > >> > > > > > >> > > delete
>> > > > >> > > > > > > > >> > > > > > >> > > > > the
>> > > > >> > > > > > > > >> > > > > > >> > > > > > > topic immediately like
>> > we'd
>> > > > >> want to
>> > > > >> > > for
>> > > > >> > > > > > this
>> > > > >> > > > > > > > KIP.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > > This makes me wonder
>> if we
>> > > > want
>> > > > >> > > tagged
>> > > > >> > > > > > fields
>> > > > >> > > > > > > > on
>> > > > >> > > > > > > > >> all
>> > > > >> > > > > > > > >> > > the
>> > > > >> > > > > > > > >> > > > > > >> requests
>> > > > >> > > > > > > > >> > > > > > >> > > > after
>> > > > >> > > > > > > > >> > > > > > >> > > > > > > all. Let me know your
>> > > > thoughts!
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > > Justine
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > > On Tue, Sep 15, 2020 at
>> > 1:03
>> > > > PM
>> > > > >> > > Justine
>> > > > >> > > > > > > Olshan
>> > > > >> > > > > > > > <
>> > > > >> > > > > > > > >> > > > > > >> > > jolshan@confluent.io
>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > > wrote:
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Hi all,
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Jun brought up a good
>> > point
>> > > > in
>> > > > >> his
>> > > > >> > > > last
>> > > > >> > > > > > > email
>> > > > >> > > > > > > > >> about
>> > > > >> > > > > > > > >> > > > > tagged
>> > > > >> > > > > > > > >> > > > > > >> > fields,
>> > > > >> > > > > > > > >> > > > > > >> > > > and
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> I've updated the KIP
>> to
>> > > > reflect
>> > > > >> > that
>> > > > >> > > > the
>> > > > >> > > > > > > > >> changes to
>> > > > >> > > > > > > > >> > > > > > requests
>> > > > >> > > > > > > > >> > > > > > >> and
>> > > > >> > > > > > > > >> > > > > > >> > > > > > responses
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> will be in the form of
>> > > tagged
>> > > > >> > fields
>> > > > >> > > > to
>> > > > >> > > > > > > avoid
>> > > > >> > > > > > > > >> > > changing
>> > > > >> > > > > > > > >> > > > > IBP.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Jun: I plan on
>> sending a
>> > > > >> followup
>> > > > >> > > > email
>> > > > >> > > > > to
>> > > > >> > > > > > > > >> address
>> > > > >> > > > > > > > >> > > some
>> > > > >> > > > > > > > >> > > > > of
>> > > > >> > > > > > > > >> > > > > > >> the
>> > > > >> > > > > > > > >> > > > > > >> > > other
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> points.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Thanks,
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Justine
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >> On Mon, Sep 14, 2020
>> at
>> > > 4:25
>> > > > PM
>> > > > >> > Jun
>> > > > >> > > > Rao
>> > > > >> > > > > <
>> > > > >> > > > > > > > >> > > > > jun@confluent.io>
>> > > > >> > > > > > > > >> > > > > > >> > wrote:
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Hi, Justine,
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Thanks for the
>> updated
>> > > KIP.
>> > > > A
>> > > > >> few
>> > > > >> > > > > > comments
>> > > > >> > > > > > > > >> below.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 10. LeaderAndIsr
>> > Response:
>> > > > Do
>> > > > >> we
>> > > > >> > > need
>> > > > >> > > > > the
>> > > > >> > > > > > > > topic
>> > > > >> > > > > > > > >> > > name?
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 11. For the changed
>> > > > >> > > request/response,
>> > > > >> > > > > > other
>> > > > >> > > > > > > > >> than
>> > > > >> > > > > > > > >> > > > > > >> LeaderAndIsr,
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> UpdateMetadata,
>> > Metadata,
>> > > do
>> > > > >> we
>> > > > >> > > need
>> > > > >> > > > to
>> > > > >> > > > > > > > include
>> > > > >> > > > > > > > >> > the
>> > > > >> > > > > > > > >> > > > > topic
>> > > > >> > > > > > > > >> > > > > > >> name?
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 12. It seems that
>> > upgrades
>> > > > >> don't
>> > > > >> > > > > require
>> > > > >> > > > > > > IBP.
>> > > > >> > > > > > > > >> Does
>> > > > >> > > > > > > > >> > > > that
>> > > > >> > > > > > > > >> > > > > > mean
>> > > > >> > > > > > > > >> > > > > > >> > the
>> > > > >> > > > > > > > >> > > > > > >> > > > new
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> fields
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> in all the
>> > > request/response
>> > > > >> are
>> > > > >> > > added
>> > > > >> > > > > as
>> > > > >> > > > > > > > tagged
>> > > > >> > > > > > > > >> > > fields
>> > > > >> > > > > > > > >> > > > > > >> without
>> > > > >> > > > > > > > >> > > > > > >> > > > > bumping
>> > > > >> > > > > > > > >> > > > > > >> > > > > > up
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> the request version?
>> It
>> > > > would
>> > > > >> be
>> > > > >> > > > useful
>> > > > >> > > > > > to
>> > > > >> > > > > > > > make
>> > > > >> > > > > > > > >> > that
>> > > > >> > > > > > > > >> > > > > > clear.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 13. Partition
>> Metadata
>> > > file:
>> > > > >> Do
>> > > > >> > we
>> > > > >> > > > need
>> > > > >> > > > > > to
>> > > > >> > > > > > > > >> include
>> > > > >> > > > > > > > >> > > the
>> > > > >> > > > > > > > >> > > > > > topic
>> > > > >> > > > > > > > >> > > > > > >> > name
>> > > > >> > > > > > > > >> > > > > > >> > > > and
>> > > > >> > > > > > > > >> > > > > > >> > > > > > the
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> partition id since
>> they
>> > > are
>> > > > >> > implied
>> > > > >> > > > in
>> > > > >> > > > > > the
>> > > > >> > > > > > > > >> > directory
>> > > > >> > > > > > > > >> > > > > name?
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 14. In the JBOD
>> mode, we
>> > > > >> support
>> > > > >> > > > > moving a
>> > > > >> > > > > > > > >> > > partition's
>> > > > >> > > > > > > > >> > > > > data
>> > > > >> > > > > > > > >> > > > > > >> from
>> > > > >> > > > > > > > >> > > > > > >> > > one
>> > > > >> > > > > > > > >> > > > > > >> > > > > > disk
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> to
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> another. Will the new
>> > > > >> partition
>> > > > >> > > > > metadata
>> > > > >> > > > > > > file
>> > > > >> > > > > > > > >> be
>> > > > >> > > > > > > > >> > > > copied
>> > > > >> > > > > > > > >> > > > > > >> during
>> > > > >> > > > > > > > >> > > > > > >> > > that
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> process?
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 15. The KIP says
>> "Remove
>> > > > >> deleted
>> > > > >> > > > topics
>> > > > >> > > > > > > from
>> > > > >> > > > > > > > >> > > replicas
>> > > > >> > > > > > > > >> > > > by
>> > > > >> > > > > > > > >> > > > > > >> > sending
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> StopReplicaRequest V2
>> > for
>> > > > any
>> > > > >> > > topics
>> > > > >> > > > > > which
>> > > > >> > > > > > > do
>> > > > >> > > > > > > > >> not
>> > > > >> > > > > > > > >> > > > > contain
>> > > > >> > > > > > > > >> > > > > > a
>> > > > >> > > > > > > > >> > > > > > >> > topic
>> > > > >> > > > > > > > >> > > > > > >> > > > ID,
>> > > > >> > > > > > > > >> > > > > > >> > > > > > and
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> V3 for any topics
>> which
>> > do
>> > > > >> > contain
>> > > > >> > > a
>> > > > >> > > > > > topic
>> > > > >> > > > > > > > >> ID.".
>> > > > >> > > > > > > > >> > > > > However,
>> > > > >> > > > > > > > >> > > > > > it
>> > > > >> > > > > > > > >> > > > > > >> > > seems
>> > > > >> > > > > > > > >> > > > > > >> > > > > the
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> updated controller
>> will
>> > > > create
>> > > > >> > all
>> > > > >> > > > > > missing
>> > > > >> > > > > > > > >> topic
>> > > > >> > > > > > > > >> > IDs
>> > > > >> > > > > > > > >> > > > > first
>> > > > >> > > > > > > > >> > > > > > >> > before
>> > > > >> > > > > > > > >> > > > > > >> > > > > doing
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> other actions. So, is
>> > > > >> > > > > StopReplicaRequest
>> > > > >> > > > > > V2
>> > > > >> > > > > > > > >> > needed?
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Jun
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> On Fri, Sep 11, 2020
>> at
>> > > > 10:31
>> > > > >> AM
>> > > > >> > > John
>> > > > >> > > > > > > > Roesler <
>> > > > >> > > > > > > > >> > > > > > >> > > vvcephei@apache.org
>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> wrote:
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > Thanks, Justine!
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > Your response seems
>> > > > >> compelling
>> > > > >> > to
>> > > > >> > > > me.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > -John
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > On Fri, 2020-09-11
>> at
>> > > > 10:17
>> > > > >> > > -0700,
>> > > > >> > > > > > > Justine
>> > > > >> > > > > > > > >> > Olshan
>> > > > >> > > > > > > > >> > > > > wrote:
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Hello all,
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Thanks for
>> > continuing
>> > > > the
>> > > > >> > > > > > discussion! I
>> > > > >> > > > > > > > >> have a
>> > > > >> > > > > > > > >> > > few
>> > > > >> > > > > > > > >> > > > > > >> > responses
>> > > > >> > > > > > > > >> > > > > > >> > > to
>> > > > >> > > > > > > > >> > > > > > >> > > > > > your
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > points.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Tom: You are
>> correct
>> > > in
>> > > > >> that
>> > > > >> > > this
>> > > > >> > > > > KIP
>> > > > >> > > > > > > has
>> > > > >> > > > > > > > >> not
>> > > > >> > > > > > > > >> > > > > > mentioned
>> > > > >> > > > > > > > >> > > > > > >> the
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
>> > DeleteTopicsRequest. I
>> > > > >> think
>> > > > >> > > that
>> > > > >> > > > > > this
>> > > > >> > > > > > > > >> would
>> > > > >> > > > > > > > >> > be
>> > > > >> > > > > > > > >> > > > out
>> > > > >> > > > > > > > >> > > > > of
>> > > > >> > > > > > > > >> > > > > > >> > scope
>> > > > >> > > > > > > > >> > > > > > >> > > > for
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> now, but
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > may be something
>> > worth
>> > > > >> adding
>> > > > >> > > in
>> > > > >> > > > > the
>> > > > >> > > > > > > > >> future.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > John: We did
>> > consider
>> > > > >> > sequence
>> > > > >> > > > ids,
>> > > > >> > > > > > but
>> > > > >> > > > > > > > >> there
>> > > > >> > > > > > > > >> > > are
>> > > > >> > > > > > > > >> > > > a
>> > > > >> > > > > > > > >> > > > > > few
>> > > > >> > > > > > > > >> > > > > > >> > > reasons
>> > > > >> > > > > > > > >> > > > > > >> > > > > to
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> favor
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > UUIDs. There are
>> > > several
>> > > > >> > cases
>> > > > >> > > > > where
>> > > > >> > > > > > > > topics
>> > > > >> > > > > > > > >> > from
>> > > > >> > > > > > > > >> > > > > > >> different
>> > > > >> > > > > > > > >> > > > > > >> > > > > clusters
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> may
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > interact now and
>> in
>> > > the
>> > > > >> > future.
>> > > > >> > > > For
>> > > > >> > > > > > > > >> example,
>> > > > >> > > > > > > > >> > > > Mirror
>> > > > >> > > > > > > > >> > > > > > >> Maker 2
>> > > > >> > > > > > > > >> > > > > > >> > > may
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> benefit
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > from being able
>> to
>> > > > detect
>> > > > >> > when
>> > > > >> > > a
>> > > > >> > > > > > > cluster
>> > > > >> > > > > > > > >> being
>> > > > >> > > > > > > > >> > > > > > mirrored
>> > > > >> > > > > > > > >> > > > > > >> is
>> > > > >> > > > > > > > >> > > > > > >> > > > > deleted
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> and
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > recreated and
>> > globally
>> > > > >> unique
>> > > > >> > > > > > > identifiers
>> > > > >> > > > > > > > >> > would
>> > > > >> > > > > > > > >> > > > make
>> > > > >> > > > > > > > >> > > > > > >> > > resolving
>> > > > >> > > > > > > > >> > > > > > >> > > > > > issues
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > easier than
>> sequence
>> > > IDs
>> > > > >> > which
>> > > > >> > > > may
>> > > > >> > > > > > > > collide
>> > > > >> > > > > > > > >> > > between
>> > > > >> > > > > > > > >> > > > > > >> > clusters.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > KIP-405
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > (tiered storage)
>> > will
>> > > > also
>> > > > >> > > > benefit
>> > > > >> > > > > > from
>> > > > >> > > > > > > > >> > globally
>> > > > >> > > > > > > > >> > > > > > unique
>> > > > >> > > > > > > > >> > > > > > >> IDs
>> > > > >> > > > > > > > >> > > > > > >> > > as
>> > > > >> > > > > > > > >> > > > > > >> > > > > > shared
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > buckets may be
>> used
>> > > > >> between
>> > > > >> > > > > clusters.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Globally unique
>> IDs
>> > > > would
>> > > > >> > also
>> > > > >> > > > make
>> > > > >> > > > > > > > >> > > functionality
>> > > > >> > > > > > > > >> > > > > like
>> > > > >> > > > > > > > >> > > > > > >> > moving
>> > > > >> > > > > > > > >> > > > > > >> > > > > > topics
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > between disparate
>> > > > clusters
>> > > > >> > > easier
>> > > > >> > > > > in
>> > > > >> > > > > > > the
>> > > > >> > > > > > > > >> > future,
>> > > > >> > > > > > > > >> > > > > > >> simplify
>> > > > >> > > > > > > > >> > > > > > >> > any
>> > > > >> > > > > > > > >> > > > > > >> > > > > > future
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > implementations
>> of
>> > > > backups
>> > > > >> > and
>> > > > >> > > > > > > restores,
>> > > > >> > > > > > > > >> and
>> > > > >> > > > > > > > >> > > more.
>> > > > >> > > > > > > > >> > > > > In
>> > > > >> > > > > > > > >> > > > > > >> > > general,
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> unique IDs
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > would ensure that
>> > the
>> > > > >> source
>> > > > >> > > > > cluster
>> > > > >> > > > > > > > >> topics do
>> > > > >> > > > > > > > >> > > not
>> > > > >> > > > > > > > >> > > > > > >> conflict
>> > > > >> > > > > > > > >> > > > > > >> > > > with
>> > > > >> > > > > > > > >> > > > > > >> > > > > > the
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > destination
>> cluster
>> > > > >> topics.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > If we were to use
>> > > > sequence
>> > > > >> > ids,
>> > > > >> > > > we
>> > > > >> > > > > > > would
>> > > > >> > > > > > > > >> need
>> > > > >> > > > > > > > >> > > > > > >> sufficiently
>> > > > >> > > > > > > > >> > > > > > >> > > > large
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> cluster
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > ids to be stored
>> > with
>> > > > the
>> > > > >> > topic
>> > > > >> > > > > > > > >> identifiers or
>> > > > >> > > > > > > > >> > > we
>> > > > >> > > > > > > > >> > > > > run
>> > > > >> > > > > > > > >> > > > > > >> the
>> > > > >> > > > > > > > >> > > > > > >> > > risk
>> > > > >> > > > > > > > >> > > > > > >> > > > of
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > collisions. This
>> > will
>> > > > >> give up
>> > > > >> > > any
>> > > > >> > > > > > > > >> advantage in
>> > > > >> > > > > > > > >> > > > > > >> compactness
>> > > > >> > > > > > > > >> > > > > > >> > > that
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> sequence
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > numbers may
>> bring.
>> > > Given
>> > > > >> > these
>> > > > >> > > > > > > > advantages I
>> > > > >> > > > > > > > >> > > think
>> > > > >> > > > > > > > >> > > > it
>> > > > >> > > > > > > > >> > > > > > >> makes
>> > > > >> > > > > > > > >> > > > > > >> > > > sense
>> > > > >> > > > > > > > >> > > > > > >> > > > > to
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> use
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > UUIDs.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Gokul: This is an
>> > > > >> interesting
>> > > > >> > > > idea,
>> > > > >> > > > > > but
>> > > > >> > > > > > > > >> this
>> > > > >> > > > > > > > >> > is
>> > > > >> > > > > > > > >> > > a
>> > > > >> > > > > > > > >> > > > > > >> breaking
>> > > > >> > > > > > > > >> > > > > > >> > > > > change.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Out of
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > scope for now,
>> but
>> > > maybe
>> > > > >> > worth
>> > > > >> > > > > > > discussing
>> > > > >> > > > > > > > >> in
>> > > > >> > > > > > > > >> > the
>> > > > >> > > > > > > > >> > > > > > future.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Hope this
>> explains
>> > > some
>> > > > of
>> > > > >> > the
>> > > > >> > > > > > > decisions,
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Justine
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > On Fri, Sep 11,
>> 2020
>> > > at
>> > > > >> 8:27
>> > > > >> > AM
>> > > > >> > > > > Gokul
>> > > > >> > > > > > > > >> Ramanan
>> > > > >> > > > > > > > >> > > > > > >> Subramanian <
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
>> > gokul2411s@gmail.com>
>> > > > >> wrote:
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Hi.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Thanks for the
>> > KIP.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Have you
>> thought
>> > > about
>> > > > >> > > whether
>> > > > >> > > > it
>> > > > >> > > > > > > makes
>> > > > >> > > > > > > > >> > sense
>> > > > >> > > > > > > > >> > > to
>> > > > >> > > > > > > > >> > > > > > >> support
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> authorizing a
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > principal for a
>> > > topic
>> > > > ID
>> > > > >> > > rather
>> > > > >> > > > > > than
>> > > > >> > > > > > > a
>> > > > >> > > > > > > > >> topic
>> > > > >> > > > > > > > >> > > > name
>> > > > >> > > > > > > > >> > > > > to
>> > > > >> > > > > > > > >> > > > > > >> > > achieve
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> tighter
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > security?
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Or is the
>> topic ID
>> > > > >> > > > fundamentally
>> > > > >> > > > > an
>> > > > >> > > > > > > > >> internal
>> > > > >> > > > > > > > >> > > > > detail
>> > > > >> > > > > > > > >> > > > > > >> > similar
>> > > > >> > > > > > > > >> > > > > > >> > > > to
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> epochs
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > used
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > in a bunch of
>> > other
>> > > > >> places
>> > > > >> > in
>> > > > >> > > > > > Kafka?
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Thanks.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > On Fri, Sep 11,
>> > 2020
>> > > > at
>> > > > >> > 4:06
>> > > > >> > > PM
>> > > > >> > > > > > John
>> > > > >> > > > > > > > >> > Roesler <
>> > > > >> > > > > > > > >> > > > > > >> > > > > > vvcephei@apache.org>
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > wrote:
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Hello
>> Justine,
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Thanks for
>> the
>> > > KIP!
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > I happen to
>> have
>> > > > been
>> > > > >> > > > > confronted
>> > > > >> > > > > > > > >> recently
>> > > > >> > > > > > > > >> > > with
>> > > > >> > > > > > > > >> > > > > the
>> > > > >> > > > > > > > >> > > > > > >> need
>> > > > >> > > > > > > > >> > > > > > >> > > to
>> > > > >> > > > > > > > >> > > > > > >> > > > > keep
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > track of
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > a
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > large number
>> of
>> > > > >> topics as
>> > > > >> > > > > > compactly
>> > > > >> > > > > > > > as
>> > > > >> > > > > > > > >> > > > > possible. I
>> > > > >> > > > > > > > >> > > > > > >> was
>> > > > >> > > > > > > > >> > > > > > >> > > > going
>> > > > >> > > > > > > > >> > > > > > >> > > > > to
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> come
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > up
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > with some
>> way to
>> > > > >> > dictionary
>> > > > >> > > > > > encode
>> > > > >> > > > > > > > the
>> > > > >> > > > > > > > >> > topic
>> > > > >> > > > > > > > >> > > > > names
>> > > > >> > > > > > > > >> > > > > > >> as
>> > > > >> > > > > > > > >> > > > > > >> > > > > integers,
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> but
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > this
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > seems much
>> > better!
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Apologies if
>> > this
>> > > > has
>> > > > >> > been
>> > > > >> > > > > raised
>> > > > >> > > > > > > > >> before,
>> > > > >> > > > > > > > >> > > but
>> > > > >> > > > > > > > >> > > > > I’m
>> > > > >> > > > > > > > >> > > > > > >> > > wondering
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> about the
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > choice of
>> UUID
>> > vs
>> > > > >> > sequence
>> > > > >> > > > > number
>> > > > >> > > > > > > for
>> > > > >> > > > > > > > >> the
>> > > > >> > > > > > > > >> > > ids.
>> > > > >> > > > > > > > >> > > > > > >> > Typically,
>> > > > >> > > > > > > > >> > > > > > >> > > > > I’ve
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> seen
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > UUIDs
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > in two
>> > situations:
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 1. When
>> > processes
>> > > > >> need to
>> > > > >> > > > > > generate
>> > > > >> > > > > > > > >> > > > non-colliding
>> > > > >> > > > > > > > >> > > > > > >> > > > identifiers
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> without
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > coordination.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 2. When the
>> > > > identifier
>> > > > >> > > needs
>> > > > >> > > > to
>> > > > >> > > > > > be
>> > > > >> > > > > > > > >> > > > “universally
>> > > > >> > > > > > > > >> > > > > > >> > unique”;
>> > > > >> > > > > > > > >> > > > > > >> > > > > I.e.,
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> the
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > identifier
>> needs
>> > > to
>> > > > >> > > > distinguish
>> > > > >> > > > > > the
>> > > > >> > > > > > > > >> entity
>> > > > >> > > > > > > > >> > > > from
>> > > > >> > > > > > > > >> > > > > > all
>> > > > >> > > > > > > > >> > > > > > >> > other
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> entities
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > that
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > could ever
>> > exist.
>> > > > >> This is
>> > > > >> > > > > useful
>> > > > >> > > > > > in
>> > > > >> > > > > > > > >> cases
>> > > > >> > > > > > > > >> > > > where
>> > > > >> > > > > > > > >> > > > > > >> > entities
>> > > > >> > > > > > > > >> > > > > > >> > > > from
>> > > > >> > > > > > > > >> > > > > > >> > > > > > all
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > kinds
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > of
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > systems get
>> > mixed
>> > > > >> > together,
>> > > > >> > > > > such
>> > > > >> > > > > > as
>> > > > >> > > > > > > > >> when
>> > > > >> > > > > > > > >> > > > dumping
>> > > > >> > > > > > > > >> > > > > > >> logs
>> > > > >> > > > > > > > >> > > > > > >> > > from
>> > > > >> > > > > > > > >> > > > > > >> > > > > all
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > processes
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > in
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a company
>> into a
>> > > > >> common
>> > > > >> > > > system.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Maybe I’m
>> being
>> > > > >> > > > short-sighted,
>> > > > >> > > > > > but
>> > > > >> > > > > > > it
>> > > > >> > > > > > > > >> > > doesn’t
>> > > > >> > > > > > > > >> > > > > seem
>> > > > >> > > > > > > > >> > > > > > >> like
>> > > > >> > > > > > > > >> > > > > > >> > > > > either
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> really
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > applies
>> here. It
>> > > > seems
>> > > > >> > like
>> > > > >> > > > the
>> > > > >> > > > > > > > brokers
>> > > > >> > > > > > > > >> > > could
>> > > > >> > > > > > > > >> > > > > and
>> > > > >> > > > > > > > >> > > > > > >> would
>> > > > >> > > > > > > > >> > > > > > >> > > > > achieve
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > consensus
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > when
>> creating a
>> > > > topic
>> > > > >> > > anyway,
>> > > > >> > > > > > which
>> > > > >> > > > > > > > is
>> > > > >> > > > > > > > >> all
>> > > > >> > > > > > > > >> > > > > that’s
>> > > > >> > > > > > > > >> > > > > > >> > > required
>> > > > >> > > > > > > > >> > > > > > >> > > > to
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > generate
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > non-colliding
>> > > > sequence
>> > > > >> > ids.
>> > > > >> > > > For
>> > > > >> > > > > > the
>> > > > >> > > > > > > > >> > second,
>> > > > >> > > > > > > > >> > > as
>> > > > >> > > > > > > > >> > > > > you
>> > > > >> > > > > > > > >> > > > > > >> > > mention,
>> > > > >> > > > > > > > >> > > > > > >> > > > > we
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> could
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > assign
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a UUID to the
>> > > > cluster
>> > > > >> as
>> > > > >> > a
>> > > > >> > > > > whole,
>> > > > >> > > > > > > > which
>> > > > >> > > > > > > > >> > > would
>> > > > >> > > > > > > > >> > > > > > render
>> > > > >> > > > > > > > >> > > > > > >> > any
>> > > > >> > > > > > > > >> > > > > > >> > > > > > resource
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > scoped
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > the broker
>> > > > universally
>> > > > >> > > unique
>> > > > >> > > > > as
>> > > > >> > > > > > > > well.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > The reason I
>> > > mention
>> > > > >> this
>> > > > >> > > is
>> > > > >> > > > > > that,
>> > > > >> > > > > > > > >> > although
>> > > > >> > > > > > > > >> > > a
>> > > > >> > > > > > > > >> > > > > UUID
>> > > > >> > > > > > > > >> > > > > > >> is
>> > > > >> > > > > > > > >> > > > > > >> > way
>> > > > >> > > > > > > > >> > > > > > >> > > > > more
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > compact
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > than topic
>> > names,
>> > > > it’s
>> > > > >> > > still
>> > > > >> > > > 16
>> > > > >> > > > > > > > bytes.
>> > > > >> > > > > > > > >> In
>> > > > >> > > > > > > > >> > > > > > contrast,
>> > > > >> > > > > > > > >> > > > > > >> a
>> > > > >> > > > > > > > >> > > > > > >> > > > 4-byte
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> integer
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > sequence id
>> > would
>> > > > give
>> > > > >> > us 4
>> > > > >> > > > > > billion
>> > > > >> > > > > > > > >> unique
>> > > > >> > > > > > > > >> > > > > topics
>> > > > >> > > > > > > > >> > > > > > >> per
>> > > > >> > > > > > > > >> > > > > > >> > > > > cluster,
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> which
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > seems
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > like enough
>> ;)
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Considering
>> the
>> > > > >> number of
>> > > > >> > > > > > different
>> > > > >> > > > > > > > >> times
>> > > > >> > > > > > > > >> > > > these
>> > > > >> > > > > > > > >> > > > > > >> topic
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> identifiers are
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > sent
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > over the
>> wire or
>> > > > >> stored
>> > > > >> > in
>> > > > >> > > > > > memory,
>> > > > >> > > > > > > it
>> > > > >> > > > > > > > >> > seems
>> > > > >> > > > > > > > >> > > > like
>> > > > >> > > > > > > > >> > > > > > it
>> > > > >> > > > > > > > >> > > > > > >> > might
>> > > > >> > > > > > > > >> > > > > > >> > > > be
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> worth
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > the
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > additional 4x
>> > > space
>> > > > >> > > savings.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > What do you
>> > think
>> > > > >> about
>> > > > >> > > this?
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Thanks,
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > John
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > On Fri, Sep
>> 11,
>> > > > 2020,
>> > > > >> at
>> > > > >> > > > 03:20,
>> > > > >> > > > > > Tom
>> > > > >> > > > > > > > >> > Bentley
>> > > > >> > > > > > > > >> > > > > wrote:
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Hi Justine,
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > This looks
>> > like
>> > > a
>> > > > >> very
>> > > > >> > > > > welcome
>> > > > >> > > > > > > > >> > > improvement.
>> > > > >> > > > > > > > >> > > > > > >> Thanks!
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Maybe I
>> missed
>> > > it,
>> > > > >> but
>> > > > >> > > the
>> > > > >> > > > > KIP
>> > > > >> > > > > > > > >> doesn't
>> > > > >> > > > > > > > >> > > seem
>> > > > >> > > > > > > > >> > > > to
>> > > > >> > > > > > > > >> > > > > > >> > mention
>> > > > >> > > > > > > > >> > > > > > >> > > > > > changing
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
>> > > > DeleteTopicsRequest
>> > > > >> to
>> > > > >> > > > > identify
>> > > > >> > > > > > > the
>> > > > >> > > > > > > > >> > topic
>> > > > >> > > > > > > > >> > > > > using
>> > > > >> > > > > > > > >> > > > > > an
>> > > > >> > > > > > > > >> > > > > > >> > id.
>> > > > >> > > > > > > > >> > > > > > >> > > > > Maybe
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > that's out
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > of
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > scope, but
>> > > > >> > > > > DeleteTopicsRequest
>> > > > >> > > > > > is
>> > > > >> > > > > > > > not
>> > > > >> > > > > > > > >> > > listed
>> > > > >> > > > > > > > >> > > > > > among
>> > > > >> > > > > > > > >> > > > > > >> > the
>> > > > >> > > > > > > > >> > > > > > >> > > > > Future
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Work
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > APIs
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > either.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Kind
>> regards,
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Tom
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > On Thu, Sep
>> > 10,
>> > > > >> 2020 at
>> > > > >> > > > 3:59
>> > > > >> > > > > PM
>> > > > >> > > > > > > > >> Satish
>> > > > >> > > > > > > > >> > > > > Duggana <
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
>> > > > >> satish.duggana@gmail.com
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > wrote:
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Thanks
>> > > > >> Lucas/Justine
>> > > > >> > > for
>> > > > >> > > > > the
>> > > > >> > > > > > > nice
>> > > > >> > > > > > > > >> KIP.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > It has
>> > several
>> > > > >> > benefits
>> > > > >> > > > > which
>> > > > >> > > > > > > > also
>> > > > >> > > > > > > > >> > > include
>> > > > >> > > > > > > > >> > > > > > >> > > simplifying
>> > > > >> > > > > > > > >> > > > > > >> > > > > the
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> topic
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > deletion
>> > > process
>> > > > >> by
>> > > > >> > > > > > controller
>> > > > >> > > > > > > > and
>> > > > >> > > > > > > > >> > logs
>> > > > >> > > > > > > > >> > > > > > cleanup
>> > > > >> > > > > > > > >> > > > > > >> by
>> > > > >> > > > > > > > >> > > > > > >> > > > > brokers
>> > > > >> > > > > > > > >> > > > > > >> > > > > > in
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > corner
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > cases.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Best,
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Satish.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > On Wed,
>> Sep
>> > 9,
>> > > > >> 2020
>> > > > >> > at
>> > > > >> > > > > 10:07
>> > > > >> > > > > > PM
>> > > > >> > > > > > > > >> > Justine
>> > > > >> > > > > > > > >> > > > > > Olshan <
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
>> jolshan@confluent.io
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > wrote:
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Hello
>> all,
>> > > > it's
>> > > > >> > been
>> > > > >> > > > > > almost a
>> > > > >> > > > > > > > >> year!
>> > > > >> > > > > > > > >> > > I've
>> > > > >> > > > > > > > >> > > > > > made
>> > > > >> > > > > > > > >> > > > > > >> > some
>> > > > >> > > > > > > > >> > > > > > >> > > > > > changes
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> to
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > this
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > KIP
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > and hope
>> to
>> > > > >> continue
>> > > > >> > > the
>> > > > >> > > > > > > > >> discussion.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > One of
>> the
>> > > > main
>> > > > >> > > changes
>> > > > >> > > > > > I've
>> > > > >> > > > > > > > >> added
>> > > > >> > > > > > > > >> > is
>> > > > >> > > > > > > > >> > > > now
>> > > > >> > > > > > > > >> > > > > > the
>> > > > >> > > > > > > > >> > > > > > >> > > > metadata
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> response
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > will
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > include
>> the
>> > > > topic
>> > > > >> ID
>> > > > >> > > (as
>> > > > >> > > > > > Colin
>> > > > >> > > > > > > > >> > > suggested).
>> > > > >> > > > > > > > >> > > > > > >> Clients
>> > > > >> > > > > > > > >> > > > > > >> > > can
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> obtain the
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > topicID
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > of a
>> given
>> > > topic
>> > > > >> > > through
>> > > > >> > > > a
>> > > > >> > > > > > > > >> > > > TopicDescription.
>> > > > >> > > > > > > > >> > > > > > The
>> > > > >> > > > > > > > >> > > > > > >> > > > topicId
>> > > > >> > > > > > > > >> > > > > > >> > > > > > will
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > also be
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > included
>> > with
>> > > > the
>> > > > >> > > > > > > UpdateMetadata
>> > > > >> > > > > > > > >> > > request.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Let me
>> > know
>> > > > what
>> > > > >> > you
>> > > > >> > > > all
>> > > > >> > > > > > > think.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Thank
>> you,
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Justine
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > On
>> > > 2019/09/13
>> > > > >> > > 16:38:26,
>> > > > >> > > > > > > "Colin
>> > > > >> > > > > > > > >> > > McCabe" <
>> > > > >> > > > > > > > >> > > > > > >> > > > > > cmccabe@apache.org
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > wrote:
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Hi
>> > Lucas,
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
>> Thanks
>> > for
>> > > > >> > tackling
>> > > > >> > > > > this.
>> > > > >> > > > > > > > >> Topic
>> > > > >> > > > > > > > >> > IDs
>> > > > >> > > > > > > > >> > > > > are a
>> > > > >> > > > > > > > >> > > > > > >> > great
>> > > > >> > > > > > > > >> > > > > > >> > > > > idea,
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> and
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > this
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > is
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > really
>> good
>> > > > >> writeup.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > For
>> > > > >> > > > > > > /brokers/topics/[topic],
>> > > > >> > > > > > > > >> the
>> > > > >> > > > > > > > >> > > > schema
>> > > > >> > > > > > > > >> > > > > > >> version
>> > > > >> > > > > > > > >> > > > > > >> > > > > should
>> > > > >> > > > > > > > >> > > > > > >> > > > > > be
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > bumped
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > version
>> 3,
>> > > > rather
>> > > > >> > than
>> > > > >> > > 2.
>> > > > >> > > > > > > > KIP-455
>> > > > >> > > > > > > > >> > > bumped
>> > > > >> > > > > > > > >> > > > > the
>> > > > >> > > > > > > > >> > > > > > >> > version
>> > > > >> > > > > > > > >> > > > > > >> > > > of
>> > > > >> > > > > > > > >> > > > > > >> > > > > > this
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > znode
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 2
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > already
>> :)
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Given
>> > that
>> > > > >> we're
>> > > > >> > > > going
>> > > > >> > > > > to
>> > > > >> > > > > > > be
>> > > > >> > > > > > > > >> > seeing
>> > > > >> > > > > > > > >> > > > > these
>> > > > >> > > > > > > > >> > > > > > >> > things
>> > > > >> > > > > > > > >> > > > > > >> > > as
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> strings
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > as
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > lot
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > (in
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > logs, in
>> > > > >> ZooKeeper,
>> > > > >> > on
>> > > > >> > > > the
>> > > > >> > > > > > > > >> > command-line,
>> > > > >> > > > > > > > >> > > > > > etc.),
>> > > > >> > > > > > > > >> > > > > > >> > does
>> > > > >> > > > > > > > >> > > > > > >> > > it
>> > > > >> > > > > > > > >> > > > > > >> > > > > > make
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > sense to
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > use
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > base64
>> when
>> > > > >> > converting
>> > > > >> > > > them
>> > > > >> > > > > > to
>> > > > >> > > > > > > > >> > strings?
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Here
>> is
>> > an
>> > > > >> > example
>> > > > >> > > of
>> > > > >> > > > > the
>> > > > >> > > > > > > hex
>> > > > >> > > > > > > > >> > > > > > >> representation:
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
>> > > > >> > > > > > > > >> > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > And
>> here
>> > > is
>> > > > an
>> > > > >> > > > example
>> > > > >> > > > > in
>> > > > >> > > > > > > > >> base64.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
>> > > > >> > > > b8tRS7h4TJ2Vt43Dp85v2A
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > The
>> > base64
>> > > > >> > version
>> > > > >> > > > > saves
>> > > > >> > > > > > 15
>> > > > >> > > > > > > > >> > letters
>> > > > >> > > > > > > > >> > > > (to
>> > > > >> > > > > > > > >> > > > > be
>> > > > >> > > > > > > > >> > > > > > >> > fair,
>> > > > >> > > > > > > > >> > > > > > >> > > 4
>> > > > >> > > > > > > > >> > > > > > >> > > > of
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> those
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > were
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > dashes
>> that
>> > we
>> > > > >> could
>> > > > >> > > have
>> > > > >> > > > > > > elided
>> > > > >> > > > > > > > in
>> > > > >> > > > > > > > >> > the
>> > > > >> > > > > > > > >> > > > hex
>> > > > >> > > > > > > > >> > > > > > >> > > > > > representation.)
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
>> Another
>> > > > thing
>> > > > >> to
>> > > > >> > > > > consider
>> > > > >> > > > > > > is
>> > > > >> > > > > > > > >> that
>> > > > >> > > > > > > > >> > we
>> > > > >> > > > > > > > >> > > > > > should
>> > > > >> > > > > > > > >> > > > > > >> > > specify
>> > > > >> > > > > > > > >> > > > > > >> > > > > > that
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> the
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
>> all-zeroes
>> > > UUID
>> > > > is
>> > > > >> > not
>> > > > >> > > a
>> > > > >> > > > > > valid
>> > > > >> > > > > > > > >> topic
>> > > > >> > > > > > > > >> > > UUID.
>> > > > >> > > > > > > > >> > > > > >  We
>> > > > >> > > > > > > > >> > > > > > >> > can't
>> > > > >> > > > > > > > >> > > > > > >> > > > use
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> null
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > for
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > this
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > because
>> we
>> > > can't
>> > > > >> > pass a
>> > > > >> > > > > null
>> > > > >> > > > > > > UUID
>> > > > >> > > > > > > > >> over
>> > > > >> > > > > > > > >> > > the
>> > > > >> > > > > > > > >> > > > > RPC
>> > > > >> > > > > > > > >> > > > > > >> > > protocol
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> (there
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > is no
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > special
>> > > pattern
>> > > > >> for
>> > > > >> > > null,
>> > > > >> > > > > nor
>> > > > >> > > > > > > do
>> > > > >> > > > > > > > we
>> > > > >> > > > > > > > >> > want
>> > > > >> > > > > > > > >> > > > to
>> > > > >> > > > > > > > >> > > > > > >> waste
>> > > > >> > > > > > > > >> > > > > > >> > > space
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> reserving
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > such
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > pattern.)
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
>> Maybe I
>> > > > missed
>> > > > >> > it,
>> > > > >> > > > but
>> > > > >> > > > > > did
>> > > > >> > > > > > > > you
>> > > > >> > > > > > > > >> > > > describe
>> > > > >> > > > > > > > >> > > > > > >> > > "migration
>> > > > >> > > > > > > > >> > > > > > >> > > > > > of...
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > existing
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > topic[s]
>> > > without
>> > > > >> > topic
>> > > > >> > > > IDs"
>> > > > >> > > > > > in
>> > > > >> > > > > > > > >> detail
>> > > > >> > > > > > > > >> > in
>> > > > >> > > > > > > > >> > > > any
>> > > > >> > > > > > > > >> > > > > > >> > section?
>> > > > >> > > > > > > > >> > > > > > >> > > > It
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> seems
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > like
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > when
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > the new
>> > > > controller
>> > > > >> > > > becomes
>> > > > >> > > > > > > > active,
>> > > > >> > > > > > > > >> it
>> > > > >> > > > > > > > >> > > > should
>> > > > >> > > > > > > > >> > > > > > >> just
>> > > > >> > > > > > > > >> > > > > > >> > > > > generate
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> random
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > UUIDs for
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > these,
>> and
>> > > write
>> > > > >> the
>> > > > >> > > > random
>> > > > >> > > > > > > UUIDs
>> > > > >> > > > > > > > >> back
>> > > > >> > > > > > > > >> > > to
>> > > > >> > > > > > > > >> > > > > > >> > ZooKeeper.
>> > > > >> > > > > > > > >> > > > > > >> > > > It
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> would be
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > good
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > to
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > spell
>> that
>> > > out.
>> > > > >> We
>> > > > >> > > > should
>> > > > >> > > > > > make
>> > > > >> > > > > > > > it
>> > > > >> > > > > > > > >> > clear
>> > > > >> > > > > > > > >> > > > > that
>> > > > >> > > > > > > > >> > > > > > >> this
>> > > > >> > > > > > > > >> > > > > > >> > > > > happens
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > regardless
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > of
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > the
>> > > inter-broker
>> > > > >> > > protocol
>> > > > >> > > > > > > version
>> > > > >> > > > > > > > >> > (it's
>> > > > >> > > > > > > > >> > > a
>> > > > >> > > > > > > > >> > > > > > >> > compatible
>> > > > >> > > > > > > > >> > > > > > >> > > > > > change).
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
>> > > > >> > > "LeaderAndIsrRequests
>> > > > >> > > > > > > > >> including an
>> > > > >> > > > > > > > >> > > > > > >> > > > is_every_partition
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> flag"
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > seems a
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > bit
>> wordy.
>> > > Can
>> > > > we
>> > > > >> > just
>> > > > >> > > > > call
>> > > > >> > > > > > > > these
>> > > > >> > > > > > > > >> > "full
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
>> LeaderAndIsrRequests"?
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > Then
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > the
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > RPC field
>> > > could
>> > > > be
>> > > > >> > > named
>> > > > >> > > > > > > "full".
>> > > > >> > > > > > > > >> > Also,
>> > > > >> > > > > > > > >> > > it
>> > > > >> > > > > > > > >> > > > > > would
>> > > > >> > > > > > > > >> > > > > > >> > > > probably
>> > > > >> > > > > > > > >> > > > > > >> > > > > > be
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > better
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > for the
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > RPC
>> field to
>> > > be
>> > > > an
>> > > > >> > enum
>> > > > >> > > > of
>> > > > >> > > > > {
>> > > > >> > > > > > > > >> > > UNSPECIFIED,
>> > > > >> > > > > > > > >> > > > > > >> > > INCREMENTAL,
>> > > > >> > > > > > > > >> > > > > > >> > > > > FULL
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> }, so
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > that
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > we
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > can
>> cleanly
>> > > > handle
>> > > > >> > old
>> > > > >> > > > > > versions
>> > > > >> > > > > > > > (by
>> > > > >> > > > > > > > >> > > > treating
>> > > > >> > > > > > > > >> > > > > > >> them
>> > > > >> > > > > > > > >> > > > > > >> > as
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> UNSPECIFIED)
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > In
>> the
>> > > > >> > > > > > LeaderAndIsrRequest
>> > > > >> > > > > > > > >> > section,
>> > > > >> > > > > > > > >> > > > you
>> > > > >> > > > > > > > >> > > > > > >> write
>> > > > >> > > > > > > > >> > > > > > >> > "A
>> > > > >> > > > > > > > >> > > > > > >> > > > > final
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > deletion
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > event
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > will be
>> > > > secheduled
>> > > > >> > for
>> > > > >> > > X
>> > > > >> > > > ms
>> > > > >> > > > > > > after
>> > > > >> > > > > > > > >> the
>> > > > >> > > > > > > > >> > > > > > >> > > > LeaderAndIsrRequest
>> > > > >> > > > > > > > >> > > > > > >> > > > > > was
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > first
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
>> received..."
>> > > I
>> > > > >> guess
>> > > > >> > > > the X
>> > > > >> > > > > > > was a
>> > > > >> > > > > > > > >> > > > > placeholder
>> > > > >> > > > > > > > >> > > > > > >> that
>> > > > >> > > > > > > > >> > > > > > >> > > you
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> intended
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > replace
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > before
>> > > posting?
>> > > > :)
>> > > > >> > In
>> > > > >> > > > any
>> > > > >> > > > > > > case,
>> > > > >> > > > > > > > >> this
>> > > > >> > > > > > > > >> > > > seems
>> > > > >> > > > > > > > >> > > > > > like
>> > > > >> > > > > > > > >> > > > > > >> > the
>> > > > >> > > > > > > > >> > > > > > >> > > > kind
>> > > > >> > > > > > > > >> > > > > > >> > > > > > of
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > thing
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > we'd
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > want a
>> > > > >> configuration
>> > > > >> > > for.
>> > > > >> > > > > > > Let's
>> > > > >> > > > > > > > >> > > describe
>> > > > >> > > > > > > > >> > > > > that
>> > > > >> > > > > > > > >> > > > > > >> > > > > > configuration
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> key
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > somewhere
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > in this
>> KIP,
>> > > > >> > including
>> > > > >> > > > what
>> > > > >> > > > > > its
>> > > > >> > > > > > > > >> > default
>> > > > >> > > > > > > > >> > > > > value
>> > > > >> > > > > > > > >> > > > > > >> is.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > We
>> > should
>> > > > >> > probably
>> > > > >> > > > also
>> > > > >> > > > > > > log a
>> > > > >> > > > > > > > >> > bunch
>> > > > >> > > > > > > > >> > > of
>> > > > >> > > > > > > > >> > > > > > >> messages
>> > > > >> > > > > > > > >> > > > > > >> > > at
>> > > > >> > > > > > > > >> > > > > > >> > > > > WARN
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> level
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > when
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
>> something is
>> > > > >> > scheduled
>> > > > >> > > > for
>> > > > >> > > > > > > > >> deletion,
>> > > > >> > > > > > > > >> > as
>> > > > >> > > > > > > > >> > > > > well.
>> > > > >> > > > > > > > >> > > > > > >> > (Maybe
>> > > > >> > > > > > > > >> > > > > > >> > > > > this
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> was
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > assumed, but
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > it would
>> be
>> > > good
>> > > > >> to
>> > > > >> > > > mention
>> > > > >> > > > > > > it).
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > I
>> feel
>> > > like
>> > > > >> there
>> > > > >> > > > are a
>> > > > >> > > > > > few
>> > > > >> > > > > > > > >> > sections
>> > > > >> > > > > > > > >> > > > > that
>> > > > >> > > > > > > > >> > > > > > >> > should
>> > > > >> > > > > > > > >> > > > > > >> > > be
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> moved to
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > "rejected
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
>> > alternatives."
>> > > > >> For
>> > > > >> > > > > example,
>> > > > >> > > > > > in
>> > > > >> > > > > > > > the
>> > > > >> > > > > > > > >> > > > > > DeleteTopics
>> > > > >> > > > > > > > >> > > > > > >> > > > section,
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> since
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > we're
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > not
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > going to
>> do
>> > > > >> option 1
>> > > > >> > or
>> > > > >> > > > 2,
>> > > > >> > > > > > > these
>> > > > >> > > > > > > > >> > should
>> > > > >> > > > > > > > >> > > be
>> > > > >> > > > > > > > >> > > > > > moved
>> > > > >> > > > > > > > >> > > > > > >> > into
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> "rejected
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
>> > alternatives,"
>> > > > >> > rather
>> > > > >> > > > than
>> > > > >> > > > > > > > >> appearing
>> > > > >> > > > > > > > >> > > > > inline.
>> > > > >> > > > > > > > >> > > > > > >> > > Another
>> > > > >> > > > > > > > >> > > > > > >> > > > > case
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> is
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > the
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > "Should
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > we remove
>> > > topic
>> > > > >> name
>> > > > >> > > from
>> > > > >> > > > > the
>> > > > >> > > > > > > > >> protocol
>> > > > >> > > > > > > > >> > > > where
>> > > > >> > > > > > > > >> > > > > > >> > > possible"
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> section.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > This
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > is
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > clearly
>> > > > >> discussing a
>> > > > >> > > > design
>> > > > >> > > > > > > > >> > alternative
>> > > > >> > > > > > > > >> > > > that
>> > > > >> > > > > > > > >> > > > > > >> we're
>> > > > >> > > > > > > > >> > > > > > >> > > not
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> proposing
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
>> implement:
>> > > > >> removing
>> > > > >> > the
>> > > > >> > > > > topic
>> > > > >> > > > > > > > name
>> > > > >> > > > > > > > >> > from
>> > > > >> > > > > > > > >> > > > > those
>> > > > >> > > > > > > > >> > > > > > >> > > > protocols.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Is it
>> > > really
>> > > > >> > > > necessary
>> > > > >> > > > > to
>> > > > >> > > > > > > > have
>> > > > >> > > > > > > > >> a
>> > > > >> > > > > > > > >> > new
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
>> > > /admin/delete_topics_by_id
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > path
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > in
>> > ZooKeeper?
>> > > > It
>> > > > >> > seems
>> > > > >> > > > > like
>> > > > >> > > > > > we
>> > > > >> > > > > > > > >> don't
>> > > > >> > > > > > > > >> > > > really
>> > > > >> > > > > > > > >> > > > > > >> need
>> > > > >> > > > > > > > >> > > > > > >> > > this.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Whenever
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > there is
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > a new
>> > > > controller,
>> > > > >> > we'll
>> > > > >> > > > > send
>> > > > >> > > > > > > out
>> > > > >> > > > > > > > >> full
>> > > > >> > > > > > > > >> > > > > > >> > > > > LeaderAndIsrRequests
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> which
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > will
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > trigger
>> the
>> > > > stale
>> > > > >> > > topics
>> > > > >> > > > to
>> > > > >> > > > > > be
>> > > > >> > > > > > > > >> cleaned
>> > > > >> > > > > > > > >> > > up.
>> > > > >> > > > > > > > >> > > > > >  The
>> > > > >> > > > > > > > >> > > > > > >> > > active
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > controller
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > will
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > also send
>> > the
>> > > > full
>> > > > >> > > > > > > > >> LeaderAndIsrRequest
>> > > > >> > > > > > > > >> > > to
>> > > > >> > > > > > > > >> > > > > > >> brokers
>> > > > >> > > > > > > > >> > > > > > >> > > that
>> > > > >> > > > > > > > >> > > > > > >> > > > > are
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> just
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > starting
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > up.
>> So we
>> > > > don't
>> > > > >> > > really
>> > > > >> > > > > > need
>> > > > >> > > > > > > > this
>> > > > >> > > > > > > > >> > kind
>> > > > >> > > > > > > > >> > > > of
>> > > > >> > > > > > > > >> > > > > > >> > two-phase
>> > > > >> > > > > > > > >> > > > > > >> > > > > > commit
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > (send
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > out
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
>> > > > >> StopReplicasRequest,
>> > > > >> > > get
>> > > > >> > > > > ACKs
>> > > > >> > > > > > > > from
>> > > > >> > > > > > > > >> all
>> > > > >> > > > > > > > >> > > > > nodes,
>> > > > >> > > > > > > > >> > > > > > >> > commit
>> > > > >> > > > > > > > >> > > > > > >> > > by
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> removing
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
>> > > > >> /admin/delete_topics
>> > > > >> > > > node)
>> > > > >> > > > > > any
>> > > > >> > > > > > > > >> more.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > You
>> > > mention
>> > > > >> that
>> > > > >> > > > > > > FetchRequest
>> > > > >> > > > > > > > >> will
>> > > > >> > > > > > > > >> > > now
>> > > > >> > > > > > > > >> > > > > > >> include
>> > > > >> > > > > > > > >> > > > > > >> > > UUID
>> > > > >> > > > > > > > >> > > > > > >> > > > > to
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> avoid
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > issues
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > where
>> > requests
>> > > > are
>> > > > >> > made
>> > > > >> > > > to
>> > > > >> > > > > > > stale
>> > > > >> > > > > > > > >> > > > partitions.
>> > > > >> > > > > > > > >> > > > > > >> > > However,
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> adding a
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > UUID
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
>> > > MetadataRequest
>> > > > is
>> > > > >> > > listed
>> > > > >> > > > > as
>> > > > >> > > > > > > > future
>> > > > >> > > > > > > > >> > > work,
>> > > > >> > > > > > > > >> > > > > out
>> > > > >> > > > > > > > >> > > > > > of
>> > > > >> > > > > > > > >> > > > > > >> > > scope
>> > > > >> > > > > > > > >> > > > > > >> > > > > for
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> this
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > KIP.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > How
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > will the
>> > > client
>> > > > >> learn
>> > > > >> > > > what
>> > > > >> > > > > > the
>> > > > >> > > > > > > > >> topic
>> > > > >> > > > > > > > >> > > UUID
>> > > > >> > > > > > > > >> > > > > is,
>> > > > >> > > > > > > > >> > > > > > if
>> > > > >> > > > > > > > >> > > > > > >> > the
>> > > > >> > > > > > > > >> > > > > > >> > > > > > metadata
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > response
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > doesn't
>> > > include
>> > > > >> that
>> > > > >> > > > > > > information?
>> > > > >> > > > > > > > >> It
>> > > > >> > > > > > > > >> > > > seems
>> > > > >> > > > > > > > >> > > > > > like
>> > > > >> > > > > > > > >> > > > > > >> > > adding
>> > > > >> > > > > > > > >> > > > > > >> > > > > the
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> UUID
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
>> > > MetadataResponse
>> > > > >> > would
>> > > > >> > > be
>> > > > >> > > > > an
>> > > > >> > > > > > > > >> > improvement
>> > > > >> > > > > > > > >> > > > > here
>> > > > >> > > > > > > > >> > > > > > >> that
>> > > > >> > > > > > > > >> > > > > > >> > > > might
>> > > > >> > > > > > > > >> > > > > > >> > > > > > not
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> be
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > too
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > hard to
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > make.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > best,
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Colin
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > On
>> Mon,
>> > > Sep
>> > > > 9,
>> > > > >> > > 2019,
>> > > > >> > > > at
>> > > > >> > > > > > > > 17:48,
>> > > > >> > > > > > > > >> > > Ryanne
>> > > > >> > > > > > > > >> > > > > > Dolan
>> > > > >> > > > > > > > >> > > > > > >> > > wrote:
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
>> Lucas,
>> > > > this
>> > > > >> > would
>> > > > >> > > > be
>> > > > >> > > > > > > great.
>> > > > >> > > > > > > > >> I've
>> > > > >> > > > > > > > >> > > run
>> > > > >> > > > > > > > >> > > > > > into
>> > > > >> > > > > > > > >> > > > > > >> > > issues
>> > > > >> > > > > > > > >> > > > > > >> > > > > with
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > topics
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > being
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
>> > > > resurrected
>> > > > >> > > > > > accidentally,
>> > > > >> > > > > > > > >> since
>> > > > >> > > > > > > > >> > a
>> > > > >> > > > > > > > >> > > > > client
>> > > > >> > > > > > > > >> > > > > > >> > cannot
>> > > > >> > > > > > > > >> > > > > > >> > > > > > easily
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > distinguish
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > between
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > a
>> > > deleted
>> > > > >> topic
>> > > > >> > > > and a
>> > > > >> > > > > > new
>> > > > >> > > > > > > > >> topic
>> > > > >> > > > > > > > >> > > with
>> > > > >> > > > > > > > >> > > > > the
>> > > > >> > > > > > > > >> > > > > > >> same
>> > > > >> > > > > > > > >> > > > > > >> > > > name.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > I'd
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > need
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > the
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > ID
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
>> > > accessible
>> > > > >> from
>> > > > >> > > the
>> > > > >> > > > > > > client
>> > > > >> > > > > > > > to
>> > > > >> > > > > > > > >> > > solve
>> > > > >> > > > > > > > >> > > > > that
>> > > > >> > > > > > > > >> > > > > > >> > issue,
>> > > > >> > > > > > > > >> > > > > > >> > > > but
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> this
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > is a
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > good
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > first
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
>> step.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
>> Ryanne
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > On
>> > Wed,
>> > > > Sep
>> > > > >> 4,
>> > > > >> > > 2019
>> > > > >> > > > > at
>> > > > >> > > > > > > 1:41
>> > > > >> > > > > > > > >> PM
>> > > > >> > > > > > > > >> > > Lucas
>> > > > >> > > > > > > > >> > > > > > >> > > Bradstreet <
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
>> > > lucas@confluent.io>
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > wrote:
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
>> Hi
>> > > all,
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > I
>> > > would
>> > > > >> like
>> > > > >> > to
>> > > > >> > > > > kick
>> > > > >> > > > > > > off
>> > > > >> > > > > > > > >> > > > discussion
>> > > > >> > > > > > > > >> > > > > of
>> > > > >> > > > > > > > >> > > > > > >> > > KIP-516,
>> > > > >> > > > > > > > >> > > > > > >> > > > > an
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
>> implementation
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > of topic
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
>> IDs
>> > > for
>> > > > >> > Kafka.
>> > > > >> > > > > Topic
>> > > > >> > > > > > > IDs
>> > > > >> > > > > > > > >> aim
>> > > > >> > > > > > > > >> > to
>> > > > >> > > > > > > > >> > > > > solve
>> > > > >> > > > > > > > >> > > > > > >> topic
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> uniqueness
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > problems in
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Kafka,
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
>> > where
>> > > > >> > referring
>> > > > >> > > > to
>> > > > >> > > > > a
>> > > > >> > > > > > > > topic
>> > > > >> > > > > > > > >> by
>> > > > >> > > > > > > > >> > > name
>> > > > >> > > > > > > > >> > > > > > >> alone is
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> insufficient.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Such
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > cases
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
>> > > include
>> > > > >> when
>> > > > >> > a
>> > > > >> > > > > topic
>> > > > >> > > > > > > has
>> > > > >> > > > > > > > >> been
>> > > > >> > > > > > > > >> > > > > deleted
>> > > > >> > > > > > > > >> > > > > > >> and
>> > > > >> > > > > > > > >> > > > > > >> > > > > recreated
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> with
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > the
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > same
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > name.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
>> > Unique
>> > > > >> > > > identifiers
>> > > > >> > > > > > will
>> > > > >> > > > > > > > >> help
>> > > > >> > > > > > > > >> > > > > simplify
>> > > > >> > > > > > > > >> > > > > > >> and
>> > > > >> > > > > > > > >> > > > > > >> > > > improve
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Kafka's
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > topic
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > deletion
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
>> > > process,
>> > > > >> as
>> > > > >> > > well
>> > > > >> > > > as
>> > > > >> > > > > > > > prevent
>> > > > >> > > > > > > > >> > > cases
>> > > > >> > > > > > > > >> > > > > > where
>> > > > >> > > > > > > > >> > > > > > >> > > brokers
>> > > > >> > > > > > > > >> > > > > > >> > > > > may
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > incorrectly
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > interact
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
>> with
>> > > > stale
>> > > > >> > > > versions
>> > > > >> > > > > > of
>> > > > >> > > > > > > > >> topics.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > >
>> > > > >> > > > > > > > >> > > > > > >> > >
>> > > > >> > > > > > > > >> > > > > > >> >
>> > > > >> > > > > > > > >> > > > > > >>
>> > > > >> > > > > > > > >> > > > > >
>> > > > >> > > > > > > > >> > > > >
>> > > > >> > > > > > > > >> > > >
>> > > > >> > > > > > > > >> > >
>> > > > >> > > > > > > > >> >
>> > > > >> > > > > > > > >>
>> > > > >> > > > > > > >
>> > > > >> > > > > > >
>> > > > >> > > > > >
>> > > > >> > > > >
>> > > > >> > > >
>> > > > >> > >
>> > > > >> >
>> > > > >>
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
>> > > Looking
>> > > > >> > forward
>> > > > >> > > > to
>> > > > >> > > > > > your
>> > > > >> > > > > > > > >> > > thoughts.
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
>> > Lucas
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
>> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
>> > > > >> > > > > > > > >> > > > > > >> > > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > > >
>> > > > >> > > > > > > > >> > > > > > >> > > >
>> > > > >> > > > > > > > >> > > > > > >> > >
>> > > > >> > > > > > > > >> > > > > > >> >
>> > > > >> > > > > > > > >> > > > > > >>
>> > > > >> > > > > > > > >> > > > > > >
>> > > > >> > > > > > > > >> > > > > >
>> > > > >> > > > > > > > >> > > > >
>> > > > >> > > > > > > > >> > > >
>> > > > >> > > > > > > > >> > >
>> > > > >> > > > > > > > >> >
>> > > > >> > > > > > > > >>
>> > > > >> > > > > > > > >
>> > > > >> > > > > > > >
>> > > > >> > > > > > >
>> > > > >> > > > > >
>> > > > >> > > > >
>> > > > >> > > >
>> > > > >> > >
>> > > > >> >
>> > > > >>
>> > > > >
>> > > >
>> > >
>> >
>>
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Justine Olshan <jo...@confluent.io>]

Hi Jun,
Thanks for looking at it again. Here's the new spec. (I fixed the typo in
it too.)
{"name": "id", "type": "string", "doc": option[UUID]}

Justine


On Thu, Oct 1, 2020 at 5:03 PM Jun Rao <ju...@confluent.io> wrote:

> Hi, Justine,
>
> Thanks for the update. The KIP looks good to me now. Just a minor comment
> below.
>
> 30. Perhaps "option[UUID]" can be put in the doc.
>
> Jun
>
> On Thu, Oct 1, 2020 at 3:28 PM Justine Olshan <jo...@confluent.io>
> wrote:
>
> > Hi Jun,
> > Thanks for the response!
> >
> > 30. I think I might have changed this in between. The current version
> > says:  {"name":
> > "id", "type": "option[UUID]"}, "doc": topic id}
> > I have switched to the option type to cover the migration case where a
> > TopicZNode does not yet have a topic ID.
> > I understand that due to json, this field is written as a string, so if I
> > should move the "option[uuid]" to the doc field and the type should be
> > "string" please let me know.
> >
> > 40. I've added a definition for UUID.
> > 41,42. Fixed
> >
> > Thank you,
> > Justine
> >
> > On Wed, Sep 30, 2020 at 1:15 PM Jun Rao <ju...@confluent.io> wrote:
> >
> > > Hi, Justine,
> > >
> > > Thanks for the summary. Just a few minor comments blow.
> > >
> > > 30.  {"name": "id", "type": "string", "doc": "version id"}}: The doc
> > should
> > > say UUID. The issue seems unfixed.
> > >
> > > 40. Since UUID is public facing, could you include its definition?
> > >
> > > 41. StopReplicaResponse still includes the topic field.
> > >
> > > 42. "It is unnecessary to include the name of the topic in the
> following
> > > Request/Response calls" It would be useful to include all modified
> > requests
> > > (e.g. produce) in the list below.
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > >
> > > On Wed, Sep 30, 2020 at 10:17 AM Justine Olshan <jo...@confluent.io>
> > > wrote:
> > >
> > > > Hello again,
> > > >
> > > > I've taken some time to discuss some of the remaining points brought
> up
> > > by
> > > > the previous emails offline. Here are some of the conclusions.
> > > >
> > > > 1. Directory Structure:
> > > > There was some talk about whether the directory structure should be
> > > changed
> > > > to replace all topic names with topic IDs.
> > > > This will be a useful change, but will prevent downgrades. It will be
> > > best
> > > > to wait until a major release, likely alongside KIP-500 changes that
> > will
> > > > prevent downgrades. I've updated the KIP to include this change with
> > the
> > > > note about migration and deprecation.
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-log.dirlayout
> > > >
> > > > 2. Partition Metadata file
> > > > There was some disagreement about the inclusion of the partition
> > metadata
> > > > file.
> > > > This file will be needed to persist the topic ID, especially while we
> > > still
> > > > have the old directory structure. Even after the changes, this file
> can
> > > be
> > > > useful for debugging and recovery.
> > > > Creating a single mapping file from topic names to topic IDs was
> > > > considered, but ultimately scrapped as it would not be as easy to
> > > maintain.
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PartitionMetadatafile
> > > >
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PersistingTopicIDs
> > > >
> > > > 3. Produce Protocols
> > > > After some further discussion, this replacing the topic name with
> topic
> > > ID
> > > > in these protocols has been added to the KIP.
> > > > This will cut down on the size of the protocol. Since changes to
> fetch
> > > are
> > > > included, it does make sense to update these protocols.
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-Produce
> > > >
> > > > 4. KIP-500 Compatibility
> > > > After some discussion with those involved with KIP-500, it seems best
> > to
> > > > use a sentinel topic ID for the metadata topic that is used before
> the
> > > > first controller election.
> > > > We had an issue where this metadata topic may not be assigned an ID
> > > before
> > > > utilizing the Vote and Fetch protocols. It was decided to reserve a
> > > unique
> > > > ID for this topic to be used until the controller could give the
> topic
> > a
> > > > unique ID.
> > > > Having the topic keep the sentinel ID (not be reassigned to a unique
> > ID)
> > > > was considered, but it seemed like a good idea to leave the option
> open
> > > for
> > > > the metadata topic to have a unique ID in cases where it would need
> to
> > be
> > > > differentiated from other clusters' metadata topics. (ex. tiered
> > storage)
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-CompatibilitywithKIP-500
> > > >
> > > > I've also split up the KIP into sub-tasks on the JIRA. Hopefully this
> > > will
> > > > provide a better idea about what tasks we have, and eventually
> provide
> > a
> > > > place to see what's done and what is left.
> > > > If there is a task I am missing, please let me know!
> > > > https://issues.apache.org/jira/browse/KAFKA-8872
> > > >
> > > > Of course, these decisions are not set in stone, and I would love to
> > hear
> > > > any feedback.
> > > >
> > > > Thanks,
> > > > Justine
> > > >
> > > > On Mon, Sep 28, 2020 at 11:38 AM Justine Olshan <
> jolshan@confluent.io>
> > > > wrote:
> > > >
> > > > > Hello all,
> > > > >
> > > > > I just wanted to follow up on this discussion. Did we come to an
> > > > > understanding about the directory structure?
> > > > >
> > > > > I think the biggest question here is what is acceptable to leave
> out
> > > due
> > > > > to scope vs. what is considered to be too much tech debt.
> > > > > This KIP is already pretty large with the number of changes, but it
> > > also
> > > > > makes sense to do things correctly, so I'd love to hear everyone's
> > > > thoughts.
> > > > >
> > > > > Thanks,
> > > > > Justine
> > > > >
> > > > > On Fri, Sep 25, 2020 at 8:19 AM Lucas Bradstreet <
> lucas@confluent.io
> > >
> > > > > wrote:
> > > > >
> > > > >> Hi Ismael,
> > > > >>
> > > > >> If you do not store it in a metadata file or in the directory
> > > structure
> > > > >> would you then
> > > > >> require the LeaderAndIsrRequest following startup to give you some
> > > > notion
> > > > >> of
> > > > >> topic name in memory? We would still need this information for the
> > > older
> > > > >> protocols, but
> > > > >> perhaps this is what's meant by tech debt.
> > > > >>
> > > > >> Once we're free of the old non-topicID requests then I think you
> > > > wouldn't
> > > > >> need to retain the topic name.
> > > > >> I think the ability to easily look up topic names associated with
> > > > >> partition
> > > > >> directories would still be missed
> > > > >> when diagnosing issues, though maybe it wouldn't be a deal breaker
> > > with
> > > > >> the
> > > > >> right tooling.
> > > > >>
> > > > >> Thanks,
> > > > >>
> > > > >> Lucas
> > > > >>
> > > > >> On Fri, Sep 25, 2020 at 7:55 AM Ismael Juma <is...@juma.me.uk>
> > > wrote:
> > > > >>
> > > > >> > Hi Lucas,
> > > > >> >
> > > > >> > Why would you include the name and id? I think you'd want to
> > remove
> > > > the
> > > > >> > name from the directory name right? Jason's suggestion was that
> if
> > > you
> > > > >> > remove the name from the directory, then why would you need the
> id
> > > > name
> > > > >> > mapping file?
> > > > >> >
> > > > >> > Ismael
> > > > >> >
> > > > >> > On Thu, Sep 24, 2020 at 4:24 PM Lucas Bradstreet <
> > > lucas@confluent.io>
> > > > >> > wrote:
> > > > >> >
> > > > >> > > > 2. Part of the usage of the file is to have persistent
> storage
> > > of
> > > > >> the
> > > > >> > > topic
> > > > >> > > ID and use it to compare with the ID supplied in the
> > LeaderAndIsr
> > > > >> > Request.
> > > > >> > > There is some discussion in the KIP about changes to the
> > directory
> > > > >> > > structure, but I believe directory changes were considered to
> be
> > > out
> > > > >> of
> > > > >> > > scope when the KIP was written.
> > > > >> > >
> > > > >> > >
> > > > >> > > Yeah, I was hoping to get a better understanding of why it was
> > > taken
> > > > >> out
> > > > >> > of
> > > > >> > > scope. Perhaps Lucas Bradstreet might have more insight about
> > the
> > > > >> > decision.
> > > > >> > > Basically my point is that we have to create additional
> > > > infrastructure
> > > > >> > here
> > > > >> > > to support the name/id mapping, so I wanted to understand if
> we
> > > just
> > > > >> > > consider this a sort of tech debt. It would be useful to cover
> > how
> > > > we
> > > > >> > > handle the case when this file gets corrupted. Seems like we
> > just
> > > > >> have to
> > > > >> > > trust that it matches whatever the controller tells us and
> > rewrite
> > > > it?
> > > > >> > >
> > > > >> > >
> > > > >> > > Hi Jason, Justine,
> > > > >> > >
> > > > >> > > My thought process is that we will likely need the metadata
> file
> > > > >> whether
> > > > >> > we
> > > > >> > > rename the directories or not.
> > > > >> > > Linux supports filenames of up to 255 bytes and that would not
> > be
> > > > >> enough
> > > > >> > to
> > > > >> > > support a directory name
> > > > >> > >  including both the name and topic ID. Given that fact, it
> > seemed
> > > > >> > > reasonable to add the metadata file
> > > > >> > > and leave the directory rename until some time in the future
> > > > (possibly
> > > > >> > > never).
> > > > >> > >
> > > > >> > > If the file does get corrupted I think you're right. We would
> > > either
> > > > >> have
> > > > >> > > to trust it matches what the controller tells us
> > > > >> > >  or error out and let an administrator resolve it by checking
> > > across
> > > > >> > > replicas for consistency.
> > > > >> > >
> > > > >> > > Lucas
> > > > >> > >
> > > > >> > >
> > > > >> > > On Thu, Sep 24, 2020 at 3:41 PM Jason Gustafson <
> > > jason@confluent.io
> > > > >
> > > > >> > > wrote:
> > > > >> > >
> > > > >> > > > Thanks Justine. Responses below:
> > > > >> > > >
> > > > >> > > > > 1. Yes, the directory will still be based on the topic
> > names.
> > > > >> > > > LeaderAndIsrRequest is one of the few requests that will
> still
> > > > >> contain
> > > > >> > > the
> > > > >> > > > topic name. So I think we have this covered. Sorry for
> > > confusion.
> > > > >> > > >
> > > > >> > > > Ah, you're right. My eyes passed right over the field.
> > > > >> > > >
> > > > >> > > > > 2. Part of the usage of the file is to have persistent
> > storage
> > > > of
> > > > >> the
> > > > >> > > > topic
> > > > >> > > > ID and use it to compare with the ID supplied in the
> > > LeaderAndIsr
> > > > >> > > Request.
> > > > >> > > > There is some discussion in the KIP about changes to the
> > > directory
> > > > >> > > > structure, but I believe directory changes were considered
> to
> > be
> > > > >> out of
> > > > >> > > > scope when the KIP was written.
> > > > >> > > >
> > > > >> > > > Yeah, I was hoping to get a better understanding of why it
> was
> > > > taken
> > > > >> > out
> > > > >> > > of
> > > > >> > > > scope. Perhaps Lucas Bradstreet might have more insight
> about
> > > the
> > > > >> > > decision.
> > > > >> > > > Basically my point is that we have to create additional
> > > > >> infrastructure
> > > > >> > > here
> > > > >> > > > to support the name/id mapping, so I wanted to understand if
> > we
> > > > just
> > > > >> > > > consider this a sort of tech debt. It would be useful to
> cover
> > > how
> > > > >> we
> > > > >> > > > handle the case when this file gets corrupted. Seems like we
> > > just
> > > > >> have
> > > > >> > to
> > > > >> > > > trust that it matches whatever the controller tells us and
> > > rewrite
> > > > >> it?
> > > > >> > > >
> > > > >> > > > > 3. I think this is a good point, but I again I wonder
> about
> > > the
> > > > >> scope
> > > > >> > > of
> > > > >> > > > the KIP. Most of the changes mentioned in the KIP are for
> > > > supporting
> > > > >> > > topic
> > > > >> > > > deletion and I believe that is why the produce request was
> > > listed
> > > > >> under
> > > > >> > > > future work.
> > > > >> > > >
> > > > >> > > > That's fair. I brought it up since `Fetch` is already
> > included.
> > > If
> > > > >> > we've
> > > > >> > > > got `Metadata` and `Fetch`, seems we may as well do
> `Produce`
> > > and
> > > > >> save
> > > > >> > an
> > > > >> > > > extra kip. No strong objection though if you want to leave
> it
> > > out.
> > > > >> > > >
> > > > >> > > >
> > > > >> > > > -Jason
> > > > >> > > >
> > > > >> > > >
> > > > >> > > > On Thu, Sep 24, 2020 at 3:26 PM Justine Olshan <
> > > > >> jolshan@confluent.io>
> > > > >> > > > wrote:
> > > > >> > > >
> > > > >> > > > > Hi Jason,
> > > > >> > > > >
> > > > >> > > > > Thanks for your comments.
> > > > >> > > > >
> > > > >> > > > > 1. Yes, the directory will still be based on the topic
> > names.
> > > > >> > > > > LeaderAndIsrRequest is one of the few requests that will
> > still
> > > > >> > contain
> > > > >> > > > the
> > > > >> > > > > topic name. So I think we have this covered. Sorry for
> > > > confusion.
> > > > >> > > > >
> > > > >> > > > > 2. Part of the usage of the file is to have persistent
> > storage
> > > > of
> > > > >> the
> > > > >> > > > topic
> > > > >> > > > > ID and use it to compare with the ID supplied in the
> > > > LeaderAndIsr
> > > > >> > > > Request.
> > > > >> > > > > There is some discussion in the KIP about changes to the
> > > > directory
> > > > >> > > > > structure, but I believe directory changes were considered
> > to
> > > be
> > > > >> out
> > > > >> > of
> > > > >> > > > > scope when the KIP was written.
> > > > >> > > > >
> > > > >> > > > > 3. I think this is a good point, but I again I wonder
> about
> > > the
> > > > >> scope
> > > > >> > > of
> > > > >> > > > > the KIP. Most of the changes mentioned in the KIP are for
> > > > >> supporting
> > > > >> > > > topic
> > > > >> > > > > deletion and I believe that is why the produce request was
> > > > listed
> > > > >> > under
> > > > >> > > > > future work.
> > > > >> > > > >
> > > > >> > > > > 4. This sounds like it might be a good solution, but I
> will
> > > need
> > > > >> to
> > > > >> > > > discuss
> > > > >> > > > > more with KIP-500 folks to get the details right.
> > > > >> > > > >
> > > > >> > > > > Thanks,
> > > > >> > > > > Justine
> > > > >> > > > >
> > > > >> > > > > On Thu, Sep 24, 2020 at 12:30 PM Jason Gustafson <
> > > > >> jason@confluent.io
> > > > >> > >
> > > > >> > > > > wrote:
> > > > >> > > > >
> > > > >> > > > > > Hi Justine,
> > > > >> > > > > >
> > > > >> > > > > > Thanks for picking up this work. I have a few
> > > > >> questions/comments:
> > > > >> > > > > >
> > > > >> > > > > > 1. It sounds like the directory structure is still going
> > to
> > > be
> > > > >> > based
> > > > >> > > on
> > > > >> > > > > > topic names. Do I have that right? One complication is
> > that
> > > > the
> > > > >> > > > > > LeaderAndIsr request does not include the topic name any
> > > > longer.
> > > > >> > This
> > > > >> > > > > means
> > > > >> > > > > > that a replica must wait for the UpdateMetadata request
> to
> > > > >> arrive
> > > > >> > > with
> > > > >> > > > > the
> > > > >> > > > > > topic name mapping before it can create the directory. I
> > > > wonder
> > > > >> if
> > > > >> > it
> > > > >> > > > > would
> > > > >> > > > > > be simpler to avoid assumptions on the ordering of
> > > > >> UpdateMetadata
> > > > >> > and
> > > > >> > > > let
> > > > >> > > > > > LeaderAndIsr include the topic name as well. Feels like
> we
> > > are
> > > > >> not
> > > > >> > > > saving
> > > > >> > > > > > that much by excluding it.
> > > > >> > > > > >
> > > > >> > > > > > 2. On a related note, it seems that the reason we have
> the
> > > > >> > partition
> > > > >> > > > > > metadata file is because we are not changing the
> directory
> > > > >> > structure.
> > > > >> > > > We
> > > > >> > > > > > need it so that we remember which directories map to
> which
> > > > topic
> > > > >> > id.
> > > > >> > > I
> > > > >> > > > > > think it would be useful to add some detail about why
> > > changing
> > > > >> the
> > > > >> > > > > > directory layout was rejected. Basically trying to
> > > understand
> > > > >> how
> > > > >> > > much
> > > > >> > > > > > effort we are saving by skipping this step if we have to
> > > > >> implement
> > > > >> > > this
> > > > >> > > > > > additional bookkeeping. One concern I have for example
> is
> > > that
> > > > >> the
> > > > >> > > > > > partition metadata file gets corrupt and then all bets
> are
> > > off
> > > > >> as
> > > > >> > far
> > > > >> > > > as
> > > > >> > > > > > topic consistency.
> > > > >> > > > > >
> > > > >> > > > > > 3. Is it worth adding support now for topic ids in the
> > > > `Produce`
> > > > >> > > > request
> > > > >> > > > > > now? Overhead is mentioned as one of the motivations and
> > the
> > > > >> best
> > > > >> > > APIs
> > > > >> > > > to
> > > > >> > > > > > get savings from are `Produce` and `Fetch`.
> > > > >> > > > > >
> > > > >> > > > > > 4. When it comes to bootstrapping the metadata topic
> with
> > > > >> respect
> > > > >> > to
> > > > >> > > > > > KIP-500, one suggestion would be to reserve a sentinel
> > topic
> > > > ID
> > > > >> > which
> > > > >> > > > can
> > > > >> > > > > > be used initially by new brokers. When the first
> > controller
> > > is
> > > > >> > > elected,
> > > > >> > > > > it
> > > > >> > > > > > can assign a topicId to the metadata topic. This is a
> bit
> > > > >> similar
> > > > >> > to
> > > > >> > > > how
> > > > >> > > > > we
> > > > >> > > > > > were planning to generate the clusterId.
> > > > >> > > > > >
> > > > >> > > > > > Thanks,
> > > > >> > > > > > Jason
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > > > On Thu, Sep 24, 2020 at 11:10 AM Jun Rao <
> > jun@confluent.io>
> > > > >> wrote:
> > > > >> > > > > >
> > > > >> > > > > > > Hi, Justine,
> > > > >> > > > > > >
> > > > >> > > > > > > Thanks for the updated KIP. A few more comments below.
> > > > >> > > > > > >
> > > > >> > > > > > > 30.  {"name": "id", "type": "string", "doc": "version
> > > id"}}:
> > > > >> The
> > > > >> > > doc
> > > > >> > > > > > should
> > > > >> > > > > > > say UUID.
> > > > >> > > > > > >
> > > > >> > > > > > > 31. LeaderAndIsrResponse v5 and StopReplicaResponse
> v4 :
> > > It
> > > > >> seems
> > > > >> > > > there
> > > > >> > > > > > is
> > > > >> > > > > > > no need to add topic_id at partitions level.
> > > > >> > > > > > >
> > > > >> > > > > > > 32. Regarding partition metadata file. Perhaps the key
> > can
> > > > be
> > > > >> a
> > > > >> > > > single
> > > > >> > > > > > > word, sth like the following.
> > > > >> > > > > > > version: 0
> > > > >> > > > > > > topic_id: 46bdb63f-9e8d-4a38-bf7b-ee4eb2a794e4
> > > > >> > > > > > >
> > > > >> > > > > > > 33. Another tricky thing that I realized is how to
> > support
> > > > the
> > > > >> > > > metadata
> > > > >> > > > > > > topic introduced in KIP-595. It's a bit tricky to
> > assign a
> > > > >> UUID
> > > > >> > to
> > > > >> > > > the
> > > > >> > > > > > > metadata topic since we have a chicken-and-egg
> problem.
> > > The
> > > > >> > > > controller
> > > > >> > > > > > > needs to persist the UUID in the metadata topic in
> order
> > > to
> > > > >> > assign
> > > > >> > > > one
> > > > >> > > > > > > successfully, but the metadata topic is needed to
> elect
> > a
> > > > >> > > controller
> > > > >> > > > > > > (KIP-631). So, this probably needs a bit more thought.
> > > > >> > > > > > >
> > > > >> > > > > > > Jun
> > > > >> > > > > > >
> > > > >> > > > > > > On Thu, Sep 24, 2020 at 3:04 AM Ismael Juma <
> > > > >> ismael@juma.me.uk>
> > > > >> > > > wrote:
> > > > >> > > > > > >
> > > > >> > > > > > > > Also, can we provide more details on how the
> Partition
> > > > >> Metadata
> > > > >> > > > file
> > > > >> > > > > > will
> > > > >> > > > > > > > be used?
> > > > >> > > > > > > >
> > > > >> > > > > > > > Ismael
> > > > >> > > > > > > >
> > > > >> > > > > > > > On Thu, Sep 24, 2020 at 3:01 AM Ismael Juma <
> > > > >> ismael@juma.me.uk
> > > > >> > >
> > > > >> > > > > wrote:
> > > > >> > > > > > > >
> > > > >> > > > > > > > > Hi Justine,
> > > > >> > > > > > > > >
> > > > >> > > > > > > > > I think we need to update the "Rejected
> > Alternatives"
> > > > >> section
> > > > >> > > to
> > > > >> > > > > take
> > > > >> > > > > > > > into
> > > > >> > > > > > > > > account that the proposal now removes the topic
> name
> > > > from
> > > > >> the
> > > > >> > > > fetch
> > > > >> > > > > > > > > request. Also, if we are removing it from the
> Fetch
> > > > >> request,
> > > > >> > > does
> > > > >> > > > > it
> > > > >> > > > > > > make
> > > > >> > > > > > > > > sense not to remove it from similar requests like
> > > > >> > > > > ListOffsetRequest?
> > > > >> > > > > > > > >
> > > > >> > > > > > > > > Ismael
> > > > >> > > > > > > > >
> > > > >> > > > > > > > > On Thu, Sep 24, 2020 at 2:46 AM David Jacot <
> > > > >> > > djacot@confluent.io
> > > > >> > > > >
> > > > >> > > > > > > wrote:
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >> Hi Justine,
> > > > >> > > > > > > > >>
> > > > >> > > > > > > > >> Thanks for the KIP. I finally had time to read it
> > :).
> > > > It
> > > > >> is
> > > > >> > a
> > > > >> > > > > great
> > > > >> > > > > > > > >> improvement.
> > > > >> > > > > > > > >>
> > > > >> > > > > > > > >> I have a few comments/questions:
> > > > >> > > > > > > > >>
> > > > >> > > > > > > > >> 1. It seems that the schema of the
> > StopReplicaRequest
> > > > is
> > > > >> > > > slightly
> > > > >> > > > > > > > >> outdated.
> > > > >> > > > > > > > >> We
> > > > >> > > > > > > > >> did some changes as part of KIP-570. V3 is
> already
> > > > >> organized
> > > > >> > > by
> > > > >> > > > > > > topics.
> > > > >> > > > > > > > >>
> > > > >> > > > > > > > >> 2. I just want to make sure that I understand the
> > > > >> > > reconciliation
> > > > >> > > > > > > > >> logic correctly. When
> > > > >> > > > > > > > >> an "INCREMENTAL" LeaderAndIsr Request is
> received,
> > > the
> > > > >> > broker
> > > > >> > > > will
> > > > >> > > > > > > also
> > > > >> > > > > > > > >> reconcile
> > > > >> > > > > > > > >> when the local uuid does not match the uuid in
> the
> > > > >> request,
> > > > >> > > > right?
> > > > >> > > > > > In
> > > > >> > > > > > > > this
> > > > >> > > > > > > > >> case, the
> > > > >> > > > > > > > >> local log is staged for deletion.
> > > > >> > > > > > > > >>
> > > > >> > > > > > > > >> 3. In the documentation of the `
> > > > >> delete.stale.topic.delay.ms
> > > > >> > `
> > > > >> > > > > > config,
> > > > >> > > > > > > it
> > > > >> > > > > > > > >> says "When a
> > > > >> > > > > > > > >> FULL LeaderAndIsrRequest is received..." but I
> > > suppose
> > > > it
> > > > >> > > > applies
> > > > >> > > > > to
> > > > >> > > > > > > > both
> > > > >> > > > > > > > >> types.
> > > > >> > > > > > > > >>
> > > > >> > > > > > > > >> Best,
> > > > >> > > > > > > > >> David
> > > > >> > > > > > > > >>
> > > > >> > > > > > > > >> On Thu, Sep 24, 2020 at 1:40 AM Justine Olshan <
> > > > >> > > > > > jolshan@confluent.io>
> > > > >> > > > > > > > >> wrote:
> > > > >> > > > > > > > >>
> > > > >> > > > > > > > >> > Hi Jun,
> > > > >> > > > > > > > >> >
> > > > >> > > > > > > > >> > Thanks for the comments. I apologize for some
> of
> > > the
> > > > >> typos
> > > > >> > > and
> > > > >> > > > > > > > >> confusion.
> > > > >> > > > > > > > >> > I’ve updated the KIP to fix some of the issues
> > you
> > > > >> > > mentioned.
> > > > >> > > > > > > > >> >
> > > > >> > > > > > > > >> > 20.2 I’ve changed the type to String.
> > > > >> > > > > > > > >> > 20.1/3 I’ve updated the TopicZNode to fix
> > > formatting
> > > > >> and
> > > > >> > > > reflect
> > > > >> > > > > > the
> > > > >> > > > > > > > >> latest
> > > > >> > > > > > > > >> > version before this change.
> > > > >> > > > > > > > >> >
> > > > >> > > > > > > > >> > 21. You are correct and I’ve removed this line.
> > > I’ve
> > > > >> also
> > > > >> > > > added
> > > > >> > > > > a
> > > > >> > > > > > > line
> > > > >> > > > > > > > >> > mentioning an IBP bump is necessary for
> migration
> > > > >> > > > > > > > >> >
> > > > >> > > > > > > > >> > 22. I think the wording was unclear but your
> > > summary
> > > > is
> > > > >> > what
> > > > >> > > > was
> > > > >> > > > > > > > >> intended
> > > > >> > > > > > > > >> > by this line. I’ve updated to make this point
> > more
> > > > >> clear.
> > > > >> > > > > “Remove
> > > > >> > > > > > > > >> deleted
> > > > >> > > > > > > > >> > topics from replicas by sending
> > StopReplicaRequest
> > > V3
> > > > >> > before
> > > > >> > > > the
> > > > >> > > > > > IBP
> > > > >> > > > > > > > >> bump
> > > > >> > > > > > > > >> > using the old logic, and using V4 and the new
> > logic
> > > > >> with
> > > > >> > > topic
> > > > >> > > > > IDs
> > > > >> > > > > > > > after
> > > > >> > > > > > > > >> > the IBP bump.”
> > > > >> > > > > > > > >> >
> > > > >> > > > > > > > >> > 23. I’ve removed the unspecified type from the
> > KIP
> > > > and
> > > > >> > > mention
> > > > >> > > > > > that
> > > > >> > > > > > > > IBP
> > > > >> > > > > > > > >> > will be used to handle this request. “IBP will
> be
> > > > used
> > > > >> to
> > > > >> > > > > > determine
> > > > >> > > > > > > > >> whether
> > > > >> > > > > > > > >> > this new form of the request will be used. For
> > > older
> > > > >> > > requests,
> > > > >> > > > > we
> > > > >> > > > > > > will
> > > > >> > > > > > > > >> > ignore this field and default to previous
> > > behavior.”
> > > > >> > > > > > > > >> >
> > > > >> > > > > > > > >> > 24. I’ve fixed this typo.
> > > > >> > > > > > > > >> >
> > > > >> > > > > > > > >> > 25. I've added a topics at a higher level for
> > > > >> > > > > LeaderAndIsrResponse
> > > > >> > > > > > > v5,
> > > > >> > > > > > > > >> > StopReplicaResponse v4. I've also changed
> > > > >> > StopReplicaRequest
> > > > >> > > > v4
> > > > >> > > > > to
> > > > >> > > > > > > > have
> > > > >> > > > > > > > >> > topics at a higher level.
> > > > >> > > > > > > > >> >
> > > > >> > > > > > > > >> > 26. I’ve updated forgotten_topics_data--added
> the
> > > > >> topic ID
> > > > >> > > and
> > > > >> > > > > > > removed
> > > > >> > > > > > > > >> the
> > > > >> > > > > > > > >> > topic name
> > > > >> > > > > > > > >> >
> > > > >> > > > > > > > >> > 27. I’ve decided on plain text, and I’ve added
> an
> > > > >> example.
> > > > >> > > > > > > > >> >
> > > > >> > > > > > > > >> > 28. I’ve added this idea to future work.
> > > > >> > > > > > > > >> >
> > > > >> > > > > > > > >> > Thanks again for taking a look,
> > > > >> > > > > > > > >> >
> > > > >> > > > > > > > >> > Justine
> > > > >> > > > > > > > >> >
> > > > >> > > > > > > > >> > On Wed, Sep 23, 2020 at 10:28 AM Jun Rao <
> > > > >> > jun@confluent.io>
> > > > >> > > > > > wrote:
> > > > >> > > > > > > > >> >
> > > > >> > > > > > > > >> > > Hi, Justine,
> > > > >> > > > > > > > >> > >
> > > > >> > > > > > > > >> > > Thanks for the response. Made another pass. A
> > few
> > > > >> more
> > > > >> > > > > comments
> > > > >> > > > > > > > below.
> > > > >> > > > > > > > >> > >
> > > > >> > > > > > > > >> > > 20. znode schema:
> > > > >> > > > > > > > >> > > 20.1 It seems that {"name": "version",
> "type":
> > > > "int",
> > > > >> > > "id":
> > > > >> > > > > > > "UUID",
> > > > >> > > > > > > > >> > "doc":
> > > > >> > > > > > > > >> > > "version id"} should be {"name": "version",
> > > "type":
> > > > >> > > "int"},
> > > > >> > > > > > > {"name":
> > > > >> > > > > > > > >> > "id",
> > > > >> > > > > > > > >> > > "type": "UUID", "doc": "version id"}.
> > > > >> > > > > > > > >> > > 20.2 The znode format is JSON which doesn't
> > have
> > > > UUID
> > > > >> > > type.
> > > > >> > > > So
> > > > >> > > > > > the
> > > > >> > > > > > > > >> type
> > > > >> > > > > > > > >> > > probably should be string?
> > > > >> > > > > > > > >> > > 20.3 Also, the existing format used seems
> > > outdated.
> > > > >> It
> > > > >> > > > should
> > > > >> > > > > > have
> > > > >> > > > > > > > the
> > > > >> > > > > > > > >> > > following format.
> > > > >> > > > > > > > >> > >     Json.encodeAsBytes(Map(
> > > > >> > > > > > > > >> > >       "version" -> 2,
> > > > >> > > > > > > > >> > >       "partitions" ->
> > > replicaAssignmentJson.asJava,
> > > > >> > > > > > > > >> > >       "adding_replicas" ->
> > > > >> > > > > addingReplicasAssignmentJson.asJava,
> > > > >> > > > > > > > >> > >       "removing_replicas" ->
> > > > >> > > > > > removingReplicasAssignmentJson.asJava
> > > > >> > > > > > > > >> > >     ).asJava)
> > > > >> > > > > > > > >> > >   }
> > > > >> > > > > > > > >> > >
> > > > >> > > > > > > > >> > > 21. Migration: The KIP says "The migration
> > > process
> > > > >> can
> > > > >> > > take
> > > > >> > > > > > place
> > > > >> > > > > > > > >> without
> > > > >> > > > > > > > >> > > an inter-broker protocol bump, as the format
> > > stored
> > > > >> in
> > > > >> > > > > > > > >> > > /brokers/topics/[topic] will be compatible
> with
> > > > older
> > > > >> > > broker
> > > > >> > > > > > > > >> versions."
> > > > >> > > > > > > > >> > > However, since we are bumping up the version
> of
> > > > >> > > inter-broker
> > > > >> > > > > > > > >> requests, it
> > > > >> > > > > > > > >> > > seems that we need to use IBP for migration.
> > > > >> > > > > > > > >> > >
> > > > >> > > > > > > > >> > > 22. The KIP says "Remove deleted topics from
> > > > >> replicas by
> > > > >> > > > > sending
> > > > >> > > > > > > > >> > > StopReplicaRequest V3 for any topics which do
> > not
> > > > >> > contain
> > > > >> > > a
> > > > >> > > > > > topic
> > > > >> > > > > > > > ID,
> > > > >> > > > > > > > >> and
> > > > >> > > > > > > > >> > > V4 for any topics which do contain a topic
> > ID.".
> > > > >> > However,
> > > > >> > > if
> > > > >> > > > > we
> > > > >> > > > > > > use
> > > > >> > > > > > > > >> IBP,
> > > > >> > > > > > > > >> > it
> > > > >> > > > > > > > >> > > seems that the controller will either send
> > > > >> > > > StopReplicaRequest
> > > > >> > > > > V3
> > > > >> > > > > > > > >> > > or StopReplicaRequest V4, but never mixed V3
> > and
> > > V4
> > > > >> for
> > > > >> > > > > > different
> > > > >> > > > > > > > >> topics.
> > > > >> > > > > > > > >> > > Basically, before the IBP bump, V3 will be
> > used.
> > > > >> After
> > > > >> > the
> > > > >> > > > IBP
> > > > >> > > > > > > bump,
> > > > >> > > > > > > > >> > > topicId will be created and V4 will be used.
> > > > >> > > > > > > > >> > >
> > > > >> > > > > > > > >> > > 23. Given that we depend on IBP, do we still
> > need
> > > > "0
> > > > >> > > > > > UNSPECIFIED"
> > > > >> > > > > > > > >> > > in LeaderAndIsr?
> > > > >> > > > > > > > >> > >
> > > > >> > > > > > > > >> > > 24. LeaderAndIsrResponse v5 : It still has
> the
> > > > topic
> > > > >> > > field.
> > > > >> > > > > > > > >> > >
> > > > >> > > > > > > > >> > > 25. LeaderAndIsrResponse v5,
> > StopReplicaResponse
> > > > v4:
> > > > >> > Could
> > > > >> > > > we
> > > > >> > > > > > use
> > > > >> > > > > > > > this
> > > > >> > > > > > > > >> > > opportunity to organize the response in 2
> > levels,
> > > > >> first
> > > > >> > by
> > > > >> > > > > > topic,
> > > > >> > > > > > > > >> then by
> > > > >> > > > > > > > >> > > partition, as most other requests/responses?
> > > > >> > > > > > > > >> > >
> > > > >> > > > > > > > >> > > 26. FetchRequest v13 : Should
> > > forgotten_topics_data
> > > > >> use
> > > > >> > > > > topicId
> > > > >> > > > > > > too?
> > > > >> > > > > > > > >> > >
> > > > >> > > > > > > > >> > > 27. "This file can either be plain text
> > > (key/value
> > > > >> > pairs)
> > > > >> > > or
> > > > >> > > > > > > JSON."
> > > > >> > > > > > > > >> Have
> > > > >> > > > > > > > >> > we
> > > > >> > > > > > > > >> > > decided which one to use? Also, it would be
> > > helpful
> > > > >> to
> > > > >> > > > provide
> > > > >> > > > > > an
> > > > >> > > > > > > > >> > example.
> > > > >> > > > > > > > >> > >
> > > > >> > > > > > > > >> > > 28. Future improvement: Another future
> > > improvement
> > > > >> > > > opportunity
> > > > >> > > > > > is
> > > > >> > > > > > > to
> > > > >> > > > > > > > >> use
> > > > >> > > > > > > > >> > > topicId in
> GroupMetadataManager.offsetCommitKey
> > > in
> > > > >> the
> > > > >> > > > > > > offset_commit
> > > > >> > > > > > > > >> > topic.
> > > > >> > > > > > > > >> > > This may save some space.
> > > > >> > > > > > > > >> > >
> > > > >> > > > > > > > >> > > Thanks,
> > > > >> > > > > > > > >> > >
> > > > >> > > > > > > > >> > > Jun
> > > > >> > > > > > > > >> > >
> > > > >> > > > > > > > >> > > On Wed, Sep 23, 2020 at 8:50 AM Justine
> Olshan
> > <
> > > > >> > > > > > > > jolshan@confluent.io>
> > > > >> > > > > > > > >> > > wrote:
> > > > >> > > > > > > > >> > >
> > > > >> > > > > > > > >> > > > Hi Tom,
> > > > >> > > > > > > > >> > > >
> > > > >> > > > > > > > >> > > > Thanks for the comment. I think this is a
> > > really
> > > > >> good
> > > > >> > > idea
> > > > >> > > > > and
> > > > >> > > > > > > it
> > > > >> > > > > > > > >> has
> > > > >> > > > > > > > >> > > been
> > > > >> > > > > > > > >> > > > added to the KIP under the newly added
> > tooling
> > > > >> > section.
> > > > >> > > > > > > > >> > > >
> > > > >> > > > > > > > >> > > > Thanks again,
> > > > >> > > > > > > > >> > > > Justine
> > > > >> > > > > > > > >> > > >
> > > > >> > > > > > > > >> > > > On Wed, Sep 23, 2020 at 3:17 AM Tom
> Bentley <
> > > > >> > > > > > > tbentley@redhat.com>
> > > > >> > > > > > > > >> > wrote:
> > > > >> > > > > > > > >> > > >
> > > > >> > > > > > > > >> > > > > Hi Justine,
> > > > >> > > > > > > > >> > > > >
> > > > >> > > > > > > > >> > > > > I know you started the vote thread, but
> on
> > > > >> > re-reading
> > > > >> > > > the
> > > > >> > > > > > KIP
> > > > >> > > > > > > I
> > > > >> > > > > > > > >> > noticed
> > > > >> > > > > > > > >> > > > > that although the topic id is included in
> > the
> > > > >> > > > > > MetadataResponse
> > > > >> > > > > > > > >> it's
> > > > >> > > > > > > > >> > not
> > > > >> > > > > > > > >> > > > > surfaced in the output from
> > `kafka-topics.sh
> > > > >> > > > --describe`.
> > > > >> > > > > > > Maybe
> > > > >> > > > > > > > >> that
> > > > >> > > > > > > > >> > > was
> > > > >> > > > > > > > >> > > > > intentional because ids are intentionally
> > not
> > > > >> really
> > > > >> > > > > > something
> > > > >> > > > > > > > the
> > > > >> > > > > > > > >> > user
> > > > >> > > > > > > > >> > > > > should care deeply about, but it would
> also
> > > > make
> > > > >> > life
> > > > >> > > > > harder
> > > > >> > > > > > > for
> > > > >> > > > > > > > >> > anyone
> > > > >> > > > > > > > >> > > > > debugging Kafka and this would likely get
> > > worse
> > > > >> the
> > > > >> > > more
> > > > >> > > > > > topic
> > > > >> > > > > > > > ids
> > > > >> > > > > > > > >> > got
> > > > >> > > > > > > > >> > > > > rolled out across the protocols, clients
> > etc.
> > > > It
> > > > >> > seems
> > > > >> > > > > > likely
> > > > >> > > > > > > > that
> > > > >> > > > > > > > >> > > > > `kafka-topics.sh` will eventually need
> the
> > > > >> ability
> > > > >> > to
> > > > >> > > > show
> > > > >> > > > > > the
> > > > >> > > > > > > > id
> > > > >> > > > > > > > >> of
> > > > >> > > > > > > > >> > a
> > > > >> > > > > > > > >> > > > > topic and perhaps find a topic name given
> > an
> > > > id.
> > > > >> Is
> > > > >> > > > there
> > > > >> > > > > > any
> > > > >> > > > > > > > >> reason
> > > > >> > > > > > > > >> > > not
> > > > >> > > > > > > > >> > > > to
> > > > >> > > > > > > > >> > > > > implement that in this KIP?
> > > > >> > > > > > > > >> > > > >
> > > > >> > > > > > > > >> > > > > Many thanks,
> > > > >> > > > > > > > >> > > > >
> > > > >> > > > > > > > >> > > > > Tom
> > > > >> > > > > > > > >> > > > >
> > > > >> > > > > > > > >> > > > > On Mon, Sep 21, 2020 at 9:54 PM Justine
> > > Olshan
> > > > <
> > > > >> > > > > > > > >> jolshan@confluent.io
> > > > >> > > > > > > > >> > >
> > > > >> > > > > > > > >> > > > > wrote:
> > > > >> > > > > > > > >> > > > >
> > > > >> > > > > > > > >> > > > > > Hi all,
> > > > >> > > > > > > > >> > > > > >
> > > > >> > > > > > > > >> > > > > > After thinking about it, I've decided
> to
> > > > remove
> > > > >> > the
> > > > >> > > > > topic
> > > > >> > > > > > > name
> > > > >> > > > > > > > >> from
> > > > >> > > > > > > > >> > > the
> > > > >> > > > > > > > >> > > > > > Fetch Request and Response after all.
> > Since
> > > > >> there
> > > > >> > > are
> > > > >> > > > so
> > > > >> > > > > > > many
> > > > >> > > > > > > > of
> > > > >> > > > > > > > >> > > these
> > > > >> > > > > > > > >> > > > > > requests per second, it is worth
> removing
> > > the
> > > > >> > extra
> > > > >> > > > > > > > information.
> > > > >> > > > > > > > >> > I've
> > > > >> > > > > > > > >> > > > > > updated the KIP to reflect this change.
> > > > >> > > > > > > > >> > > > > >
> > > > >> > > > > > > > >> > > > > > Please let me know if there is anything
> > > else
> > > > we
> > > > >> > > should
> > > > >> > > > > > > discuss
> > > > >> > > > > > > > >> > before
> > > > >> > > > > > > > >> > > > > > voting.
> > > > >> > > > > > > > >> > > > > >
> > > > >> > > > > > > > >> > > > > > Thank you,
> > > > >> > > > > > > > >> > > > > > Justine
> > > > >> > > > > > > > >> > > > > >
> > > > >> > > > > > > > >> > > > > > On Fri, Sep 18, 2020 at 9:46 AM Justine
> > > > Olshan
> > > > >> <
> > > > >> > > > > > > > >> > jolshan@confluent.io
> > > > >> > > > > > > > >> > > >
> > > > >> > > > > > > > >> > > > > > wrote:
> > > > >> > > > > > > > >> > > > > >
> > > > >> > > > > > > > >> > > > > > > Hi Jun,
> > > > >> > > > > > > > >> > > > > > >
> > > > >> > > > > > > > >> > > > > > > I see what you are saying. For now we
> > can
> > > > >> remove
> > > > >> > > the
> > > > >> > > > > > extra
> > > > >> > > > > > > > >> > > > information.
> > > > >> > > > > > > > >> > > > > > > I'll leave the option to add more
> > fields
> > > to
> > > > >> the
> > > > >> > > file
> > > > >> > > > > in
> > > > >> > > > > > > the
> > > > >> > > > > > > > >> > future.
> > > > >> > > > > > > > >> > > > The
> > > > >> > > > > > > > >> > > > > > KIP
> > > > >> > > > > > > > >> > > > > > > has been updated to reflect this
> > change.
> > > > >> > > > > > > > >> > > > > > >
> > > > >> > > > > > > > >> > > > > > > Thanks,
> > > > >> > > > > > > > >> > > > > > > Justine
> > > > >> > > > > > > > >> > > > > > >
> > > > >> > > > > > > > >> > > > > > > On Fri, Sep 18, 2020 at 8:46 AM Jun
> > Rao <
> > > > >> > > > > > jun@confluent.io
> > > > >> > > > > > > >
> > > > >> > > > > > > > >> > wrote:
> > > > >> > > > > > > > >> > > > > > >
> > > > >> > > > > > > > >> > > > > > >> Hi, Justine,
> > > > >> > > > > > > > >> > > > > > >>
> > > > >> > > > > > > > >> > > > > > >> Thanks for the reply.
> > > > >> > > > > > > > >> > > > > > >>
> > > > >> > > > > > > > >> > > > > > >> 13. If the log directory is the
> source
> > > of
> > > > >> > truth,
> > > > >> > > it
> > > > >> > > > > > means
> > > > >> > > > > > > > >> that
> > > > >> > > > > > > > >> > the
> > > > >> > > > > > > > >> > > > > > >> redundant info in the metadata file
> > will
> > > > be
> > > > >> > > > ignored.
> > > > >> > > > > > Then
> > > > >> > > > > > > > the
> > > > >> > > > > > > > >> > > > question
> > > > >> > > > > > > > >> > > > > > is
> > > > >> > > > > > > > >> > > > > > >> why do we need to put the redundant
> > info
> > > > in
> > > > >> the
> > > > >> > > > > > metadata
> > > > >> > > > > > > > file
> > > > >> > > > > > > > >> > now?
> > > > >> > > > > > > > >> > > > > > >>
> > > > >> > > > > > > > >> > > > > > >> Thanks,
> > > > >> > > > > > > > >> > > > > > >>
> > > > >> > > > > > > > >> > > > > > >> Jun
> > > > >> > > > > > > > >> > > > > > >>
> > > > >> > > > > > > > >> > > > > > >> On Thu, Sep 17, 2020 at 5:07 PM
> > Justine
> > > > >> Olshan
> > > > >> > <
> > > > >> > > > > > > > >> > > > jolshan@confluent.io>
> > > > >> > > > > > > > >> > > > > > >> wrote:
> > > > >> > > > > > > > >> > > > > > >>
> > > > >> > > > > > > > >> > > > > > >> > Hi Jun,
> > > > >> > > > > > > > >> > > > > > >> > Thanks for the quick response!
> > > > >> > > > > > > > >> > > > > > >> >
> > > > >> > > > > > > > >> > > > > > >> > 12. I've decided to bump up the
> > > versions
> > > > >> on
> > > > >> > the
> > > > >> > > > > > > requests
> > > > >> > > > > > > > >> and
> > > > >> > > > > > > > >> > > > updated
> > > > >> > > > > > > > >> > > > > > the
> > > > >> > > > > > > > >> > > > > > >> > KIP. I think it's good we
> thoroughly
> > > > >> > discussed
> > > > >> > > > the
> > > > >> > > > > > > > options
> > > > >> > > > > > > > >> > here,
> > > > >> > > > > > > > >> > > > so
> > > > >> > > > > > > > >> > > > > we
> > > > >> > > > > > > > >> > > > > > >> know
> > > > >> > > > > > > > >> > > > > > >> > we made a good choice. :)
> > > > >> > > > > > > > >> > > > > > >> >
> > > > >> > > > > > > > >> > > > > > >> > 13. This is an interesting
> > situation.
> > > I
> > > > >> think
> > > > >> > > if
> > > > >> > > > > this
> > > > >> > > > > > > > does
> > > > >> > > > > > > > >> > occur
> > > > >> > > > > > > > >> > > > we
> > > > >> > > > > > > > >> > > > > > >> should
> > > > >> > > > > > > > >> > > > > > >> > give a warning. I agree that it's
> > hard
> > > > to
> > > > >> > know
> > > > >> > > > the
> > > > >> > > > > > > source
> > > > >> > > > > > > > >> of
> > > > >> > > > > > > > >> > > truth
> > > > >> > > > > > > > >> > > > > for
> > > > >> > > > > > > > >> > > > > > >> sure
> > > > >> > > > > > > > >> > > > > > >> > since the directory or the file
> > could
> > > be
> > > > >> > > manually
> > > > >> > > > > > > > >> modified. I
> > > > >> > > > > > > > >> > > > guess
> > > > >> > > > > > > > >> > > > > > the
> > > > >> > > > > > > > >> > > > > > >> > directory could be used as the
> > source
> > > of
> > > > >> > truth.
> > > > >> > > > To
> > > > >> > > > > be
> > > > >> > > > > > > > >> honest,
> > > > >> > > > > > > > >> > > I'm
> > > > >> > > > > > > > >> > > > > not
> > > > >> > > > > > > > >> > > > > > >> > really sure what happens in kafka
> > when
> > > > the
> > > > >> > log
> > > > >> > > > > > > directory
> > > > >> > > > > > > > is
> > > > >> > > > > > > > >> > > > renamed
> > > > >> > > > > > > > >> > > > > > >> > manually in such a way. I'm also
> > > > >> wondering if
> > > > >> > > the
> > > > >> > > > > > > > >> situation is
> > > > >> > > > > > > > >> > > > > > >> recoverable
> > > > >> > > > > > > > >> > > > > > >> > in this scenario.
> > > > >> > > > > > > > >> > > > > > >> >
> > > > >> > > > > > > > >> > > > > > >> > Thanks,
> > > > >> > > > > > > > >> > > > > > >> > Justine
> > > > >> > > > > > > > >> > > > > > >> >
> > > > >> > > > > > > > >> > > > > > >> > On Thu, Sep 17, 2020 at 4:28 PM
> Jun
> > > Rao
> > > > <
> > > > >> > > > > > > > jun@confluent.io>
> > > > >> > > > > > > > >> > > wrote:
> > > > >> > > > > > > > >> > > > > > >> >
> > > > >> > > > > > > > >> > > > > > >> > > Hi, Justine,
> > > > >> > > > > > > > >> > > > > > >> > >
> > > > >> > > > > > > > >> > > > > > >> > > Thanks for the reply.
> > > > >> > > > > > > > >> > > > > > >> > >
> > > > >> > > > > > > > >> > > > > > >> > > 12. I don't have a strong
> > preference
> > > > >> > either.
> > > > >> > > > > > However,
> > > > >> > > > > > > > if
> > > > >> > > > > > > > >> we
> > > > >> > > > > > > > >> > > need
> > > > >> > > > > > > > >> > > > > IBP
> > > > >> > > > > > > > >> > > > > > >> > > anyway, maybe it's easier to
> just
> > > bump
> > > > >> up
> > > > >> > the
> > > > >> > > > > > version
> > > > >> > > > > > > > for
> > > > >> > > > > > > > >> > all
> > > > >> > > > > > > > >> > > > > inter
> > > > >> > > > > > > > >> > > > > > >> > broker
> > > > >> > > > > > > > >> > > > > > >> > > requests and add the topic id
> > field
> > > > as a
> > > > >> > > > regular
> > > > >> > > > > > > > field. A
> > > > >> > > > > > > > >> > > > regular
> > > > >> > > > > > > > >> > > > > > >> field
> > > > >> > > > > > > > >> > > > > > >> > is
> > > > >> > > > > > > > >> > > > > > >> > > a bit more concise in wire
> > transfer
> > > > >> than a
> > > > >> > > > > flexible
> > > > >> > > > > > > > >> field.
> > > > >> > > > > > > > >> > > > > > >> > >
> > > > >> > > > > > > > >> > > > > > >> > > 13. The confusion that I was
> > > referring
> > > > >> to
> > > > >> > is
> > > > >> > > > > > between
> > > > >> > > > > > > > the
> > > > >> > > > > > > > >> > topic
> > > > >> > > > > > > > >> > > > > name
> > > > >> > > > > > > > >> > > > > > >> and
> > > > >> > > > > > > > >> > > > > > >> > > partition number between the log
> > dir
> > > > and
> > > > >> > the
> > > > >> > > > > > metadata
> > > > >> > > > > > > > >> file.
> > > > >> > > > > > > > >> > > For
> > > > >> > > > > > > > >> > > > > > >> example,
> > > > >> > > > > > > > >> > > > > > >> > if
> > > > >> > > > > > > > >> > > > > > >> > > the log dir is topicA-1 and the
> > > > metadata
> > > > >> > file
> > > > >> > > > in
> > > > >> > > > > it
> > > > >> > > > > > > has
> > > > >> > > > > > > > >> > topicB
> > > > >> > > > > > > > >> > > > and
> > > > >> > > > > > > > >> > > > > > >> > > partition 0 (say due to a bug or
> > > > manual
> > > > >> > > > > > > modification),
> > > > >> > > > > > > > >> which
> > > > >> > > > > > > > >> > > one
> > > > >> > > > > > > > >> > > > > do
> > > > >> > > > > > > > >> > > > > > we
> > > > >> > > > > > > > >> > > > > > >> > use
> > > > >> > > > > > > > >> > > > > > >> > > as the source of truth?
> > > > >> > > > > > > > >> > > > > > >> > >
> > > > >> > > > > > > > >> > > > > > >> > > Jun
> > > > >> > > > > > > > >> > > > > > >> > >
> > > > >> > > > > > > > >> > > > > > >> > > On Thu, Sep 17, 2020 at 3:43 PM
> > > > Justine
> > > > >> > > Olshan
> > > > >> > > > <
> > > > >> > > > > > > > >> > > > > > jolshan@confluent.io>
> > > > >> > > > > > > > >> > > > > > >> > > wrote:
> > > > >> > > > > > > > >> > > > > > >> > >
> > > > >> > > > > > > > >> > > > > > >> > > > Hi Jun,
> > > > >> > > > > > > > >> > > > > > >> > > > Thanks for the comments.
> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > >> > > > > > > > >> > > > > > >> > > > 12. I bumped the
> > > LeaderAndIsrRequest
> > > > >> > > because
> > > > >> > > > I
> > > > >> > > > > > > > removed
> > > > >> > > > > > > > >> the
> > > > >> > > > > > > > >> > > > topic
> > > > >> > > > > > > > >> > > > > > >> name
> > > > >> > > > > > > > >> > > > > > >> > > field
> > > > >> > > > > > > > >> > > > > > >> > > > in the response. It may be
> > > possible
> > > > to
> > > > >> > > avoid
> > > > >> > > > > > > bumping
> > > > >> > > > > > > > >> the
> > > > >> > > > > > > > >> > > > version
> > > > >> > > > > > > > >> > > > > > >> > without
> > > > >> > > > > > > > >> > > > > > >> > > > that change, but I may be
> > missing
> > > > >> > > something.
> > > > >> > > > > > > > >> > > > > > >> > > > I believe StopReplica is
> > actually
> > > on
> > > > >> > > version
> > > > >> > > > 3
> > > > >> > > > > > now,
> > > > >> > > > > > > > but
> > > > >> > > > > > > > >> > > > because
> > > > >> > > > > > > > >> > > > > > >> > version 2
> > > > >> > > > > > > > >> > > > > > >> > > > is flexible, I kept that
> listed
> > as
> > > > >> > version
> > > > >> > > 2
> > > > >> > > > on
> > > > >> > > > > > the
> > > > >> > > > > > > > KIP
> > > > >> > > > > > > > >> > > page.
> > > > >> > > > > > > > >> > > > > > >> However,
> > > > >> > > > > > > > >> > > > > > >> > > you
> > > > >> > > > > > > > >> > > > > > >> > > > may be right in that we may
> need
> > > to
> > > > >> bump
> > > > >> > > the
> > > > >> > > > > > > version
> > > > >> > > > > > > > on
> > > > >> > > > > > > > >> > > > > > StopReplica
> > > > >> > > > > > > > >> > > > > > >> to
> > > > >> > > > > > > > >> > > > > > >> > > deal
> > > > >> > > > > > > > >> > > > > > >> > > > with deletion differently as
> > > > mentioned
> > > > >> > > > above. I
> > > > >> > > > > > > don't
> > > > >> > > > > > > > >> know
> > > > >> > > > > > > > >> > > if
> > > > >> > > > > > > > >> > > > I
> > > > >> > > > > > > > >> > > > > > >> have a
> > > > >> > > > > > > > >> > > > > > >> > > big
> > > > >> > > > > > > > >> > > > > > >> > > > preference over used tagged
> > fields
> > > > or
> > > > >> > not.
> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > >> > > > > > > > >> > > > > > >> > > > 13. I was thinking that in the
> > > case
> > > > >> where
> > > > >> > > the
> > > > >> > > > > > file
> > > > >> > > > > > > > and
> > > > >> > > > > > > > >> the
> > > > >> > > > > > > > >> > > > > request
> > > > >> > > > > > > > >> > > > > > >> > topic
> > > > >> > > > > > > > >> > > > > > >> > > > ids don't match, it means that
> > the
> > > > >> > broker's
> > > > >> > > > > > > topic/the
> > > > >> > > > > > > > >> one
> > > > >> > > > > > > > >> > in
> > > > >> > > > > > > > >> > > > the
> > > > >> > > > > > > > >> > > > > > >> file
> > > > >> > > > > > > > >> > > > > > >> > has
> > > > >> > > > > > > > >> > > > > > >> > > > been deleted. In that case, we
> > > would
> > > > >> need
> > > > >> > > to
> > > > >> > > > > > delete
> > > > >> > > > > > > > the
> > > > >> > > > > > > > >> > old
> > > > >> > > > > > > > >> > > > > topic
> > > > >> > > > > > > > >> > > > > > >> and
> > > > >> > > > > > > > >> > > > > > >> > > start
> > > > >> > > > > > > > >> > > > > > >> > > > receiving the new version. If
> > the
> > > > >> topic
> > > > >> > > name
> > > > >> > > > > were
> > > > >> > > > > > > to
> > > > >> > > > > > > > >> > change,
> > > > >> > > > > > > > >> > > > but
> > > > >> > > > > > > > >> > > > > > the
> > > > >> > > > > > > > >> > > > > > >> > ids
> > > > >> > > > > > > > >> > > > > > >> > > > still matched, the file would
> > also
> > > > >> need
> > > > >> > to
> > > > >> > > > > > update.
> > > > >> > > > > > > > Am I
> > > > >> > > > > > > > >> > > > missing
> > > > >> > > > > > > > >> > > > > a
> > > > >> > > > > > > > >> > > > > > >> case
> > > > >> > > > > > > > >> > > > > > >> > > > where the file would be
> correct
> > > and
> > > > >> not
> > > > >> > the
> > > > >> > > > > > > request?
> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > >> > > > > > > > >> > > > > > >> > > > Thanks,
> > > > >> > > > > > > > >> > > > > > >> > > > Justine
> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > >> > > > > > > > >> > > > > > >> > > > On Thu, Sep 17, 2020 at 3:18
> PM
> > > Jun
> > > > >> Rao <
> > > > >> > > > > > > > >> jun@confluent.io
> > > > >> > > > > > > > >> > >
> > > > >> > > > > > > > >> > > > > wrote:
> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > Hi, Justine,
> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > Thanks for the reply. A
> couple
> > > of
> > > > >> more
> > > > >> > > > > comments
> > > > >> > > > > > > > >> below.
> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > 12. ListOffset and
> > > OffsetForLeader
> > > > >> > > > currently
> > > > >> > > > > > > don't
> > > > >> > > > > > > > >> > support
> > > > >> > > > > > > > >> > > > > > >> flexible
> > > > >> > > > > > > > >> > > > > > >> > > > fields.
> > > > >> > > > > > > > >> > > > > > >> > > > > So, we have to bump up the
> > > version
> > > > >> > number
> > > > >> > > > and
> > > > >> > > > > > use
> > > > >> > > > > > > > >> IBP at
> > > > >> > > > > > > > >> > > > least
> > > > >> > > > > > > > >> > > > > > for
> > > > >> > > > > > > > >> > > > > > >> > > these
> > > > >> > > > > > > > >> > > > > > >> > > > > two requests. Note that it
> > seems
> > > > >> 2.7.0
> > > > >> > > will
> > > > >> > > > > > > require
> > > > >> > > > > > > > >> IBP
> > > > >> > > > > > > > >> > > > anyway
> > > > >> > > > > > > > >> > > > > > >> > because
> > > > >> > > > > > > > >> > > > > > >> > > of
> > > > >> > > > > > > > >> > > > > > >> > > > > changes in KAFKA-10435.
> Also,
> > it
> > > > >> seems
> > > > >> > > that
> > > > >> > > > > the
> > > > >> > > > > > > > >> version
> > > > >> > > > > > > > >> > > for
> > > > >> > > > > > > > >> > > > > > >> > > > > LeaderAndIsrRequest and
> > > > StopReplica
> > > > >> are
> > > > >> > > > > bumped
> > > > >> > > > > > > even
> > > > >> > > > > > > > >> > though
> > > > >> > > > > > > > >> > > > we
> > > > >> > > > > > > > >> > > > > > only
> > > > >> > > > > > > > >> > > > > > >> > > added
> > > > >> > > > > > > > >> > > > > > >> > > > a
> > > > >> > > > > > > > >> > > > > > >> > > > > tagged field. But since IBP
> is
> > > > >> needed
> > > > >> > > > anyway,
> > > > >> > > > > > we
> > > > >> > > > > > > > may
> > > > >> > > > > > > > >> > want
> > > > >> > > > > > > > >> > > to
> > > > >> > > > > > > > >> > > > > > >> revisit
> > > > >> > > > > > > > >> > > > > > >> > > the
> > > > >> > > > > > > > >> > > > > > >> > > > > overall tagged field choice.
> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > 13. The only downside is the
> > > > >> potential
> > > > >> > > > > > confusion
> > > > >> > > > > > > on
> > > > >> > > > > > > > >> > which
> > > > >> > > > > > > > >> > > > one
> > > > >> > > > > > > > >> > > > > is
> > > > >> > > > > > > > >> > > > > > >> the
> > > > >> > > > > > > > >> > > > > > >> > > > source
> > > > >> > > > > > > > >> > > > > > >> > > > > of truth if they don't
> match.
> > > > >> Another
> > > > >> > > > option
> > > > >> > > > > is
> > > > >> > > > > > > to
> > > > >> > > > > > > > >> > include
> > > > >> > > > > > > > >> > > > > those
> > > > >> > > > > > > > >> > > > > > >> > fields
> > > > >> > > > > > > > >> > > > > > >> > > > in
> > > > >> > > > > > > > >> > > > > > >> > > > > the metadata file when we
> > > actually
> > > > >> > change
> > > > >> > > > the
> > > > >> > > > > > > > >> directory
> > > > >> > > > > > > > >> > > > > > structure.
> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > Thanks,
> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > Jun
> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > On Thu, Sep 17, 2020 at 2:01
> > PM
> > > > >> Justine
> > > > >> > > > > Olshan
> > > > >> > > > > > <
> > > > >> > > > > > > > >> > > > > > >> jolshan@confluent.io
> > > > >> > > > > > > > >> > > > > > >> > >
> > > > >> > > > > > > > >> > > > > > >> > > > > wrote:
> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > Hello all,
> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > I've thought some more
> about
> > > > >> removing
> > > > >> > > the
> > > > >> > > > > > topic
> > > > >> > > > > > > > >> name
> > > > >> > > > > > > > >> > > field
> > > > >> > > > > > > > >> > > > > > from
> > > > >> > > > > > > > >> > > > > > >> > some
> > > > >> > > > > > > > >> > > > > > >> > > of
> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > >> > > > > > > > >> > > > > > >> > > > > > requests. On closer
> > inspection
> > > > of
> > > > >> the
> > > > >> > > > > > > > >> > > requests/responses,
> > > > >> > > > > > > > >> > > > it
> > > > >> > > > > > > > >> > > > > > >> seems
> > > > >> > > > > > > > >> > > > > > >> > > that
> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > >> > > > > > > > >> > > > > > >> > > > > > internal changes would be
> > much
> > > > >> larger
> > > > >> > > > than
> > > > >> > > > > I
> > > > >> > > > > > > > >> expected.
> > > > >> > > > > > > > >> > > > Some
> > > > >> > > > > > > > >> > > > > > >> > protocols
> > > > >> > > > > > > > >> > > > > > >> > > > > > involve clients, so they
> > would
> > > > >> > require
> > > > >> > > > > > changes
> > > > >> > > > > > > > too.
> > > > >> > > > > > > > >> > I'm
> > > > >> > > > > > > > >> > > > > > thinking
> > > > >> > > > > > > > >> > > > > > >> > that
> > > > >> > > > > > > > >> > > > > > >> > > > for
> > > > >> > > > > > > > >> > > > > > >> > > > > > now, removing the topic
> name
> > > > from
> > > > >> > these
> > > > >> > > > > > > requests
> > > > >> > > > > > > > >> and
> > > > >> > > > > > > > >> > > > > responses
> > > > >> > > > > > > > >> > > > > > >> are
> > > > >> > > > > > > > >> > > > > > >> > > out
> > > > >> > > > > > > > >> > > > > > >> > > > of
> > > > >> > > > > > > > >> > > > > > >> > > > > > scope.
> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > I have decided to just
> keep
> > > the
> > > > >> > change
> > > > >> > > > > > > > >> > > > LeaderAndIsrResponse
> > > > >> > > > > > > > >> > > > > to
> > > > >> > > > > > > > >> > > > > > >> > remove
> > > > >> > > > > > > > >> > > > > > >> > > > the
> > > > >> > > > > > > > >> > > > > > >> > > > > > topic name, and have
> updated
> > > the
> > > > >> KIP
> > > > >> > to
> > > > >> > > > > > reflect
> > > > >> > > > > > > > >> this
> > > > >> > > > > > > > >> > > > > change. I
> > > > >> > > > > > > > >> > > > > > >> have
> > > > >> > > > > > > > >> > > > > > >> > > > also
> > > > >> > > > > > > > >> > > > > > >> > > > > > mentioned the other
> requests
> > > and
> > > > >> > > > responses
> > > > >> > > > > in
> > > > >> > > > > > > > >> future
> > > > >> > > > > > > > >> > > work.
> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > I'm hoping to start the
> > voting
> > > > >> > process
> > > > >> > > > > soon,
> > > > >> > > > > > so
> > > > >> > > > > > > > >> let me
> > > > >> > > > > > > > >> > > > know
> > > > >> > > > > > > > >> > > > > if
> > > > >> > > > > > > > >> > > > > > >> > there
> > > > >> > > > > > > > >> > > > > > >> > > is
> > > > >> > > > > > > > >> > > > > > >> > > > > > anything else we should
> > > discuss.
> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > Thank you,
> > > > >> > > > > > > > >> > > > > > >> > > > > > Justine
> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > On Tue, Sep 15, 2020 at
> 3:57
> > > PM
> > > > >> > Justine
> > > > >> > > > > > Olshan
> > > > >> > > > > > > <
> > > > >> > > > > > > > >> > > > > > >> > jolshan@confluent.io
> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > wrote:
> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > > Hello again,
> > > > >> > > > > > > > >> > > > > > >> > > > > > > To follow up on some of
> > the
> > > > >> other
> > > > >> > > > > comments:
> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > > 10/11) We can remove the
> > > topic
> > > > >> name
> > > > >> > > > from
> > > > >> > > > > > > these
> > > > >> > > > > > > > >> > > > > > >> > requests/responses,
> > > > >> > > > > > > > >> > > > > > >> > > > and
> > > > >> > > > > > > > >> > > > > > >> > > > > > > that means we will just
> > have
> > > > to
> > > > >> > make
> > > > >> > > a
> > > > >> > > > > few
> > > > >> > > > > > > > >> internal
> > > > >> > > > > > > > >> > > > > changes
> > > > >> > > > > > > > >> > > > > > to
> > > > >> > > > > > > > >> > > > > > >> > make
> > > > >> > > > > > > > >> > > > > > >> > > > > > > partitions accessible by
> > > topic
> > > > >> id
> > > > >> > and
> > > > >> > > > > > > > partition.
> > > > >> > > > > > > > >> I
> > > > >> > > > > > > > >> > can
> > > > >> > > > > > > > >> > > > > > update
> > > > >> > > > > > > > >> > > > > > >> the
> > > > >> > > > > > > > >> > > > > > >> > > KIP
> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > >> > > > > > > > >> > > > > > >> > > > > > > remove them unless
> anyone
> > > > thinks
> > > > >> > they
> > > > >> > > > > > should
> > > > >> > > > > > > > >> stay.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > > 12) Addressed in the
> > > previous
> > > > >> > email.
> > > > >> > > > I've
> > > > >> > > > > > > > updated
> > > > >> > > > > > > > >> > the
> > > > >> > > > > > > > >> > > > KIP
> > > > >> > > > > > > > >> > > > > to
> > > > >> > > > > > > > >> > > > > > >> > > include
> > > > >> > > > > > > > >> > > > > > >> > > > > > > tagged fields for the
> > > requests
> > > > >> and
> > > > >> > > > > > responses.
> > > > >> > > > > > > > >> (More
> > > > >> > > > > > > > >> > on
> > > > >> > > > > > > > >> > > > > that
> > > > >> > > > > > > > >> > > > > > >> > below)
> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > > 13) I think part of the
> > idea
> > > > for
> > > > >> > > > > including
> > > > >> > > > > > > this
> > > > >> > > > > > > > >> > > > > information
> > > > >> > > > > > > > >> > > > > > >> is to
> > > > >> > > > > > > > >> > > > > > >> > > > > prepare
> > > > >> > > > > > > > >> > > > > > >> > > > > > > for future changes.
> > Perhaps
> > > > the
> > > > >> > > > directory
> > > > >> > > > > > > > >> structure
> > > > >> > > > > > > > >> > > > might
> > > > >> > > > > > > > >> > > > > > >> change
> > > > >> > > > > > > > >> > > > > > >> > > from
> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> topicName_partitionNumber
> > to
> > > > >> > > something
> > > > >> > > > > like
> > > > >> > > > > > > > >> > > > > > >> > > topicID_partitionNumber.
> > > > >> > > > > > > > >> > > > > > >> > > > > Then
> > > > >> > > > > > > > >> > > > > > >> > > > > > > it would be useful to
> have
> > > the
> > > > >> > topic
> > > > >> > > > name
> > > > >> > > > > > in
> > > > >> > > > > > > > the
> > > > >> > > > > > > > >> > file
> > > > >> > > > > > > > >> > > > > since
> > > > >> > > > > > > > >> > > > > > it
> > > > >> > > > > > > > >> > > > > > >> > > would
> > > > >> > > > > > > > >> > > > > > >> > > > > not
> > > > >> > > > > > > > >> > > > > > >> > > > > > be
> > > > >> > > > > > > > >> > > > > > >> > > > > > > in the directory
> > structure.
> > > > >> > > Supporting
> > > > >> > > > > > topic
> > > > >> > > > > > > > >> renames
> > > > >> > > > > > > > >> > > > might
> > > > >> > > > > > > > >> > > > > > be
> > > > >> > > > > > > > >> > > > > > >> > > easier
> > > > >> > > > > > > > >> > > > > > >> > > > if
> > > > >> > > > > > > > >> > > > > > >> > > > > > the
> > > > >> > > > > > > > >> > > > > > >> > > > > > > other fields are
> included.
> > > > Would
> > > > >> > > there
> > > > >> > > > be
> > > > >> > > > > > any
> > > > >> > > > > > > > >> > > downsides
> > > > >> > > > > > > > >> > > > to
> > > > >> > > > > > > > >> > > > > > >> > > including
> > > > >> > > > > > > > >> > > > > > >> > > > > this
> > > > >> > > > > > > > >> > > > > > >> > > > > > > information?
> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > > 14)  Yes, we would need
> to
> > > > copy
> > > > >> the
> > > > >> > > > > > partition
> > > > >> > > > > > > > >> > metadata
> > > > >> > > > > > > > >> > > > > file
> > > > >> > > > > > > > >> > > > > > in
> > > > >> > > > > > > > >> > > > > > >> > this
> > > > >> > > > > > > > >> > > > > > >> > > > > > > process. I've updated
> the
> > > KIP
> > > > to
> > > > >> > > > include
> > > > >> > > > > > > this.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > > 15) I believe Lucas
> meant
> > v1
> > > > >> and v2
> > > > >> > > > here.
> > > > >> > > > > > He
> > > > >> > > > > > > > was
> > > > >> > > > > > > > >> > > > referring
> > > > >> > > > > > > > >> > > > > > to
> > > > >> > > > > > > > >> > > > > > >> how
> > > > >> > > > > > > > >> > > > > > >> > > the
> > > > >> > > > > > > > >> > > > > > >> > > > > > > requests would fall
> under
> > > > >> different
> > > > >> > > IBP
> > > > >> > > > > and
> > > > >> > > > > > > > meant
> > > > >> > > > > > > > >> > that
> > > > >> > > > > > > > >> > > > > older
> > > > >> > > > > > > > >> > > > > > >> > > brokers
> > > > >> > > > > > > > >> > > > > > >> > > > > > would
> > > > >> > > > > > > > >> > > > > > >> > > > > > > have to use the older
> > > version
> > > > of
> > > > >> > the
> > > > >> > > > > > request
> > > > >> > > > > > > > and
> > > > >> > > > > > > > >> the
> > > > >> > > > > > > > >> > > > > > existing
> > > > >> > > > > > > > >> > > > > > >> > topic
> > > > >> > > > > > > > >> > > > > > >> > > > > > > deletion process. At
> > first,
> > > it
> > > > >> > seemed
> > > > >> > > > > like
> > > > >> > > > > > > > tagged
> > > > >> > > > > > > > >> > > fields
> > > > >> > > > > > > > >> > > > > > would
> > > > >> > > > > > > > >> > > > > > >> > > > resolve
> > > > >> > > > > > > > >> > > > > > >> > > > > > > the IBP issue. However,
> we
> > > may
> > > > >> need
> > > > >> > > IBP
> > > > >> > > > > for
> > > > >> > > > > > > > this
> > > > >> > > > > > > > >> > > request
> > > > >> > > > > > > > >> > > > > > after
> > > > >> > > > > > > > >> > > > > > >> > all
> > > > >> > > > > > > > >> > > > > > >> > > > > since
> > > > >> > > > > > > > >> > > > > > >> > > > > > > the controller handles
> the
> > > > topic
> > > > >> > > > deletion
> > > > >> > > > > > > > >> > differently
> > > > >> > > > > > > > >> > > > > > >> depending
> > > > >> > > > > > > > >> > > > > > >> > on
> > > > >> > > > > > > > >> > > > > > >> > > > the
> > > > >> > > > > > > > >> > > > > > >> > > > > > IBP
> > > > >> > > > > > > > >> > > > > > >> > > > > > > version. In an older
> > > version,
> > > > we
> > > > >> > > can't
> > > > >> > > > > just
> > > > >> > > > > > > > send
> > > > >> > > > > > > > >> a
> > > > >> > > > > > > > >> > > > > > StopReplica
> > > > >> > > > > > > > >> > > > > > >> > > delete
> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > >> > > > > > > > >> > > > > > >> > > > > > > topic immediately like
> > we'd
> > > > >> want to
> > > > >> > > for
> > > > >> > > > > > this
> > > > >> > > > > > > > KIP.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > > This makes me wonder if
> we
> > > > want
> > > > >> > > tagged
> > > > >> > > > > > fields
> > > > >> > > > > > > > on
> > > > >> > > > > > > > >> all
> > > > >> > > > > > > > >> > > the
> > > > >> > > > > > > > >> > > > > > >> requests
> > > > >> > > > > > > > >> > > > > > >> > > > after
> > > > >> > > > > > > > >> > > > > > >> > > > > > > all. Let me know your
> > > > thoughts!
> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > > Justine
> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > > On Tue, Sep 15, 2020 at
> > 1:03
> > > > PM
> > > > >> > > Justine
> > > > >> > > > > > > Olshan
> > > > >> > > > > > > > <
> > > > >> > > > > > > > >> > > > > > >> > > jolshan@confluent.io
> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > > wrote:
> > > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Hi all,
> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Jun brought up a good
> > point
> > > > in
> > > > >> his
> > > > >> > > > last
> > > > >> > > > > > > email
> > > > >> > > > > > > > >> about
> > > > >> > > > > > > > >> > > > > tagged
> > > > >> > > > > > > > >> > > > > > >> > fields,
> > > > >> > > > > > > > >> > > > > > >> > > > and
> > > > >> > > > > > > > >> > > > > > >> > > > > > >> I've updated the KIP to
> > > > reflect
> > > > >> > that
> > > > >> > > > the
> > > > >> > > > > > > > >> changes to
> > > > >> > > > > > > > >> > > > > > requests
> > > > >> > > > > > > > >> > > > > > >> and
> > > > >> > > > > > > > >> > > > > > >> > > > > > responses
> > > > >> > > > > > > > >> > > > > > >> > > > > > >> will be in the form of
> > > tagged
> > > > >> > fields
> > > > >> > > > to
> > > > >> > > > > > > avoid
> > > > >> > > > > > > > >> > > changing
> > > > >> > > > > > > > >> > > > > IBP.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Jun: I plan on sending
> a
> > > > >> followup
> > > > >> > > > email
> > > > >> > > > > to
> > > > >> > > > > > > > >> address
> > > > >> > > > > > > > >> > > some
> > > > >> > > > > > > > >> > > > > of
> > > > >> > > > > > > > >> > > > > > >> the
> > > > >> > > > > > > > >> > > > > > >> > > other
> > > > >> > > > > > > > >> > > > > > >> > > > > > >> points.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Thanks,
> > > > >> > > > > > > > >> > > > > > >> > > > > > >> Justine
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > >> > > > > > > > >> > > > > > >> > > > > > >> On Mon, Sep 14, 2020 at
> > > 4:25
> > > > PM
> > > > >> > Jun
> > > > >> > > > Rao
> > > > >> > > > > <
> > > > >> > > > > > > > >> > > > > jun@confluent.io>
> > > > >> > > > > > > > >> > > > > > >> > wrote:
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Hi, Justine,
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Thanks for the updated
> > > KIP.
> > > > A
> > > > >> few
> > > > >> > > > > > comments
> > > > >> > > > > > > > >> below.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 10. LeaderAndIsr
> > Response:
> > > > Do
> > > > >> we
> > > > >> > > need
> > > > >> > > > > the
> > > > >> > > > > > > > topic
> > > > >> > > > > > > > >> > > name?
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 11. For the changed
> > > > >> > > request/response,
> > > > >> > > > > > other
> > > > >> > > > > > > > >> than
> > > > >> > > > > > > > >> > > > > > >> LeaderAndIsr,
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> UpdateMetadata,
> > Metadata,
> > > do
> > > > >> we
> > > > >> > > need
> > > > >> > > > to
> > > > >> > > > > > > > include
> > > > >> > > > > > > > >> > the
> > > > >> > > > > > > > >> > > > > topic
> > > > >> > > > > > > > >> > > > > > >> name?
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 12. It seems that
> > upgrades
> > > > >> don't
> > > > >> > > > > require
> > > > >> > > > > > > IBP.
> > > > >> > > > > > > > >> Does
> > > > >> > > > > > > > >> > > > that
> > > > >> > > > > > > > >> > > > > > mean
> > > > >> > > > > > > > >> > > > > > >> > the
> > > > >> > > > > > > > >> > > > > > >> > > > new
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> fields
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> in all the
> > > request/response
> > > > >> are
> > > > >> > > added
> > > > >> > > > > as
> > > > >> > > > > > > > tagged
> > > > >> > > > > > > > >> > > fields
> > > > >> > > > > > > > >> > > > > > >> without
> > > > >> > > > > > > > >> > > > > > >> > > > > bumping
> > > > >> > > > > > > > >> > > > > > >> > > > > > up
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> the request version?
> It
> > > > would
> > > > >> be
> > > > >> > > > useful
> > > > >> > > > > > to
> > > > >> > > > > > > > make
> > > > >> > > > > > > > >> > that
> > > > >> > > > > > > > >> > > > > > clear.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 13. Partition Metadata
> > > file:
> > > > >> Do
> > > > >> > we
> > > > >> > > > need
> > > > >> > > > > > to
> > > > >> > > > > > > > >> include
> > > > >> > > > > > > > >> > > the
> > > > >> > > > > > > > >> > > > > > topic
> > > > >> > > > > > > > >> > > > > > >> > name
> > > > >> > > > > > > > >> > > > > > >> > > > and
> > > > >> > > > > > > > >> > > > > > >> > > > > > the
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> partition id since
> they
> > > are
> > > > >> > implied
> > > > >> > > > in
> > > > >> > > > > > the
> > > > >> > > > > > > > >> > directory
> > > > >> > > > > > > > >> > > > > name?
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 14. In the JBOD mode,
> we
> > > > >> support
> > > > >> > > > > moving a
> > > > >> > > > > > > > >> > > partition's
> > > > >> > > > > > > > >> > > > > data
> > > > >> > > > > > > > >> > > > > > >> from
> > > > >> > > > > > > > >> > > > > > >> > > one
> > > > >> > > > > > > > >> > > > > > >> > > > > > disk
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> to
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> another. Will the new
> > > > >> partition
> > > > >> > > > > metadata
> > > > >> > > > > > > file
> > > > >> > > > > > > > >> be
> > > > >> > > > > > > > >> > > > copied
> > > > >> > > > > > > > >> > > > > > >> during
> > > > >> > > > > > > > >> > > > > > >> > > that
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> process?
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 15. The KIP says
> "Remove
> > > > >> deleted
> > > > >> > > > topics
> > > > >> > > > > > > from
> > > > >> > > > > > > > >> > > replicas
> > > > >> > > > > > > > >> > > > by
> > > > >> > > > > > > > >> > > > > > >> > sending
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> StopReplicaRequest V2
> > for
> > > > any
> > > > >> > > topics
> > > > >> > > > > > which
> > > > >> > > > > > > do
> > > > >> > > > > > > > >> not
> > > > >> > > > > > > > >> > > > > contain
> > > > >> > > > > > > > >> > > > > > a
> > > > >> > > > > > > > >> > > > > > >> > topic
> > > > >> > > > > > > > >> > > > > > >> > > > ID,
> > > > >> > > > > > > > >> > > > > > >> > > > > > and
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> V3 for any topics
> which
> > do
> > > > >> > contain
> > > > >> > > a
> > > > >> > > > > > topic
> > > > >> > > > > > > > >> ID.".
> > > > >> > > > > > > > >> > > > > However,
> > > > >> > > > > > > > >> > > > > > it
> > > > >> > > > > > > > >> > > > > > >> > > seems
> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> updated controller
> will
> > > > create
> > > > >> > all
> > > > >> > > > > > missing
> > > > >> > > > > > > > >> topic
> > > > >> > > > > > > > >> > IDs
> > > > >> > > > > > > > >> > > > > first
> > > > >> > > > > > > > >> > > > > > >> > before
> > > > >> > > > > > > > >> > > > > > >> > > > > doing
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> other actions. So, is
> > > > >> > > > > StopReplicaRequest
> > > > >> > > > > > V2
> > > > >> > > > > > > > >> > needed?
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Jun
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> On Fri, Sep 11, 2020
> at
> > > > 10:31
> > > > >> AM
> > > > >> > > John
> > > > >> > > > > > > > Roesler <
> > > > >> > > > > > > > >> > > > > > >> > > vvcephei@apache.org
> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> wrote:
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > Thanks, Justine!
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > Your response seems
> > > > >> compelling
> > > > >> > to
> > > > >> > > > me.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > -John
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > On Fri, 2020-09-11
> at
> > > > 10:17
> > > > >> > > -0700,
> > > > >> > > > > > > Justine
> > > > >> > > > > > > > >> > Olshan
> > > > >> > > > > > > > >> > > > > wrote:
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Hello all,
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Thanks for
> > continuing
> > > > the
> > > > >> > > > > > discussion! I
> > > > >> > > > > > > > >> have a
> > > > >> > > > > > > > >> > > few
> > > > >> > > > > > > > >> > > > > > >> > responses
> > > > >> > > > > > > > >> > > > > > >> > > to
> > > > >> > > > > > > > >> > > > > > >> > > > > > your
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > points.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Tom: You are
> correct
> > > in
> > > > >> that
> > > > >> > > this
> > > > >> > > > > KIP
> > > > >> > > > > > > has
> > > > >> > > > > > > > >> not
> > > > >> > > > > > > > >> > > > > > mentioned
> > > > >> > > > > > > > >> > > > > > >> the
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > DeleteTopicsRequest. I
> > > > >> think
> > > > >> > > that
> > > > >> > > > > > this
> > > > >> > > > > > > > >> would
> > > > >> > > > > > > > >> > be
> > > > >> > > > > > > > >> > > > out
> > > > >> > > > > > > > >> > > > > of
> > > > >> > > > > > > > >> > > > > > >> > scope
> > > > >> > > > > > > > >> > > > > > >> > > > for
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> now, but
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > may be something
> > worth
> > > > >> adding
> > > > >> > > in
> > > > >> > > > > the
> > > > >> > > > > > > > >> future.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > John: We did
> > consider
> > > > >> > sequence
> > > > >> > > > ids,
> > > > >> > > > > > but
> > > > >> > > > > > > > >> there
> > > > >> > > > > > > > >> > > are
> > > > >> > > > > > > > >> > > > a
> > > > >> > > > > > > > >> > > > > > few
> > > > >> > > > > > > > >> > > > > > >> > > reasons
> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> favor
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > UUIDs. There are
> > > several
> > > > >> > cases
> > > > >> > > > > where
> > > > >> > > > > > > > topics
> > > > >> > > > > > > > >> > from
> > > > >> > > > > > > > >> > > > > > >> different
> > > > >> > > > > > > > >> > > > > > >> > > > > clusters
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> may
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > interact now and
> in
> > > the
> > > > >> > future.
> > > > >> > > > For
> > > > >> > > > > > > > >> example,
> > > > >> > > > > > > > >> > > > Mirror
> > > > >> > > > > > > > >> > > > > > >> Maker 2
> > > > >> > > > > > > > >> > > > > > >> > > may
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> benefit
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > from being able to
> > > > detect
> > > > >> > when
> > > > >> > > a
> > > > >> > > > > > > cluster
> > > > >> > > > > > > > >> being
> > > > >> > > > > > > > >> > > > > > mirrored
> > > > >> > > > > > > > >> > > > > > >> is
> > > > >> > > > > > > > >> > > > > > >> > > > > deleted
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> and
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > recreated and
> > globally
> > > > >> unique
> > > > >> > > > > > > identifiers
> > > > >> > > > > > > > >> > would
> > > > >> > > > > > > > >> > > > make
> > > > >> > > > > > > > >> > > > > > >> > > resolving
> > > > >> > > > > > > > >> > > > > > >> > > > > > issues
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > easier than
> sequence
> > > IDs
> > > > >> > which
> > > > >> > > > may
> > > > >> > > > > > > > collide
> > > > >> > > > > > > > >> > > between
> > > > >> > > > > > > > >> > > > > > >> > clusters.
> > > > >> > > > > > > > >> > > > > > >> > > > > > KIP-405
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > (tiered storage)
> > will
> > > > also
> > > > >> > > > benefit
> > > > >> > > > > > from
> > > > >> > > > > > > > >> > globally
> > > > >> > > > > > > > >> > > > > > unique
> > > > >> > > > > > > > >> > > > > > >> IDs
> > > > >> > > > > > > > >> > > > > > >> > > as
> > > > >> > > > > > > > >> > > > > > >> > > > > > shared
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > buckets may be
> used
> > > > >> between
> > > > >> > > > > clusters.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Globally unique
> IDs
> > > > would
> > > > >> > also
> > > > >> > > > make
> > > > >> > > > > > > > >> > > functionality
> > > > >> > > > > > > > >> > > > > like
> > > > >> > > > > > > > >> > > > > > >> > moving
> > > > >> > > > > > > > >> > > > > > >> > > > > > topics
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > between disparate
> > > > clusters
> > > > >> > > easier
> > > > >> > > > > in
> > > > >> > > > > > > the
> > > > >> > > > > > > > >> > future,
> > > > >> > > > > > > > >> > > > > > >> simplify
> > > > >> > > > > > > > >> > > > > > >> > any
> > > > >> > > > > > > > >> > > > > > >> > > > > > future
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > implementations of
> > > > backups
> > > > >> > and
> > > > >> > > > > > > restores,
> > > > >> > > > > > > > >> and
> > > > >> > > > > > > > >> > > more.
> > > > >> > > > > > > > >> > > > > In
> > > > >> > > > > > > > >> > > > > > >> > > general,
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> unique IDs
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > would ensure that
> > the
> > > > >> source
> > > > >> > > > > cluster
> > > > >> > > > > > > > >> topics do
> > > > >> > > > > > > > >> > > not
> > > > >> > > > > > > > >> > > > > > >> conflict
> > > > >> > > > > > > > >> > > > > > >> > > > with
> > > > >> > > > > > > > >> > > > > > >> > > > > > the
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > destination
> cluster
> > > > >> topics.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > If we were to use
> > > > sequence
> > > > >> > ids,
> > > > >> > > > we
> > > > >> > > > > > > would
> > > > >> > > > > > > > >> need
> > > > >> > > > > > > > >> > > > > > >> sufficiently
> > > > >> > > > > > > > >> > > > > > >> > > > large
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> cluster
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > ids to be stored
> > with
> > > > the
> > > > >> > topic
> > > > >> > > > > > > > >> identifiers or
> > > > >> > > > > > > > >> > > we
> > > > >> > > > > > > > >> > > > > run
> > > > >> > > > > > > > >> > > > > > >> the
> > > > >> > > > > > > > >> > > > > > >> > > risk
> > > > >> > > > > > > > >> > > > > > >> > > > of
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > collisions. This
> > will
> > > > >> give up
> > > > >> > > any
> > > > >> > > > > > > > >> advantage in
> > > > >> > > > > > > > >> > > > > > >> compactness
> > > > >> > > > > > > > >> > > > > > >> > > that
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> sequence
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > numbers may bring.
> > > Given
> > > > >> > these
> > > > >> > > > > > > > advantages I
> > > > >> > > > > > > > >> > > think
> > > > >> > > > > > > > >> > > > it
> > > > >> > > > > > > > >> > > > > > >> makes
> > > > >> > > > > > > > >> > > > > > >> > > > sense
> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> use
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > UUIDs.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Gokul: This is an
> > > > >> interesting
> > > > >> > > > idea,
> > > > >> > > > > > but
> > > > >> > > > > > > > >> this
> > > > >> > > > > > > > >> > is
> > > > >> > > > > > > > >> > > a
> > > > >> > > > > > > > >> > > > > > >> breaking
> > > > >> > > > > > > > >> > > > > > >> > > > > change.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Out of
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > scope for now, but
> > > maybe
> > > > >> > worth
> > > > >> > > > > > > discussing
> > > > >> > > > > > > > >> in
> > > > >> > > > > > > > >> > the
> > > > >> > > > > > > > >> > > > > > future.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Hope this explains
> > > some
> > > > of
> > > > >> > the
> > > > >> > > > > > > decisions,
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Justine
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > On Fri, Sep 11,
> 2020
> > > at
> > > > >> 8:27
> > > > >> > AM
> > > > >> > > > > Gokul
> > > > >> > > > > > > > >> Ramanan
> > > > >> > > > > > > > >> > > > > > >> Subramanian <
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > gokul2411s@gmail.com>
> > > > >> wrote:
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Hi.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Thanks for the
> > KIP.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Have you thought
> > > about
> > > > >> > > whether
> > > > >> > > > it
> > > > >> > > > > > > makes
> > > > >> > > > > > > > >> > sense
> > > > >> > > > > > > > >> > > to
> > > > >> > > > > > > > >> > > > > > >> support
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> authorizing a
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > principal for a
> > > topic
> > > > ID
> > > > >> > > rather
> > > > >> > > > > > than
> > > > >> > > > > > > a
> > > > >> > > > > > > > >> topic
> > > > >> > > > > > > > >> > > > name
> > > > >> > > > > > > > >> > > > > to
> > > > >> > > > > > > > >> > > > > > >> > > achieve
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> tighter
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > security?
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Or is the topic
> ID
> > > > >> > > > fundamentally
> > > > >> > > > > an
> > > > >> > > > > > > > >> internal
> > > > >> > > > > > > > >> > > > > detail
> > > > >> > > > > > > > >> > > > > > >> > similar
> > > > >> > > > > > > > >> > > > > > >> > > > to
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> epochs
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > used
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > in a bunch of
> > other
> > > > >> places
> > > > >> > in
> > > > >> > > > > > Kafka?
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Thanks.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > On Fri, Sep 11,
> > 2020
> > > > at
> > > > >> > 4:06
> > > > >> > > PM
> > > > >> > > > > > John
> > > > >> > > > > > > > >> > Roesler <
> > > > >> > > > > > > > >> > > > > > >> > > > > > vvcephei@apache.org>
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > wrote:
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Hello Justine,
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Thanks for the
> > > KIP!
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > I happen to
> have
> > > > been
> > > > >> > > > > confronted
> > > > >> > > > > > > > >> recently
> > > > >> > > > > > > > >> > > with
> > > > >> > > > > > > > >> > > > > the
> > > > >> > > > > > > > >> > > > > > >> need
> > > > >> > > > > > > > >> > > > > > >> > > to
> > > > >> > > > > > > > >> > > > > > >> > > > > keep
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > track of
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > a
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > large number
> of
> > > > >> topics as
> > > > >> > > > > > compactly
> > > > >> > > > > > > > as
> > > > >> > > > > > > > >> > > > > possible. I
> > > > >> > > > > > > > >> > > > > > >> was
> > > > >> > > > > > > > >> > > > > > >> > > > going
> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> come
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > up
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > with some way
> to
> > > > >> > dictionary
> > > > >> > > > > > encode
> > > > >> > > > > > > > the
> > > > >> > > > > > > > >> > topic
> > > > >> > > > > > > > >> > > > > names
> > > > >> > > > > > > > >> > > > > > >> as
> > > > >> > > > > > > > >> > > > > > >> > > > > integers,
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> but
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > this
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > seems much
> > better!
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Apologies if
> > this
> > > > has
> > > > >> > been
> > > > >> > > > > raised
> > > > >> > > > > > > > >> before,
> > > > >> > > > > > > > >> > > but
> > > > >> > > > > > > > >> > > > > I’m
> > > > >> > > > > > > > >> > > > > > >> > > wondering
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> about the
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > choice of UUID
> > vs
> > > > >> > sequence
> > > > >> > > > > number
> > > > >> > > > > > > for
> > > > >> > > > > > > > >> the
> > > > >> > > > > > > > >> > > ids.
> > > > >> > > > > > > > >> > > > > > >> > Typically,
> > > > >> > > > > > > > >> > > > > > >> > > > > I’ve
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> seen
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > UUIDs
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > in two
> > situations:
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 1. When
> > processes
> > > > >> need to
> > > > >> > > > > > generate
> > > > >> > > > > > > > >> > > > non-colliding
> > > > >> > > > > > > > >> > > > > > >> > > > identifiers
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> without
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > coordination.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 2. When the
> > > > identifier
> > > > >> > > needs
> > > > >> > > > to
> > > > >> > > > > > be
> > > > >> > > > > > > > >> > > > “universally
> > > > >> > > > > > > > >> > > > > > >> > unique”;
> > > > >> > > > > > > > >> > > > > > >> > > > > I.e.,
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> the
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > identifier
> needs
> > > to
> > > > >> > > > distinguish
> > > > >> > > > > > the
> > > > >> > > > > > > > >> entity
> > > > >> > > > > > > > >> > > > from
> > > > >> > > > > > > > >> > > > > > all
> > > > >> > > > > > > > >> > > > > > >> > other
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> entities
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > that
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > could ever
> > exist.
> > > > >> This is
> > > > >> > > > > useful
> > > > >> > > > > > in
> > > > >> > > > > > > > >> cases
> > > > >> > > > > > > > >> > > > where
> > > > >> > > > > > > > >> > > > > > >> > entities
> > > > >> > > > > > > > >> > > > > > >> > > > from
> > > > >> > > > > > > > >> > > > > > >> > > > > > all
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > kinds
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > of
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > systems get
> > mixed
> > > > >> > together,
> > > > >> > > > > such
> > > > >> > > > > > as
> > > > >> > > > > > > > >> when
> > > > >> > > > > > > > >> > > > dumping
> > > > >> > > > > > > > >> > > > > > >> logs
> > > > >> > > > > > > > >> > > > > > >> > > from
> > > > >> > > > > > > > >> > > > > > >> > > > > all
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > processes
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > in
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a company
> into a
> > > > >> common
> > > > >> > > > system.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Maybe I’m
> being
> > > > >> > > > short-sighted,
> > > > >> > > > > > but
> > > > >> > > > > > > it
> > > > >> > > > > > > > >> > > doesn’t
> > > > >> > > > > > > > >> > > > > seem
> > > > >> > > > > > > > >> > > > > > >> like
> > > > >> > > > > > > > >> > > > > > >> > > > > either
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> really
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > applies here.
> It
> > > > seems
> > > > >> > like
> > > > >> > > > the
> > > > >> > > > > > > > brokers
> > > > >> > > > > > > > >> > > could
> > > > >> > > > > > > > >> > > > > and
> > > > >> > > > > > > > >> > > > > > >> would
> > > > >> > > > > > > > >> > > > > > >> > > > > achieve
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > consensus
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > when creating
> a
> > > > topic
> > > > >> > > anyway,
> > > > >> > > > > > which
> > > > >> > > > > > > > is
> > > > >> > > > > > > > >> all
> > > > >> > > > > > > > >> > > > > that’s
> > > > >> > > > > > > > >> > > > > > >> > > required
> > > > >> > > > > > > > >> > > > > > >> > > > to
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > generate
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > non-colliding
> > > > sequence
> > > > >> > ids.
> > > > >> > > > For
> > > > >> > > > > > the
> > > > >> > > > > > > > >> > second,
> > > > >> > > > > > > > >> > > as
> > > > >> > > > > > > > >> > > > > you
> > > > >> > > > > > > > >> > > > > > >> > > mention,
> > > > >> > > > > > > > >> > > > > > >> > > > > we
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> could
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > assign
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a UUID to the
> > > > cluster
> > > > >> as
> > > > >> > a
> > > > >> > > > > whole,
> > > > >> > > > > > > > which
> > > > >> > > > > > > > >> > > would
> > > > >> > > > > > > > >> > > > > > render
> > > > >> > > > > > > > >> > > > > > >> > any
> > > > >> > > > > > > > >> > > > > > >> > > > > > resource
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > scoped
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > the broker
> > > > universally
> > > > >> > > unique
> > > > >> > > > > as
> > > > >> > > > > > > > well.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > The reason I
> > > mention
> > > > >> this
> > > > >> > > is
> > > > >> > > > > > that,
> > > > >> > > > > > > > >> > although
> > > > >> > > > > > > > >> > > a
> > > > >> > > > > > > > >> > > > > UUID
> > > > >> > > > > > > > >> > > > > > >> is
> > > > >> > > > > > > > >> > > > > > >> > way
> > > > >> > > > > > > > >> > > > > > >> > > > > more
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > compact
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > than topic
> > names,
> > > > it’s
> > > > >> > > still
> > > > >> > > > 16
> > > > >> > > > > > > > bytes.
> > > > >> > > > > > > > >> In
> > > > >> > > > > > > > >> > > > > > contrast,
> > > > >> > > > > > > > >> > > > > > >> a
> > > > >> > > > > > > > >> > > > > > >> > > > 4-byte
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> integer
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > sequence id
> > would
> > > > give
> > > > >> > us 4
> > > > >> > > > > > billion
> > > > >> > > > > > > > >> unique
> > > > >> > > > > > > > >> > > > > topics
> > > > >> > > > > > > > >> > > > > > >> per
> > > > >> > > > > > > > >> > > > > > >> > > > > cluster,
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> which
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > seems
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > like enough ;)
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Considering
> the
> > > > >> number of
> > > > >> > > > > > different
> > > > >> > > > > > > > >> times
> > > > >> > > > > > > > >> > > > these
> > > > >> > > > > > > > >> > > > > > >> topic
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> identifiers are
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > sent
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > over the wire
> or
> > > > >> stored
> > > > >> > in
> > > > >> > > > > > memory,
> > > > >> > > > > > > it
> > > > >> > > > > > > > >> > seems
> > > > >> > > > > > > > >> > > > like
> > > > >> > > > > > > > >> > > > > > it
> > > > >> > > > > > > > >> > > > > > >> > might
> > > > >> > > > > > > > >> > > > > > >> > > > be
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> worth
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > the
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > additional 4x
> > > space
> > > > >> > > savings.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > What do you
> > think
> > > > >> about
> > > > >> > > this?
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Thanks,
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > John
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > On Fri, Sep
> 11,
> > > > 2020,
> > > > >> at
> > > > >> > > > 03:20,
> > > > >> > > > > > Tom
> > > > >> > > > > > > > >> > Bentley
> > > > >> > > > > > > > >> > > > > wrote:
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Hi Justine,
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > This looks
> > like
> > > a
> > > > >> very
> > > > >> > > > > welcome
> > > > >> > > > > > > > >> > > improvement.
> > > > >> > > > > > > > >> > > > > > >> Thanks!
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Maybe I
> missed
> > > it,
> > > > >> but
> > > > >> > > the
> > > > >> > > > > KIP
> > > > >> > > > > > > > >> doesn't
> > > > >> > > > > > > > >> > > seem
> > > > >> > > > > > > > >> > > > to
> > > > >> > > > > > > > >> > > > > > >> > mention
> > > > >> > > > > > > > >> > > > > > >> > > > > > changing
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > DeleteTopicsRequest
> > > > >> to
> > > > >> > > > > identify
> > > > >> > > > > > > the
> > > > >> > > > > > > > >> > topic
> > > > >> > > > > > > > >> > > > > using
> > > > >> > > > > > > > >> > > > > > an
> > > > >> > > > > > > > >> > > > > > >> > id.
> > > > >> > > > > > > > >> > > > > > >> > > > > Maybe
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > that's out
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > of
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > scope, but
> > > > >> > > > > DeleteTopicsRequest
> > > > >> > > > > > is
> > > > >> > > > > > > > not
> > > > >> > > > > > > > >> > > listed
> > > > >> > > > > > > > >> > > > > > among
> > > > >> > > > > > > > >> > > > > > >> > the
> > > > >> > > > > > > > >> > > > > > >> > > > > Future
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Work
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > APIs
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > either.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Kind
> regards,
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Tom
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > On Thu, Sep
> > 10,
> > > > >> 2020 at
> > > > >> > > > 3:59
> > > > >> > > > > PM
> > > > >> > > > > > > > >> Satish
> > > > >> > > > > > > > >> > > > > Duggana <
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > >> satish.duggana@gmail.com
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > wrote:
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Thanks
> > > > >> Lucas/Justine
> > > > >> > > for
> > > > >> > > > > the
> > > > >> > > > > > > nice
> > > > >> > > > > > > > >> KIP.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > It has
> > several
> > > > >> > benefits
> > > > >> > > > > which
> > > > >> > > > > > > > also
> > > > >> > > > > > > > >> > > include
> > > > >> > > > > > > > >> > > > > > >> > > simplifying
> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> topic
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > deletion
> > > process
> > > > >> by
> > > > >> > > > > > controller
> > > > >> > > > > > > > and
> > > > >> > > > > > > > >> > logs
> > > > >> > > > > > > > >> > > > > > cleanup
> > > > >> > > > > > > > >> > > > > > >> by
> > > > >> > > > > > > > >> > > > > > >> > > > > brokers
> > > > >> > > > > > > > >> > > > > > >> > > > > > in
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > corner
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > cases.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Best,
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Satish.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > On Wed,
> Sep
> > 9,
> > > > >> 2020
> > > > >> > at
> > > > >> > > > > 10:07
> > > > >> > > > > > PM
> > > > >> > > > > > > > >> > Justine
> > > > >> > > > > > > > >> > > > > > Olshan <
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> jolshan@confluent.io
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > wrote:
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Hello
> all,
> > > > it's
> > > > >> > been
> > > > >> > > > > > almost a
> > > > >> > > > > > > > >> year!
> > > > >> > > > > > > > >> > > I've
> > > > >> > > > > > > > >> > > > > > made
> > > > >> > > > > > > > >> > > > > > >> > some
> > > > >> > > > > > > > >> > > > > > >> > > > > > changes
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> to
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > this
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > KIP
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > and hope
> to
> > > > >> continue
> > > > >> > > the
> > > > >> > > > > > > > >> discussion.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > One of
> the
> > > > main
> > > > >> > > changes
> > > > >> > > > > > I've
> > > > >> > > > > > > > >> added
> > > > >> > > > > > > > >> > is
> > > > >> > > > > > > > >> > > > now
> > > > >> > > > > > > > >> > > > > > the
> > > > >> > > > > > > > >> > > > > > >> > > > metadata
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> response
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > will
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > include
> the
> > > > topic
> > > > >> ID
> > > > >> > > (as
> > > > >> > > > > > Colin
> > > > >> > > > > > > > >> > > suggested).
> > > > >> > > > > > > > >> > > > > > >> Clients
> > > > >> > > > > > > > >> > > > > > >> > > can
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> obtain the
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > topicID
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > of a given
> > > topic
> > > > >> > > through
> > > > >> > > > a
> > > > >> > > > > > > > >> > > > TopicDescription.
> > > > >> > > > > > > > >> > > > > > The
> > > > >> > > > > > > > >> > > > > > >> > > > topicId
> > > > >> > > > > > > > >> > > > > > >> > > > > > will
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > also be
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > included
> > with
> > > > the
> > > > >> > > > > > > UpdateMetadata
> > > > >> > > > > > > > >> > > request.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Let me
> > know
> > > > what
> > > > >> > you
> > > > >> > > > all
> > > > >> > > > > > > think.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Thank
> you,
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Justine
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > On
> > > 2019/09/13
> > > > >> > > 16:38:26,
> > > > >> > > > > > > "Colin
> > > > >> > > > > > > > >> > > McCabe" <
> > > > >> > > > > > > > >> > > > > > >> > > > > > cmccabe@apache.org
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > wrote:
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Hi
> > Lucas,
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Thanks
> > for
> > > > >> > tackling
> > > > >> > > > > this.
> > > > >> > > > > > > > >> Topic
> > > > >> > > > > > > > >> > IDs
> > > > >> > > > > > > > >> > > > > are a
> > > > >> > > > > > > > >> > > > > > >> > great
> > > > >> > > > > > > > >> > > > > > >> > > > > idea,
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> and
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > this
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > is
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > really
> good
> > > > >> writeup.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > For
> > > > >> > > > > > > /brokers/topics/[topic],
> > > > >> > > > > > > > >> the
> > > > >> > > > > > > > >> > > > schema
> > > > >> > > > > > > > >> > > > > > >> version
> > > > >> > > > > > > > >> > > > > > >> > > > > should
> > > > >> > > > > > > > >> > > > > > >> > > > > > be
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > bumped
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > version 3,
> > > > rather
> > > > >> > than
> > > > >> > > 2.
> > > > >> > > > > > > > KIP-455
> > > > >> > > > > > > > >> > > bumped
> > > > >> > > > > > > > >> > > > > the
> > > > >> > > > > > > > >> > > > > > >> > version
> > > > >> > > > > > > > >> > > > > > >> > > > of
> > > > >> > > > > > > > >> > > > > > >> > > > > > this
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > znode
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 2
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > already :)
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Given
> > that
> > > > >> we're
> > > > >> > > > going
> > > > >> > > > > to
> > > > >> > > > > > > be
> > > > >> > > > > > > > >> > seeing
> > > > >> > > > > > > > >> > > > > these
> > > > >> > > > > > > > >> > > > > > >> > things
> > > > >> > > > > > > > >> > > > > > >> > > as
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> strings
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > as
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > lot
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > (in
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > logs, in
> > > > >> ZooKeeper,
> > > > >> > on
> > > > >> > > > the
> > > > >> > > > > > > > >> > command-line,
> > > > >> > > > > > > > >> > > > > > etc.),
> > > > >> > > > > > > > >> > > > > > >> > does
> > > > >> > > > > > > > >> > > > > > >> > > it
> > > > >> > > > > > > > >> > > > > > >> > > > > > make
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > sense to
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > use
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > base64
> when
> > > > >> > converting
> > > > >> > > > them
> > > > >> > > > > > to
> > > > >> > > > > > > > >> > strings?
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Here
> is
> > an
> > > > >> > example
> > > > >> > > of
> > > > >> > > > > the
> > > > >> > > > > > > hex
> > > > >> > > > > > > > >> > > > > > >> representation:
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > > >> > > > > > > > >> > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > And
> here
> > > is
> > > > an
> > > > >> > > > example
> > > > >> > > > > in
> > > > >> > > > > > > > >> base64.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > > >> > > > b8tRS7h4TJ2Vt43Dp85v2A
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > The
> > base64
> > > > >> > version
> > > > >> > > > > saves
> > > > >> > > > > > 15
> > > > >> > > > > > > > >> > letters
> > > > >> > > > > > > > >> > > > (to
> > > > >> > > > > > > > >> > > > > be
> > > > >> > > > > > > > >> > > > > > >> > fair,
> > > > >> > > > > > > > >> > > > > > >> > > 4
> > > > >> > > > > > > > >> > > > > > >> > > > of
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> those
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > were
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > dashes
> that
> > we
> > > > >> could
> > > > >> > > have
> > > > >> > > > > > > elided
> > > > >> > > > > > > > in
> > > > >> > > > > > > > >> > the
> > > > >> > > > > > > > >> > > > hex
> > > > >> > > > > > > > >> > > > > > >> > > > > > representation.)
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> Another
> > > > thing
> > > > >> to
> > > > >> > > > > consider
> > > > >> > > > > > > is
> > > > >> > > > > > > > >> that
> > > > >> > > > > > > > >> > we
> > > > >> > > > > > > > >> > > > > > should
> > > > >> > > > > > > > >> > > > > > >> > > specify
> > > > >> > > > > > > > >> > > > > > >> > > > > > that
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> the
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > all-zeroes
> > > UUID
> > > > is
> > > > >> > not
> > > > >> > > a
> > > > >> > > > > > valid
> > > > >> > > > > > > > >> topic
> > > > >> > > > > > > > >> > > UUID.
> > > > >> > > > > > > > >> > > > > >  We
> > > > >> > > > > > > > >> > > > > > >> > can't
> > > > >> > > > > > > > >> > > > > > >> > > > use
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> null
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > for
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > this
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > because we
> > > can't
> > > > >> > pass a
> > > > >> > > > > null
> > > > >> > > > > > > UUID
> > > > >> > > > > > > > >> over
> > > > >> > > > > > > > >> > > the
> > > > >> > > > > > > > >> > > > > RPC
> > > > >> > > > > > > > >> > > > > > >> > > protocol
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> (there
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > is no
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > special
> > > pattern
> > > > >> for
> > > > >> > > null,
> > > > >> > > > > nor
> > > > >> > > > > > > do
> > > > >> > > > > > > > we
> > > > >> > > > > > > > >> > want
> > > > >> > > > > > > > >> > > > to
> > > > >> > > > > > > > >> > > > > > >> waste
> > > > >> > > > > > > > >> > > > > > >> > > space
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> reserving
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > such
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > pattern.)
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Maybe
> I
> > > > missed
> > > > >> > it,
> > > > >> > > > but
> > > > >> > > > > > did
> > > > >> > > > > > > > you
> > > > >> > > > > > > > >> > > > describe
> > > > >> > > > > > > > >> > > > > > >> > > "migration
> > > > >> > > > > > > > >> > > > > > >> > > > > > of...
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > existing
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > topic[s]
> > > without
> > > > >> > topic
> > > > >> > > > IDs"
> > > > >> > > > > > in
> > > > >> > > > > > > > >> detail
> > > > >> > > > > > > > >> > in
> > > > >> > > > > > > > >> > > > any
> > > > >> > > > > > > > >> > > > > > >> > section?
> > > > >> > > > > > > > >> > > > > > >> > > > It
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> seems
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > like
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > when
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > the new
> > > > controller
> > > > >> > > > becomes
> > > > >> > > > > > > > active,
> > > > >> > > > > > > > >> it
> > > > >> > > > > > > > >> > > > should
> > > > >> > > > > > > > >> > > > > > >> just
> > > > >> > > > > > > > >> > > > > > >> > > > > generate
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> random
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > UUIDs for
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > these, and
> > > write
> > > > >> the
> > > > >> > > > random
> > > > >> > > > > > > UUIDs
> > > > >> > > > > > > > >> back
> > > > >> > > > > > > > >> > > to
> > > > >> > > > > > > > >> > > > > > >> > ZooKeeper.
> > > > >> > > > > > > > >> > > > > > >> > > > It
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> would be
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > good
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > to
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > spell that
> > > out.
> > > > >> We
> > > > >> > > > should
> > > > >> > > > > > make
> > > > >> > > > > > > > it
> > > > >> > > > > > > > >> > clear
> > > > >> > > > > > > > >> > > > > that
> > > > >> > > > > > > > >> > > > > > >> this
> > > > >> > > > > > > > >> > > > > > >> > > > > happens
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > regardless
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > of
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > the
> > > inter-broker
> > > > >> > > protocol
> > > > >> > > > > > > version
> > > > >> > > > > > > > >> > (it's
> > > > >> > > > > > > > >> > > a
> > > > >> > > > > > > > >> > > > > > >> > compatible
> > > > >> > > > > > > > >> > > > > > >> > > > > > change).
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > > >> > > "LeaderAndIsrRequests
> > > > >> > > > > > > > >> including an
> > > > >> > > > > > > > >> > > > > > >> > > > is_every_partition
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> flag"
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > seems a
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > bit wordy.
> > > Can
> > > > we
> > > > >> > just
> > > > >> > > > > call
> > > > >> > > > > > > > these
> > > > >> > > > > > > > >> > "full
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> LeaderAndIsrRequests"?
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > Then
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > the
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > RPC field
> > > could
> > > > be
> > > > >> > > named
> > > > >> > > > > > > "full".
> > > > >> > > > > > > > >> > Also,
> > > > >> > > > > > > > >> > > it
> > > > >> > > > > > > > >> > > > > > would
> > > > >> > > > > > > > >> > > > > > >> > > > probably
> > > > >> > > > > > > > >> > > > > > >> > > > > > be
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > better
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > for the
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > RPC field
> to
> > > be
> > > > an
> > > > >> > enum
> > > > >> > > > of
> > > > >> > > > > {
> > > > >> > > > > > > > >> > > UNSPECIFIED,
> > > > >> > > > > > > > >> > > > > > >> > > INCREMENTAL,
> > > > >> > > > > > > > >> > > > > > >> > > > > FULL
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> }, so
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > that
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > we
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > can
> cleanly
> > > > handle
> > > > >> > old
> > > > >> > > > > > versions
> > > > >> > > > > > > > (by
> > > > >> > > > > > > > >> > > > treating
> > > > >> > > > > > > > >> > > > > > >> them
> > > > >> > > > > > > > >> > > > > > >> > as
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> UNSPECIFIED)
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > In the
> > > > >> > > > > > LeaderAndIsrRequest
> > > > >> > > > > > > > >> > section,
> > > > >> > > > > > > > >> > > > you
> > > > >> > > > > > > > >> > > > > > >> write
> > > > >> > > > > > > > >> > > > > > >> > "A
> > > > >> > > > > > > > >> > > > > > >> > > > > final
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > deletion
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > event
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > will be
> > > > secheduled
> > > > >> > for
> > > > >> > > X
> > > > >> > > > ms
> > > > >> > > > > > > after
> > > > >> > > > > > > > >> the
> > > > >> > > > > > > > >> > > > > > >> > > > LeaderAndIsrRequest
> > > > >> > > > > > > > >> > > > > > >> > > > > > was
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > first
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> received..."
> > > I
> > > > >> guess
> > > > >> > > > the X
> > > > >> > > > > > > was a
> > > > >> > > > > > > > >> > > > > placeholder
> > > > >> > > > > > > > >> > > > > > >> that
> > > > >> > > > > > > > >> > > > > > >> > > you
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> intended
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > replace
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > before
> > > posting?
> > > > :)
> > > > >> > In
> > > > >> > > > any
> > > > >> > > > > > > case,
> > > > >> > > > > > > > >> this
> > > > >> > > > > > > > >> > > > seems
> > > > >> > > > > > > > >> > > > > > like
> > > > >> > > > > > > > >> > > > > > >> > the
> > > > >> > > > > > > > >> > > > > > >> > > > kind
> > > > >> > > > > > > > >> > > > > > >> > > > > > of
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > thing
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > we'd
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > want a
> > > > >> configuration
> > > > >> > > for.
> > > > >> > > > > > > Let's
> > > > >> > > > > > > > >> > > describe
> > > > >> > > > > > > > >> > > > > that
> > > > >> > > > > > > > >> > > > > > >> > > > > > configuration
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> key
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > somewhere
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > in this
> KIP,
> > > > >> > including
> > > > >> > > > what
> > > > >> > > > > > its
> > > > >> > > > > > > > >> > default
> > > > >> > > > > > > > >> > > > > value
> > > > >> > > > > > > > >> > > > > > >> is.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > We
> > should
> > > > >> > probably
> > > > >> > > > also
> > > > >> > > > > > > log a
> > > > >> > > > > > > > >> > bunch
> > > > >> > > > > > > > >> > > of
> > > > >> > > > > > > > >> > > > > > >> messages
> > > > >> > > > > > > > >> > > > > > >> > > at
> > > > >> > > > > > > > >> > > > > > >> > > > > WARN
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> level
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > when
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > something
> is
> > > > >> > scheduled
> > > > >> > > > for
> > > > >> > > > > > > > >> deletion,
> > > > >> > > > > > > > >> > as
> > > > >> > > > > > > > >> > > > > well.
> > > > >> > > > > > > > >> > > > > > >> > (Maybe
> > > > >> > > > > > > > >> > > > > > >> > > > > this
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> was
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > assumed, but
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > it would
> be
> > > good
> > > > >> to
> > > > >> > > > mention
> > > > >> > > > > > > it).
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > I feel
> > > like
> > > > >> there
> > > > >> > > > are a
> > > > >> > > > > > few
> > > > >> > > > > > > > >> > sections
> > > > >> > > > > > > > >> > > > > that
> > > > >> > > > > > > > >> > > > > > >> > should
> > > > >> > > > > > > > >> > > > > > >> > > be
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> moved to
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > "rejected
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > alternatives."
> > > > >> For
> > > > >> > > > > example,
> > > > >> > > > > > in
> > > > >> > > > > > > > the
> > > > >> > > > > > > > >> > > > > > DeleteTopics
> > > > >> > > > > > > > >> > > > > > >> > > > section,
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> since
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > we're
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > not
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > going to
> do
> > > > >> option 1
> > > > >> > or
> > > > >> > > > 2,
> > > > >> > > > > > > these
> > > > >> > > > > > > > >> > should
> > > > >> > > > > > > > >> > > be
> > > > >> > > > > > > > >> > > > > > moved
> > > > >> > > > > > > > >> > > > > > >> > into
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> "rejected
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > alternatives,"
> > > > >> > rather
> > > > >> > > > than
> > > > >> > > > > > > > >> appearing
> > > > >> > > > > > > > >> > > > > inline.
> > > > >> > > > > > > > >> > > > > > >> > > Another
> > > > >> > > > > > > > >> > > > > > >> > > > > case
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> is
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > the
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > "Should
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > we remove
> > > topic
> > > > >> name
> > > > >> > > from
> > > > >> > > > > the
> > > > >> > > > > > > > >> protocol
> > > > >> > > > > > > > >> > > > where
> > > > >> > > > > > > > >> > > > > > >> > > possible"
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> section.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > This
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > is
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > clearly
> > > > >> discussing a
> > > > >> > > > design
> > > > >> > > > > > > > >> > alternative
> > > > >> > > > > > > > >> > > > that
> > > > >> > > > > > > > >> > > > > > >> we're
> > > > >> > > > > > > > >> > > > > > >> > > not
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> proposing
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > implement:
> > > > >> removing
> > > > >> > the
> > > > >> > > > > topic
> > > > >> > > > > > > > name
> > > > >> > > > > > > > >> > from
> > > > >> > > > > > > > >> > > > > those
> > > > >> > > > > > > > >> > > > > > >> > > > protocols.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Is it
> > > really
> > > > >> > > > necessary
> > > > >> > > > > to
> > > > >> > > > > > > > have
> > > > >> > > > > > > > >> a
> > > > >> > > > > > > > >> > new
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > /admin/delete_topics_by_id
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > path
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > in
> > ZooKeeper?
> > > > It
> > > > >> > seems
> > > > >> > > > > like
> > > > >> > > > > > we
> > > > >> > > > > > > > >> don't
> > > > >> > > > > > > > >> > > > really
> > > > >> > > > > > > > >> > > > > > >> need
> > > > >> > > > > > > > >> > > > > > >> > > this.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Whenever
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > there is
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > a new
> > > > controller,
> > > > >> > we'll
> > > > >> > > > > send
> > > > >> > > > > > > out
> > > > >> > > > > > > > >> full
> > > > >> > > > > > > > >> > > > > > >> > > > > LeaderAndIsrRequests
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> which
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > will
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > trigger
> the
> > > > stale
> > > > >> > > topics
> > > > >> > > > to
> > > > >> > > > > > be
> > > > >> > > > > > > > >> cleaned
> > > > >> > > > > > > > >> > > up.
> > > > >> > > > > > > > >> > > > > >  The
> > > > >> > > > > > > > >> > > > > > >> > > active
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > controller
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > will
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > also send
> > the
> > > > full
> > > > >> > > > > > > > >> LeaderAndIsrRequest
> > > > >> > > > > > > > >> > > to
> > > > >> > > > > > > > >> > > > > > >> brokers
> > > > >> > > > > > > > >> > > > > > >> > > that
> > > > >> > > > > > > > >> > > > > > >> > > > > are
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> just
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > starting
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > up.    So
> we
> > > > don't
> > > > >> > > really
> > > > >> > > > > > need
> > > > >> > > > > > > > this
> > > > >> > > > > > > > >> > kind
> > > > >> > > > > > > > >> > > > of
> > > > >> > > > > > > > >> > > > > > >> > two-phase
> > > > >> > > > > > > > >> > > > > > >> > > > > > commit
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > (send
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > out
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > >> StopReplicasRequest,
> > > > >> > > get
> > > > >> > > > > ACKs
> > > > >> > > > > > > > from
> > > > >> > > > > > > > >> all
> > > > >> > > > > > > > >> > > > > nodes,
> > > > >> > > > > > > > >> > > > > > >> > commit
> > > > >> > > > > > > > >> > > > > > >> > > by
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> removing
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > >> /admin/delete_topics
> > > > >> > > > node)
> > > > >> > > > > > any
> > > > >> > > > > > > > >> more.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > You
> > > mention
> > > > >> that
> > > > >> > > > > > > FetchRequest
> > > > >> > > > > > > > >> will
> > > > >> > > > > > > > >> > > now
> > > > >> > > > > > > > >> > > > > > >> include
> > > > >> > > > > > > > >> > > > > > >> > > UUID
> > > > >> > > > > > > > >> > > > > > >> > > > > to
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> avoid
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > issues
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > where
> > requests
> > > > are
> > > > >> > made
> > > > >> > > > to
> > > > >> > > > > > > stale
> > > > >> > > > > > > > >> > > > partitions.
> > > > >> > > > > > > > >> > > > > > >> > > However,
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> adding a
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > UUID
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > MetadataRequest
> > > > is
> > > > >> > > listed
> > > > >> > > > > as
> > > > >> > > > > > > > future
> > > > >> > > > > > > > >> > > work,
> > > > >> > > > > > > > >> > > > > out
> > > > >> > > > > > > > >> > > > > > of
> > > > >> > > > > > > > >> > > > > > >> > > scope
> > > > >> > > > > > > > >> > > > > > >> > > > > for
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> this
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > KIP.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > How
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > will the
> > > client
> > > > >> learn
> > > > >> > > > what
> > > > >> > > > > > the
> > > > >> > > > > > > > >> topic
> > > > >> > > > > > > > >> > > UUID
> > > > >> > > > > > > > >> > > > > is,
> > > > >> > > > > > > > >> > > > > > if
> > > > >> > > > > > > > >> > > > > > >> > the
> > > > >> > > > > > > > >> > > > > > >> > > > > > metadata
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > response
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > doesn't
> > > include
> > > > >> that
> > > > >> > > > > > > information?
> > > > >> > > > > > > > >> It
> > > > >> > > > > > > > >> > > > seems
> > > > >> > > > > > > > >> > > > > > like
> > > > >> > > > > > > > >> > > > > > >> > > adding
> > > > >> > > > > > > > >> > > > > > >> > > > > the
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> UUID
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > MetadataResponse
> > > > >> > would
> > > > >> > > be
> > > > >> > > > > an
> > > > >> > > > > > > > >> > improvement
> > > > >> > > > > > > > >> > > > > here
> > > > >> > > > > > > > >> > > > > > >> that
> > > > >> > > > > > > > >> > > > > > >> > > > might
> > > > >> > > > > > > > >> > > > > > >> > > > > > not
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> be
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > too
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > hard to
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > make.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > best,
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Colin
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > On
> Mon,
> > > Sep
> > > > 9,
> > > > >> > > 2019,
> > > > >> > > > at
> > > > >> > > > > > > > 17:48,
> > > > >> > > > > > > > >> > > Ryanne
> > > > >> > > > > > > > >> > > > > > Dolan
> > > > >> > > > > > > > >> > > > > > >> > > wrote:
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> Lucas,
> > > > this
> > > > >> > would
> > > > >> > > > be
> > > > >> > > > > > > great.
> > > > >> > > > > > > > >> I've
> > > > >> > > > > > > > >> > > run
> > > > >> > > > > > > > >> > > > > > into
> > > > >> > > > > > > > >> > > > > > >> > > issues
> > > > >> > > > > > > > >> > > > > > >> > > > > with
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > topics
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > being
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> > > > resurrected
> > > > >> > > > > > accidentally,
> > > > >> > > > > > > > >> since
> > > > >> > > > > > > > >> > a
> > > > >> > > > > > > > >> > > > > client
> > > > >> > > > > > > > >> > > > > > >> > cannot
> > > > >> > > > > > > > >> > > > > > >> > > > > > easily
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > distinguish
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > between
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > a
> > > deleted
> > > > >> topic
> > > > >> > > > and a
> > > > >> > > > > > new
> > > > >> > > > > > > > >> topic
> > > > >> > > > > > > > >> > > with
> > > > >> > > > > > > > >> > > > > the
> > > > >> > > > > > > > >> > > > > > >> same
> > > > >> > > > > > > > >> > > > > > >> > > > name.
> > > > >> > > > > > > > >> > > > > > >> > > > > > I'd
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > need
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > the
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > ID
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> > > accessible
> > > > >> from
> > > > >> > > the
> > > > >> > > > > > > client
> > > > >> > > > > > > > to
> > > > >> > > > > > > > >> > > solve
> > > > >> > > > > > > > >> > > > > that
> > > > >> > > > > > > > >> > > > > > >> > issue,
> > > > >> > > > > > > > >> > > > > > >> > > > but
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> this
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > is a
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > good
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > first
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> step.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> Ryanne
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > On
> > Wed,
> > > > Sep
> > > > >> 4,
> > > > >> > > 2019
> > > > >> > > > > at
> > > > >> > > > > > > 1:41
> > > > >> > > > > > > > >> PM
> > > > >> > > > > > > > >> > > Lucas
> > > > >> > > > > > > > >> > > > > > >> > > Bradstreet <
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > lucas@confluent.io>
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > wrote:
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Hi
> > > all,
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > I
> > > would
> > > > >> like
> > > > >> > to
> > > > >> > > > > kick
> > > > >> > > > > > > off
> > > > >> > > > > > > > >> > > > discussion
> > > > >> > > > > > > > >> > > > > of
> > > > >> > > > > > > > >> > > > > > >> > > KIP-516,
> > > > >> > > > > > > > >> > > > > > >> > > > > an
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > implementation
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > of topic
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> IDs
> > > for
> > > > >> > Kafka.
> > > > >> > > > > Topic
> > > > >> > > > > > > IDs
> > > > >> > > > > > > > >> aim
> > > > >> > > > > > > > >> > to
> > > > >> > > > > > > > >> > > > > solve
> > > > >> > > > > > > > >> > > > > > >> topic
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> uniqueness
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > problems in
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Kafka,
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > where
> > > > >> > referring
> > > > >> > > > to
> > > > >> > > > > a
> > > > >> > > > > > > > topic
> > > > >> > > > > > > > >> by
> > > > >> > > > > > > > >> > > name
> > > > >> > > > > > > > >> > > > > > >> alone is
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> insufficient.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Such
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > cases
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > include
> > > > >> when
> > > > >> > a
> > > > >> > > > > topic
> > > > >> > > > > > > has
> > > > >> > > > > > > > >> been
> > > > >> > > > > > > > >> > > > > deleted
> > > > >> > > > > > > > >> > > > > > >> and
> > > > >> > > > > > > > >> > > > > > >> > > > > recreated
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> with
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > the
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > same
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > name.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > Unique
> > > > >> > > > identifiers
> > > > >> > > > > > will
> > > > >> > > > > > > > >> help
> > > > >> > > > > > > > >> > > > > simplify
> > > > >> > > > > > > > >> > > > > > >> and
> > > > >> > > > > > > > >> > > > > > >> > > > improve
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Kafka's
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > topic
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > deletion
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > process,
> > > > >> as
> > > > >> > > well
> > > > >> > > > as
> > > > >> > > > > > > > prevent
> > > > >> > > > > > > > >> > > cases
> > > > >> > > > > > > > >> > > > > > where
> > > > >> > > > > > > > >> > > > > > >> > > brokers
> > > > >> > > > > > > > >> > > > > > >> > > > > may
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > incorrectly
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > interact
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> with
> > > > stale
> > > > >> > > > versions
> > > > >> > > > > > of
> > > > >> > > > > > > > >> topics.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > >> > > > > > > > >> > > > > > >> > >
> > > > >> > > > > > > > >> > > > > > >> >
> > > > >> > > > > > > > >> > > > > > >>
> > > > >> > > > > > > > >> > > > > >
> > > > >> > > > > > > > >> > > > >
> > > > >> > > > > > > > >> > > >
> > > > >> > > > > > > > >> > >
> > > > >> > > > > > > > >> >
> > > > >> > > > > > > > >>
> > > > >> > > > > > > >
> > > > >> > > > > > >
> > > > >> > > > > >
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > Looking
> > > > >> > forward
> > > > >> > > > to
> > > > >> > > > > > your
> > > > >> > > > > > > > >> > > thoughts.
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > Lucas
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > > >> > > > > > > > >> > > > > > >> > > > > >
> > > > >> > > > > > > > >> > > > > > >> > > > >
> > > > >> > > > > > > > >> > > > > > >> > > >
> > > > >> > > > > > > > >> > > > > > >> > >
> > > > >> > > > > > > > >> > > > > > >> >
> > > > >> > > > > > > > >> > > > > > >>
> > > > >> > > > > > > > >> > > > > > >
> > > > >> > > > > > > > >> > > > > >
> > > > >> > > > > > > > >> > > > >
> > > > >> > > > > > > > >> > > >
> > > > >> > > > > > > > >> > >
> > > > >> > > > > > > > >> >
> > > > >> > > > > > > > >>
> > > > >> > > > > > > > >
> > > > >> > > > > > > >
> > > > >> > > > > > >
> > > > >> > > > > >
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Jun Rao <ju...@confluent.io>]

Hi, Justine,

Thanks for the update. The KIP looks good to me now. Just a minor comment
below.

30. Perhaps "option[UUID]" can be put in the doc.

Jun

On Thu, Oct 1, 2020 at 3:28 PM Justine Olshan <jo...@confluent.io> wrote:

> Hi Jun,
> Thanks for the response!
>
> 30. I think I might have changed this in between. The current version
> says:  {"name":
> "id", "type": "option[UUID]"}, "doc": topic id}
> I have switched to the option type to cover the migration case where a
> TopicZNode does not yet have a topic ID.
> I understand that due to json, this field is written as a string, so if I
> should move the "option[uuid]" to the doc field and the type should be
> "string" please let me know.
>
> 40. I've added a definition for UUID.
> 41,42. Fixed
>
> Thank you,
> Justine
>
> On Wed, Sep 30, 2020 at 1:15 PM Jun Rao <ju...@confluent.io> wrote:
>
> > Hi, Justine,
> >
> > Thanks for the summary. Just a few minor comments blow.
> >
> > 30.  {"name": "id", "type": "string", "doc": "version id"}}: The doc
> should
> > say UUID. The issue seems unfixed.
> >
> > 40. Since UUID is public facing, could you include its definition?
> >
> > 41. StopReplicaResponse still includes the topic field.
> >
> > 42. "It is unnecessary to include the name of the topic in the following
> > Request/Response calls" It would be useful to include all modified
> requests
> > (e.g. produce) in the list below.
> >
> > Thanks,
> >
> > Jun
> >
> >
> > On Wed, Sep 30, 2020 at 10:17 AM Justine Olshan <jo...@confluent.io>
> > wrote:
> >
> > > Hello again,
> > >
> > > I've taken some time to discuss some of the remaining points brought up
> > by
> > > the previous emails offline. Here are some of the conclusions.
> > >
> > > 1. Directory Structure:
> > > There was some talk about whether the directory structure should be
> > changed
> > > to replace all topic names with topic IDs.
> > > This will be a useful change, but will prevent downgrades. It will be
> > best
> > > to wait until a major release, likely alongside KIP-500 changes that
> will
> > > prevent downgrades. I've updated the KIP to include this change with
> the
> > > note about migration and deprecation.
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-log.dirlayout
> > >
> > > 2. Partition Metadata file
> > > There was some disagreement about the inclusion of the partition
> metadata
> > > file.
> > > This file will be needed to persist the topic ID, especially while we
> > still
> > > have the old directory structure. Even after the changes, this file can
> > be
> > > useful for debugging and recovery.
> > > Creating a single mapping file from topic names to topic IDs was
> > > considered, but ultimately scrapped as it would not be as easy to
> > maintain.
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PartitionMetadatafile
> > >
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PersistingTopicIDs
> > >
> > > 3. Produce Protocols
> > > After some further discussion, this replacing the topic name with topic
> > ID
> > > in these protocols has been added to the KIP.
> > > This will cut down on the size of the protocol. Since changes to fetch
> > are
> > > included, it does make sense to update these protocols.
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-Produce
> > >
> > > 4. KIP-500 Compatibility
> > > After some discussion with those involved with KIP-500, it seems best
> to
> > > use a sentinel topic ID for the metadata topic that is used before the
> > > first controller election.
> > > We had an issue where this metadata topic may not be assigned an ID
> > before
> > > utilizing the Vote and Fetch protocols. It was decided to reserve a
> > unique
> > > ID for this topic to be used until the controller could give the topic
> a
> > > unique ID.
> > > Having the topic keep the sentinel ID (not be reassigned to a unique
> ID)
> > > was considered, but it seemed like a good idea to leave the option open
> > for
> > > the metadata topic to have a unique ID in cases where it would need to
> be
> > > differentiated from other clusters' metadata topics. (ex. tiered
> storage)
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-CompatibilitywithKIP-500
> > >
> > > I've also split up the KIP into sub-tasks on the JIRA. Hopefully this
> > will
> > > provide a better idea about what tasks we have, and eventually provide
> a
> > > place to see what's done and what is left.
> > > If there is a task I am missing, please let me know!
> > > https://issues.apache.org/jira/browse/KAFKA-8872
> > >
> > > Of course, these decisions are not set in stone, and I would love to
> hear
> > > any feedback.
> > >
> > > Thanks,
> > > Justine
> > >
> > > On Mon, Sep 28, 2020 at 11:38 AM Justine Olshan <jo...@confluent.io>
> > > wrote:
> > >
> > > > Hello all,
> > > >
> > > > I just wanted to follow up on this discussion. Did we come to an
> > > > understanding about the directory structure?
> > > >
> > > > I think the biggest question here is what is acceptable to leave out
> > due
> > > > to scope vs. what is considered to be too much tech debt.
> > > > This KIP is already pretty large with the number of changes, but it
> > also
> > > > makes sense to do things correctly, so I'd love to hear everyone's
> > > thoughts.
> > > >
> > > > Thanks,
> > > > Justine
> > > >
> > > > On Fri, Sep 25, 2020 at 8:19 AM Lucas Bradstreet <lucas@confluent.io
> >
> > > > wrote:
> > > >
> > > >> Hi Ismael,
> > > >>
> > > >> If you do not store it in a metadata file or in the directory
> > structure
> > > >> would you then
> > > >> require the LeaderAndIsrRequest following startup to give you some
> > > notion
> > > >> of
> > > >> topic name in memory? We would still need this information for the
> > older
> > > >> protocols, but
> > > >> perhaps this is what's meant by tech debt.
> > > >>
> > > >> Once we're free of the old non-topicID requests then I think you
> > > wouldn't
> > > >> need to retain the topic name.
> > > >> I think the ability to easily look up topic names associated with
> > > >> partition
> > > >> directories would still be missed
> > > >> when diagnosing issues, though maybe it wouldn't be a deal breaker
> > with
> > > >> the
> > > >> right tooling.
> > > >>
> > > >> Thanks,
> > > >>
> > > >> Lucas
> > > >>
> > > >> On Fri, Sep 25, 2020 at 7:55 AM Ismael Juma <is...@juma.me.uk>
> > wrote:
> > > >>
> > > >> > Hi Lucas,
> > > >> >
> > > >> > Why would you include the name and id? I think you'd want to
> remove
> > > the
> > > >> > name from the directory name right? Jason's suggestion was that if
> > you
> > > >> > remove the name from the directory, then why would you need the id
> > > name
> > > >> > mapping file?
> > > >> >
> > > >> > Ismael
> > > >> >
> > > >> > On Thu, Sep 24, 2020 at 4:24 PM Lucas Bradstreet <
> > lucas@confluent.io>
> > > >> > wrote:
> > > >> >
> > > >> > > > 2. Part of the usage of the file is to have persistent storage
> > of
> > > >> the
> > > >> > > topic
> > > >> > > ID and use it to compare with the ID supplied in the
> LeaderAndIsr
> > > >> > Request.
> > > >> > > There is some discussion in the KIP about changes to the
> directory
> > > >> > > structure, but I believe directory changes were considered to be
> > out
> > > >> of
> > > >> > > scope when the KIP was written.
> > > >> > >
> > > >> > >
> > > >> > > Yeah, I was hoping to get a better understanding of why it was
> > taken
> > > >> out
> > > >> > of
> > > >> > > scope. Perhaps Lucas Bradstreet might have more insight about
> the
> > > >> > decision.
> > > >> > > Basically my point is that we have to create additional
> > > infrastructure
> > > >> > here
> > > >> > > to support the name/id mapping, so I wanted to understand if we
> > just
> > > >> > > consider this a sort of tech debt. It would be useful to cover
> how
> > > we
> > > >> > > handle the case when this file gets corrupted. Seems like we
> just
> > > >> have to
> > > >> > > trust that it matches whatever the controller tells us and
> rewrite
> > > it?
> > > >> > >
> > > >> > >
> > > >> > > Hi Jason, Justine,
> > > >> > >
> > > >> > > My thought process is that we will likely need the metadata file
> > > >> whether
> > > >> > we
> > > >> > > rename the directories or not.
> > > >> > > Linux supports filenames of up to 255 bytes and that would not
> be
> > > >> enough
> > > >> > to
> > > >> > > support a directory name
> > > >> > >  including both the name and topic ID. Given that fact, it
> seemed
> > > >> > > reasonable to add the metadata file
> > > >> > > and leave the directory rename until some time in the future
> > > (possibly
> > > >> > > never).
> > > >> > >
> > > >> > > If the file does get corrupted I think you're right. We would
> > either
> > > >> have
> > > >> > > to trust it matches what the controller tells us
> > > >> > >  or error out and let an administrator resolve it by checking
> > across
> > > >> > > replicas for consistency.
> > > >> > >
> > > >> > > Lucas
> > > >> > >
> > > >> > >
> > > >> > > On Thu, Sep 24, 2020 at 3:41 PM Jason Gustafson <
> > jason@confluent.io
> > > >
> > > >> > > wrote:
> > > >> > >
> > > >> > > > Thanks Justine. Responses below:
> > > >> > > >
> > > >> > > > > 1. Yes, the directory will still be based on the topic
> names.
> > > >> > > > LeaderAndIsrRequest is one of the few requests that will still
> > > >> contain
> > > >> > > the
> > > >> > > > topic name. So I think we have this covered. Sorry for
> > confusion.
> > > >> > > >
> > > >> > > > Ah, you're right. My eyes passed right over the field.
> > > >> > > >
> > > >> > > > > 2. Part of the usage of the file is to have persistent
> storage
> > > of
> > > >> the
> > > >> > > > topic
> > > >> > > > ID and use it to compare with the ID supplied in the
> > LeaderAndIsr
> > > >> > > Request.
> > > >> > > > There is some discussion in the KIP about changes to the
> > directory
> > > >> > > > structure, but I believe directory changes were considered to
> be
> > > >> out of
> > > >> > > > scope when the KIP was written.
> > > >> > > >
> > > >> > > > Yeah, I was hoping to get a better understanding of why it was
> > > taken
> > > >> > out
> > > >> > > of
> > > >> > > > scope. Perhaps Lucas Bradstreet might have more insight about
> > the
> > > >> > > decision.
> > > >> > > > Basically my point is that we have to create additional
> > > >> infrastructure
> > > >> > > here
> > > >> > > > to support the name/id mapping, so I wanted to understand if
> we
> > > just
> > > >> > > > consider this a sort of tech debt. It would be useful to cover
> > how
> > > >> we
> > > >> > > > handle the case when this file gets corrupted. Seems like we
> > just
> > > >> have
> > > >> > to
> > > >> > > > trust that it matches whatever the controller tells us and
> > rewrite
> > > >> it?
> > > >> > > >
> > > >> > > > > 3. I think this is a good point, but I again I wonder about
> > the
> > > >> scope
> > > >> > > of
> > > >> > > > the KIP. Most of the changes mentioned in the KIP are for
> > > supporting
> > > >> > > topic
> > > >> > > > deletion and I believe that is why the produce request was
> > listed
> > > >> under
> > > >> > > > future work.
> > > >> > > >
> > > >> > > > That's fair. I brought it up since `Fetch` is already
> included.
> > If
> > > >> > we've
> > > >> > > > got `Metadata` and `Fetch`, seems we may as well do `Produce`
> > and
> > > >> save
> > > >> > an
> > > >> > > > extra kip. No strong objection though if you want to leave it
> > out.
> > > >> > > >
> > > >> > > >
> > > >> > > > -Jason
> > > >> > > >
> > > >> > > >
> > > >> > > > On Thu, Sep 24, 2020 at 3:26 PM Justine Olshan <
> > > >> jolshan@confluent.io>
> > > >> > > > wrote:
> > > >> > > >
> > > >> > > > > Hi Jason,
> > > >> > > > >
> > > >> > > > > Thanks for your comments.
> > > >> > > > >
> > > >> > > > > 1. Yes, the directory will still be based on the topic
> names.
> > > >> > > > > LeaderAndIsrRequest is one of the few requests that will
> still
> > > >> > contain
> > > >> > > > the
> > > >> > > > > topic name. So I think we have this covered. Sorry for
> > > confusion.
> > > >> > > > >
> > > >> > > > > 2. Part of the usage of the file is to have persistent
> storage
> > > of
> > > >> the
> > > >> > > > topic
> > > >> > > > > ID and use it to compare with the ID supplied in the
> > > LeaderAndIsr
> > > >> > > > Request.
> > > >> > > > > There is some discussion in the KIP about changes to the
> > > directory
> > > >> > > > > structure, but I believe directory changes were considered
> to
> > be
> > > >> out
> > > >> > of
> > > >> > > > > scope when the KIP was written.
> > > >> > > > >
> > > >> > > > > 3. I think this is a good point, but I again I wonder about
> > the
> > > >> scope
> > > >> > > of
> > > >> > > > > the KIP. Most of the changes mentioned in the KIP are for
> > > >> supporting
> > > >> > > > topic
> > > >> > > > > deletion and I believe that is why the produce request was
> > > listed
> > > >> > under
> > > >> > > > > future work.
> > > >> > > > >
> > > >> > > > > 4. This sounds like it might be a good solution, but I will
> > need
> > > >> to
> > > >> > > > discuss
> > > >> > > > > more with KIP-500 folks to get the details right.
> > > >> > > > >
> > > >> > > > > Thanks,
> > > >> > > > > Justine
> > > >> > > > >
> > > >> > > > > On Thu, Sep 24, 2020 at 12:30 PM Jason Gustafson <
> > > >> jason@confluent.io
> > > >> > >
> > > >> > > > > wrote:
> > > >> > > > >
> > > >> > > > > > Hi Justine,
> > > >> > > > > >
> > > >> > > > > > Thanks for picking up this work. I have a few
> > > >> questions/comments:
> > > >> > > > > >
> > > >> > > > > > 1. It sounds like the directory structure is still going
> to
> > be
> > > >> > based
> > > >> > > on
> > > >> > > > > > topic names. Do I have that right? One complication is
> that
> > > the
> > > >> > > > > > LeaderAndIsr request does not include the topic name any
> > > longer.
> > > >> > This
> > > >> > > > > means
> > > >> > > > > > that a replica must wait for the UpdateMetadata request to
> > > >> arrive
> > > >> > > with
> > > >> > > > > the
> > > >> > > > > > topic name mapping before it can create the directory. I
> > > wonder
> > > >> if
> > > >> > it
> > > >> > > > > would
> > > >> > > > > > be simpler to avoid assumptions on the ordering of
> > > >> UpdateMetadata
> > > >> > and
> > > >> > > > let
> > > >> > > > > > LeaderAndIsr include the topic name as well. Feels like we
> > are
> > > >> not
> > > >> > > > saving
> > > >> > > > > > that much by excluding it.
> > > >> > > > > >
> > > >> > > > > > 2. On a related note, it seems that the reason we have the
> > > >> > partition
> > > >> > > > > > metadata file is because we are not changing the directory
> > > >> > structure.
> > > >> > > > We
> > > >> > > > > > need it so that we remember which directories map to which
> > > topic
> > > >> > id.
> > > >> > > I
> > > >> > > > > > think it would be useful to add some detail about why
> > changing
> > > >> the
> > > >> > > > > > directory layout was rejected. Basically trying to
> > understand
> > > >> how
> > > >> > > much
> > > >> > > > > > effort we are saving by skipping this step if we have to
> > > >> implement
> > > >> > > this
> > > >> > > > > > additional bookkeeping. One concern I have for example is
> > that
> > > >> the
> > > >> > > > > > partition metadata file gets corrupt and then all bets are
> > off
> > > >> as
> > > >> > far
> > > >> > > > as
> > > >> > > > > > topic consistency.
> > > >> > > > > >
> > > >> > > > > > 3. Is it worth adding support now for topic ids in the
> > > `Produce`
> > > >> > > > request
> > > >> > > > > > now? Overhead is mentioned as one of the motivations and
> the
> > > >> best
> > > >> > > APIs
> > > >> > > > to
> > > >> > > > > > get savings from are `Produce` and `Fetch`.
> > > >> > > > > >
> > > >> > > > > > 4. When it comes to bootstrapping the metadata topic with
> > > >> respect
> > > >> > to
> > > >> > > > > > KIP-500, one suggestion would be to reserve a sentinel
> topic
> > > ID
> > > >> > which
> > > >> > > > can
> > > >> > > > > > be used initially by new brokers. When the first
> controller
> > is
> > > >> > > elected,
> > > >> > > > > it
> > > >> > > > > > can assign a topicId to the metadata topic. This is a bit
> > > >> similar
> > > >> > to
> > > >> > > > how
> > > >> > > > > we
> > > >> > > > > > were planning to generate the clusterId.
> > > >> > > > > >
> > > >> > > > > > Thanks,
> > > >> > > > > > Jason
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > > On Thu, Sep 24, 2020 at 11:10 AM Jun Rao <
> jun@confluent.io>
> > > >> wrote:
> > > >> > > > > >
> > > >> > > > > > > Hi, Justine,
> > > >> > > > > > >
> > > >> > > > > > > Thanks for the updated KIP. A few more comments below.
> > > >> > > > > > >
> > > >> > > > > > > 30.  {"name": "id", "type": "string", "doc": "version
> > id"}}:
> > > >> The
> > > >> > > doc
> > > >> > > > > > should
> > > >> > > > > > > say UUID.
> > > >> > > > > > >
> > > >> > > > > > > 31. LeaderAndIsrResponse v5 and StopReplicaResponse v4 :
> > It
> > > >> seems
> > > >> > > > there
> > > >> > > > > > is
> > > >> > > > > > > no need to add topic_id at partitions level.
> > > >> > > > > > >
> > > >> > > > > > > 32. Regarding partition metadata file. Perhaps the key
> can
> > > be
> > > >> a
> > > >> > > > single
> > > >> > > > > > > word, sth like the following.
> > > >> > > > > > > version: 0
> > > >> > > > > > > topic_id: 46bdb63f-9e8d-4a38-bf7b-ee4eb2a794e4
> > > >> > > > > > >
> > > >> > > > > > > 33. Another tricky thing that I realized is how to
> support
> > > the
> > > >> > > > metadata
> > > >> > > > > > > topic introduced in KIP-595. It's a bit tricky to
> assign a
> > > >> UUID
> > > >> > to
> > > >> > > > the
> > > >> > > > > > > metadata topic since we have a chicken-and-egg problem.
> > The
> > > >> > > > controller
> > > >> > > > > > > needs to persist the UUID in the metadata topic in order
> > to
> > > >> > assign
> > > >> > > > one
> > > >> > > > > > > successfully, but the metadata topic is needed to elect
> a
> > > >> > > controller
> > > >> > > > > > > (KIP-631). So, this probably needs a bit more thought.
> > > >> > > > > > >
> > > >> > > > > > > Jun
> > > >> > > > > > >
> > > >> > > > > > > On Thu, Sep 24, 2020 at 3:04 AM Ismael Juma <
> > > >> ismael@juma.me.uk>
> > > >> > > > wrote:
> > > >> > > > > > >
> > > >> > > > > > > > Also, can we provide more details on how the Partition
> > > >> Metadata
> > > >> > > > file
> > > >> > > > > > will
> > > >> > > > > > > > be used?
> > > >> > > > > > > >
> > > >> > > > > > > > Ismael
> > > >> > > > > > > >
> > > >> > > > > > > > On Thu, Sep 24, 2020 at 3:01 AM Ismael Juma <
> > > >> ismael@juma.me.uk
> > > >> > >
> > > >> > > > > wrote:
> > > >> > > > > > > >
> > > >> > > > > > > > > Hi Justine,
> > > >> > > > > > > > >
> > > >> > > > > > > > > I think we need to update the "Rejected
> Alternatives"
> > > >> section
> > > >> > > to
> > > >> > > > > take
> > > >> > > > > > > > into
> > > >> > > > > > > > > account that the proposal now removes the topic name
> > > from
> > > >> the
> > > >> > > > fetch
> > > >> > > > > > > > > request. Also, if we are removing it from the Fetch
> > > >> request,
> > > >> > > does
> > > >> > > > > it
> > > >> > > > > > > make
> > > >> > > > > > > > > sense not to remove it from similar requests like
> > > >> > > > > ListOffsetRequest?
> > > >> > > > > > > > >
> > > >> > > > > > > > > Ismael
> > > >> > > > > > > > >
> > > >> > > > > > > > > On Thu, Sep 24, 2020 at 2:46 AM David Jacot <
> > > >> > > djacot@confluent.io
> > > >> > > > >
> > > >> > > > > > > wrote:
> > > >> > > > > > > > >
> > > >> > > > > > > > >> Hi Justine,
> > > >> > > > > > > > >>
> > > >> > > > > > > > >> Thanks for the KIP. I finally had time to read it
> :).
> > > It
> > > >> is
> > > >> > a
> > > >> > > > > great
> > > >> > > > > > > > >> improvement.
> > > >> > > > > > > > >>
> > > >> > > > > > > > >> I have a few comments/questions:
> > > >> > > > > > > > >>
> > > >> > > > > > > > >> 1. It seems that the schema of the
> StopReplicaRequest
> > > is
> > > >> > > > slightly
> > > >> > > > > > > > >> outdated.
> > > >> > > > > > > > >> We
> > > >> > > > > > > > >> did some changes as part of KIP-570. V3 is already
> > > >> organized
> > > >> > > by
> > > >> > > > > > > topics.
> > > >> > > > > > > > >>
> > > >> > > > > > > > >> 2. I just want to make sure that I understand the
> > > >> > > reconciliation
> > > >> > > > > > > > >> logic correctly. When
> > > >> > > > > > > > >> an "INCREMENTAL" LeaderAndIsr Request is received,
> > the
> > > >> > broker
> > > >> > > > will
> > > >> > > > > > > also
> > > >> > > > > > > > >> reconcile
> > > >> > > > > > > > >> when the local uuid does not match the uuid in the
> > > >> request,
> > > >> > > > right?
> > > >> > > > > > In
> > > >> > > > > > > > this
> > > >> > > > > > > > >> case, the
> > > >> > > > > > > > >> local log is staged for deletion.
> > > >> > > > > > > > >>
> > > >> > > > > > > > >> 3. In the documentation of the `
> > > >> delete.stale.topic.delay.ms
> > > >> > `
> > > >> > > > > > config,
> > > >> > > > > > > it
> > > >> > > > > > > > >> says "When a
> > > >> > > > > > > > >> FULL LeaderAndIsrRequest is received..." but I
> > suppose
> > > it
> > > >> > > > applies
> > > >> > > > > to
> > > >> > > > > > > > both
> > > >> > > > > > > > >> types.
> > > >> > > > > > > > >>
> > > >> > > > > > > > >> Best,
> > > >> > > > > > > > >> David
> > > >> > > > > > > > >>
> > > >> > > > > > > > >> On Thu, Sep 24, 2020 at 1:40 AM Justine Olshan <
> > > >> > > > > > jolshan@confluent.io>
> > > >> > > > > > > > >> wrote:
> > > >> > > > > > > > >>
> > > >> > > > > > > > >> > Hi Jun,
> > > >> > > > > > > > >> >
> > > >> > > > > > > > >> > Thanks for the comments. I apologize for some of
> > the
> > > >> typos
> > > >> > > and
> > > >> > > > > > > > >> confusion.
> > > >> > > > > > > > >> > I’ve updated the KIP to fix some of the issues
> you
> > > >> > > mentioned.
> > > >> > > > > > > > >> >
> > > >> > > > > > > > >> > 20.2 I’ve changed the type to String.
> > > >> > > > > > > > >> > 20.1/3 I’ve updated the TopicZNode to fix
> > formatting
> > > >> and
> > > >> > > > reflect
> > > >> > > > > > the
> > > >> > > > > > > > >> latest
> > > >> > > > > > > > >> > version before this change.
> > > >> > > > > > > > >> >
> > > >> > > > > > > > >> > 21. You are correct and I’ve removed this line.
> > I’ve
> > > >> also
> > > >> > > > added
> > > >> > > > > a
> > > >> > > > > > > line
> > > >> > > > > > > > >> > mentioning an IBP bump is necessary for migration
> > > >> > > > > > > > >> >
> > > >> > > > > > > > >> > 22. I think the wording was unclear but your
> > summary
> > > is
> > > >> > what
> > > >> > > > was
> > > >> > > > > > > > >> intended
> > > >> > > > > > > > >> > by this line. I’ve updated to make this point
> more
> > > >> clear.
> > > >> > > > > “Remove
> > > >> > > > > > > > >> deleted
> > > >> > > > > > > > >> > topics from replicas by sending
> StopReplicaRequest
> > V3
> > > >> > before
> > > >> > > > the
> > > >> > > > > > IBP
> > > >> > > > > > > > >> bump
> > > >> > > > > > > > >> > using the old logic, and using V4 and the new
> logic
> > > >> with
> > > >> > > topic
> > > >> > > > > IDs
> > > >> > > > > > > > after
> > > >> > > > > > > > >> > the IBP bump.”
> > > >> > > > > > > > >> >
> > > >> > > > > > > > >> > 23. I’ve removed the unspecified type from the
> KIP
> > > and
> > > >> > > mention
> > > >> > > > > > that
> > > >> > > > > > > > IBP
> > > >> > > > > > > > >> > will be used to handle this request. “IBP will be
> > > used
> > > >> to
> > > >> > > > > > determine
> > > >> > > > > > > > >> whether
> > > >> > > > > > > > >> > this new form of the request will be used. For
> > older
> > > >> > > requests,
> > > >> > > > > we
> > > >> > > > > > > will
> > > >> > > > > > > > >> > ignore this field and default to previous
> > behavior.”
> > > >> > > > > > > > >> >
> > > >> > > > > > > > >> > 24. I’ve fixed this typo.
> > > >> > > > > > > > >> >
> > > >> > > > > > > > >> > 25. I've added a topics at a higher level for
> > > >> > > > > LeaderAndIsrResponse
> > > >> > > > > > > v5,
> > > >> > > > > > > > >> > StopReplicaResponse v4. I've also changed
> > > >> > StopReplicaRequest
> > > >> > > > v4
> > > >> > > > > to
> > > >> > > > > > > > have
> > > >> > > > > > > > >> > topics at a higher level.
> > > >> > > > > > > > >> >
> > > >> > > > > > > > >> > 26. I’ve updated forgotten_topics_data--added the
> > > >> topic ID
> > > >> > > and
> > > >> > > > > > > removed
> > > >> > > > > > > > >> the
> > > >> > > > > > > > >> > topic name
> > > >> > > > > > > > >> >
> > > >> > > > > > > > >> > 27. I’ve decided on plain text, and I’ve added an
> > > >> example.
> > > >> > > > > > > > >> >
> > > >> > > > > > > > >> > 28. I’ve added this idea to future work.
> > > >> > > > > > > > >> >
> > > >> > > > > > > > >> > Thanks again for taking a look,
> > > >> > > > > > > > >> >
> > > >> > > > > > > > >> > Justine
> > > >> > > > > > > > >> >
> > > >> > > > > > > > >> > On Wed, Sep 23, 2020 at 10:28 AM Jun Rao <
> > > >> > jun@confluent.io>
> > > >> > > > > > wrote:
> > > >> > > > > > > > >> >
> > > >> > > > > > > > >> > > Hi, Justine,
> > > >> > > > > > > > >> > >
> > > >> > > > > > > > >> > > Thanks for the response. Made another pass. A
> few
> > > >> more
> > > >> > > > > comments
> > > >> > > > > > > > below.
> > > >> > > > > > > > >> > >
> > > >> > > > > > > > >> > > 20. znode schema:
> > > >> > > > > > > > >> > > 20.1 It seems that {"name": "version", "type":
> > > "int",
> > > >> > > "id":
> > > >> > > > > > > "UUID",
> > > >> > > > > > > > >> > "doc":
> > > >> > > > > > > > >> > > "version id"} should be {"name": "version",
> > "type":
> > > >> > > "int"},
> > > >> > > > > > > {"name":
> > > >> > > > > > > > >> > "id",
> > > >> > > > > > > > >> > > "type": "UUID", "doc": "version id"}.
> > > >> > > > > > > > >> > > 20.2 The znode format is JSON which doesn't
> have
> > > UUID
> > > >> > > type.
> > > >> > > > So
> > > >> > > > > > the
> > > >> > > > > > > > >> type
> > > >> > > > > > > > >> > > probably should be string?
> > > >> > > > > > > > >> > > 20.3 Also, the existing format used seems
> > outdated.
> > > >> It
> > > >> > > > should
> > > >> > > > > > have
> > > >> > > > > > > > the
> > > >> > > > > > > > >> > > following format.
> > > >> > > > > > > > >> > >     Json.encodeAsBytes(Map(
> > > >> > > > > > > > >> > >       "version" -> 2,
> > > >> > > > > > > > >> > >       "partitions" ->
> > replicaAssignmentJson.asJava,
> > > >> > > > > > > > >> > >       "adding_replicas" ->
> > > >> > > > > addingReplicasAssignmentJson.asJava,
> > > >> > > > > > > > >> > >       "removing_replicas" ->
> > > >> > > > > > removingReplicasAssignmentJson.asJava
> > > >> > > > > > > > >> > >     ).asJava)
> > > >> > > > > > > > >> > >   }
> > > >> > > > > > > > >> > >
> > > >> > > > > > > > >> > > 21. Migration: The KIP says "The migration
> > process
> > > >> can
> > > >> > > take
> > > >> > > > > > place
> > > >> > > > > > > > >> without
> > > >> > > > > > > > >> > > an inter-broker protocol bump, as the format
> > stored
> > > >> in
> > > >> > > > > > > > >> > > /brokers/topics/[topic] will be compatible with
> > > older
> > > >> > > broker
> > > >> > > > > > > > >> versions."
> > > >> > > > > > > > >> > > However, since we are bumping up the version of
> > > >> > > inter-broker
> > > >> > > > > > > > >> requests, it
> > > >> > > > > > > > >> > > seems that we need to use IBP for migration.
> > > >> > > > > > > > >> > >
> > > >> > > > > > > > >> > > 22. The KIP says "Remove deleted topics from
> > > >> replicas by
> > > >> > > > > sending
> > > >> > > > > > > > >> > > StopReplicaRequest V3 for any topics which do
> not
> > > >> > contain
> > > >> > > a
> > > >> > > > > > topic
> > > >> > > > > > > > ID,
> > > >> > > > > > > > >> and
> > > >> > > > > > > > >> > > V4 for any topics which do contain a topic
> ID.".
> > > >> > However,
> > > >> > > if
> > > >> > > > > we
> > > >> > > > > > > use
> > > >> > > > > > > > >> IBP,
> > > >> > > > > > > > >> > it
> > > >> > > > > > > > >> > > seems that the controller will either send
> > > >> > > > StopReplicaRequest
> > > >> > > > > V3
> > > >> > > > > > > > >> > > or StopReplicaRequest V4, but never mixed V3
> and
> > V4
> > > >> for
> > > >> > > > > > different
> > > >> > > > > > > > >> topics.
> > > >> > > > > > > > >> > > Basically, before the IBP bump, V3 will be
> used.
> > > >> After
> > > >> > the
> > > >> > > > IBP
> > > >> > > > > > > bump,
> > > >> > > > > > > > >> > > topicId will be created and V4 will be used.
> > > >> > > > > > > > >> > >
> > > >> > > > > > > > >> > > 23. Given that we depend on IBP, do we still
> need
> > > "0
> > > >> > > > > > UNSPECIFIED"
> > > >> > > > > > > > >> > > in LeaderAndIsr?
> > > >> > > > > > > > >> > >
> > > >> > > > > > > > >> > > 24. LeaderAndIsrResponse v5 : It still has the
> > > topic
> > > >> > > field.
> > > >> > > > > > > > >> > >
> > > >> > > > > > > > >> > > 25. LeaderAndIsrResponse v5,
> StopReplicaResponse
> > > v4:
> > > >> > Could
> > > >> > > > we
> > > >> > > > > > use
> > > >> > > > > > > > this
> > > >> > > > > > > > >> > > opportunity to organize the response in 2
> levels,
> > > >> first
> > > >> > by
> > > >> > > > > > topic,
> > > >> > > > > > > > >> then by
> > > >> > > > > > > > >> > > partition, as most other requests/responses?
> > > >> > > > > > > > >> > >
> > > >> > > > > > > > >> > > 26. FetchRequest v13 : Should
> > forgotten_topics_data
> > > >> use
> > > >> > > > > topicId
> > > >> > > > > > > too?
> > > >> > > > > > > > >> > >
> > > >> > > > > > > > >> > > 27. "This file can either be plain text
> > (key/value
> > > >> > pairs)
> > > >> > > or
> > > >> > > > > > > JSON."
> > > >> > > > > > > > >> Have
> > > >> > > > > > > > >> > we
> > > >> > > > > > > > >> > > decided which one to use? Also, it would be
> > helpful
> > > >> to
> > > >> > > > provide
> > > >> > > > > > an
> > > >> > > > > > > > >> > example.
> > > >> > > > > > > > >> > >
> > > >> > > > > > > > >> > > 28. Future improvement: Another future
> > improvement
> > > >> > > > opportunity
> > > >> > > > > > is
> > > >> > > > > > > to
> > > >> > > > > > > > >> use
> > > >> > > > > > > > >> > > topicId in GroupMetadataManager.offsetCommitKey
> > in
> > > >> the
> > > >> > > > > > > offset_commit
> > > >> > > > > > > > >> > topic.
> > > >> > > > > > > > >> > > This may save some space.
> > > >> > > > > > > > >> > >
> > > >> > > > > > > > >> > > Thanks,
> > > >> > > > > > > > >> > >
> > > >> > > > > > > > >> > > Jun
> > > >> > > > > > > > >> > >
> > > >> > > > > > > > >> > > On Wed, Sep 23, 2020 at 8:50 AM Justine Olshan
> <
> > > >> > > > > > > > jolshan@confluent.io>
> > > >> > > > > > > > >> > > wrote:
> > > >> > > > > > > > >> > >
> > > >> > > > > > > > >> > > > Hi Tom,
> > > >> > > > > > > > >> > > >
> > > >> > > > > > > > >> > > > Thanks for the comment. I think this is a
> > really
> > > >> good
> > > >> > > idea
> > > >> > > > > and
> > > >> > > > > > > it
> > > >> > > > > > > > >> has
> > > >> > > > > > > > >> > > been
> > > >> > > > > > > > >> > > > added to the KIP under the newly added
> tooling
> > > >> > section.
> > > >> > > > > > > > >> > > >
> > > >> > > > > > > > >> > > > Thanks again,
> > > >> > > > > > > > >> > > > Justine
> > > >> > > > > > > > >> > > >
> > > >> > > > > > > > >> > > > On Wed, Sep 23, 2020 at 3:17 AM Tom Bentley <
> > > >> > > > > > > tbentley@redhat.com>
> > > >> > > > > > > > >> > wrote:
> > > >> > > > > > > > >> > > >
> > > >> > > > > > > > >> > > > > Hi Justine,
> > > >> > > > > > > > >> > > > >
> > > >> > > > > > > > >> > > > > I know you started the vote thread, but on
> > > >> > re-reading
> > > >> > > > the
> > > >> > > > > > KIP
> > > >> > > > > > > I
> > > >> > > > > > > > >> > noticed
> > > >> > > > > > > > >> > > > > that although the topic id is included in
> the
> > > >> > > > > > MetadataResponse
> > > >> > > > > > > > >> it's
> > > >> > > > > > > > >> > not
> > > >> > > > > > > > >> > > > > surfaced in the output from
> `kafka-topics.sh
> > > >> > > > --describe`.
> > > >> > > > > > > Maybe
> > > >> > > > > > > > >> that
> > > >> > > > > > > > >> > > was
> > > >> > > > > > > > >> > > > > intentional because ids are intentionally
> not
> > > >> really
> > > >> > > > > > something
> > > >> > > > > > > > the
> > > >> > > > > > > > >> > user
> > > >> > > > > > > > >> > > > > should care deeply about, but it would also
> > > make
> > > >> > life
> > > >> > > > > harder
> > > >> > > > > > > for
> > > >> > > > > > > > >> > anyone
> > > >> > > > > > > > >> > > > > debugging Kafka and this would likely get
> > worse
> > > >> the
> > > >> > > more
> > > >> > > > > > topic
> > > >> > > > > > > > ids
> > > >> > > > > > > > >> > got
> > > >> > > > > > > > >> > > > > rolled out across the protocols, clients
> etc.
> > > It
> > > >> > seems
> > > >> > > > > > likely
> > > >> > > > > > > > that
> > > >> > > > > > > > >> > > > > `kafka-topics.sh` will eventually need the
> > > >> ability
> > > >> > to
> > > >> > > > show
> > > >> > > > > > the
> > > >> > > > > > > > id
> > > >> > > > > > > > >> of
> > > >> > > > > > > > >> > a
> > > >> > > > > > > > >> > > > > topic and perhaps find a topic name given
> an
> > > id.
> > > >> Is
> > > >> > > > there
> > > >> > > > > > any
> > > >> > > > > > > > >> reason
> > > >> > > > > > > > >> > > not
> > > >> > > > > > > > >> > > > to
> > > >> > > > > > > > >> > > > > implement that in this KIP?
> > > >> > > > > > > > >> > > > >
> > > >> > > > > > > > >> > > > > Many thanks,
> > > >> > > > > > > > >> > > > >
> > > >> > > > > > > > >> > > > > Tom
> > > >> > > > > > > > >> > > > >
> > > >> > > > > > > > >> > > > > On Mon, Sep 21, 2020 at 9:54 PM Justine
> > Olshan
> > > <
> > > >> > > > > > > > >> jolshan@confluent.io
> > > >> > > > > > > > >> > >
> > > >> > > > > > > > >> > > > > wrote:
> > > >> > > > > > > > >> > > > >
> > > >> > > > > > > > >> > > > > > Hi all,
> > > >> > > > > > > > >> > > > > >
> > > >> > > > > > > > >> > > > > > After thinking about it, I've decided to
> > > remove
> > > >> > the
> > > >> > > > > topic
> > > >> > > > > > > name
> > > >> > > > > > > > >> from
> > > >> > > > > > > > >> > > the
> > > >> > > > > > > > >> > > > > > Fetch Request and Response after all.
> Since
> > > >> there
> > > >> > > are
> > > >> > > > so
> > > >> > > > > > > many
> > > >> > > > > > > > of
> > > >> > > > > > > > >> > > these
> > > >> > > > > > > > >> > > > > > requests per second, it is worth removing
> > the
> > > >> > extra
> > > >> > > > > > > > information.
> > > >> > > > > > > > >> > I've
> > > >> > > > > > > > >> > > > > > updated the KIP to reflect this change.
> > > >> > > > > > > > >> > > > > >
> > > >> > > > > > > > >> > > > > > Please let me know if there is anything
> > else
> > > we
> > > >> > > should
> > > >> > > > > > > discuss
> > > >> > > > > > > > >> > before
> > > >> > > > > > > > >> > > > > > voting.
> > > >> > > > > > > > >> > > > > >
> > > >> > > > > > > > >> > > > > > Thank you,
> > > >> > > > > > > > >> > > > > > Justine
> > > >> > > > > > > > >> > > > > >
> > > >> > > > > > > > >> > > > > > On Fri, Sep 18, 2020 at 9:46 AM Justine
> > > Olshan
> > > >> <
> > > >> > > > > > > > >> > jolshan@confluent.io
> > > >> > > > > > > > >> > > >
> > > >> > > > > > > > >> > > > > > wrote:
> > > >> > > > > > > > >> > > > > >
> > > >> > > > > > > > >> > > > > > > Hi Jun,
> > > >> > > > > > > > >> > > > > > >
> > > >> > > > > > > > >> > > > > > > I see what you are saying. For now we
> can
> > > >> remove
> > > >> > > the
> > > >> > > > > > extra
> > > >> > > > > > > > >> > > > information.
> > > >> > > > > > > > >> > > > > > > I'll leave the option to add more
> fields
> > to
> > > >> the
> > > >> > > file
> > > >> > > > > in
> > > >> > > > > > > the
> > > >> > > > > > > > >> > future.
> > > >> > > > > > > > >> > > > The
> > > >> > > > > > > > >> > > > > > KIP
> > > >> > > > > > > > >> > > > > > > has been updated to reflect this
> change.
> > > >> > > > > > > > >> > > > > > >
> > > >> > > > > > > > >> > > > > > > Thanks,
> > > >> > > > > > > > >> > > > > > > Justine
> > > >> > > > > > > > >> > > > > > >
> > > >> > > > > > > > >> > > > > > > On Fri, Sep 18, 2020 at 8:46 AM Jun
> Rao <
> > > >> > > > > > jun@confluent.io
> > > >> > > > > > > >
> > > >> > > > > > > > >> > wrote:
> > > >> > > > > > > > >> > > > > > >
> > > >> > > > > > > > >> > > > > > >> Hi, Justine,
> > > >> > > > > > > > >> > > > > > >>
> > > >> > > > > > > > >> > > > > > >> Thanks for the reply.
> > > >> > > > > > > > >> > > > > > >>
> > > >> > > > > > > > >> > > > > > >> 13. If the log directory is the source
> > of
> > > >> > truth,
> > > >> > > it
> > > >> > > > > > means
> > > >> > > > > > > > >> that
> > > >> > > > > > > > >> > the
> > > >> > > > > > > > >> > > > > > >> redundant info in the metadata file
> will
> > > be
> > > >> > > > ignored.
> > > >> > > > > > Then
> > > >> > > > > > > > the
> > > >> > > > > > > > >> > > > question
> > > >> > > > > > > > >> > > > > > is
> > > >> > > > > > > > >> > > > > > >> why do we need to put the redundant
> info
> > > in
> > > >> the
> > > >> > > > > > metadata
> > > >> > > > > > > > file
> > > >> > > > > > > > >> > now?
> > > >> > > > > > > > >> > > > > > >>
> > > >> > > > > > > > >> > > > > > >> Thanks,
> > > >> > > > > > > > >> > > > > > >>
> > > >> > > > > > > > >> > > > > > >> Jun
> > > >> > > > > > > > >> > > > > > >>
> > > >> > > > > > > > >> > > > > > >> On Thu, Sep 17, 2020 at 5:07 PM
> Justine
> > > >> Olshan
> > > >> > <
> > > >> > > > > > > > >> > > > jolshan@confluent.io>
> > > >> > > > > > > > >> > > > > > >> wrote:
> > > >> > > > > > > > >> > > > > > >>
> > > >> > > > > > > > >> > > > > > >> > Hi Jun,
> > > >> > > > > > > > >> > > > > > >> > Thanks for the quick response!
> > > >> > > > > > > > >> > > > > > >> >
> > > >> > > > > > > > >> > > > > > >> > 12. I've decided to bump up the
> > versions
> > > >> on
> > > >> > the
> > > >> > > > > > > requests
> > > >> > > > > > > > >> and
> > > >> > > > > > > > >> > > > updated
> > > >> > > > > > > > >> > > > > > the
> > > >> > > > > > > > >> > > > > > >> > KIP. I think it's good we thoroughly
> > > >> > discussed
> > > >> > > > the
> > > >> > > > > > > > options
> > > >> > > > > > > > >> > here,
> > > >> > > > > > > > >> > > > so
> > > >> > > > > > > > >> > > > > we
> > > >> > > > > > > > >> > > > > > >> know
> > > >> > > > > > > > >> > > > > > >> > we made a good choice. :)
> > > >> > > > > > > > >> > > > > > >> >
> > > >> > > > > > > > >> > > > > > >> > 13. This is an interesting
> situation.
> > I
> > > >> think
> > > >> > > if
> > > >> > > > > this
> > > >> > > > > > > > does
> > > >> > > > > > > > >> > occur
> > > >> > > > > > > > >> > > > we
> > > >> > > > > > > > >> > > > > > >> should
> > > >> > > > > > > > >> > > > > > >> > give a warning. I agree that it's
> hard
> > > to
> > > >> > know
> > > >> > > > the
> > > >> > > > > > > source
> > > >> > > > > > > > >> of
> > > >> > > > > > > > >> > > truth
> > > >> > > > > > > > >> > > > > for
> > > >> > > > > > > > >> > > > > > >> sure
> > > >> > > > > > > > >> > > > > > >> > since the directory or the file
> could
> > be
> > > >> > > manually
> > > >> > > > > > > > >> modified. I
> > > >> > > > > > > > >> > > > guess
> > > >> > > > > > > > >> > > > > > the
> > > >> > > > > > > > >> > > > > > >> > directory could be used as the
> source
> > of
> > > >> > truth.
> > > >> > > > To
> > > >> > > > > be
> > > >> > > > > > > > >> honest,
> > > >> > > > > > > > >> > > I'm
> > > >> > > > > > > > >> > > > > not
> > > >> > > > > > > > >> > > > > > >> > really sure what happens in kafka
> when
> > > the
> > > >> > log
> > > >> > > > > > > directory
> > > >> > > > > > > > is
> > > >> > > > > > > > >> > > > renamed
> > > >> > > > > > > > >> > > > > > >> > manually in such a way. I'm also
> > > >> wondering if
> > > >> > > the
> > > >> > > > > > > > >> situation is
> > > >> > > > > > > > >> > > > > > >> recoverable
> > > >> > > > > > > > >> > > > > > >> > in this scenario.
> > > >> > > > > > > > >> > > > > > >> >
> > > >> > > > > > > > >> > > > > > >> > Thanks,
> > > >> > > > > > > > >> > > > > > >> > Justine
> > > >> > > > > > > > >> > > > > > >> >
> > > >> > > > > > > > >> > > > > > >> > On Thu, Sep 17, 2020 at 4:28 PM Jun
> > Rao
> > > <
> > > >> > > > > > > > jun@confluent.io>
> > > >> > > > > > > > >> > > wrote:
> > > >> > > > > > > > >> > > > > > >> >
> > > >> > > > > > > > >> > > > > > >> > > Hi, Justine,
> > > >> > > > > > > > >> > > > > > >> > >
> > > >> > > > > > > > >> > > > > > >> > > Thanks for the reply.
> > > >> > > > > > > > >> > > > > > >> > >
> > > >> > > > > > > > >> > > > > > >> > > 12. I don't have a strong
> preference
> > > >> > either.
> > > >> > > > > > However,
> > > >> > > > > > > > if
> > > >> > > > > > > > >> we
> > > >> > > > > > > > >> > > need
> > > >> > > > > > > > >> > > > > IBP
> > > >> > > > > > > > >> > > > > > >> > > anyway, maybe it's easier to just
> > bump
> > > >> up
> > > >> > the
> > > >> > > > > > version
> > > >> > > > > > > > for
> > > >> > > > > > > > >> > all
> > > >> > > > > > > > >> > > > > inter
> > > >> > > > > > > > >> > > > > > >> > broker
> > > >> > > > > > > > >> > > > > > >> > > requests and add the topic id
> field
> > > as a
> > > >> > > > regular
> > > >> > > > > > > > field. A
> > > >> > > > > > > > >> > > > regular
> > > >> > > > > > > > >> > > > > > >> field
> > > >> > > > > > > > >> > > > > > >> > is
> > > >> > > > > > > > >> > > > > > >> > > a bit more concise in wire
> transfer
> > > >> than a
> > > >> > > > > flexible
> > > >> > > > > > > > >> field.
> > > >> > > > > > > > >> > > > > > >> > >
> > > >> > > > > > > > >> > > > > > >> > > 13. The confusion that I was
> > referring
> > > >> to
> > > >> > is
> > > >> > > > > > between
> > > >> > > > > > > > the
> > > >> > > > > > > > >> > topic
> > > >> > > > > > > > >> > > > > name
> > > >> > > > > > > > >> > > > > > >> and
> > > >> > > > > > > > >> > > > > > >> > > partition number between the log
> dir
> > > and
> > > >> > the
> > > >> > > > > > metadata
> > > >> > > > > > > > >> file.
> > > >> > > > > > > > >> > > For
> > > >> > > > > > > > >> > > > > > >> example,
> > > >> > > > > > > > >> > > > > > >> > if
> > > >> > > > > > > > >> > > > > > >> > > the log dir is topicA-1 and the
> > > metadata
> > > >> > file
> > > >> > > > in
> > > >> > > > > it
> > > >> > > > > > > has
> > > >> > > > > > > > >> > topicB
> > > >> > > > > > > > >> > > > and
> > > >> > > > > > > > >> > > > > > >> > > partition 0 (say due to a bug or
> > > manual
> > > >> > > > > > > modification),
> > > >> > > > > > > > >> which
> > > >> > > > > > > > >> > > one
> > > >> > > > > > > > >> > > > > do
> > > >> > > > > > > > >> > > > > > we
> > > >> > > > > > > > >> > > > > > >> > use
> > > >> > > > > > > > >> > > > > > >> > > as the source of truth?
> > > >> > > > > > > > >> > > > > > >> > >
> > > >> > > > > > > > >> > > > > > >> > > Jun
> > > >> > > > > > > > >> > > > > > >> > >
> > > >> > > > > > > > >> > > > > > >> > > On Thu, Sep 17, 2020 at 3:43 PM
> > > Justine
> > > >> > > Olshan
> > > >> > > > <
> > > >> > > > > > > > >> > > > > > jolshan@confluent.io>
> > > >> > > > > > > > >> > > > > > >> > > wrote:
> > > >> > > > > > > > >> > > > > > >> > >
> > > >> > > > > > > > >> > > > > > >> > > > Hi Jun,
> > > >> > > > > > > > >> > > > > > >> > > > Thanks for the comments.
> > > >> > > > > > > > >> > > > > > >> > > >
> > > >> > > > > > > > >> > > > > > >> > > > 12. I bumped the
> > LeaderAndIsrRequest
> > > >> > > because
> > > >> > > > I
> > > >> > > > > > > > removed
> > > >> > > > > > > > >> the
> > > >> > > > > > > > >> > > > topic
> > > >> > > > > > > > >> > > > > > >> name
> > > >> > > > > > > > >> > > > > > >> > > field
> > > >> > > > > > > > >> > > > > > >> > > > in the response. It may be
> > possible
> > > to
> > > >> > > avoid
> > > >> > > > > > > bumping
> > > >> > > > > > > > >> the
> > > >> > > > > > > > >> > > > version
> > > >> > > > > > > > >> > > > > > >> > without
> > > >> > > > > > > > >> > > > > > >> > > > that change, but I may be
> missing
> > > >> > > something.
> > > >> > > > > > > > >> > > > > > >> > > > I believe StopReplica is
> actually
> > on
> > > >> > > version
> > > >> > > > 3
> > > >> > > > > > now,
> > > >> > > > > > > > but
> > > >> > > > > > > > >> > > > because
> > > >> > > > > > > > >> > > > > > >> > version 2
> > > >> > > > > > > > >> > > > > > >> > > > is flexible, I kept that listed
> as
> > > >> > version
> > > >> > > 2
> > > >> > > > on
> > > >> > > > > > the
> > > >> > > > > > > > KIP
> > > >> > > > > > > > >> > > page.
> > > >> > > > > > > > >> > > > > > >> However,
> > > >> > > > > > > > >> > > > > > >> > > you
> > > >> > > > > > > > >> > > > > > >> > > > may be right in that we may need
> > to
> > > >> bump
> > > >> > > the
> > > >> > > > > > > version
> > > >> > > > > > > > on
> > > >> > > > > > > > >> > > > > > StopReplica
> > > >> > > > > > > > >> > > > > > >> to
> > > >> > > > > > > > >> > > > > > >> > > deal
> > > >> > > > > > > > >> > > > > > >> > > > with deletion differently as
> > > mentioned
> > > >> > > > above. I
> > > >> > > > > > > don't
> > > >> > > > > > > > >> know
> > > >> > > > > > > > >> > > if
> > > >> > > > > > > > >> > > > I
> > > >> > > > > > > > >> > > > > > >> have a
> > > >> > > > > > > > >> > > > > > >> > > big
> > > >> > > > > > > > >> > > > > > >> > > > preference over used tagged
> fields
> > > or
> > > >> > not.
> > > >> > > > > > > > >> > > > > > >> > > >
> > > >> > > > > > > > >> > > > > > >> > > > 13. I was thinking that in the
> > case
> > > >> where
> > > >> > > the
> > > >> > > > > > file
> > > >> > > > > > > > and
> > > >> > > > > > > > >> the
> > > >> > > > > > > > >> > > > > request
> > > >> > > > > > > > >> > > > > > >> > topic
> > > >> > > > > > > > >> > > > > > >> > > > ids don't match, it means that
> the
> > > >> > broker's
> > > >> > > > > > > topic/the
> > > >> > > > > > > > >> one
> > > >> > > > > > > > >> > in
> > > >> > > > > > > > >> > > > the
> > > >> > > > > > > > >> > > > > > >> file
> > > >> > > > > > > > >> > > > > > >> > has
> > > >> > > > > > > > >> > > > > > >> > > > been deleted. In that case, we
> > would
> > > >> need
> > > >> > > to
> > > >> > > > > > delete
> > > >> > > > > > > > the
> > > >> > > > > > > > >> > old
> > > >> > > > > > > > >> > > > > topic
> > > >> > > > > > > > >> > > > > > >> and
> > > >> > > > > > > > >> > > > > > >> > > start
> > > >> > > > > > > > >> > > > > > >> > > > receiving the new version. If
> the
> > > >> topic
> > > >> > > name
> > > >> > > > > were
> > > >> > > > > > > to
> > > >> > > > > > > > >> > change,
> > > >> > > > > > > > >> > > > but
> > > >> > > > > > > > >> > > > > > the
> > > >> > > > > > > > >> > > > > > >> > ids
> > > >> > > > > > > > >> > > > > > >> > > > still matched, the file would
> also
> > > >> need
> > > >> > to
> > > >> > > > > > update.
> > > >> > > > > > > > Am I
> > > >> > > > > > > > >> > > > missing
> > > >> > > > > > > > >> > > > > a
> > > >> > > > > > > > >> > > > > > >> case
> > > >> > > > > > > > >> > > > > > >> > > > where the file would be correct
> > and
> > > >> not
> > > >> > the
> > > >> > > > > > > request?
> > > >> > > > > > > > >> > > > > > >> > > >
> > > >> > > > > > > > >> > > > > > >> > > > Thanks,
> > > >> > > > > > > > >> > > > > > >> > > > Justine
> > > >> > > > > > > > >> > > > > > >> > > >
> > > >> > > > > > > > >> > > > > > >> > > > On Thu, Sep 17, 2020 at 3:18 PM
> > Jun
> > > >> Rao <
> > > >> > > > > > > > >> jun@confluent.io
> > > >> > > > > > > > >> > >
> > > >> > > > > > > > >> > > > > wrote:
> > > >> > > > > > > > >> > > > > > >> > > >
> > > >> > > > > > > > >> > > > > > >> > > > > Hi, Justine,
> > > >> > > > > > > > >> > > > > > >> > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > Thanks for the reply. A couple
> > of
> > > >> more
> > > >> > > > > comments
> > > >> > > > > > > > >> below.
> > > >> > > > > > > > >> > > > > > >> > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > 12. ListOffset and
> > OffsetForLeader
> > > >> > > > currently
> > > >> > > > > > > don't
> > > >> > > > > > > > >> > support
> > > >> > > > > > > > >> > > > > > >> flexible
> > > >> > > > > > > > >> > > > > > >> > > > fields.
> > > >> > > > > > > > >> > > > > > >> > > > > So, we have to bump up the
> > version
> > > >> > number
> > > >> > > > and
> > > >> > > > > > use
> > > >> > > > > > > > >> IBP at
> > > >> > > > > > > > >> > > > least
> > > >> > > > > > > > >> > > > > > for
> > > >> > > > > > > > >> > > > > > >> > > these
> > > >> > > > > > > > >> > > > > > >> > > > > two requests. Note that it
> seems
> > > >> 2.7.0
> > > >> > > will
> > > >> > > > > > > require
> > > >> > > > > > > > >> IBP
> > > >> > > > > > > > >> > > > anyway
> > > >> > > > > > > > >> > > > > > >> > because
> > > >> > > > > > > > >> > > > > > >> > > of
> > > >> > > > > > > > >> > > > > > >> > > > > changes in KAFKA-10435. Also,
> it
> > > >> seems
> > > >> > > that
> > > >> > > > > the
> > > >> > > > > > > > >> version
> > > >> > > > > > > > >> > > for
> > > >> > > > > > > > >> > > > > > >> > > > > LeaderAndIsrRequest and
> > > StopReplica
> > > >> are
> > > >> > > > > bumped
> > > >> > > > > > > even
> > > >> > > > > > > > >> > though
> > > >> > > > > > > > >> > > > we
> > > >> > > > > > > > >> > > > > > only
> > > >> > > > > > > > >> > > > > > >> > > added
> > > >> > > > > > > > >> > > > > > >> > > > a
> > > >> > > > > > > > >> > > > > > >> > > > > tagged field. But since IBP is
> > > >> needed
> > > >> > > > anyway,
> > > >> > > > > > we
> > > >> > > > > > > > may
> > > >> > > > > > > > >> > want
> > > >> > > > > > > > >> > > to
> > > >> > > > > > > > >> > > > > > >> revisit
> > > >> > > > > > > > >> > > > > > >> > > the
> > > >> > > > > > > > >> > > > > > >> > > > > overall tagged field choice.
> > > >> > > > > > > > >> > > > > > >> > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > 13. The only downside is the
> > > >> potential
> > > >> > > > > > confusion
> > > >> > > > > > > on
> > > >> > > > > > > > >> > which
> > > >> > > > > > > > >> > > > one
> > > >> > > > > > > > >> > > > > is
> > > >> > > > > > > > >> > > > > > >> the
> > > >> > > > > > > > >> > > > > > >> > > > source
> > > >> > > > > > > > >> > > > > > >> > > > > of truth if they don't match.
> > > >> Another
> > > >> > > > option
> > > >> > > > > is
> > > >> > > > > > > to
> > > >> > > > > > > > >> > include
> > > >> > > > > > > > >> > > > > those
> > > >> > > > > > > > >> > > > > > >> > fields
> > > >> > > > > > > > >> > > > > > >> > > > in
> > > >> > > > > > > > >> > > > > > >> > > > > the metadata file when we
> > actually
> > > >> > change
> > > >> > > > the
> > > >> > > > > > > > >> directory
> > > >> > > > > > > > >> > > > > > structure.
> > > >> > > > > > > > >> > > > > > >> > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > Thanks,
> > > >> > > > > > > > >> > > > > > >> > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > Jun
> > > >> > > > > > > > >> > > > > > >> > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > On Thu, Sep 17, 2020 at 2:01
> PM
> > > >> Justine
> > > >> > > > > Olshan
> > > >> > > > > > <
> > > >> > > > > > > > >> > > > > > >> jolshan@confluent.io
> > > >> > > > > > > > >> > > > > > >> > >
> > > >> > > > > > > > >> > > > > > >> > > > > wrote:
> > > >> > > > > > > > >> > > > > > >> > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > Hello all,
> > > >> > > > > > > > >> > > > > > >> > > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > I've thought some more about
> > > >> removing
> > > >> > > the
> > > >> > > > > > topic
> > > >> > > > > > > > >> name
> > > >> > > > > > > > >> > > field
> > > >> > > > > > > > >> > > > > > from
> > > >> > > > > > > > >> > > > > > >> > some
> > > >> > > > > > > > >> > > > > > >> > > of
> > > >> > > > > > > > >> > > > > > >> > > > > the
> > > >> > > > > > > > >> > > > > > >> > > > > > requests. On closer
> inspection
> > > of
> > > >> the
> > > >> > > > > > > > >> > > requests/responses,
> > > >> > > > > > > > >> > > > it
> > > >> > > > > > > > >> > > > > > >> seems
> > > >> > > > > > > > >> > > > > > >> > > that
> > > >> > > > > > > > >> > > > > > >> > > > > the
> > > >> > > > > > > > >> > > > > > >> > > > > > internal changes would be
> much
> > > >> larger
> > > >> > > > than
> > > >> > > > > I
> > > >> > > > > > > > >> expected.
> > > >> > > > > > > > >> > > > Some
> > > >> > > > > > > > >> > > > > > >> > protocols
> > > >> > > > > > > > >> > > > > > >> > > > > > involve clients, so they
> would
> > > >> > require
> > > >> > > > > > changes
> > > >> > > > > > > > too.
> > > >> > > > > > > > >> > I'm
> > > >> > > > > > > > >> > > > > > thinking
> > > >> > > > > > > > >> > > > > > >> > that
> > > >> > > > > > > > >> > > > > > >> > > > for
> > > >> > > > > > > > >> > > > > > >> > > > > > now, removing the topic name
> > > from
> > > >> > these
> > > >> > > > > > > requests
> > > >> > > > > > > > >> and
> > > >> > > > > > > > >> > > > > responses
> > > >> > > > > > > > >> > > > > > >> are
> > > >> > > > > > > > >> > > > > > >> > > out
> > > >> > > > > > > > >> > > > > > >> > > > of
> > > >> > > > > > > > >> > > > > > >> > > > > > scope.
> > > >> > > > > > > > >> > > > > > >> > > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > I have decided to just keep
> > the
> > > >> > change
> > > >> > > > > > > > >> > > > LeaderAndIsrResponse
> > > >> > > > > > > > >> > > > > to
> > > >> > > > > > > > >> > > > > > >> > remove
> > > >> > > > > > > > >> > > > > > >> > > > the
> > > >> > > > > > > > >> > > > > > >> > > > > > topic name, and have updated
> > the
> > > >> KIP
> > > >> > to
> > > >> > > > > > reflect
> > > >> > > > > > > > >> this
> > > >> > > > > > > > >> > > > > change. I
> > > >> > > > > > > > >> > > > > > >> have
> > > >> > > > > > > > >> > > > > > >> > > > also
> > > >> > > > > > > > >> > > > > > >> > > > > > mentioned the other requests
> > and
> > > >> > > > responses
> > > >> > > > > in
> > > >> > > > > > > > >> future
> > > >> > > > > > > > >> > > work.
> > > >> > > > > > > > >> > > > > > >> > > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > I'm hoping to start the
> voting
> > > >> > process
> > > >> > > > > soon,
> > > >> > > > > > so
> > > >> > > > > > > > >> let me
> > > >> > > > > > > > >> > > > know
> > > >> > > > > > > > >> > > > > if
> > > >> > > > > > > > >> > > > > > >> > there
> > > >> > > > > > > > >> > > > > > >> > > is
> > > >> > > > > > > > >> > > > > > >> > > > > > anything else we should
> > discuss.
> > > >> > > > > > > > >> > > > > > >> > > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > Thank you,
> > > >> > > > > > > > >> > > > > > >> > > > > > Justine
> > > >> > > > > > > > >> > > > > > >> > > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > On Tue, Sep 15, 2020 at 3:57
> > PM
> > > >> > Justine
> > > >> > > > > > Olshan
> > > >> > > > > > > <
> > > >> > > > > > > > >> > > > > > >> > jolshan@confluent.io
> > > >> > > > > > > > >> > > > > > >> > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > wrote:
> > > >> > > > > > > > >> > > > > > >> > > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > > Hello again,
> > > >> > > > > > > > >> > > > > > >> > > > > > > To follow up on some of
> the
> > > >> other
> > > >> > > > > comments:
> > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > > 10/11) We can remove the
> > topic
> > > >> name
> > > >> > > > from
> > > >> > > > > > > these
> > > >> > > > > > > > >> > > > > > >> > requests/responses,
> > > >> > > > > > > > >> > > > > > >> > > > and
> > > >> > > > > > > > >> > > > > > >> > > > > > > that means we will just
> have
> > > to
> > > >> > make
> > > >> > > a
> > > >> > > > > few
> > > >> > > > > > > > >> internal
> > > >> > > > > > > > >> > > > > changes
> > > >> > > > > > > > >> > > > > > to
> > > >> > > > > > > > >> > > > > > >> > make
> > > >> > > > > > > > >> > > > > > >> > > > > > > partitions accessible by
> > topic
> > > >> id
> > > >> > and
> > > >> > > > > > > > partition.
> > > >> > > > > > > > >> I
> > > >> > > > > > > > >> > can
> > > >> > > > > > > > >> > > > > > update
> > > >> > > > > > > > >> > > > > > >> the
> > > >> > > > > > > > >> > > > > > >> > > KIP
> > > >> > > > > > > > >> > > > > > >> > > > > to
> > > >> > > > > > > > >> > > > > > >> > > > > > > remove them unless anyone
> > > thinks
> > > >> > they
> > > >> > > > > > should
> > > >> > > > > > > > >> stay.
> > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > > 12) Addressed in the
> > previous
> > > >> > email.
> > > >> > > > I've
> > > >> > > > > > > > updated
> > > >> > > > > > > > >> > the
> > > >> > > > > > > > >> > > > KIP
> > > >> > > > > > > > >> > > > > to
> > > >> > > > > > > > >> > > > > > >> > > include
> > > >> > > > > > > > >> > > > > > >> > > > > > > tagged fields for the
> > requests
> > > >> and
> > > >> > > > > > responses.
> > > >> > > > > > > > >> (More
> > > >> > > > > > > > >> > on
> > > >> > > > > > > > >> > > > > that
> > > >> > > > > > > > >> > > > > > >> > below)
> > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > > 13) I think part of the
> idea
> > > for
> > > >> > > > > including
> > > >> > > > > > > this
> > > >> > > > > > > > >> > > > > information
> > > >> > > > > > > > >> > > > > > >> is to
> > > >> > > > > > > > >> > > > > > >> > > > > prepare
> > > >> > > > > > > > >> > > > > > >> > > > > > > for future changes.
> Perhaps
> > > the
> > > >> > > > directory
> > > >> > > > > > > > >> structure
> > > >> > > > > > > > >> > > > might
> > > >> > > > > > > > >> > > > > > >> change
> > > >> > > > > > > > >> > > > > > >> > > from
> > > >> > > > > > > > >> > > > > > >> > > > > > > topicName_partitionNumber
> to
> > > >> > > something
> > > >> > > > > like
> > > >> > > > > > > > >> > > > > > >> > > topicID_partitionNumber.
> > > >> > > > > > > > >> > > > > > >> > > > > Then
> > > >> > > > > > > > >> > > > > > >> > > > > > > it would be useful to have
> > the
> > > >> > topic
> > > >> > > > name
> > > >> > > > > > in
> > > >> > > > > > > > the
> > > >> > > > > > > > >> > file
> > > >> > > > > > > > >> > > > > since
> > > >> > > > > > > > >> > > > > > it
> > > >> > > > > > > > >> > > > > > >> > > would
> > > >> > > > > > > > >> > > > > > >> > > > > not
> > > >> > > > > > > > >> > > > > > >> > > > > > be
> > > >> > > > > > > > >> > > > > > >> > > > > > > in the directory
> structure.
> > > >> > > Supporting
> > > >> > > > > > topic
> > > >> > > > > > > > >> renames
> > > >> > > > > > > > >> > > > might
> > > >> > > > > > > > >> > > > > > be
> > > >> > > > > > > > >> > > > > > >> > > easier
> > > >> > > > > > > > >> > > > > > >> > > > if
> > > >> > > > > > > > >> > > > > > >> > > > > > the
> > > >> > > > > > > > >> > > > > > >> > > > > > > other fields are included.
> > > Would
> > > >> > > there
> > > >> > > > be
> > > >> > > > > > any
> > > >> > > > > > > > >> > > downsides
> > > >> > > > > > > > >> > > > to
> > > >> > > > > > > > >> > > > > > >> > > including
> > > >> > > > > > > > >> > > > > > >> > > > > this
> > > >> > > > > > > > >> > > > > > >> > > > > > > information?
> > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > > 14)  Yes, we would need to
> > > copy
> > > >> the
> > > >> > > > > > partition
> > > >> > > > > > > > >> > metadata
> > > >> > > > > > > > >> > > > > file
> > > >> > > > > > > > >> > > > > > in
> > > >> > > > > > > > >> > > > > > >> > this
> > > >> > > > > > > > >> > > > > > >> > > > > > > process. I've updated the
> > KIP
> > > to
> > > >> > > > include
> > > >> > > > > > > this.
> > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > > 15) I believe Lucas meant
> v1
> > > >> and v2
> > > >> > > > here.
> > > >> > > > > > He
> > > >> > > > > > > > was
> > > >> > > > > > > > >> > > > referring
> > > >> > > > > > > > >> > > > > > to
> > > >> > > > > > > > >> > > > > > >> how
> > > >> > > > > > > > >> > > > > > >> > > the
> > > >> > > > > > > > >> > > > > > >> > > > > > > requests would fall under
> > > >> different
> > > >> > > IBP
> > > >> > > > > and
> > > >> > > > > > > > meant
> > > >> > > > > > > > >> > that
> > > >> > > > > > > > >> > > > > older
> > > >> > > > > > > > >> > > > > > >> > > brokers
> > > >> > > > > > > > >> > > > > > >> > > > > > would
> > > >> > > > > > > > >> > > > > > >> > > > > > > have to use the older
> > version
> > > of
> > > >> > the
> > > >> > > > > > request
> > > >> > > > > > > > and
> > > >> > > > > > > > >> the
> > > >> > > > > > > > >> > > > > > existing
> > > >> > > > > > > > >> > > > > > >> > topic
> > > >> > > > > > > > >> > > > > > >> > > > > > > deletion process. At
> first,
> > it
> > > >> > seemed
> > > >> > > > > like
> > > >> > > > > > > > tagged
> > > >> > > > > > > > >> > > fields
> > > >> > > > > > > > >> > > > > > would
> > > >> > > > > > > > >> > > > > > >> > > > resolve
> > > >> > > > > > > > >> > > > > > >> > > > > > > the IBP issue. However, we
> > may
> > > >> need
> > > >> > > IBP
> > > >> > > > > for
> > > >> > > > > > > > this
> > > >> > > > > > > > >> > > request
> > > >> > > > > > > > >> > > > > > after
> > > >> > > > > > > > >> > > > > > >> > all
> > > >> > > > > > > > >> > > > > > >> > > > > since
> > > >> > > > > > > > >> > > > > > >> > > > > > > the controller handles the
> > > topic
> > > >> > > > deletion
> > > >> > > > > > > > >> > differently
> > > >> > > > > > > > >> > > > > > >> depending
> > > >> > > > > > > > >> > > > > > >> > on
> > > >> > > > > > > > >> > > > > > >> > > > the
> > > >> > > > > > > > >> > > > > > >> > > > > > IBP
> > > >> > > > > > > > >> > > > > > >> > > > > > > version. In an older
> > version,
> > > we
> > > >> > > can't
> > > >> > > > > just
> > > >> > > > > > > > send
> > > >> > > > > > > > >> a
> > > >> > > > > > > > >> > > > > > StopReplica
> > > >> > > > > > > > >> > > > > > >> > > delete
> > > >> > > > > > > > >> > > > > > >> > > > > the
> > > >> > > > > > > > >> > > > > > >> > > > > > > topic immediately like
> we'd
> > > >> want to
> > > >> > > for
> > > >> > > > > > this
> > > >> > > > > > > > KIP.
> > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > > This makes me wonder if we
> > > want
> > > >> > > tagged
> > > >> > > > > > fields
> > > >> > > > > > > > on
> > > >> > > > > > > > >> all
> > > >> > > > > > > > >> > > the
> > > >> > > > > > > > >> > > > > > >> requests
> > > >> > > > > > > > >> > > > > > >> > > > after
> > > >> > > > > > > > >> > > > > > >> > > > > > > all. Let me know your
> > > thoughts!
> > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > > Justine
> > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > > On Tue, Sep 15, 2020 at
> 1:03
> > > PM
> > > >> > > Justine
> > > >> > > > > > > Olshan
> > > >> > > > > > > > <
> > > >> > > > > > > > >> > > > > > >> > > jolshan@confluent.io
> > > >> > > > > > > > >> > > > > > >> > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > > wrote:
> > > >> > > > > > > > >> > > > > > >> > > > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >> Hi all,
> > > >> > > > > > > > >> > > > > > >> > > > > > >> Jun brought up a good
> point
> > > in
> > > >> his
> > > >> > > > last
> > > >> > > > > > > email
> > > >> > > > > > > > >> about
> > > >> > > > > > > > >> > > > > tagged
> > > >> > > > > > > > >> > > > > > >> > fields,
> > > >> > > > > > > > >> > > > > > >> > > > and
> > > >> > > > > > > > >> > > > > > >> > > > > > >> I've updated the KIP to
> > > reflect
> > > >> > that
> > > >> > > > the
> > > >> > > > > > > > >> changes to
> > > >> > > > > > > > >> > > > > > requests
> > > >> > > > > > > > >> > > > > > >> and
> > > >> > > > > > > > >> > > > > > >> > > > > > responses
> > > >> > > > > > > > >> > > > > > >> > > > > > >> will be in the form of
> > tagged
> > > >> > fields
> > > >> > > > to
> > > >> > > > > > > avoid
> > > >> > > > > > > > >> > > changing
> > > >> > > > > > > > >> > > > > IBP.
> > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > >> > > > > > > > >> > > > > > >> > > > > > >> Jun: I plan on sending a
> > > >> followup
> > > >> > > > email
> > > >> > > > > to
> > > >> > > > > > > > >> address
> > > >> > > > > > > > >> > > some
> > > >> > > > > > > > >> > > > > of
> > > >> > > > > > > > >> > > > > > >> the
> > > >> > > > > > > > >> > > > > > >> > > other
> > > >> > > > > > > > >> > > > > > >> > > > > > >> points.
> > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > >> > > > > > > > >> > > > > > >> > > > > > >> Thanks,
> > > >> > > > > > > > >> > > > > > >> > > > > > >> Justine
> > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > >> > > > > > > > >> > > > > > >> > > > > > >> On Mon, Sep 14, 2020 at
> > 4:25
> > > PM
> > > >> > Jun
> > > >> > > > Rao
> > > >> > > > > <
> > > >> > > > > > > > >> > > > > jun@confluent.io>
> > > >> > > > > > > > >> > > > > > >> > wrote:
> > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> Hi, Justine,
> > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> Thanks for the updated
> > KIP.
> > > A
> > > >> few
> > > >> > > > > > comments
> > > >> > > > > > > > >> below.
> > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> 10. LeaderAndIsr
> Response:
> > > Do
> > > >> we
> > > >> > > need
> > > >> > > > > the
> > > >> > > > > > > > topic
> > > >> > > > > > > > >> > > name?
> > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> 11. For the changed
> > > >> > > request/response,
> > > >> > > > > > other
> > > >> > > > > > > > >> than
> > > >> > > > > > > > >> > > > > > >> LeaderAndIsr,
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> UpdateMetadata,
> Metadata,
> > do
> > > >> we
> > > >> > > need
> > > >> > > > to
> > > >> > > > > > > > include
> > > >> > > > > > > > >> > the
> > > >> > > > > > > > >> > > > > topic
> > > >> > > > > > > > >> > > > > > >> name?
> > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> 12. It seems that
> upgrades
> > > >> don't
> > > >> > > > > require
> > > >> > > > > > > IBP.
> > > >> > > > > > > > >> Does
> > > >> > > > > > > > >> > > > that
> > > >> > > > > > > > >> > > > > > mean
> > > >> > > > > > > > >> > > > > > >> > the
> > > >> > > > > > > > >> > > > > > >> > > > new
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> fields
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> in all the
> > request/response
> > > >> are
> > > >> > > added
> > > >> > > > > as
> > > >> > > > > > > > tagged
> > > >> > > > > > > > >> > > fields
> > > >> > > > > > > > >> > > > > > >> without
> > > >> > > > > > > > >> > > > > > >> > > > > bumping
> > > >> > > > > > > > >> > > > > > >> > > > > > up
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> the request version? It
> > > would
> > > >> be
> > > >> > > > useful
> > > >> > > > > > to
> > > >> > > > > > > > make
> > > >> > > > > > > > >> > that
> > > >> > > > > > > > >> > > > > > clear.
> > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> 13. Partition Metadata
> > file:
> > > >> Do
> > > >> > we
> > > >> > > > need
> > > >> > > > > > to
> > > >> > > > > > > > >> include
> > > >> > > > > > > > >> > > the
> > > >> > > > > > > > >> > > > > > topic
> > > >> > > > > > > > >> > > > > > >> > name
> > > >> > > > > > > > >> > > > > > >> > > > and
> > > >> > > > > > > > >> > > > > > >> > > > > > the
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> partition id since they
> > are
> > > >> > implied
> > > >> > > > in
> > > >> > > > > > the
> > > >> > > > > > > > >> > directory
> > > >> > > > > > > > >> > > > > name?
> > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> 14. In the JBOD mode, we
> > > >> support
> > > >> > > > > moving a
> > > >> > > > > > > > >> > > partition's
> > > >> > > > > > > > >> > > > > data
> > > >> > > > > > > > >> > > > > > >> from
> > > >> > > > > > > > >> > > > > > >> > > one
> > > >> > > > > > > > >> > > > > > >> > > > > > disk
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> to
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> another. Will the new
> > > >> partition
> > > >> > > > > metadata
> > > >> > > > > > > file
> > > >> > > > > > > > >> be
> > > >> > > > > > > > >> > > > copied
> > > >> > > > > > > > >> > > > > > >> during
> > > >> > > > > > > > >> > > > > > >> > > that
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> process?
> > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> 15. The KIP says "Remove
> > > >> deleted
> > > >> > > > topics
> > > >> > > > > > > from
> > > >> > > > > > > > >> > > replicas
> > > >> > > > > > > > >> > > > by
> > > >> > > > > > > > >> > > > > > >> > sending
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> StopReplicaRequest V2
> for
> > > any
> > > >> > > topics
> > > >> > > > > > which
> > > >> > > > > > > do
> > > >> > > > > > > > >> not
> > > >> > > > > > > > >> > > > > contain
> > > >> > > > > > > > >> > > > > > a
> > > >> > > > > > > > >> > > > > > >> > topic
> > > >> > > > > > > > >> > > > > > >> > > > ID,
> > > >> > > > > > > > >> > > > > > >> > > > > > and
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> V3 for any topics which
> do
> > > >> > contain
> > > >> > > a
> > > >> > > > > > topic
> > > >> > > > > > > > >> ID.".
> > > >> > > > > > > > >> > > > > However,
> > > >> > > > > > > > >> > > > > > it
> > > >> > > > > > > > >> > > > > > >> > > seems
> > > >> > > > > > > > >> > > > > > >> > > > > the
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> updated controller will
> > > create
> > > >> > all
> > > >> > > > > > missing
> > > >> > > > > > > > >> topic
> > > >> > > > > > > > >> > IDs
> > > >> > > > > > > > >> > > > > first
> > > >> > > > > > > > >> > > > > > >> > before
> > > >> > > > > > > > >> > > > > > >> > > > > doing
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> other actions. So, is
> > > >> > > > > StopReplicaRequest
> > > >> > > > > > V2
> > > >> > > > > > > > >> > needed?
> > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> Jun
> > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> On Fri, Sep 11, 2020 at
> > > 10:31
> > > >> AM
> > > >> > > John
> > > >> > > > > > > > Roesler <
> > > >> > > > > > > > >> > > > > > >> > > vvcephei@apache.org
> > > >> > > > > > > > >> > > > > > >> > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> wrote:
> > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > Thanks, Justine!
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > Your response seems
> > > >> compelling
> > > >> > to
> > > >> > > > me.
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > -John
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > On Fri, 2020-09-11 at
> > > 10:17
> > > >> > > -0700,
> > > >> > > > > > > Justine
> > > >> > > > > > > > >> > Olshan
> > > >> > > > > > > > >> > > > > wrote:
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Hello all,
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Thanks for
> continuing
> > > the
> > > >> > > > > > discussion! I
> > > >> > > > > > > > >> have a
> > > >> > > > > > > > >> > > few
> > > >> > > > > > > > >> > > > > > >> > responses
> > > >> > > > > > > > >> > > > > > >> > > to
> > > >> > > > > > > > >> > > > > > >> > > > > > your
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > points.
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Tom: You are correct
> > in
> > > >> that
> > > >> > > this
> > > >> > > > > KIP
> > > >> > > > > > > has
> > > >> > > > > > > > >> not
> > > >> > > > > > > > >> > > > > > mentioned
> > > >> > > > > > > > >> > > > > > >> the
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> DeleteTopicsRequest. I
> > > >> think
> > > >> > > that
> > > >> > > > > > this
> > > >> > > > > > > > >> would
> > > >> > > > > > > > >> > be
> > > >> > > > > > > > >> > > > out
> > > >> > > > > > > > >> > > > > of
> > > >> > > > > > > > >> > > > > > >> > scope
> > > >> > > > > > > > >> > > > > > >> > > > for
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> now, but
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > may be something
> worth
> > > >> adding
> > > >> > > in
> > > >> > > > > the
> > > >> > > > > > > > >> future.
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > John: We did
> consider
> > > >> > sequence
> > > >> > > > ids,
> > > >> > > > > > but
> > > >> > > > > > > > >> there
> > > >> > > > > > > > >> > > are
> > > >> > > > > > > > >> > > > a
> > > >> > > > > > > > >> > > > > > few
> > > >> > > > > > > > >> > > > > > >> > > reasons
> > > >> > > > > > > > >> > > > > > >> > > > > to
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> favor
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > UUIDs. There are
> > several
> > > >> > cases
> > > >> > > > > where
> > > >> > > > > > > > topics
> > > >> > > > > > > > >> > from
> > > >> > > > > > > > >> > > > > > >> different
> > > >> > > > > > > > >> > > > > > >> > > > > clusters
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> may
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > interact now and in
> > the
> > > >> > future.
> > > >> > > > For
> > > >> > > > > > > > >> example,
> > > >> > > > > > > > >> > > > Mirror
> > > >> > > > > > > > >> > > > > > >> Maker 2
> > > >> > > > > > > > >> > > > > > >> > > may
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> benefit
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > from being able to
> > > detect
> > > >> > when
> > > >> > > a
> > > >> > > > > > > cluster
> > > >> > > > > > > > >> being
> > > >> > > > > > > > >> > > > > > mirrored
> > > >> > > > > > > > >> > > > > > >> is
> > > >> > > > > > > > >> > > > > > >> > > > > deleted
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> and
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > recreated and
> globally
> > > >> unique
> > > >> > > > > > > identifiers
> > > >> > > > > > > > >> > would
> > > >> > > > > > > > >> > > > make
> > > >> > > > > > > > >> > > > > > >> > > resolving
> > > >> > > > > > > > >> > > > > > >> > > > > > issues
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > easier than sequence
> > IDs
> > > >> > which
> > > >> > > > may
> > > >> > > > > > > > collide
> > > >> > > > > > > > >> > > between
> > > >> > > > > > > > >> > > > > > >> > clusters.
> > > >> > > > > > > > >> > > > > > >> > > > > > KIP-405
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > (tiered storage)
> will
> > > also
> > > >> > > > benefit
> > > >> > > > > > from
> > > >> > > > > > > > >> > globally
> > > >> > > > > > > > >> > > > > > unique
> > > >> > > > > > > > >> > > > > > >> IDs
> > > >> > > > > > > > >> > > > > > >> > > as
> > > >> > > > > > > > >> > > > > > >> > > > > > shared
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > buckets may be used
> > > >> between
> > > >> > > > > clusters.
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Globally unique IDs
> > > would
> > > >> > also
> > > >> > > > make
> > > >> > > > > > > > >> > > functionality
> > > >> > > > > > > > >> > > > > like
> > > >> > > > > > > > >> > > > > > >> > moving
> > > >> > > > > > > > >> > > > > > >> > > > > > topics
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > between disparate
> > > clusters
> > > >> > > easier
> > > >> > > > > in
> > > >> > > > > > > the
> > > >> > > > > > > > >> > future,
> > > >> > > > > > > > >> > > > > > >> simplify
> > > >> > > > > > > > >> > > > > > >> > any
> > > >> > > > > > > > >> > > > > > >> > > > > > future
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > implementations of
> > > backups
> > > >> > and
> > > >> > > > > > > restores,
> > > >> > > > > > > > >> and
> > > >> > > > > > > > >> > > more.
> > > >> > > > > > > > >> > > > > In
> > > >> > > > > > > > >> > > > > > >> > > general,
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> unique IDs
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > would ensure that
> the
> > > >> source
> > > >> > > > > cluster
> > > >> > > > > > > > >> topics do
> > > >> > > > > > > > >> > > not
> > > >> > > > > > > > >> > > > > > >> conflict
> > > >> > > > > > > > >> > > > > > >> > > > with
> > > >> > > > > > > > >> > > > > > >> > > > > > the
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > destination cluster
> > > >> topics.
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > If we were to use
> > > sequence
> > > >> > ids,
> > > >> > > > we
> > > >> > > > > > > would
> > > >> > > > > > > > >> need
> > > >> > > > > > > > >> > > > > > >> sufficiently
> > > >> > > > > > > > >> > > > > > >> > > > large
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> cluster
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > ids to be stored
> with
> > > the
> > > >> > topic
> > > >> > > > > > > > >> identifiers or
> > > >> > > > > > > > >> > > we
> > > >> > > > > > > > >> > > > > run
> > > >> > > > > > > > >> > > > > > >> the
> > > >> > > > > > > > >> > > > > > >> > > risk
> > > >> > > > > > > > >> > > > > > >> > > > of
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > collisions. This
> will
> > > >> give up
> > > >> > > any
> > > >> > > > > > > > >> advantage in
> > > >> > > > > > > > >> > > > > > >> compactness
> > > >> > > > > > > > >> > > > > > >> > > that
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> sequence
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > numbers may bring.
> > Given
> > > >> > these
> > > >> > > > > > > > advantages I
> > > >> > > > > > > > >> > > think
> > > >> > > > > > > > >> > > > it
> > > >> > > > > > > > >> > > > > > >> makes
> > > >> > > > > > > > >> > > > > > >> > > > sense
> > > >> > > > > > > > >> > > > > > >> > > > > to
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> use
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > UUIDs.
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Gokul: This is an
> > > >> interesting
> > > >> > > > idea,
> > > >> > > > > > but
> > > >> > > > > > > > >> this
> > > >> > > > > > > > >> > is
> > > >> > > > > > > > >> > > a
> > > >> > > > > > > > >> > > > > > >> breaking
> > > >> > > > > > > > >> > > > > > >> > > > > change.
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> Out of
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > scope for now, but
> > maybe
> > > >> > worth
> > > >> > > > > > > discussing
> > > >> > > > > > > > >> in
> > > >> > > > > > > > >> > the
> > > >> > > > > > > > >> > > > > > future.
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Hope this explains
> > some
> > > of
> > > >> > the
> > > >> > > > > > > decisions,
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Justine
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > On Fri, Sep 11, 2020
> > at
> > > >> 8:27
> > > >> > AM
> > > >> > > > > Gokul
> > > >> > > > > > > > >> Ramanan
> > > >> > > > > > > > >> > > > > > >> Subramanian <
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> gokul2411s@gmail.com>
> > > >> wrote:
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Hi.
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Thanks for the
> KIP.
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Have you thought
> > about
> > > >> > > whether
> > > >> > > > it
> > > >> > > > > > > makes
> > > >> > > > > > > > >> > sense
> > > >> > > > > > > > >> > > to
> > > >> > > > > > > > >> > > > > > >> support
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> authorizing a
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > principal for a
> > topic
> > > ID
> > > >> > > rather
> > > >> > > > > > than
> > > >> > > > > > > a
> > > >> > > > > > > > >> topic
> > > >> > > > > > > > >> > > > name
> > > >> > > > > > > > >> > > > > to
> > > >> > > > > > > > >> > > > > > >> > > achieve
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> tighter
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > security?
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Or is the topic ID
> > > >> > > > fundamentally
> > > >> > > > > an
> > > >> > > > > > > > >> internal
> > > >> > > > > > > > >> > > > > detail
> > > >> > > > > > > > >> > > > > > >> > similar
> > > >> > > > > > > > >> > > > > > >> > > > to
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> epochs
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > used
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > in a bunch of
> other
> > > >> places
> > > >> > in
> > > >> > > > > > Kafka?
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Thanks.
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > On Fri, Sep 11,
> 2020
> > > at
> > > >> > 4:06
> > > >> > > PM
> > > >> > > > > > John
> > > >> > > > > > > > >> > Roesler <
> > > >> > > > > > > > >> > > > > > >> > > > > > vvcephei@apache.org>
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > wrote:
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Hello Justine,
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Thanks for the
> > KIP!
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > I happen to have
> > > been
> > > >> > > > > confronted
> > > >> > > > > > > > >> recently
> > > >> > > > > > > > >> > > with
> > > >> > > > > > > > >> > > > > the
> > > >> > > > > > > > >> > > > > > >> need
> > > >> > > > > > > > >> > > > > > >> > > to
> > > >> > > > > > > > >> > > > > > >> > > > > keep
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > track of
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > a
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > large number of
> > > >> topics as
> > > >> > > > > > compactly
> > > >> > > > > > > > as
> > > >> > > > > > > > >> > > > > possible. I
> > > >> > > > > > > > >> > > > > > >> was
> > > >> > > > > > > > >> > > > > > >> > > > going
> > > >> > > > > > > > >> > > > > > >> > > > > to
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> come
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > up
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > with some way to
> > > >> > dictionary
> > > >> > > > > > encode
> > > >> > > > > > > > the
> > > >> > > > > > > > >> > topic
> > > >> > > > > > > > >> > > > > names
> > > >> > > > > > > > >> > > > > > >> as
> > > >> > > > > > > > >> > > > > > >> > > > > integers,
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> but
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > this
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > seems much
> better!
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Apologies if
> this
> > > has
> > > >> > been
> > > >> > > > > raised
> > > >> > > > > > > > >> before,
> > > >> > > > > > > > >> > > but
> > > >> > > > > > > > >> > > > > I’m
> > > >> > > > > > > > >> > > > > > >> > > wondering
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> about the
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > choice of UUID
> vs
> > > >> > sequence
> > > >> > > > > number
> > > >> > > > > > > for
> > > >> > > > > > > > >> the
> > > >> > > > > > > > >> > > ids.
> > > >> > > > > > > > >> > > > > > >> > Typically,
> > > >> > > > > > > > >> > > > > > >> > > > > I’ve
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> seen
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > UUIDs
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > in two
> situations:
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 1. When
> processes
> > > >> need to
> > > >> > > > > > generate
> > > >> > > > > > > > >> > > > non-colliding
> > > >> > > > > > > > >> > > > > > >> > > > identifiers
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> without
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > coordination.
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 2. When the
> > > identifier
> > > >> > > needs
> > > >> > > > to
> > > >> > > > > > be
> > > >> > > > > > > > >> > > > “universally
> > > >> > > > > > > > >> > > > > > >> > unique”;
> > > >> > > > > > > > >> > > > > > >> > > > > I.e.,
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> the
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > identifier needs
> > to
> > > >> > > > distinguish
> > > >> > > > > > the
> > > >> > > > > > > > >> entity
> > > >> > > > > > > > >> > > > from
> > > >> > > > > > > > >> > > > > > all
> > > >> > > > > > > > >> > > > > > >> > other
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> entities
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > that
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > could ever
> exist.
> > > >> This is
> > > >> > > > > useful
> > > >> > > > > > in
> > > >> > > > > > > > >> cases
> > > >> > > > > > > > >> > > > where
> > > >> > > > > > > > >> > > > > > >> > entities
> > > >> > > > > > > > >> > > > > > >> > > > from
> > > >> > > > > > > > >> > > > > > >> > > > > > all
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > kinds
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > of
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > systems get
> mixed
> > > >> > together,
> > > >> > > > > such
> > > >> > > > > > as
> > > >> > > > > > > > >> when
> > > >> > > > > > > > >> > > > dumping
> > > >> > > > > > > > >> > > > > > >> logs
> > > >> > > > > > > > >> > > > > > >> > > from
> > > >> > > > > > > > >> > > > > > >> > > > > all
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > processes
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > in
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a company into a
> > > >> common
> > > >> > > > system.
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Maybe I’m being
> > > >> > > > short-sighted,
> > > >> > > > > > but
> > > >> > > > > > > it
> > > >> > > > > > > > >> > > doesn’t
> > > >> > > > > > > > >> > > > > seem
> > > >> > > > > > > > >> > > > > > >> like
> > > >> > > > > > > > >> > > > > > >> > > > > either
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> really
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > applies here. It
> > > seems
> > > >> > like
> > > >> > > > the
> > > >> > > > > > > > brokers
> > > >> > > > > > > > >> > > could
> > > >> > > > > > > > >> > > > > and
> > > >> > > > > > > > >> > > > > > >> would
> > > >> > > > > > > > >> > > > > > >> > > > > achieve
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > consensus
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > when creating a
> > > topic
> > > >> > > anyway,
> > > >> > > > > > which
> > > >> > > > > > > > is
> > > >> > > > > > > > >> all
> > > >> > > > > > > > >> > > > > that’s
> > > >> > > > > > > > >> > > > > > >> > > required
> > > >> > > > > > > > >> > > > > > >> > > > to
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > generate
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > non-colliding
> > > sequence
> > > >> > ids.
> > > >> > > > For
> > > >> > > > > > the
> > > >> > > > > > > > >> > second,
> > > >> > > > > > > > >> > > as
> > > >> > > > > > > > >> > > > > you
> > > >> > > > > > > > >> > > > > > >> > > mention,
> > > >> > > > > > > > >> > > > > > >> > > > > we
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> could
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > assign
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a UUID to the
> > > cluster
> > > >> as
> > > >> > a
> > > >> > > > > whole,
> > > >> > > > > > > > which
> > > >> > > > > > > > >> > > would
> > > >> > > > > > > > >> > > > > > render
> > > >> > > > > > > > >> > > > > > >> > any
> > > >> > > > > > > > >> > > > > > >> > > > > > resource
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > scoped
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > the broker
> > > universally
> > > >> > > unique
> > > >> > > > > as
> > > >> > > > > > > > well.
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > The reason I
> > mention
> > > >> this
> > > >> > > is
> > > >> > > > > > that,
> > > >> > > > > > > > >> > although
> > > >> > > > > > > > >> > > a
> > > >> > > > > > > > >> > > > > UUID
> > > >> > > > > > > > >> > > > > > >> is
> > > >> > > > > > > > >> > > > > > >> > way
> > > >> > > > > > > > >> > > > > > >> > > > > more
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > compact
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > than topic
> names,
> > > it’s
> > > >> > > still
> > > >> > > > 16
> > > >> > > > > > > > bytes.
> > > >> > > > > > > > >> In
> > > >> > > > > > > > >> > > > > > contrast,
> > > >> > > > > > > > >> > > > > > >> a
> > > >> > > > > > > > >> > > > > > >> > > > 4-byte
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> integer
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > sequence id
> would
> > > give
> > > >> > us 4
> > > >> > > > > > billion
> > > >> > > > > > > > >> unique
> > > >> > > > > > > > >> > > > > topics
> > > >> > > > > > > > >> > > > > > >> per
> > > >> > > > > > > > >> > > > > > >> > > > > cluster,
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> which
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > seems
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > like enough ;)
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Considering the
> > > >> number of
> > > >> > > > > > different
> > > >> > > > > > > > >> times
> > > >> > > > > > > > >> > > > these
> > > >> > > > > > > > >> > > > > > >> topic
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> identifiers are
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > sent
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > over the wire or
> > > >> stored
> > > >> > in
> > > >> > > > > > memory,
> > > >> > > > > > > it
> > > >> > > > > > > > >> > seems
> > > >> > > > > > > > >> > > > like
> > > >> > > > > > > > >> > > > > > it
> > > >> > > > > > > > >> > > > > > >> > might
> > > >> > > > > > > > >> > > > > > >> > > > be
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> worth
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > the
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > additional 4x
> > space
> > > >> > > savings.
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > What do you
> think
> > > >> about
> > > >> > > this?
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Thanks,
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > John
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > On Fri, Sep 11,
> > > 2020,
> > > >> at
> > > >> > > > 03:20,
> > > >> > > > > > Tom
> > > >> > > > > > > > >> > Bentley
> > > >> > > > > > > > >> > > > > wrote:
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Hi Justine,
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > This looks
> like
> > a
> > > >> very
> > > >> > > > > welcome
> > > >> > > > > > > > >> > > improvement.
> > > >> > > > > > > > >> > > > > > >> Thanks!
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Maybe I missed
> > it,
> > > >> but
> > > >> > > the
> > > >> > > > > KIP
> > > >> > > > > > > > >> doesn't
> > > >> > > > > > > > >> > > seem
> > > >> > > > > > > > >> > > > to
> > > >> > > > > > > > >> > > > > > >> > mention
> > > >> > > > > > > > >> > > > > > >> > > > > > changing
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > DeleteTopicsRequest
> > > >> to
> > > >> > > > > identify
> > > >> > > > > > > the
> > > >> > > > > > > > >> > topic
> > > >> > > > > > > > >> > > > > using
> > > >> > > > > > > > >> > > > > > an
> > > >> > > > > > > > >> > > > > > >> > id.
> > > >> > > > > > > > >> > > > > > >> > > > > Maybe
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > that's out
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > of
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > scope, but
> > > >> > > > > DeleteTopicsRequest
> > > >> > > > > > is
> > > >> > > > > > > > not
> > > >> > > > > > > > >> > > listed
> > > >> > > > > > > > >> > > > > > among
> > > >> > > > > > > > >> > > > > > >> > the
> > > >> > > > > > > > >> > > > > > >> > > > > Future
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> Work
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > APIs
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > either.
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Kind regards,
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Tom
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > On Thu, Sep
> 10,
> > > >> 2020 at
> > > >> > > > 3:59
> > > >> > > > > PM
> > > >> > > > > > > > >> Satish
> > > >> > > > > > > > >> > > > > Duggana <
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > >> satish.duggana@gmail.com
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > wrote:
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Thanks
> > > >> Lucas/Justine
> > > >> > > for
> > > >> > > > > the
> > > >> > > > > > > nice
> > > >> > > > > > > > >> KIP.
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > It has
> several
> > > >> > benefits
> > > >> > > > > which
> > > >> > > > > > > > also
> > > >> > > > > > > > >> > > include
> > > >> > > > > > > > >> > > > > > >> > > simplifying
> > > >> > > > > > > > >> > > > > > >> > > > > the
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> topic
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > deletion
> > process
> > > >> by
> > > >> > > > > > controller
> > > >> > > > > > > > and
> > > >> > > > > > > > >> > logs
> > > >> > > > > > > > >> > > > > > cleanup
> > > >> > > > > > > > >> > > > > > >> by
> > > >> > > > > > > > >> > > > > > >> > > > > brokers
> > > >> > > > > > > > >> > > > > > >> > > > > > in
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > corner
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > cases.
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Best,
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Satish.
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > On Wed, Sep
> 9,
> > > >> 2020
> > > >> > at
> > > >> > > > > 10:07
> > > >> > > > > > PM
> > > >> > > > > > > > >> > Justine
> > > >> > > > > > > > >> > > > > > Olshan <
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > jolshan@confluent.io
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > wrote:
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Hello all,
> > > it's
> > > >> > been
> > > >> > > > > > almost a
> > > >> > > > > > > > >> year!
> > > >> > > > > > > > >> > > I've
> > > >> > > > > > > > >> > > > > > made
> > > >> > > > > > > > >> > > > > > >> > some
> > > >> > > > > > > > >> > > > > > >> > > > > > changes
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> to
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > this
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > KIP
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > and hope to
> > > >> continue
> > > >> > > the
> > > >> > > > > > > > >> discussion.
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > One of the
> > > main
> > > >> > > changes
> > > >> > > > > > I've
> > > >> > > > > > > > >> added
> > > >> > > > > > > > >> > is
> > > >> > > > > > > > >> > > > now
> > > >> > > > > > > > >> > > > > > the
> > > >> > > > > > > > >> > > > > > >> > > > metadata
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> response
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > will
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > include the
> > > topic
> > > >> ID
> > > >> > > (as
> > > >> > > > > > Colin
> > > >> > > > > > > > >> > > suggested).
> > > >> > > > > > > > >> > > > > > >> Clients
> > > >> > > > > > > > >> > > > > > >> > > can
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> obtain the
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > topicID
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > of a given
> > topic
> > > >> > > through
> > > >> > > > a
> > > >> > > > > > > > >> > > > TopicDescription.
> > > >> > > > > > > > >> > > > > > The
> > > >> > > > > > > > >> > > > > > >> > > > topicId
> > > >> > > > > > > > >> > > > > > >> > > > > > will
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > also be
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > included
> with
> > > the
> > > >> > > > > > > UpdateMetadata
> > > >> > > > > > > > >> > > request.
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Let me
> know
> > > what
> > > >> > you
> > > >> > > > all
> > > >> > > > > > > think.
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Thank you,
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Justine
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > On
> > 2019/09/13
> > > >> > > 16:38:26,
> > > >> > > > > > > "Colin
> > > >> > > > > > > > >> > > McCabe" <
> > > >> > > > > > > > >> > > > > > >> > > > > > cmccabe@apache.org
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > wrote:
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Hi
> Lucas,
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Thanks
> for
> > > >> > tackling
> > > >> > > > > this.
> > > >> > > > > > > > >> Topic
> > > >> > > > > > > > >> > IDs
> > > >> > > > > > > > >> > > > > are a
> > > >> > > > > > > > >> > > > > > >> > great
> > > >> > > > > > > > >> > > > > > >> > > > > idea,
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> and
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > this
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > is
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > really good
> > > >> writeup.
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > For
> > > >> > > > > > > /brokers/topics/[topic],
> > > >> > > > > > > > >> the
> > > >> > > > > > > > >> > > > schema
> > > >> > > > > > > > >> > > > > > >> version
> > > >> > > > > > > > >> > > > > > >> > > > > should
> > > >> > > > > > > > >> > > > > > >> > > > > > be
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > bumped
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > version 3,
> > > rather
> > > >> > than
> > > >> > > 2.
> > > >> > > > > > > > KIP-455
> > > >> > > > > > > > >> > > bumped
> > > >> > > > > > > > >> > > > > the
> > > >> > > > > > > > >> > > > > > >> > version
> > > >> > > > > > > > >> > > > > > >> > > > of
> > > >> > > > > > > > >> > > > > > >> > > > > > this
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > znode
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 2
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > already :)
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Given
> that
> > > >> we're
> > > >> > > > going
> > > >> > > > > to
> > > >> > > > > > > be
> > > >> > > > > > > > >> > seeing
> > > >> > > > > > > > >> > > > > these
> > > >> > > > > > > > >> > > > > > >> > things
> > > >> > > > > > > > >> > > > > > >> > > as
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> strings
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > as
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > lot
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > (in
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > logs, in
> > > >> ZooKeeper,
> > > >> > on
> > > >> > > > the
> > > >> > > > > > > > >> > command-line,
> > > >> > > > > > > > >> > > > > > etc.),
> > > >> > > > > > > > >> > > > > > >> > does
> > > >> > > > > > > > >> > > > > > >> > > it
> > > >> > > > > > > > >> > > > > > >> > > > > > make
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > sense to
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > use
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > base64 when
> > > >> > converting
> > > >> > > > them
> > > >> > > > > > to
> > > >> > > > > > > > >> > strings?
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Here is
> an
> > > >> > example
> > > >> > > of
> > > >> > > > > the
> > > >> > > > > > > hex
> > > >> > > > > > > > >> > > > > > >> representation:
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > >> > > > > > > > >> > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > And here
> > is
> > > an
> > > >> > > > example
> > > >> > > > > in
> > > >> > > > > > > > >> base64.
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > >> > > > b8tRS7h4TJ2Vt43Dp85v2A
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > The
> base64
> > > >> > version
> > > >> > > > > saves
> > > >> > > > > > 15
> > > >> > > > > > > > >> > letters
> > > >> > > > > > > > >> > > > (to
> > > >> > > > > > > > >> > > > > be
> > > >> > > > > > > > >> > > > > > >> > fair,
> > > >> > > > > > > > >> > > > > > >> > > 4
> > > >> > > > > > > > >> > > > > > >> > > > of
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> those
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > were
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > dashes that
> we
> > > >> could
> > > >> > > have
> > > >> > > > > > > elided
> > > >> > > > > > > > in
> > > >> > > > > > > > >> > the
> > > >> > > > > > > > >> > > > hex
> > > >> > > > > > > > >> > > > > > >> > > > > > representation.)
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Another
> > > thing
> > > >> to
> > > >> > > > > consider
> > > >> > > > > > > is
> > > >> > > > > > > > >> that
> > > >> > > > > > > > >> > we
> > > >> > > > > > > > >> > > > > > should
> > > >> > > > > > > > >> > > > > > >> > > specify
> > > >> > > > > > > > >> > > > > > >> > > > > > that
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> the
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > all-zeroes
> > UUID
> > > is
> > > >> > not
> > > >> > > a
> > > >> > > > > > valid
> > > >> > > > > > > > >> topic
> > > >> > > > > > > > >> > > UUID.
> > > >> > > > > > > > >> > > > > >  We
> > > >> > > > > > > > >> > > > > > >> > can't
> > > >> > > > > > > > >> > > > > > >> > > > use
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> null
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > for
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > this
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > because we
> > can't
> > > >> > pass a
> > > >> > > > > null
> > > >> > > > > > > UUID
> > > >> > > > > > > > >> over
> > > >> > > > > > > > >> > > the
> > > >> > > > > > > > >> > > > > RPC
> > > >> > > > > > > > >> > > > > > >> > > protocol
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> (there
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > is no
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > special
> > pattern
> > > >> for
> > > >> > > null,
> > > >> > > > > nor
> > > >> > > > > > > do
> > > >> > > > > > > > we
> > > >> > > > > > > > >> > want
> > > >> > > > > > > > >> > > > to
> > > >> > > > > > > > >> > > > > > >> waste
> > > >> > > > > > > > >> > > > > > >> > > space
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> reserving
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > such
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > pattern.)
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Maybe I
> > > missed
> > > >> > it,
> > > >> > > > but
> > > >> > > > > > did
> > > >> > > > > > > > you
> > > >> > > > > > > > >> > > > describe
> > > >> > > > > > > > >> > > > > > >> > > "migration
> > > >> > > > > > > > >> > > > > > >> > > > > > of...
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > existing
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > topic[s]
> > without
> > > >> > topic
> > > >> > > > IDs"
> > > >> > > > > > in
> > > >> > > > > > > > >> detail
> > > >> > > > > > > > >> > in
> > > >> > > > > > > > >> > > > any
> > > >> > > > > > > > >> > > > > > >> > section?
> > > >> > > > > > > > >> > > > > > >> > > > It
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> seems
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > like
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > when
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > the new
> > > controller
> > > >> > > > becomes
> > > >> > > > > > > > active,
> > > >> > > > > > > > >> it
> > > >> > > > > > > > >> > > > should
> > > >> > > > > > > > >> > > > > > >> just
> > > >> > > > > > > > >> > > > > > >> > > > > generate
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> random
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > UUIDs for
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > these, and
> > write
> > > >> the
> > > >> > > > random
> > > >> > > > > > > UUIDs
> > > >> > > > > > > > >> back
> > > >> > > > > > > > >> > > to
> > > >> > > > > > > > >> > > > > > >> > ZooKeeper.
> > > >> > > > > > > > >> > > > > > >> > > > It
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> would be
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > good
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > to
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > spell that
> > out.
> > > >> We
> > > >> > > > should
> > > >> > > > > > make
> > > >> > > > > > > > it
> > > >> > > > > > > > >> > clear
> > > >> > > > > > > > >> > > > > that
> > > >> > > > > > > > >> > > > > > >> this
> > > >> > > > > > > > >> > > > > > >> > > > > happens
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > regardless
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > of
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > the
> > inter-broker
> > > >> > > protocol
> > > >> > > > > > > version
> > > >> > > > > > > > >> > (it's
> > > >> > > > > > > > >> > > a
> > > >> > > > > > > > >> > > > > > >> > compatible
> > > >> > > > > > > > >> > > > > > >> > > > > > change).
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > >> > > "LeaderAndIsrRequests
> > > >> > > > > > > > >> including an
> > > >> > > > > > > > >> > > > > > >> > > > is_every_partition
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> flag"
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > seems a
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > bit wordy.
> > Can
> > > we
> > > >> > just
> > > >> > > > > call
> > > >> > > > > > > > these
> > > >> > > > > > > > >> > "full
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> LeaderAndIsrRequests"?
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > Then
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > the
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > RPC field
> > could
> > > be
> > > >> > > named
> > > >> > > > > > > "full".
> > > >> > > > > > > > >> > Also,
> > > >> > > > > > > > >> > > it
> > > >> > > > > > > > >> > > > > > would
> > > >> > > > > > > > >> > > > > > >> > > > probably
> > > >> > > > > > > > >> > > > > > >> > > > > > be
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > better
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > for the
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > RPC field to
> > be
> > > an
> > > >> > enum
> > > >> > > > of
> > > >> > > > > {
> > > >> > > > > > > > >> > > UNSPECIFIED,
> > > >> > > > > > > > >> > > > > > >> > > INCREMENTAL,
> > > >> > > > > > > > >> > > > > > >> > > > > FULL
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> }, so
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > that
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > we
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > can cleanly
> > > handle
> > > >> > old
> > > >> > > > > > versions
> > > >> > > > > > > > (by
> > > >> > > > > > > > >> > > > treating
> > > >> > > > > > > > >> > > > > > >> them
> > > >> > > > > > > > >> > > > > > >> > as
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> UNSPECIFIED)
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > In the
> > > >> > > > > > LeaderAndIsrRequest
> > > >> > > > > > > > >> > section,
> > > >> > > > > > > > >> > > > you
> > > >> > > > > > > > >> > > > > > >> write
> > > >> > > > > > > > >> > > > > > >> > "A
> > > >> > > > > > > > >> > > > > > >> > > > > final
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > deletion
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > event
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > will be
> > > secheduled
> > > >> > for
> > > >> > > X
> > > >> > > > ms
> > > >> > > > > > > after
> > > >> > > > > > > > >> the
> > > >> > > > > > > > >> > > > > > >> > > > LeaderAndIsrRequest
> > > >> > > > > > > > >> > > > > > >> > > > > > was
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > first
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > received..."
> > I
> > > >> guess
> > > >> > > > the X
> > > >> > > > > > > was a
> > > >> > > > > > > > >> > > > > placeholder
> > > >> > > > > > > > >> > > > > > >> that
> > > >> > > > > > > > >> > > > > > >> > > you
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> intended
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > replace
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > before
> > posting?
> > > :)
> > > >> > In
> > > >> > > > any
> > > >> > > > > > > case,
> > > >> > > > > > > > >> this
> > > >> > > > > > > > >> > > > seems
> > > >> > > > > > > > >> > > > > > like
> > > >> > > > > > > > >> > > > > > >> > the
> > > >> > > > > > > > >> > > > > > >> > > > kind
> > > >> > > > > > > > >> > > > > > >> > > > > > of
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > thing
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > we'd
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > want a
> > > >> configuration
> > > >> > > for.
> > > >> > > > > > > Let's
> > > >> > > > > > > > >> > > describe
> > > >> > > > > > > > >> > > > > that
> > > >> > > > > > > > >> > > > > > >> > > > > > configuration
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> key
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > somewhere
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > in this KIP,
> > > >> > including
> > > >> > > > what
> > > >> > > > > > its
> > > >> > > > > > > > >> > default
> > > >> > > > > > > > >> > > > > value
> > > >> > > > > > > > >> > > > > > >> is.
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > We
> should
> > > >> > probably
> > > >> > > > also
> > > >> > > > > > > log a
> > > >> > > > > > > > >> > bunch
> > > >> > > > > > > > >> > > of
> > > >> > > > > > > > >> > > > > > >> messages
> > > >> > > > > > > > >> > > > > > >> > > at
> > > >> > > > > > > > >> > > > > > >> > > > > WARN
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> level
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > when
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > something is
> > > >> > scheduled
> > > >> > > > for
> > > >> > > > > > > > >> deletion,
> > > >> > > > > > > > >> > as
> > > >> > > > > > > > >> > > > > well.
> > > >> > > > > > > > >> > > > > > >> > (Maybe
> > > >> > > > > > > > >> > > > > > >> > > > > this
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> was
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > assumed, but
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > it would be
> > good
> > > >> to
> > > >> > > > mention
> > > >> > > > > > > it).
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > I feel
> > like
> > > >> there
> > > >> > > > are a
> > > >> > > > > > few
> > > >> > > > > > > > >> > sections
> > > >> > > > > > > > >> > > > > that
> > > >> > > > > > > > >> > > > > > >> > should
> > > >> > > > > > > > >> > > > > > >> > > be
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> moved to
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > "rejected
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> alternatives."
> > > >> For
> > > >> > > > > example,
> > > >> > > > > > in
> > > >> > > > > > > > the
> > > >> > > > > > > > >> > > > > > DeleteTopics
> > > >> > > > > > > > >> > > > > > >> > > > section,
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> since
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > we're
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > not
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > going to do
> > > >> option 1
> > > >> > or
> > > >> > > > 2,
> > > >> > > > > > > these
> > > >> > > > > > > > >> > should
> > > >> > > > > > > > >> > > be
> > > >> > > > > > > > >> > > > > > moved
> > > >> > > > > > > > >> > > > > > >> > into
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> "rejected
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> alternatives,"
> > > >> > rather
> > > >> > > > than
> > > >> > > > > > > > >> appearing
> > > >> > > > > > > > >> > > > > inline.
> > > >> > > > > > > > >> > > > > > >> > > Another
> > > >> > > > > > > > >> > > > > > >> > > > > case
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> is
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > the
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > "Should
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > we remove
> > topic
> > > >> name
> > > >> > > from
> > > >> > > > > the
> > > >> > > > > > > > >> protocol
> > > >> > > > > > > > >> > > > where
> > > >> > > > > > > > >> > > > > > >> > > possible"
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> section.
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > This
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > is
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > clearly
> > > >> discussing a
> > > >> > > > design
> > > >> > > > > > > > >> > alternative
> > > >> > > > > > > > >> > > > that
> > > >> > > > > > > > >> > > > > > >> we're
> > > >> > > > > > > > >> > > > > > >> > > not
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> proposing
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > implement:
> > > >> removing
> > > >> > the
> > > >> > > > > topic
> > > >> > > > > > > > name
> > > >> > > > > > > > >> > from
> > > >> > > > > > > > >> > > > > those
> > > >> > > > > > > > >> > > > > > >> > > > protocols.
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Is it
> > really
> > > >> > > > necessary
> > > >> > > > > to
> > > >> > > > > > > > have
> > > >> > > > > > > > >> a
> > > >> > > > > > > > >> > new
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > /admin/delete_topics_by_id
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > path
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > in
> ZooKeeper?
> > > It
> > > >> > seems
> > > >> > > > > like
> > > >> > > > > > we
> > > >> > > > > > > > >> don't
> > > >> > > > > > > > >> > > > really
> > > >> > > > > > > > >> > > > > > >> need
> > > >> > > > > > > > >> > > > > > >> > > this.
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> Whenever
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > there is
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > a new
> > > controller,
> > > >> > we'll
> > > >> > > > > send
> > > >> > > > > > > out
> > > >> > > > > > > > >> full
> > > >> > > > > > > > >> > > > > > >> > > > > LeaderAndIsrRequests
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> which
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > will
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > trigger the
> > > stale
> > > >> > > topics
> > > >> > > > to
> > > >> > > > > > be
> > > >> > > > > > > > >> cleaned
> > > >> > > > > > > > >> > > up.
> > > >> > > > > > > > >> > > > > >  The
> > > >> > > > > > > > >> > > > > > >> > > active
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > controller
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > will
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > also send
> the
> > > full
> > > >> > > > > > > > >> LeaderAndIsrRequest
> > > >> > > > > > > > >> > > to
> > > >> > > > > > > > >> > > > > > >> brokers
> > > >> > > > > > > > >> > > > > > >> > > that
> > > >> > > > > > > > >> > > > > > >> > > > > are
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> just
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > starting
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > up.    So we
> > > don't
> > > >> > > really
> > > >> > > > > > need
> > > >> > > > > > > > this
> > > >> > > > > > > > >> > kind
> > > >> > > > > > > > >> > > > of
> > > >> > > > > > > > >> > > > > > >> > two-phase
> > > >> > > > > > > > >> > > > > > >> > > > > > commit
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > (send
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > out
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > >> StopReplicasRequest,
> > > >> > > get
> > > >> > > > > ACKs
> > > >> > > > > > > > from
> > > >> > > > > > > > >> all
> > > >> > > > > > > > >> > > > > nodes,
> > > >> > > > > > > > >> > > > > > >> > commit
> > > >> > > > > > > > >> > > > > > >> > > by
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> removing
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > >> /admin/delete_topics
> > > >> > > > node)
> > > >> > > > > > any
> > > >> > > > > > > > >> more.
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > You
> > mention
> > > >> that
> > > >> > > > > > > FetchRequest
> > > >> > > > > > > > >> will
> > > >> > > > > > > > >> > > now
> > > >> > > > > > > > >> > > > > > >> include
> > > >> > > > > > > > >> > > > > > >> > > UUID
> > > >> > > > > > > > >> > > > > > >> > > > > to
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> avoid
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > issues
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > where
> requests
> > > are
> > > >> > made
> > > >> > > > to
> > > >> > > > > > > stale
> > > >> > > > > > > > >> > > > partitions.
> > > >> > > > > > > > >> > > > > > >> > > However,
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> adding a
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > UUID
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > MetadataRequest
> > > is
> > > >> > > listed
> > > >> > > > > as
> > > >> > > > > > > > future
> > > >> > > > > > > > >> > > work,
> > > >> > > > > > > > >> > > > > out
> > > >> > > > > > > > >> > > > > > of
> > > >> > > > > > > > >> > > > > > >> > > scope
> > > >> > > > > > > > >> > > > > > >> > > > > for
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> this
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > KIP.
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > How
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > will the
> > client
> > > >> learn
> > > >> > > > what
> > > >> > > > > > the
> > > >> > > > > > > > >> topic
> > > >> > > > > > > > >> > > UUID
> > > >> > > > > > > > >> > > > > is,
> > > >> > > > > > > > >> > > > > > if
> > > >> > > > > > > > >> > > > > > >> > the
> > > >> > > > > > > > >> > > > > > >> > > > > > metadata
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > response
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > doesn't
> > include
> > > >> that
> > > >> > > > > > > information?
> > > >> > > > > > > > >> It
> > > >> > > > > > > > >> > > > seems
> > > >> > > > > > > > >> > > > > > like
> > > >> > > > > > > > >> > > > > > >> > > adding
> > > >> > > > > > > > >> > > > > > >> > > > > the
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> UUID
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > MetadataResponse
> > > >> > would
> > > >> > > be
> > > >> > > > > an
> > > >> > > > > > > > >> > improvement
> > > >> > > > > > > > >> > > > > here
> > > >> > > > > > > > >> > > > > > >> that
> > > >> > > > > > > > >> > > > > > >> > > > might
> > > >> > > > > > > > >> > > > > > >> > > > > > not
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> be
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > too
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > hard to
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > make.
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > best,
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Colin
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > On Mon,
> > Sep
> > > 9,
> > > >> > > 2019,
> > > >> > > > at
> > > >> > > > > > > > 17:48,
> > > >> > > > > > > > >> > > Ryanne
> > > >> > > > > > > > >> > > > > > Dolan
> > > >> > > > > > > > >> > > > > > >> > > wrote:
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > Lucas,
> > > this
> > > >> > would
> > > >> > > > be
> > > >> > > > > > > great.
> > > >> > > > > > > > >> I've
> > > >> > > > > > > > >> > > run
> > > >> > > > > > > > >> > > > > > into
> > > >> > > > > > > > >> > > > > > >> > > issues
> > > >> > > > > > > > >> > > > > > >> > > > > with
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > topics
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > being
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> > > resurrected
> > > >> > > > > > accidentally,
> > > >> > > > > > > > >> since
> > > >> > > > > > > > >> > a
> > > >> > > > > > > > >> > > > > client
> > > >> > > > > > > > >> > > > > > >> > cannot
> > > >> > > > > > > > >> > > > > > >> > > > > > easily
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > distinguish
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > between
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > a
> > deleted
> > > >> topic
> > > >> > > > and a
> > > >> > > > > > new
> > > >> > > > > > > > >> topic
> > > >> > > > > > > > >> > > with
> > > >> > > > > > > > >> > > > > the
> > > >> > > > > > > > >> > > > > > >> same
> > > >> > > > > > > > >> > > > > > >> > > > name.
> > > >> > > > > > > > >> > > > > > >> > > > > > I'd
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > need
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > the
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > ID
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> > accessible
> > > >> from
> > > >> > > the
> > > >> > > > > > > client
> > > >> > > > > > > > to
> > > >> > > > > > > > >> > > solve
> > > >> > > > > > > > >> > > > > that
> > > >> > > > > > > > >> > > > > > >> > issue,
> > > >> > > > > > > > >> > > > > > >> > > > but
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> this
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > is a
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > good
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > first
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > step.
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > Ryanne
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > On
> Wed,
> > > Sep
> > > >> 4,
> > > >> > > 2019
> > > >> > > > > at
> > > >> > > > > > > 1:41
> > > >> > > > > > > > >> PM
> > > >> > > > > > > > >> > > Lucas
> > > >> > > > > > > > >> > > > > > >> > > Bradstreet <
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > lucas@confluent.io>
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > wrote:
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Hi
> > all,
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > I
> > would
> > > >> like
> > > >> > to
> > > >> > > > > kick
> > > >> > > > > > > off
> > > >> > > > > > > > >> > > > discussion
> > > >> > > > > > > > >> > > > > of
> > > >> > > > > > > > >> > > > > > >> > > KIP-516,
> > > >> > > > > > > > >> > > > > > >> > > > > an
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > implementation
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > of topic
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > IDs
> > for
> > > >> > Kafka.
> > > >> > > > > Topic
> > > >> > > > > > > IDs
> > > >> > > > > > > > >> aim
> > > >> > > > > > > > >> > to
> > > >> > > > > > > > >> > > > > solve
> > > >> > > > > > > > >> > > > > > >> topic
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> uniqueness
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > problems in
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Kafka,
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> where
> > > >> > referring
> > > >> > > > to
> > > >> > > > > a
> > > >> > > > > > > > topic
> > > >> > > > > > > > >> by
> > > >> > > > > > > > >> > > name
> > > >> > > > > > > > >> > > > > > >> alone is
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> insufficient.
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Such
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > cases
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > include
> > > >> when
> > > >> > a
> > > >> > > > > topic
> > > >> > > > > > > has
> > > >> > > > > > > > >> been
> > > >> > > > > > > > >> > > > > deleted
> > > >> > > > > > > > >> > > > > > >> and
> > > >> > > > > > > > >> > > > > > >> > > > > recreated
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> with
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > the
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > same
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > name.
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> Unique
> > > >> > > > identifiers
> > > >> > > > > > will
> > > >> > > > > > > > >> help
> > > >> > > > > > > > >> > > > > simplify
> > > >> > > > > > > > >> > > > > > >> and
> > > >> > > > > > > > >> > > > > > >> > > > improve
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> Kafka's
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > topic
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > deletion
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > process,
> > > >> as
> > > >> > > well
> > > >> > > > as
> > > >> > > > > > > > prevent
> > > >> > > > > > > > >> > > cases
> > > >> > > > > > > > >> > > > > > where
> > > >> > > > > > > > >> > > > > > >> > > brokers
> > > >> > > > > > > > >> > > > > > >> > > > > may
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > incorrectly
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > interact
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > with
> > > stale
> > > >> > > > versions
> > > >> > > > > > of
> > > >> > > > > > > > >> topics.
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > >> > > > > > > > >> > > > > > >> > > > > >
> > > >> > > > > > > > >> > > > > > >> > > > >
> > > >> > > > > > > > >> > > > > > >> > > >
> > > >> > > > > > > > >> > > > > > >> > >
> > > >> > > > > > > > >> > > > > > >> >
> > > >> > > > > > > > >> > > > > > >>
> > > >> > > > > > > > >> > > > > >
> > > >> > > > > > > > >> > > > >
> > > >> > > > > > > > >> > > >
> > > >> > > > > > > > >> > >
> > > >> > > > > > > > >> >
> > > >> > > > > > > > >>
> > > >> > > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > Looking
> > > >> > forward
> > > >> > > > to
> > > >> > > > > > your
> > > >> > > > > > > > >> > > thoughts.
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> Lucas
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > > >> > > > > > > > >> > > > > > >> > > > > > >>
> > > >> > > > > > > > >> > > > > > >> > > > > >
> > > >> > > > > > > > >> > > > > > >> > > > >
> > > >> > > > > > > > >> > > > > > >> > > >
> > > >> > > > > > > > >> > > > > > >> > >
> > > >> > > > > > > > >> > > > > > >> >
> > > >> > > > > > > > >> > > > > > >>
> > > >> > > > > > > > >> > > > > > >
> > > >> > > > > > > > >> > > > > >
> > > >> > > > > > > > >> > > > >
> > > >> > > > > > > > >> > > >
> > > >> > > > > > > > >> > >
> > > >> > > > > > > > >> >
> > > >> > > > > > > > >>
> > > >> > > > > > > > >
> > > >> > > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > > >
> > >
> >
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Justine Olshan <jo...@confluent.io>]

Hi Jun,
Thanks for the response!

30. I think I might have changed this in between. The current version
says:  {"name":
"id", "type": "option[UUID]"}, "doc": topic id}
I have switched to the option type to cover the migration case where a
TopicZNode does not yet have a topic ID.
I understand that due to json, this field is written as a string, so if I
should move the "option[uuid]" to the doc field and the type should be
"string" please let me know.

40. I've added a definition for UUID.
41,42. Fixed

Thank you,
Justine

On Wed, Sep 30, 2020 at 1:15 PM Jun Rao <ju...@confluent.io> wrote:

> Hi, Justine,
>
> Thanks for the summary. Just a few minor comments blow.
>
> 30.  {"name": "id", "type": "string", "doc": "version id"}}: The doc should
> say UUID. The issue seems unfixed.
>
> 40. Since UUID is public facing, could you include its definition?
>
> 41. StopReplicaResponse still includes the topic field.
>
> 42. "It is unnecessary to include the name of the topic in the following
> Request/Response calls" It would be useful to include all modified requests
> (e.g. produce) in the list below.
>
> Thanks,
>
> Jun
>
>
> On Wed, Sep 30, 2020 at 10:17 AM Justine Olshan <jo...@confluent.io>
> wrote:
>
> > Hello again,
> >
> > I've taken some time to discuss some of the remaining points brought up
> by
> > the previous emails offline. Here are some of the conclusions.
> >
> > 1. Directory Structure:
> > There was some talk about whether the directory structure should be
> changed
> > to replace all topic names with topic IDs.
> > This will be a useful change, but will prevent downgrades. It will be
> best
> > to wait until a major release, likely alongside KIP-500 changes that will
> > prevent downgrades. I've updated the KIP to include this change with the
> > note about migration and deprecation.
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-log.dirlayout
> >
> > 2. Partition Metadata file
> > There was some disagreement about the inclusion of the partition metadata
> > file.
> > This file will be needed to persist the topic ID, especially while we
> still
> > have the old directory structure. Even after the changes, this file can
> be
> > useful for debugging and recovery.
> > Creating a single mapping file from topic names to topic IDs was
> > considered, but ultimately scrapped as it would not be as easy to
> maintain.
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PartitionMetadatafile
> >
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PersistingTopicIDs
> >
> > 3. Produce Protocols
> > After some further discussion, this replacing the topic name with topic
> ID
> > in these protocols has been added to the KIP.
> > This will cut down on the size of the protocol. Since changes to fetch
> are
> > included, it does make sense to update these protocols.
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-Produce
> >
> > 4. KIP-500 Compatibility
> > After some discussion with those involved with KIP-500, it seems best to
> > use a sentinel topic ID for the metadata topic that is used before the
> > first controller election.
> > We had an issue where this metadata topic may not be assigned an ID
> before
> > utilizing the Vote and Fetch protocols. It was decided to reserve a
> unique
> > ID for this topic to be used until the controller could give the topic a
> > unique ID.
> > Having the topic keep the sentinel ID (not be reassigned to a unique ID)
> > was considered, but it seemed like a good idea to leave the option open
> for
> > the metadata topic to have a unique ID in cases where it would need to be
> > differentiated from other clusters' metadata topics. (ex. tiered storage)
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-CompatibilitywithKIP-500
> >
> > I've also split up the KIP into sub-tasks on the JIRA. Hopefully this
> will
> > provide a better idea about what tasks we have, and eventually provide a
> > place to see what's done and what is left.
> > If there is a task I am missing, please let me know!
> > https://issues.apache.org/jira/browse/KAFKA-8872
> >
> > Of course, these decisions are not set in stone, and I would love to hear
> > any feedback.
> >
> > Thanks,
> > Justine
> >
> > On Mon, Sep 28, 2020 at 11:38 AM Justine Olshan <jo...@confluent.io>
> > wrote:
> >
> > > Hello all,
> > >
> > > I just wanted to follow up on this discussion. Did we come to an
> > > understanding about the directory structure?
> > >
> > > I think the biggest question here is what is acceptable to leave out
> due
> > > to scope vs. what is considered to be too much tech debt.
> > > This KIP is already pretty large with the number of changes, but it
> also
> > > makes sense to do things correctly, so I'd love to hear everyone's
> > thoughts.
> > >
> > > Thanks,
> > > Justine
> > >
> > > On Fri, Sep 25, 2020 at 8:19 AM Lucas Bradstreet <lu...@confluent.io>
> > > wrote:
> > >
> > >> Hi Ismael,
> > >>
> > >> If you do not store it in a metadata file or in the directory
> structure
> > >> would you then
> > >> require the LeaderAndIsrRequest following startup to give you some
> > notion
> > >> of
> > >> topic name in memory? We would still need this information for the
> older
> > >> protocols, but
> > >> perhaps this is what's meant by tech debt.
> > >>
> > >> Once we're free of the old non-topicID requests then I think you
> > wouldn't
> > >> need to retain the topic name.
> > >> I think the ability to easily look up topic names associated with
> > >> partition
> > >> directories would still be missed
> > >> when diagnosing issues, though maybe it wouldn't be a deal breaker
> with
> > >> the
> > >> right tooling.
> > >>
> > >> Thanks,
> > >>
> > >> Lucas
> > >>
> > >> On Fri, Sep 25, 2020 at 7:55 AM Ismael Juma <is...@juma.me.uk>
> wrote:
> > >>
> > >> > Hi Lucas,
> > >> >
> > >> > Why would you include the name and id? I think you'd want to remove
> > the
> > >> > name from the directory name right? Jason's suggestion was that if
> you
> > >> > remove the name from the directory, then why would you need the id
> > name
> > >> > mapping file?
> > >> >
> > >> > Ismael
> > >> >
> > >> > On Thu, Sep 24, 2020 at 4:24 PM Lucas Bradstreet <
> lucas@confluent.io>
> > >> > wrote:
> > >> >
> > >> > > > 2. Part of the usage of the file is to have persistent storage
> of
> > >> the
> > >> > > topic
> > >> > > ID and use it to compare with the ID supplied in the LeaderAndIsr
> > >> > Request.
> > >> > > There is some discussion in the KIP about changes to the directory
> > >> > > structure, but I believe directory changes were considered to be
> out
> > >> of
> > >> > > scope when the KIP was written.
> > >> > >
> > >> > >
> > >> > > Yeah, I was hoping to get a better understanding of why it was
> taken
> > >> out
> > >> > of
> > >> > > scope. Perhaps Lucas Bradstreet might have more insight about the
> > >> > decision.
> > >> > > Basically my point is that we have to create additional
> > infrastructure
> > >> > here
> > >> > > to support the name/id mapping, so I wanted to understand if we
> just
> > >> > > consider this a sort of tech debt. It would be useful to cover how
> > we
> > >> > > handle the case when this file gets corrupted. Seems like we just
> > >> have to
> > >> > > trust that it matches whatever the controller tells us and rewrite
> > it?
> > >> > >
> > >> > >
> > >> > > Hi Jason, Justine,
> > >> > >
> > >> > > My thought process is that we will likely need the metadata file
> > >> whether
> > >> > we
> > >> > > rename the directories or not.
> > >> > > Linux supports filenames of up to 255 bytes and that would not be
> > >> enough
> > >> > to
> > >> > > support a directory name
> > >> > >  including both the name and topic ID. Given that fact, it seemed
> > >> > > reasonable to add the metadata file
> > >> > > and leave the directory rename until some time in the future
> > (possibly
> > >> > > never).
> > >> > >
> > >> > > If the file does get corrupted I think you're right. We would
> either
> > >> have
> > >> > > to trust it matches what the controller tells us
> > >> > >  or error out and let an administrator resolve it by checking
> across
> > >> > > replicas for consistency.
> > >> > >
> > >> > > Lucas
> > >> > >
> > >> > >
> > >> > > On Thu, Sep 24, 2020 at 3:41 PM Jason Gustafson <
> jason@confluent.io
> > >
> > >> > > wrote:
> > >> > >
> > >> > > > Thanks Justine. Responses below:
> > >> > > >
> > >> > > > > 1. Yes, the directory will still be based on the topic names.
> > >> > > > LeaderAndIsrRequest is one of the few requests that will still
> > >> contain
> > >> > > the
> > >> > > > topic name. So I think we have this covered. Sorry for
> confusion.
> > >> > > >
> > >> > > > Ah, you're right. My eyes passed right over the field.
> > >> > > >
> > >> > > > > 2. Part of the usage of the file is to have persistent storage
> > of
> > >> the
> > >> > > > topic
> > >> > > > ID and use it to compare with the ID supplied in the
> LeaderAndIsr
> > >> > > Request.
> > >> > > > There is some discussion in the KIP about changes to the
> directory
> > >> > > > structure, but I believe directory changes were considered to be
> > >> out of
> > >> > > > scope when the KIP was written.
> > >> > > >
> > >> > > > Yeah, I was hoping to get a better understanding of why it was
> > taken
> > >> > out
> > >> > > of
> > >> > > > scope. Perhaps Lucas Bradstreet might have more insight about
> the
> > >> > > decision.
> > >> > > > Basically my point is that we have to create additional
> > >> infrastructure
> > >> > > here
> > >> > > > to support the name/id mapping, so I wanted to understand if we
> > just
> > >> > > > consider this a sort of tech debt. It would be useful to cover
> how
> > >> we
> > >> > > > handle the case when this file gets corrupted. Seems like we
> just
> > >> have
> > >> > to
> > >> > > > trust that it matches whatever the controller tells us and
> rewrite
> > >> it?
> > >> > > >
> > >> > > > > 3. I think this is a good point, but I again I wonder about
> the
> > >> scope
> > >> > > of
> > >> > > > the KIP. Most of the changes mentioned in the KIP are for
> > supporting
> > >> > > topic
> > >> > > > deletion and I believe that is why the produce request was
> listed
> > >> under
> > >> > > > future work.
> > >> > > >
> > >> > > > That's fair. I brought it up since `Fetch` is already included.
> If
> > >> > we've
> > >> > > > got `Metadata` and `Fetch`, seems we may as well do `Produce`
> and
> > >> save
> > >> > an
> > >> > > > extra kip. No strong objection though if you want to leave it
> out.
> > >> > > >
> > >> > > >
> > >> > > > -Jason
> > >> > > >
> > >> > > >
> > >> > > > On Thu, Sep 24, 2020 at 3:26 PM Justine Olshan <
> > >> jolshan@confluent.io>
> > >> > > > wrote:
> > >> > > >
> > >> > > > > Hi Jason,
> > >> > > > >
> > >> > > > > Thanks for your comments.
> > >> > > > >
> > >> > > > > 1. Yes, the directory will still be based on the topic names.
> > >> > > > > LeaderAndIsrRequest is one of the few requests that will still
> > >> > contain
> > >> > > > the
> > >> > > > > topic name. So I think we have this covered. Sorry for
> > confusion.
> > >> > > > >
> > >> > > > > 2. Part of the usage of the file is to have persistent storage
> > of
> > >> the
> > >> > > > topic
> > >> > > > > ID and use it to compare with the ID supplied in the
> > LeaderAndIsr
> > >> > > > Request.
> > >> > > > > There is some discussion in the KIP about changes to the
> > directory
> > >> > > > > structure, but I believe directory changes were considered to
> be
> > >> out
> > >> > of
> > >> > > > > scope when the KIP was written.
> > >> > > > >
> > >> > > > > 3. I think this is a good point, but I again I wonder about
> the
> > >> scope
> > >> > > of
> > >> > > > > the KIP. Most of the changes mentioned in the KIP are for
> > >> supporting
> > >> > > > topic
> > >> > > > > deletion and I believe that is why the produce request was
> > listed
> > >> > under
> > >> > > > > future work.
> > >> > > > >
> > >> > > > > 4. This sounds like it might be a good solution, but I will
> need
> > >> to
> > >> > > > discuss
> > >> > > > > more with KIP-500 folks to get the details right.
> > >> > > > >
> > >> > > > > Thanks,
> > >> > > > > Justine
> > >> > > > >
> > >> > > > > On Thu, Sep 24, 2020 at 12:30 PM Jason Gustafson <
> > >> jason@confluent.io
> > >> > >
> > >> > > > > wrote:
> > >> > > > >
> > >> > > > > > Hi Justine,
> > >> > > > > >
> > >> > > > > > Thanks for picking up this work. I have a few
> > >> questions/comments:
> > >> > > > > >
> > >> > > > > > 1. It sounds like the directory structure is still going to
> be
> > >> > based
> > >> > > on
> > >> > > > > > topic names. Do I have that right? One complication is that
> > the
> > >> > > > > > LeaderAndIsr request does not include the topic name any
> > longer.
> > >> > This
> > >> > > > > means
> > >> > > > > > that a replica must wait for the UpdateMetadata request to
> > >> arrive
> > >> > > with
> > >> > > > > the
> > >> > > > > > topic name mapping before it can create the directory. I
> > wonder
> > >> if
> > >> > it
> > >> > > > > would
> > >> > > > > > be simpler to avoid assumptions on the ordering of
> > >> UpdateMetadata
> > >> > and
> > >> > > > let
> > >> > > > > > LeaderAndIsr include the topic name as well. Feels like we
> are
> > >> not
> > >> > > > saving
> > >> > > > > > that much by excluding it.
> > >> > > > > >
> > >> > > > > > 2. On a related note, it seems that the reason we have the
> > >> > partition
> > >> > > > > > metadata file is because we are not changing the directory
> > >> > structure.
> > >> > > > We
> > >> > > > > > need it so that we remember which directories map to which
> > topic
> > >> > id.
> > >> > > I
> > >> > > > > > think it would be useful to add some detail about why
> changing
> > >> the
> > >> > > > > > directory layout was rejected. Basically trying to
> understand
> > >> how
> > >> > > much
> > >> > > > > > effort we are saving by skipping this step if we have to
> > >> implement
> > >> > > this
> > >> > > > > > additional bookkeeping. One concern I have for example is
> that
> > >> the
> > >> > > > > > partition metadata file gets corrupt and then all bets are
> off
> > >> as
> > >> > far
> > >> > > > as
> > >> > > > > > topic consistency.
> > >> > > > > >
> > >> > > > > > 3. Is it worth adding support now for topic ids in the
> > `Produce`
> > >> > > > request
> > >> > > > > > now? Overhead is mentioned as one of the motivations and the
> > >> best
> > >> > > APIs
> > >> > > > to
> > >> > > > > > get savings from are `Produce` and `Fetch`.
> > >> > > > > >
> > >> > > > > > 4. When it comes to bootstrapping the metadata topic with
> > >> respect
> > >> > to
> > >> > > > > > KIP-500, one suggestion would be to reserve a sentinel topic
> > ID
> > >> > which
> > >> > > > can
> > >> > > > > > be used initially by new brokers. When the first controller
> is
> > >> > > elected,
> > >> > > > > it
> > >> > > > > > can assign a topicId to the metadata topic. This is a bit
> > >> similar
> > >> > to
> > >> > > > how
> > >> > > > > we
> > >> > > > > > were planning to generate the clusterId.
> > >> > > > > >
> > >> > > > > > Thanks,
> > >> > > > > > Jason
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > On Thu, Sep 24, 2020 at 11:10 AM Jun Rao <ju...@confluent.io>
> > >> wrote:
> > >> > > > > >
> > >> > > > > > > Hi, Justine,
> > >> > > > > > >
> > >> > > > > > > Thanks for the updated KIP. A few more comments below.
> > >> > > > > > >
> > >> > > > > > > 30.  {"name": "id", "type": "string", "doc": "version
> id"}}:
> > >> The
> > >> > > doc
> > >> > > > > > should
> > >> > > > > > > say UUID.
> > >> > > > > > >
> > >> > > > > > > 31. LeaderAndIsrResponse v5 and StopReplicaResponse v4 :
> It
> > >> seems
> > >> > > > there
> > >> > > > > > is
> > >> > > > > > > no need to add topic_id at partitions level.
> > >> > > > > > >
> > >> > > > > > > 32. Regarding partition metadata file. Perhaps the key can
> > be
> > >> a
> > >> > > > single
> > >> > > > > > > word, sth like the following.
> > >> > > > > > > version: 0
> > >> > > > > > > topic_id: 46bdb63f-9e8d-4a38-bf7b-ee4eb2a794e4
> > >> > > > > > >
> > >> > > > > > > 33. Another tricky thing that I realized is how to support
> > the
> > >> > > > metadata
> > >> > > > > > > topic introduced in KIP-595. It's a bit tricky to assign a
> > >> UUID
> > >> > to
> > >> > > > the
> > >> > > > > > > metadata topic since we have a chicken-and-egg problem.
> The
> > >> > > > controller
> > >> > > > > > > needs to persist the UUID in the metadata topic in order
> to
> > >> > assign
> > >> > > > one
> > >> > > > > > > successfully, but the metadata topic is needed to elect a
> > >> > > controller
> > >> > > > > > > (KIP-631). So, this probably needs a bit more thought.
> > >> > > > > > >
> > >> > > > > > > Jun
> > >> > > > > > >
> > >> > > > > > > On Thu, Sep 24, 2020 at 3:04 AM Ismael Juma <
> > >> ismael@juma.me.uk>
> > >> > > > wrote:
> > >> > > > > > >
> > >> > > > > > > > Also, can we provide more details on how the Partition
> > >> Metadata
> > >> > > > file
> > >> > > > > > will
> > >> > > > > > > > be used?
> > >> > > > > > > >
> > >> > > > > > > > Ismael
> > >> > > > > > > >
> > >> > > > > > > > On Thu, Sep 24, 2020 at 3:01 AM Ismael Juma <
> > >> ismael@juma.me.uk
> > >> > >
> > >> > > > > wrote:
> > >> > > > > > > >
> > >> > > > > > > > > Hi Justine,
> > >> > > > > > > > >
> > >> > > > > > > > > I think we need to update the "Rejected Alternatives"
> > >> section
> > >> > > to
> > >> > > > > take
> > >> > > > > > > > into
> > >> > > > > > > > > account that the proposal now removes the topic name
> > from
> > >> the
> > >> > > > fetch
> > >> > > > > > > > > request. Also, if we are removing it from the Fetch
> > >> request,
> > >> > > does
> > >> > > > > it
> > >> > > > > > > make
> > >> > > > > > > > > sense not to remove it from similar requests like
> > >> > > > > ListOffsetRequest?
> > >> > > > > > > > >
> > >> > > > > > > > > Ismael
> > >> > > > > > > > >
> > >> > > > > > > > > On Thu, Sep 24, 2020 at 2:46 AM David Jacot <
> > >> > > djacot@confluent.io
> > >> > > > >
> > >> > > > > > > wrote:
> > >> > > > > > > > >
> > >> > > > > > > > >> Hi Justine,
> > >> > > > > > > > >>
> > >> > > > > > > > >> Thanks for the KIP. I finally had time to read it :).
> > It
> > >> is
> > >> > a
> > >> > > > > great
> > >> > > > > > > > >> improvement.
> > >> > > > > > > > >>
> > >> > > > > > > > >> I have a few comments/questions:
> > >> > > > > > > > >>
> > >> > > > > > > > >> 1. It seems that the schema of the StopReplicaRequest
> > is
> > >> > > > slightly
> > >> > > > > > > > >> outdated.
> > >> > > > > > > > >> We
> > >> > > > > > > > >> did some changes as part of KIP-570. V3 is already
> > >> organized
> > >> > > by
> > >> > > > > > > topics.
> > >> > > > > > > > >>
> > >> > > > > > > > >> 2. I just want to make sure that I understand the
> > >> > > reconciliation
> > >> > > > > > > > >> logic correctly. When
> > >> > > > > > > > >> an "INCREMENTAL" LeaderAndIsr Request is received,
> the
> > >> > broker
> > >> > > > will
> > >> > > > > > > also
> > >> > > > > > > > >> reconcile
> > >> > > > > > > > >> when the local uuid does not match the uuid in the
> > >> request,
> > >> > > > right?
> > >> > > > > > In
> > >> > > > > > > > this
> > >> > > > > > > > >> case, the
> > >> > > > > > > > >> local log is staged for deletion.
> > >> > > > > > > > >>
> > >> > > > > > > > >> 3. In the documentation of the `
> > >> delete.stale.topic.delay.ms
> > >> > `
> > >> > > > > > config,
> > >> > > > > > > it
> > >> > > > > > > > >> says "When a
> > >> > > > > > > > >> FULL LeaderAndIsrRequest is received..." but I
> suppose
> > it
> > >> > > > applies
> > >> > > > > to
> > >> > > > > > > > both
> > >> > > > > > > > >> types.
> > >> > > > > > > > >>
> > >> > > > > > > > >> Best,
> > >> > > > > > > > >> David
> > >> > > > > > > > >>
> > >> > > > > > > > >> On Thu, Sep 24, 2020 at 1:40 AM Justine Olshan <
> > >> > > > > > jolshan@confluent.io>
> > >> > > > > > > > >> wrote:
> > >> > > > > > > > >>
> > >> > > > > > > > >> > Hi Jun,
> > >> > > > > > > > >> >
> > >> > > > > > > > >> > Thanks for the comments. I apologize for some of
> the
> > >> typos
> > >> > > and
> > >> > > > > > > > >> confusion.
> > >> > > > > > > > >> > I’ve updated the KIP to fix some of the issues you
> > >> > > mentioned.
> > >> > > > > > > > >> >
> > >> > > > > > > > >> > 20.2 I’ve changed the type to String.
> > >> > > > > > > > >> > 20.1/3 I’ve updated the TopicZNode to fix
> formatting
> > >> and
> > >> > > > reflect
> > >> > > > > > the
> > >> > > > > > > > >> latest
> > >> > > > > > > > >> > version before this change.
> > >> > > > > > > > >> >
> > >> > > > > > > > >> > 21. You are correct and I’ve removed this line.
> I’ve
> > >> also
> > >> > > > added
> > >> > > > > a
> > >> > > > > > > line
> > >> > > > > > > > >> > mentioning an IBP bump is necessary for migration
> > >> > > > > > > > >> >
> > >> > > > > > > > >> > 22. I think the wording was unclear but your
> summary
> > is
> > >> > what
> > >> > > > was
> > >> > > > > > > > >> intended
> > >> > > > > > > > >> > by this line. I’ve updated to make this point more
> > >> clear.
> > >> > > > > “Remove
> > >> > > > > > > > >> deleted
> > >> > > > > > > > >> > topics from replicas by sending StopReplicaRequest
> V3
> > >> > before
> > >> > > > the
> > >> > > > > > IBP
> > >> > > > > > > > >> bump
> > >> > > > > > > > >> > using the old logic, and using V4 and the new logic
> > >> with
> > >> > > topic
> > >> > > > > IDs
> > >> > > > > > > > after
> > >> > > > > > > > >> > the IBP bump.”
> > >> > > > > > > > >> >
> > >> > > > > > > > >> > 23. I’ve removed the unspecified type from the KIP
> > and
> > >> > > mention
> > >> > > > > > that
> > >> > > > > > > > IBP
> > >> > > > > > > > >> > will be used to handle this request. “IBP will be
> > used
> > >> to
> > >> > > > > > determine
> > >> > > > > > > > >> whether
> > >> > > > > > > > >> > this new form of the request will be used. For
> older
> > >> > > requests,
> > >> > > > > we
> > >> > > > > > > will
> > >> > > > > > > > >> > ignore this field and default to previous
> behavior.”
> > >> > > > > > > > >> >
> > >> > > > > > > > >> > 24. I’ve fixed this typo.
> > >> > > > > > > > >> >
> > >> > > > > > > > >> > 25. I've added a topics at a higher level for
> > >> > > > > LeaderAndIsrResponse
> > >> > > > > > > v5,
> > >> > > > > > > > >> > StopReplicaResponse v4. I've also changed
> > >> > StopReplicaRequest
> > >> > > > v4
> > >> > > > > to
> > >> > > > > > > > have
> > >> > > > > > > > >> > topics at a higher level.
> > >> > > > > > > > >> >
> > >> > > > > > > > >> > 26. I’ve updated forgotten_topics_data--added the
> > >> topic ID
> > >> > > and
> > >> > > > > > > removed
> > >> > > > > > > > >> the
> > >> > > > > > > > >> > topic name
> > >> > > > > > > > >> >
> > >> > > > > > > > >> > 27. I’ve decided on plain text, and I’ve added an
> > >> example.
> > >> > > > > > > > >> >
> > >> > > > > > > > >> > 28. I’ve added this idea to future work.
> > >> > > > > > > > >> >
> > >> > > > > > > > >> > Thanks again for taking a look,
> > >> > > > > > > > >> >
> > >> > > > > > > > >> > Justine
> > >> > > > > > > > >> >
> > >> > > > > > > > >> > On Wed, Sep 23, 2020 at 10:28 AM Jun Rao <
> > >> > jun@confluent.io>
> > >> > > > > > wrote:
> > >> > > > > > > > >> >
> > >> > > > > > > > >> > > Hi, Justine,
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> > > Thanks for the response. Made another pass. A few
> > >> more
> > >> > > > > comments
> > >> > > > > > > > below.
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> > > 20. znode schema:
> > >> > > > > > > > >> > > 20.1 It seems that {"name": "version", "type":
> > "int",
> > >> > > "id":
> > >> > > > > > > "UUID",
> > >> > > > > > > > >> > "doc":
> > >> > > > > > > > >> > > "version id"} should be {"name": "version",
> "type":
> > >> > > "int"},
> > >> > > > > > > {"name":
> > >> > > > > > > > >> > "id",
> > >> > > > > > > > >> > > "type": "UUID", "doc": "version id"}.
> > >> > > > > > > > >> > > 20.2 The znode format is JSON which doesn't have
> > UUID
> > >> > > type.
> > >> > > > So
> > >> > > > > > the
> > >> > > > > > > > >> type
> > >> > > > > > > > >> > > probably should be string?
> > >> > > > > > > > >> > > 20.3 Also, the existing format used seems
> outdated.
> > >> It
> > >> > > > should
> > >> > > > > > have
> > >> > > > > > > > the
> > >> > > > > > > > >> > > following format.
> > >> > > > > > > > >> > >     Json.encodeAsBytes(Map(
> > >> > > > > > > > >> > >       "version" -> 2,
> > >> > > > > > > > >> > >       "partitions" ->
> replicaAssignmentJson.asJava,
> > >> > > > > > > > >> > >       "adding_replicas" ->
> > >> > > > > addingReplicasAssignmentJson.asJava,
> > >> > > > > > > > >> > >       "removing_replicas" ->
> > >> > > > > > removingReplicasAssignmentJson.asJava
> > >> > > > > > > > >> > >     ).asJava)
> > >> > > > > > > > >> > >   }
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> > > 21. Migration: The KIP says "The migration
> process
> > >> can
> > >> > > take
> > >> > > > > > place
> > >> > > > > > > > >> without
> > >> > > > > > > > >> > > an inter-broker protocol bump, as the format
> stored
> > >> in
> > >> > > > > > > > >> > > /brokers/topics/[topic] will be compatible with
> > older
> > >> > > broker
> > >> > > > > > > > >> versions."
> > >> > > > > > > > >> > > However, since we are bumping up the version of
> > >> > > inter-broker
> > >> > > > > > > > >> requests, it
> > >> > > > > > > > >> > > seems that we need to use IBP for migration.
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> > > 22. The KIP says "Remove deleted topics from
> > >> replicas by
> > >> > > > > sending
> > >> > > > > > > > >> > > StopReplicaRequest V3 for any topics which do not
> > >> > contain
> > >> > > a
> > >> > > > > > topic
> > >> > > > > > > > ID,
> > >> > > > > > > > >> and
> > >> > > > > > > > >> > > V4 for any topics which do contain a topic ID.".
> > >> > However,
> > >> > > if
> > >> > > > > we
> > >> > > > > > > use
> > >> > > > > > > > >> IBP,
> > >> > > > > > > > >> > it
> > >> > > > > > > > >> > > seems that the controller will either send
> > >> > > > StopReplicaRequest
> > >> > > > > V3
> > >> > > > > > > > >> > > or StopReplicaRequest V4, but never mixed V3 and
> V4
> > >> for
> > >> > > > > > different
> > >> > > > > > > > >> topics.
> > >> > > > > > > > >> > > Basically, before the IBP bump, V3 will be used.
> > >> After
> > >> > the
> > >> > > > IBP
> > >> > > > > > > bump,
> > >> > > > > > > > >> > > topicId will be created and V4 will be used.
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> > > 23. Given that we depend on IBP, do we still need
> > "0
> > >> > > > > > UNSPECIFIED"
> > >> > > > > > > > >> > > in LeaderAndIsr?
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> > > 24. LeaderAndIsrResponse v5 : It still has the
> > topic
> > >> > > field.
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> > > 25. LeaderAndIsrResponse v5, StopReplicaResponse
> > v4:
> > >> > Could
> > >> > > > we
> > >> > > > > > use
> > >> > > > > > > > this
> > >> > > > > > > > >> > > opportunity to organize the response in 2 levels,
> > >> first
> > >> > by
> > >> > > > > > topic,
> > >> > > > > > > > >> then by
> > >> > > > > > > > >> > > partition, as most other requests/responses?
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> > > 26. FetchRequest v13 : Should
> forgotten_topics_data
> > >> use
> > >> > > > > topicId
> > >> > > > > > > too?
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> > > 27. "This file can either be plain text
> (key/value
> > >> > pairs)
> > >> > > or
> > >> > > > > > > JSON."
> > >> > > > > > > > >> Have
> > >> > > > > > > > >> > we
> > >> > > > > > > > >> > > decided which one to use? Also, it would be
> helpful
> > >> to
> > >> > > > provide
> > >> > > > > > an
> > >> > > > > > > > >> > example.
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> > > 28. Future improvement: Another future
> improvement
> > >> > > > opportunity
> > >> > > > > > is
> > >> > > > > > > to
> > >> > > > > > > > >> use
> > >> > > > > > > > >> > > topicId in GroupMetadataManager.offsetCommitKey
> in
> > >> the
> > >> > > > > > > offset_commit
> > >> > > > > > > > >> > topic.
> > >> > > > > > > > >> > > This may save some space.
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> > > Thanks,
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> > > Jun
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> > > On Wed, Sep 23, 2020 at 8:50 AM Justine Olshan <
> > >> > > > > > > > jolshan@confluent.io>
> > >> > > > > > > > >> > > wrote:
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> > > > Hi Tom,
> > >> > > > > > > > >> > > >
> > >> > > > > > > > >> > > > Thanks for the comment. I think this is a
> really
> > >> good
> > >> > > idea
> > >> > > > > and
> > >> > > > > > > it
> > >> > > > > > > > >> has
> > >> > > > > > > > >> > > been
> > >> > > > > > > > >> > > > added to the KIP under the newly added tooling
> > >> > section.
> > >> > > > > > > > >> > > >
> > >> > > > > > > > >> > > > Thanks again,
> > >> > > > > > > > >> > > > Justine
> > >> > > > > > > > >> > > >
> > >> > > > > > > > >> > > > On Wed, Sep 23, 2020 at 3:17 AM Tom Bentley <
> > >> > > > > > > tbentley@redhat.com>
> > >> > > > > > > > >> > wrote:
> > >> > > > > > > > >> > > >
> > >> > > > > > > > >> > > > > Hi Justine,
> > >> > > > > > > > >> > > > >
> > >> > > > > > > > >> > > > > I know you started the vote thread, but on
> > >> > re-reading
> > >> > > > the
> > >> > > > > > KIP
> > >> > > > > > > I
> > >> > > > > > > > >> > noticed
> > >> > > > > > > > >> > > > > that although the topic id is included in the
> > >> > > > > > MetadataResponse
> > >> > > > > > > > >> it's
> > >> > > > > > > > >> > not
> > >> > > > > > > > >> > > > > surfaced in the output from `kafka-topics.sh
> > >> > > > --describe`.
> > >> > > > > > > Maybe
> > >> > > > > > > > >> that
> > >> > > > > > > > >> > > was
> > >> > > > > > > > >> > > > > intentional because ids are intentionally not
> > >> really
> > >> > > > > > something
> > >> > > > > > > > the
> > >> > > > > > > > >> > user
> > >> > > > > > > > >> > > > > should care deeply about, but it would also
> > make
> > >> > life
> > >> > > > > harder
> > >> > > > > > > for
> > >> > > > > > > > >> > anyone
> > >> > > > > > > > >> > > > > debugging Kafka and this would likely get
> worse
> > >> the
> > >> > > more
> > >> > > > > > topic
> > >> > > > > > > > ids
> > >> > > > > > > > >> > got
> > >> > > > > > > > >> > > > > rolled out across the protocols, clients etc.
> > It
> > >> > seems
> > >> > > > > > likely
> > >> > > > > > > > that
> > >> > > > > > > > >> > > > > `kafka-topics.sh` will eventually need the
> > >> ability
> > >> > to
> > >> > > > show
> > >> > > > > > the
> > >> > > > > > > > id
> > >> > > > > > > > >> of
> > >> > > > > > > > >> > a
> > >> > > > > > > > >> > > > > topic and perhaps find a topic name given an
> > id.
> > >> Is
> > >> > > > there
> > >> > > > > > any
> > >> > > > > > > > >> reason
> > >> > > > > > > > >> > > not
> > >> > > > > > > > >> > > > to
> > >> > > > > > > > >> > > > > implement that in this KIP?
> > >> > > > > > > > >> > > > >
> > >> > > > > > > > >> > > > > Many thanks,
> > >> > > > > > > > >> > > > >
> > >> > > > > > > > >> > > > > Tom
> > >> > > > > > > > >> > > > >
> > >> > > > > > > > >> > > > > On Mon, Sep 21, 2020 at 9:54 PM Justine
> Olshan
> > <
> > >> > > > > > > > >> jolshan@confluent.io
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> > > > > wrote:
> > >> > > > > > > > >> > > > >
> > >> > > > > > > > >> > > > > > Hi all,
> > >> > > > > > > > >> > > > > >
> > >> > > > > > > > >> > > > > > After thinking about it, I've decided to
> > remove
> > >> > the
> > >> > > > > topic
> > >> > > > > > > name
> > >> > > > > > > > >> from
> > >> > > > > > > > >> > > the
> > >> > > > > > > > >> > > > > > Fetch Request and Response after all. Since
> > >> there
> > >> > > are
> > >> > > > so
> > >> > > > > > > many
> > >> > > > > > > > of
> > >> > > > > > > > >> > > these
> > >> > > > > > > > >> > > > > > requests per second, it is worth removing
> the
> > >> > extra
> > >> > > > > > > > information.
> > >> > > > > > > > >> > I've
> > >> > > > > > > > >> > > > > > updated the KIP to reflect this change.
> > >> > > > > > > > >> > > > > >
> > >> > > > > > > > >> > > > > > Please let me know if there is anything
> else
> > we
> > >> > > should
> > >> > > > > > > discuss
> > >> > > > > > > > >> > before
> > >> > > > > > > > >> > > > > > voting.
> > >> > > > > > > > >> > > > > >
> > >> > > > > > > > >> > > > > > Thank you,
> > >> > > > > > > > >> > > > > > Justine
> > >> > > > > > > > >> > > > > >
> > >> > > > > > > > >> > > > > > On Fri, Sep 18, 2020 at 9:46 AM Justine
> > Olshan
> > >> <
> > >> > > > > > > > >> > jolshan@confluent.io
> > >> > > > > > > > >> > > >
> > >> > > > > > > > >> > > > > > wrote:
> > >> > > > > > > > >> > > > > >
> > >> > > > > > > > >> > > > > > > Hi Jun,
> > >> > > > > > > > >> > > > > > >
> > >> > > > > > > > >> > > > > > > I see what you are saying. For now we can
> > >> remove
> > >> > > the
> > >> > > > > > extra
> > >> > > > > > > > >> > > > information.
> > >> > > > > > > > >> > > > > > > I'll leave the option to add more fields
> to
> > >> the
> > >> > > file
> > >> > > > > in
> > >> > > > > > > the
> > >> > > > > > > > >> > future.
> > >> > > > > > > > >> > > > The
> > >> > > > > > > > >> > > > > > KIP
> > >> > > > > > > > >> > > > > > > has been updated to reflect this change.
> > >> > > > > > > > >> > > > > > >
> > >> > > > > > > > >> > > > > > > Thanks,
> > >> > > > > > > > >> > > > > > > Justine
> > >> > > > > > > > >> > > > > > >
> > >> > > > > > > > >> > > > > > > On Fri, Sep 18, 2020 at 8:46 AM Jun Rao <
> > >> > > > > > jun@confluent.io
> > >> > > > > > > >
> > >> > > > > > > > >> > wrote:
> > >> > > > > > > > >> > > > > > >
> > >> > > > > > > > >> > > > > > >> Hi, Justine,
> > >> > > > > > > > >> > > > > > >>
> > >> > > > > > > > >> > > > > > >> Thanks for the reply.
> > >> > > > > > > > >> > > > > > >>
> > >> > > > > > > > >> > > > > > >> 13. If the log directory is the source
> of
> > >> > truth,
> > >> > > it
> > >> > > > > > means
> > >> > > > > > > > >> that
> > >> > > > > > > > >> > the
> > >> > > > > > > > >> > > > > > >> redundant info in the metadata file will
> > be
> > >> > > > ignored.
> > >> > > > > > Then
> > >> > > > > > > > the
> > >> > > > > > > > >> > > > question
> > >> > > > > > > > >> > > > > > is
> > >> > > > > > > > >> > > > > > >> why do we need to put the redundant info
> > in
> > >> the
> > >> > > > > > metadata
> > >> > > > > > > > file
> > >> > > > > > > > >> > now?
> > >> > > > > > > > >> > > > > > >>
> > >> > > > > > > > >> > > > > > >> Thanks,
> > >> > > > > > > > >> > > > > > >>
> > >> > > > > > > > >> > > > > > >> Jun
> > >> > > > > > > > >> > > > > > >>
> > >> > > > > > > > >> > > > > > >> On Thu, Sep 17, 2020 at 5:07 PM Justine
> > >> Olshan
> > >> > <
> > >> > > > > > > > >> > > > jolshan@confluent.io>
> > >> > > > > > > > >> > > > > > >> wrote:
> > >> > > > > > > > >> > > > > > >>
> > >> > > > > > > > >> > > > > > >> > Hi Jun,
> > >> > > > > > > > >> > > > > > >> > Thanks for the quick response!
> > >> > > > > > > > >> > > > > > >> >
> > >> > > > > > > > >> > > > > > >> > 12. I've decided to bump up the
> versions
> > >> on
> > >> > the
> > >> > > > > > > requests
> > >> > > > > > > > >> and
> > >> > > > > > > > >> > > > updated
> > >> > > > > > > > >> > > > > > the
> > >> > > > > > > > >> > > > > > >> > KIP. I think it's good we thoroughly
> > >> > discussed
> > >> > > > the
> > >> > > > > > > > options
> > >> > > > > > > > >> > here,
> > >> > > > > > > > >> > > > so
> > >> > > > > > > > >> > > > > we
> > >> > > > > > > > >> > > > > > >> know
> > >> > > > > > > > >> > > > > > >> > we made a good choice. :)
> > >> > > > > > > > >> > > > > > >> >
> > >> > > > > > > > >> > > > > > >> > 13. This is an interesting situation.
> I
> > >> think
> > >> > > if
> > >> > > > > this
> > >> > > > > > > > does
> > >> > > > > > > > >> > occur
> > >> > > > > > > > >> > > > we
> > >> > > > > > > > >> > > > > > >> should
> > >> > > > > > > > >> > > > > > >> > give a warning. I agree that it's hard
> > to
> > >> > know
> > >> > > > the
> > >> > > > > > > source
> > >> > > > > > > > >> of
> > >> > > > > > > > >> > > truth
> > >> > > > > > > > >> > > > > for
> > >> > > > > > > > >> > > > > > >> sure
> > >> > > > > > > > >> > > > > > >> > since the directory or the file could
> be
> > >> > > manually
> > >> > > > > > > > >> modified. I
> > >> > > > > > > > >> > > > guess
> > >> > > > > > > > >> > > > > > the
> > >> > > > > > > > >> > > > > > >> > directory could be used as the source
> of
> > >> > truth.
> > >> > > > To
> > >> > > > > be
> > >> > > > > > > > >> honest,
> > >> > > > > > > > >> > > I'm
> > >> > > > > > > > >> > > > > not
> > >> > > > > > > > >> > > > > > >> > really sure what happens in kafka when
> > the
> > >> > log
> > >> > > > > > > directory
> > >> > > > > > > > is
> > >> > > > > > > > >> > > > renamed
> > >> > > > > > > > >> > > > > > >> > manually in such a way. I'm also
> > >> wondering if
> > >> > > the
> > >> > > > > > > > >> situation is
> > >> > > > > > > > >> > > > > > >> recoverable
> > >> > > > > > > > >> > > > > > >> > in this scenario.
> > >> > > > > > > > >> > > > > > >> >
> > >> > > > > > > > >> > > > > > >> > Thanks,
> > >> > > > > > > > >> > > > > > >> > Justine
> > >> > > > > > > > >> > > > > > >> >
> > >> > > > > > > > >> > > > > > >> > On Thu, Sep 17, 2020 at 4:28 PM Jun
> Rao
> > <
> > >> > > > > > > > jun@confluent.io>
> > >> > > > > > > > >> > > wrote:
> > >> > > > > > > > >> > > > > > >> >
> > >> > > > > > > > >> > > > > > >> > > Hi, Justine,
> > >> > > > > > > > >> > > > > > >> > >
> > >> > > > > > > > >> > > > > > >> > > Thanks for the reply.
> > >> > > > > > > > >> > > > > > >> > >
> > >> > > > > > > > >> > > > > > >> > > 12. I don't have a strong preference
> > >> > either.
> > >> > > > > > However,
> > >> > > > > > > > if
> > >> > > > > > > > >> we
> > >> > > > > > > > >> > > need
> > >> > > > > > > > >> > > > > IBP
> > >> > > > > > > > >> > > > > > >> > > anyway, maybe it's easier to just
> bump
> > >> up
> > >> > the
> > >> > > > > > version
> > >> > > > > > > > for
> > >> > > > > > > > >> > all
> > >> > > > > > > > >> > > > > inter
> > >> > > > > > > > >> > > > > > >> > broker
> > >> > > > > > > > >> > > > > > >> > > requests and add the topic id field
> > as a
> > >> > > > regular
> > >> > > > > > > > field. A
> > >> > > > > > > > >> > > > regular
> > >> > > > > > > > >> > > > > > >> field
> > >> > > > > > > > >> > > > > > >> > is
> > >> > > > > > > > >> > > > > > >> > > a bit more concise in wire transfer
> > >> than a
> > >> > > > > flexible
> > >> > > > > > > > >> field.
> > >> > > > > > > > >> > > > > > >> > >
> > >> > > > > > > > >> > > > > > >> > > 13. The confusion that I was
> referring
> > >> to
> > >> > is
> > >> > > > > > between
> > >> > > > > > > > the
> > >> > > > > > > > >> > topic
> > >> > > > > > > > >> > > > > name
> > >> > > > > > > > >> > > > > > >> and
> > >> > > > > > > > >> > > > > > >> > > partition number between the log dir
> > and
> > >> > the
> > >> > > > > > metadata
> > >> > > > > > > > >> file.
> > >> > > > > > > > >> > > For
> > >> > > > > > > > >> > > > > > >> example,
> > >> > > > > > > > >> > > > > > >> > if
> > >> > > > > > > > >> > > > > > >> > > the log dir is topicA-1 and the
> > metadata
> > >> > file
> > >> > > > in
> > >> > > > > it
> > >> > > > > > > has
> > >> > > > > > > > >> > topicB
> > >> > > > > > > > >> > > > and
> > >> > > > > > > > >> > > > > > >> > > partition 0 (say due to a bug or
> > manual
> > >> > > > > > > modification),
> > >> > > > > > > > >> which
> > >> > > > > > > > >> > > one
> > >> > > > > > > > >> > > > > do
> > >> > > > > > > > >> > > > > > we
> > >> > > > > > > > >> > > > > > >> > use
> > >> > > > > > > > >> > > > > > >> > > as the source of truth?
> > >> > > > > > > > >> > > > > > >> > >
> > >> > > > > > > > >> > > > > > >> > > Jun
> > >> > > > > > > > >> > > > > > >> > >
> > >> > > > > > > > >> > > > > > >> > > On Thu, Sep 17, 2020 at 3:43 PM
> > Justine
> > >> > > Olshan
> > >> > > > <
> > >> > > > > > > > >> > > > > > jolshan@confluent.io>
> > >> > > > > > > > >> > > > > > >> > > wrote:
> > >> > > > > > > > >> > > > > > >> > >
> > >> > > > > > > > >> > > > > > >> > > > Hi Jun,
> > >> > > > > > > > >> > > > > > >> > > > Thanks for the comments.
> > >> > > > > > > > >> > > > > > >> > > >
> > >> > > > > > > > >> > > > > > >> > > > 12. I bumped the
> LeaderAndIsrRequest
> > >> > > because
> > >> > > > I
> > >> > > > > > > > removed
> > >> > > > > > > > >> the
> > >> > > > > > > > >> > > > topic
> > >> > > > > > > > >> > > > > > >> name
> > >> > > > > > > > >> > > > > > >> > > field
> > >> > > > > > > > >> > > > > > >> > > > in the response. It may be
> possible
> > to
> > >> > > avoid
> > >> > > > > > > bumping
> > >> > > > > > > > >> the
> > >> > > > > > > > >> > > > version
> > >> > > > > > > > >> > > > > > >> > without
> > >> > > > > > > > >> > > > > > >> > > > that change, but I may be missing
> > >> > > something.
> > >> > > > > > > > >> > > > > > >> > > > I believe StopReplica is actually
> on
> > >> > > version
> > >> > > > 3
> > >> > > > > > now,
> > >> > > > > > > > but
> > >> > > > > > > > >> > > > because
> > >> > > > > > > > >> > > > > > >> > version 2
> > >> > > > > > > > >> > > > > > >> > > > is flexible, I kept that listed as
> > >> > version
> > >> > > 2
> > >> > > > on
> > >> > > > > > the
> > >> > > > > > > > KIP
> > >> > > > > > > > >> > > page.
> > >> > > > > > > > >> > > > > > >> However,
> > >> > > > > > > > >> > > > > > >> > > you
> > >> > > > > > > > >> > > > > > >> > > > may be right in that we may need
> to
> > >> bump
> > >> > > the
> > >> > > > > > > version
> > >> > > > > > > > on
> > >> > > > > > > > >> > > > > > StopReplica
> > >> > > > > > > > >> > > > > > >> to
> > >> > > > > > > > >> > > > > > >> > > deal
> > >> > > > > > > > >> > > > > > >> > > > with deletion differently as
> > mentioned
> > >> > > > above. I
> > >> > > > > > > don't
> > >> > > > > > > > >> know
> > >> > > > > > > > >> > > if
> > >> > > > > > > > >> > > > I
> > >> > > > > > > > >> > > > > > >> have a
> > >> > > > > > > > >> > > > > > >> > > big
> > >> > > > > > > > >> > > > > > >> > > > preference over used tagged fields
> > or
> > >> > not.
> > >> > > > > > > > >> > > > > > >> > > >
> > >> > > > > > > > >> > > > > > >> > > > 13. I was thinking that in the
> case
> > >> where
> > >> > > the
> > >> > > > > > file
> > >> > > > > > > > and
> > >> > > > > > > > >> the
> > >> > > > > > > > >> > > > > request
> > >> > > > > > > > >> > > > > > >> > topic
> > >> > > > > > > > >> > > > > > >> > > > ids don't match, it means that the
> > >> > broker's
> > >> > > > > > > topic/the
> > >> > > > > > > > >> one
> > >> > > > > > > > >> > in
> > >> > > > > > > > >> > > > the
> > >> > > > > > > > >> > > > > > >> file
> > >> > > > > > > > >> > > > > > >> > has
> > >> > > > > > > > >> > > > > > >> > > > been deleted. In that case, we
> would
> > >> need
> > >> > > to
> > >> > > > > > delete
> > >> > > > > > > > the
> > >> > > > > > > > >> > old
> > >> > > > > > > > >> > > > > topic
> > >> > > > > > > > >> > > > > > >> and
> > >> > > > > > > > >> > > > > > >> > > start
> > >> > > > > > > > >> > > > > > >> > > > receiving the new version. If the
> > >> topic
> > >> > > name
> > >> > > > > were
> > >> > > > > > > to
> > >> > > > > > > > >> > change,
> > >> > > > > > > > >> > > > but
> > >> > > > > > > > >> > > > > > the
> > >> > > > > > > > >> > > > > > >> > ids
> > >> > > > > > > > >> > > > > > >> > > > still matched, the file would also
> > >> need
> > >> > to
> > >> > > > > > update.
> > >> > > > > > > > Am I
> > >> > > > > > > > >> > > > missing
> > >> > > > > > > > >> > > > > a
> > >> > > > > > > > >> > > > > > >> case
> > >> > > > > > > > >> > > > > > >> > > > where the file would be correct
> and
> > >> not
> > >> > the
> > >> > > > > > > request?
> > >> > > > > > > > >> > > > > > >> > > >
> > >> > > > > > > > >> > > > > > >> > > > Thanks,
> > >> > > > > > > > >> > > > > > >> > > > Justine
> > >> > > > > > > > >> > > > > > >> > > >
> > >> > > > > > > > >> > > > > > >> > > > On Thu, Sep 17, 2020 at 3:18 PM
> Jun
> > >> Rao <
> > >> > > > > > > > >> jun@confluent.io
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> > > > > wrote:
> > >> > > > > > > > >> > > > > > >> > > >
> > >> > > > > > > > >> > > > > > >> > > > > Hi, Justine,
> > >> > > > > > > > >> > > > > > >> > > > >
> > >> > > > > > > > >> > > > > > >> > > > > Thanks for the reply. A couple
> of
> > >> more
> > >> > > > > comments
> > >> > > > > > > > >> below.
> > >> > > > > > > > >> > > > > > >> > > > >
> > >> > > > > > > > >> > > > > > >> > > > > 12. ListOffset and
> OffsetForLeader
> > >> > > > currently
> > >> > > > > > > don't
> > >> > > > > > > > >> > support
> > >> > > > > > > > >> > > > > > >> flexible
> > >> > > > > > > > >> > > > > > >> > > > fields.
> > >> > > > > > > > >> > > > > > >> > > > > So, we have to bump up the
> version
> > >> > number
> > >> > > > and
> > >> > > > > > use
> > >> > > > > > > > >> IBP at
> > >> > > > > > > > >> > > > least
> > >> > > > > > > > >> > > > > > for
> > >> > > > > > > > >> > > > > > >> > > these
> > >> > > > > > > > >> > > > > > >> > > > > two requests. Note that it seems
> > >> 2.7.0
> > >> > > will
> > >> > > > > > > require
> > >> > > > > > > > >> IBP
> > >> > > > > > > > >> > > > anyway
> > >> > > > > > > > >> > > > > > >> > because
> > >> > > > > > > > >> > > > > > >> > > of
> > >> > > > > > > > >> > > > > > >> > > > > changes in KAFKA-10435. Also, it
> > >> seems
> > >> > > that
> > >> > > > > the
> > >> > > > > > > > >> version
> > >> > > > > > > > >> > > for
> > >> > > > > > > > >> > > > > > >> > > > > LeaderAndIsrRequest and
> > StopReplica
> > >> are
> > >> > > > > bumped
> > >> > > > > > > even
> > >> > > > > > > > >> > though
> > >> > > > > > > > >> > > > we
> > >> > > > > > > > >> > > > > > only
> > >> > > > > > > > >> > > > > > >> > > added
> > >> > > > > > > > >> > > > > > >> > > > a
> > >> > > > > > > > >> > > > > > >> > > > > tagged field. But since IBP is
> > >> needed
> > >> > > > anyway,
> > >> > > > > > we
> > >> > > > > > > > may
> > >> > > > > > > > >> > want
> > >> > > > > > > > >> > > to
> > >> > > > > > > > >> > > > > > >> revisit
> > >> > > > > > > > >> > > > > > >> > > the
> > >> > > > > > > > >> > > > > > >> > > > > overall tagged field choice.
> > >> > > > > > > > >> > > > > > >> > > > >
> > >> > > > > > > > >> > > > > > >> > > > > 13. The only downside is the
> > >> potential
> > >> > > > > > confusion
> > >> > > > > > > on
> > >> > > > > > > > >> > which
> > >> > > > > > > > >> > > > one
> > >> > > > > > > > >> > > > > is
> > >> > > > > > > > >> > > > > > >> the
> > >> > > > > > > > >> > > > > > >> > > > source
> > >> > > > > > > > >> > > > > > >> > > > > of truth if they don't match.
> > >> Another
> > >> > > > option
> > >> > > > > is
> > >> > > > > > > to
> > >> > > > > > > > >> > include
> > >> > > > > > > > >> > > > > those
> > >> > > > > > > > >> > > > > > >> > fields
> > >> > > > > > > > >> > > > > > >> > > > in
> > >> > > > > > > > >> > > > > > >> > > > > the metadata file when we
> actually
> > >> > change
> > >> > > > the
> > >> > > > > > > > >> directory
> > >> > > > > > > > >> > > > > > structure.
> > >> > > > > > > > >> > > > > > >> > > > >
> > >> > > > > > > > >> > > > > > >> > > > > Thanks,
> > >> > > > > > > > >> > > > > > >> > > > >
> > >> > > > > > > > >> > > > > > >> > > > > Jun
> > >> > > > > > > > >> > > > > > >> > > > >
> > >> > > > > > > > >> > > > > > >> > > > > On Thu, Sep 17, 2020 at 2:01 PM
> > >> Justine
> > >> > > > > Olshan
> > >> > > > > > <
> > >> > > > > > > > >> > > > > > >> jolshan@confluent.io
> > >> > > > > > > > >> > > > > > >> > >
> > >> > > > > > > > >> > > > > > >> > > > > wrote:
> > >> > > > > > > > >> > > > > > >> > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > Hello all,
> > >> > > > > > > > >> > > > > > >> > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > I've thought some more about
> > >> removing
> > >> > > the
> > >> > > > > > topic
> > >> > > > > > > > >> name
> > >> > > > > > > > >> > > field
> > >> > > > > > > > >> > > > > > from
> > >> > > > > > > > >> > > > > > >> > some
> > >> > > > > > > > >> > > > > > >> > > of
> > >> > > > > > > > >> > > > > > >> > > > > the
> > >> > > > > > > > >> > > > > > >> > > > > > requests. On closer inspection
> > of
> > >> the
> > >> > > > > > > > >> > > requests/responses,
> > >> > > > > > > > >> > > > it
> > >> > > > > > > > >> > > > > > >> seems
> > >> > > > > > > > >> > > > > > >> > > that
> > >> > > > > > > > >> > > > > > >> > > > > the
> > >> > > > > > > > >> > > > > > >> > > > > > internal changes would be much
> > >> larger
> > >> > > > than
> > >> > > > > I
> > >> > > > > > > > >> expected.
> > >> > > > > > > > >> > > > Some
> > >> > > > > > > > >> > > > > > >> > protocols
> > >> > > > > > > > >> > > > > > >> > > > > > involve clients, so they would
> > >> > require
> > >> > > > > > changes
> > >> > > > > > > > too.
> > >> > > > > > > > >> > I'm
> > >> > > > > > > > >> > > > > > thinking
> > >> > > > > > > > >> > > > > > >> > that
> > >> > > > > > > > >> > > > > > >> > > > for
> > >> > > > > > > > >> > > > > > >> > > > > > now, removing the topic name
> > from
> > >> > these
> > >> > > > > > > requests
> > >> > > > > > > > >> and
> > >> > > > > > > > >> > > > > responses
> > >> > > > > > > > >> > > > > > >> are
> > >> > > > > > > > >> > > > > > >> > > out
> > >> > > > > > > > >> > > > > > >> > > > of
> > >> > > > > > > > >> > > > > > >> > > > > > scope.
> > >> > > > > > > > >> > > > > > >> > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > I have decided to just keep
> the
> > >> > change
> > >> > > > > > > > >> > > > LeaderAndIsrResponse
> > >> > > > > > > > >> > > > > to
> > >> > > > > > > > >> > > > > > >> > remove
> > >> > > > > > > > >> > > > > > >> > > > the
> > >> > > > > > > > >> > > > > > >> > > > > > topic name, and have updated
> the
> > >> KIP
> > >> > to
> > >> > > > > > reflect
> > >> > > > > > > > >> this
> > >> > > > > > > > >> > > > > change. I
> > >> > > > > > > > >> > > > > > >> have
> > >> > > > > > > > >> > > > > > >> > > > also
> > >> > > > > > > > >> > > > > > >> > > > > > mentioned the other requests
> and
> > >> > > > responses
> > >> > > > > in
> > >> > > > > > > > >> future
> > >> > > > > > > > >> > > work.
> > >> > > > > > > > >> > > > > > >> > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > I'm hoping to start the voting
> > >> > process
> > >> > > > > soon,
> > >> > > > > > so
> > >> > > > > > > > >> let me
> > >> > > > > > > > >> > > > know
> > >> > > > > > > > >> > > > > if
> > >> > > > > > > > >> > > > > > >> > there
> > >> > > > > > > > >> > > > > > >> > > is
> > >> > > > > > > > >> > > > > > >> > > > > > anything else we should
> discuss.
> > >> > > > > > > > >> > > > > > >> > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > Thank you,
> > >> > > > > > > > >> > > > > > >> > > > > > Justine
> > >> > > > > > > > >> > > > > > >> > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > On Tue, Sep 15, 2020 at 3:57
> PM
> > >> > Justine
> > >> > > > > > Olshan
> > >> > > > > > > <
> > >> > > > > > > > >> > > > > > >> > jolshan@confluent.io
> > >> > > > > > > > >> > > > > > >> > > >
> > >> > > > > > > > >> > > > > > >> > > > > > wrote:
> > >> > > > > > > > >> > > > > > >> > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > > Hello again,
> > >> > > > > > > > >> > > > > > >> > > > > > > To follow up on some of the
> > >> other
> > >> > > > > comments:
> > >> > > > > > > > >> > > > > > >> > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > > 10/11) We can remove the
> topic
> > >> name
> > >> > > > from
> > >> > > > > > > these
> > >> > > > > > > > >> > > > > > >> > requests/responses,
> > >> > > > > > > > >> > > > > > >> > > > and
> > >> > > > > > > > >> > > > > > >> > > > > > > that means we will just have
> > to
> > >> > make
> > >> > > a
> > >> > > > > few
> > >> > > > > > > > >> internal
> > >> > > > > > > > >> > > > > changes
> > >> > > > > > > > >> > > > > > to
> > >> > > > > > > > >> > > > > > >> > make
> > >> > > > > > > > >> > > > > > >> > > > > > > partitions accessible by
> topic
> > >> id
> > >> > and
> > >> > > > > > > > partition.
> > >> > > > > > > > >> I
> > >> > > > > > > > >> > can
> > >> > > > > > > > >> > > > > > update
> > >> > > > > > > > >> > > > > > >> the
> > >> > > > > > > > >> > > > > > >> > > KIP
> > >> > > > > > > > >> > > > > > >> > > > > to
> > >> > > > > > > > >> > > > > > >> > > > > > > remove them unless anyone
> > thinks
> > >> > they
> > >> > > > > > should
> > >> > > > > > > > >> stay.
> > >> > > > > > > > >> > > > > > >> > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > > 12) Addressed in the
> previous
> > >> > email.
> > >> > > > I've
> > >> > > > > > > > updated
> > >> > > > > > > > >> > the
> > >> > > > > > > > >> > > > KIP
> > >> > > > > > > > >> > > > > to
> > >> > > > > > > > >> > > > > > >> > > include
> > >> > > > > > > > >> > > > > > >> > > > > > > tagged fields for the
> requests
> > >> and
> > >> > > > > > responses.
> > >> > > > > > > > >> (More
> > >> > > > > > > > >> > on
> > >> > > > > > > > >> > > > > that
> > >> > > > > > > > >> > > > > > >> > below)
> > >> > > > > > > > >> > > > > > >> > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > > 13) I think part of the idea
> > for
> > >> > > > > including
> > >> > > > > > > this
> > >> > > > > > > > >> > > > > information
> > >> > > > > > > > >> > > > > > >> is to
> > >> > > > > > > > >> > > > > > >> > > > > prepare
> > >> > > > > > > > >> > > > > > >> > > > > > > for future changes. Perhaps
> > the
> > >> > > > directory
> > >> > > > > > > > >> structure
> > >> > > > > > > > >> > > > might
> > >> > > > > > > > >> > > > > > >> change
> > >> > > > > > > > >> > > > > > >> > > from
> > >> > > > > > > > >> > > > > > >> > > > > > > topicName_partitionNumber to
> > >> > > something
> > >> > > > > like
> > >> > > > > > > > >> > > > > > >> > > topicID_partitionNumber.
> > >> > > > > > > > >> > > > > > >> > > > > Then
> > >> > > > > > > > >> > > > > > >> > > > > > > it would be useful to have
> the
> > >> > topic
> > >> > > > name
> > >> > > > > > in
> > >> > > > > > > > the
> > >> > > > > > > > >> > file
> > >> > > > > > > > >> > > > > since
> > >> > > > > > > > >> > > > > > it
> > >> > > > > > > > >> > > > > > >> > > would
> > >> > > > > > > > >> > > > > > >> > > > > not
> > >> > > > > > > > >> > > > > > >> > > > > > be
> > >> > > > > > > > >> > > > > > >> > > > > > > in the directory structure.
> > >> > > Supporting
> > >> > > > > > topic
> > >> > > > > > > > >> renames
> > >> > > > > > > > >> > > > might
> > >> > > > > > > > >> > > > > > be
> > >> > > > > > > > >> > > > > > >> > > easier
> > >> > > > > > > > >> > > > > > >> > > > if
> > >> > > > > > > > >> > > > > > >> > > > > > the
> > >> > > > > > > > >> > > > > > >> > > > > > > other fields are included.
> > Would
> > >> > > there
> > >> > > > be
> > >> > > > > > any
> > >> > > > > > > > >> > > downsides
> > >> > > > > > > > >> > > > to
> > >> > > > > > > > >> > > > > > >> > > including
> > >> > > > > > > > >> > > > > > >> > > > > this
> > >> > > > > > > > >> > > > > > >> > > > > > > information?
> > >> > > > > > > > >> > > > > > >> > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > > 14)  Yes, we would need to
> > copy
> > >> the
> > >> > > > > > partition
> > >> > > > > > > > >> > metadata
> > >> > > > > > > > >> > > > > file
> > >> > > > > > > > >> > > > > > in
> > >> > > > > > > > >> > > > > > >> > this
> > >> > > > > > > > >> > > > > > >> > > > > > > process. I've updated the
> KIP
> > to
> > >> > > > include
> > >> > > > > > > this.
> > >> > > > > > > > >> > > > > > >> > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > > 15) I believe Lucas meant v1
> > >> and v2
> > >> > > > here.
> > >> > > > > > He
> > >> > > > > > > > was
> > >> > > > > > > > >> > > > referring
> > >> > > > > > > > >> > > > > > to
> > >> > > > > > > > >> > > > > > >> how
> > >> > > > > > > > >> > > > > > >> > > the
> > >> > > > > > > > >> > > > > > >> > > > > > > requests would fall under
> > >> different
> > >> > > IBP
> > >> > > > > and
> > >> > > > > > > > meant
> > >> > > > > > > > >> > that
> > >> > > > > > > > >> > > > > older
> > >> > > > > > > > >> > > > > > >> > > brokers
> > >> > > > > > > > >> > > > > > >> > > > > > would
> > >> > > > > > > > >> > > > > > >> > > > > > > have to use the older
> version
> > of
> > >> > the
> > >> > > > > > request
> > >> > > > > > > > and
> > >> > > > > > > > >> the
> > >> > > > > > > > >> > > > > > existing
> > >> > > > > > > > >> > > > > > >> > topic
> > >> > > > > > > > >> > > > > > >> > > > > > > deletion process. At first,
> it
> > >> > seemed
> > >> > > > > like
> > >> > > > > > > > tagged
> > >> > > > > > > > >> > > fields
> > >> > > > > > > > >> > > > > > would
> > >> > > > > > > > >> > > > > > >> > > > resolve
> > >> > > > > > > > >> > > > > > >> > > > > > > the IBP issue. However, we
> may
> > >> need
> > >> > > IBP
> > >> > > > > for
> > >> > > > > > > > this
> > >> > > > > > > > >> > > request
> > >> > > > > > > > >> > > > > > after
> > >> > > > > > > > >> > > > > > >> > all
> > >> > > > > > > > >> > > > > > >> > > > > since
> > >> > > > > > > > >> > > > > > >> > > > > > > the controller handles the
> > topic
> > >> > > > deletion
> > >> > > > > > > > >> > differently
> > >> > > > > > > > >> > > > > > >> depending
> > >> > > > > > > > >> > > > > > >> > on
> > >> > > > > > > > >> > > > > > >> > > > the
> > >> > > > > > > > >> > > > > > >> > > > > > IBP
> > >> > > > > > > > >> > > > > > >> > > > > > > version. In an older
> version,
> > we
> > >> > > can't
> > >> > > > > just
> > >> > > > > > > > send
> > >> > > > > > > > >> a
> > >> > > > > > > > >> > > > > > StopReplica
> > >> > > > > > > > >> > > > > > >> > > delete
> > >> > > > > > > > >> > > > > > >> > > > > the
> > >> > > > > > > > >> > > > > > >> > > > > > > topic immediately like we'd
> > >> want to
> > >> > > for
> > >> > > > > > this
> > >> > > > > > > > KIP.
> > >> > > > > > > > >> > > > > > >> > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > > This makes me wonder if we
> > want
> > >> > > tagged
> > >> > > > > > fields
> > >> > > > > > > > on
> > >> > > > > > > > >> all
> > >> > > > > > > > >> > > the
> > >> > > > > > > > >> > > > > > >> requests
> > >> > > > > > > > >> > > > > > >> > > > after
> > >> > > > > > > > >> > > > > > >> > > > > > > all. Let me know your
> > thoughts!
> > >> > > > > > > > >> > > > > > >> > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > > Justine
> > >> > > > > > > > >> > > > > > >> > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > > On Tue, Sep 15, 2020 at 1:03
> > PM
> > >> > > Justine
> > >> > > > > > > Olshan
> > >> > > > > > > > <
> > >> > > > > > > > >> > > > > > >> > > jolshan@confluent.io
> > >> > > > > > > > >> > > > > > >> > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > > wrote:
> > >> > > > > > > > >> > > > > > >> > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >> Hi all,
> > >> > > > > > > > >> > > > > > >> > > > > > >> Jun brought up a good point
> > in
> > >> his
> > >> > > > last
> > >> > > > > > > email
> > >> > > > > > > > >> about
> > >> > > > > > > > >> > > > > tagged
> > >> > > > > > > > >> > > > > > >> > fields,
> > >> > > > > > > > >> > > > > > >> > > > and
> > >> > > > > > > > >> > > > > > >> > > > > > >> I've updated the KIP to
> > reflect
> > >> > that
> > >> > > > the
> > >> > > > > > > > >> changes to
> > >> > > > > > > > >> > > > > > requests
> > >> > > > > > > > >> > > > > > >> and
> > >> > > > > > > > >> > > > > > >> > > > > > responses
> > >> > > > > > > > >> > > > > > >> > > > > > >> will be in the form of
> tagged
> > >> > fields
> > >> > > > to
> > >> > > > > > > avoid
> > >> > > > > > > > >> > > changing
> > >> > > > > > > > >> > > > > IBP.
> > >> > > > > > > > >> > > > > > >> > > > > > >>
> > >> > > > > > > > >> > > > > > >> > > > > > >> Jun: I plan on sending a
> > >> followup
> > >> > > > email
> > >> > > > > to
> > >> > > > > > > > >> address
> > >> > > > > > > > >> > > some
> > >> > > > > > > > >> > > > > of
> > >> > > > > > > > >> > > > > > >> the
> > >> > > > > > > > >> > > > > > >> > > other
> > >> > > > > > > > >> > > > > > >> > > > > > >> points.
> > >> > > > > > > > >> > > > > > >> > > > > > >>
> > >> > > > > > > > >> > > > > > >> > > > > > >> Thanks,
> > >> > > > > > > > >> > > > > > >> > > > > > >> Justine
> > >> > > > > > > > >> > > > > > >> > > > > > >>
> > >> > > > > > > > >> > > > > > >> > > > > > >> On Mon, Sep 14, 2020 at
> 4:25
> > PM
> > >> > Jun
> > >> > > > Rao
> > >> > > > > <
> > >> > > > > > > > >> > > > > jun@confluent.io>
> > >> > > > > > > > >> > > > > > >> > wrote:
> > >> > > > > > > > >> > > > > > >> > > > > > >>
> > >> > > > > > > > >> > > > > > >> > > > > > >>> Hi, Justine,
> > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > >> > > > > > > > >> > > > > > >> > > > > > >>> Thanks for the updated
> KIP.
> > A
> > >> few
> > >> > > > > > comments
> > >> > > > > > > > >> below.
> > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > >> > > > > > > > >> > > > > > >> > > > > > >>> 10. LeaderAndIsr Response:
> > Do
> > >> we
> > >> > > need
> > >> > > > > the
> > >> > > > > > > > topic
> > >> > > > > > > > >> > > name?
> > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > >> > > > > > > > >> > > > > > >> > > > > > >>> 11. For the changed
> > >> > > request/response,
> > >> > > > > > other
> > >> > > > > > > > >> than
> > >> > > > > > > > >> > > > > > >> LeaderAndIsr,
> > >> > > > > > > > >> > > > > > >> > > > > > >>> UpdateMetadata, Metadata,
> do
> > >> we
> > >> > > need
> > >> > > > to
> > >> > > > > > > > include
> > >> > > > > > > > >> > the
> > >> > > > > > > > >> > > > > topic
> > >> > > > > > > > >> > > > > > >> name?
> > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > >> > > > > > > > >> > > > > > >> > > > > > >>> 12. It seems that upgrades
> > >> don't
> > >> > > > > require
> > >> > > > > > > IBP.
> > >> > > > > > > > >> Does
> > >> > > > > > > > >> > > > that
> > >> > > > > > > > >> > > > > > mean
> > >> > > > > > > > >> > > > > > >> > the
> > >> > > > > > > > >> > > > > > >> > > > new
> > >> > > > > > > > >> > > > > > >> > > > > > >>> fields
> > >> > > > > > > > >> > > > > > >> > > > > > >>> in all the
> request/response
> > >> are
> > >> > > added
> > >> > > > > as
> > >> > > > > > > > tagged
> > >> > > > > > > > >> > > fields
> > >> > > > > > > > >> > > > > > >> without
> > >> > > > > > > > >> > > > > > >> > > > > bumping
> > >> > > > > > > > >> > > > > > >> > > > > > up
> > >> > > > > > > > >> > > > > > >> > > > > > >>> the request version? It
> > would
> > >> be
> > >> > > > useful
> > >> > > > > > to
> > >> > > > > > > > make
> > >> > > > > > > > >> > that
> > >> > > > > > > > >> > > > > > clear.
> > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > >> > > > > > > > >> > > > > > >> > > > > > >>> 13. Partition Metadata
> file:
> > >> Do
> > >> > we
> > >> > > > need
> > >> > > > > > to
> > >> > > > > > > > >> include
> > >> > > > > > > > >> > > the
> > >> > > > > > > > >> > > > > > topic
> > >> > > > > > > > >> > > > > > >> > name
> > >> > > > > > > > >> > > > > > >> > > > and
> > >> > > > > > > > >> > > > > > >> > > > > > the
> > >> > > > > > > > >> > > > > > >> > > > > > >>> partition id since they
> are
> > >> > implied
> > >> > > > in
> > >> > > > > > the
> > >> > > > > > > > >> > directory
> > >> > > > > > > > >> > > > > name?
> > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > >> > > > > > > > >> > > > > > >> > > > > > >>> 14. In the JBOD mode, we
> > >> support
> > >> > > > > moving a
> > >> > > > > > > > >> > > partition's
> > >> > > > > > > > >> > > > > data
> > >> > > > > > > > >> > > > > > >> from
> > >> > > > > > > > >> > > > > > >> > > one
> > >> > > > > > > > >> > > > > > >> > > > > > disk
> > >> > > > > > > > >> > > > > > >> > > > > > >>> to
> > >> > > > > > > > >> > > > > > >> > > > > > >>> another. Will the new
> > >> partition
> > >> > > > > metadata
> > >> > > > > > > file
> > >> > > > > > > > >> be
> > >> > > > > > > > >> > > > copied
> > >> > > > > > > > >> > > > > > >> during
> > >> > > > > > > > >> > > > > > >> > > that
> > >> > > > > > > > >> > > > > > >> > > > > > >>> process?
> > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > >> > > > > > > > >> > > > > > >> > > > > > >>> 15. The KIP says "Remove
> > >> deleted
> > >> > > > topics
> > >> > > > > > > from
> > >> > > > > > > > >> > > replicas
> > >> > > > > > > > >> > > > by
> > >> > > > > > > > >> > > > > > >> > sending
> > >> > > > > > > > >> > > > > > >> > > > > > >>> StopReplicaRequest V2 for
> > any
> > >> > > topics
> > >> > > > > > which
> > >> > > > > > > do
> > >> > > > > > > > >> not
> > >> > > > > > > > >> > > > > contain
> > >> > > > > > > > >> > > > > > a
> > >> > > > > > > > >> > > > > > >> > topic
> > >> > > > > > > > >> > > > > > >> > > > ID,
> > >> > > > > > > > >> > > > > > >> > > > > > and
> > >> > > > > > > > >> > > > > > >> > > > > > >>> V3 for any topics which do
> > >> > contain
> > >> > > a
> > >> > > > > > topic
> > >> > > > > > > > >> ID.".
> > >> > > > > > > > >> > > > > However,
> > >> > > > > > > > >> > > > > > it
> > >> > > > > > > > >> > > > > > >> > > seems
> > >> > > > > > > > >> > > > > > >> > > > > the
> > >> > > > > > > > >> > > > > > >> > > > > > >>> updated controller will
> > create
> > >> > all
> > >> > > > > > missing
> > >> > > > > > > > >> topic
> > >> > > > > > > > >> > IDs
> > >> > > > > > > > >> > > > > first
> > >> > > > > > > > >> > > > > > >> > before
> > >> > > > > > > > >> > > > > > >> > > > > doing
> > >> > > > > > > > >> > > > > > >> > > > > > >>> other actions. So, is
> > >> > > > > StopReplicaRequest
> > >> > > > > > V2
> > >> > > > > > > > >> > needed?
> > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > >> > > > > > > > >> > > > > > >> > > > > > >>> Jun
> > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > >> > > > > > > > >> > > > > > >> > > > > > >>> On Fri, Sep 11, 2020 at
> > 10:31
> > >> AM
> > >> > > John
> > >> > > > > > > > Roesler <
> > >> > > > > > > > >> > > > > > >> > > vvcephei@apache.org
> > >> > > > > > > > >> > > > > > >> > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> wrote:
> > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > Thanks, Justine!
> > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > Your response seems
> > >> compelling
> > >> > to
> > >> > > > me.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > -John
> > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > On Fri, 2020-09-11 at
> > 10:17
> > >> > > -0700,
> > >> > > > > > > Justine
> > >> > > > > > > > >> > Olshan
> > >> > > > > > > > >> > > > > wrote:
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Hello all,
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Thanks for continuing
> > the
> > >> > > > > > discussion! I
> > >> > > > > > > > >> have a
> > >> > > > > > > > >> > > few
> > >> > > > > > > > >> > > > > > >> > responses
> > >> > > > > > > > >> > > > > > >> > > to
> > >> > > > > > > > >> > > > > > >> > > > > > your
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > points.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Tom: You are correct
> in
> > >> that
> > >> > > this
> > >> > > > > KIP
> > >> > > > > > > has
> > >> > > > > > > > >> not
> > >> > > > > > > > >> > > > > > mentioned
> > >> > > > > > > > >> > > > > > >> the
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > DeleteTopicsRequest. I
> > >> think
> > >> > > that
> > >> > > > > > this
> > >> > > > > > > > >> would
> > >> > > > > > > > >> > be
> > >> > > > > > > > >> > > > out
> > >> > > > > > > > >> > > > > of
> > >> > > > > > > > >> > > > > > >> > scope
> > >> > > > > > > > >> > > > > > >> > > > for
> > >> > > > > > > > >> > > > > > >> > > > > > >>> now, but
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > may be something worth
> > >> adding
> > >> > > in
> > >> > > > > the
> > >> > > > > > > > >> future.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > John: We did consider
> > >> > sequence
> > >> > > > ids,
> > >> > > > > > but
> > >> > > > > > > > >> there
> > >> > > > > > > > >> > > are
> > >> > > > > > > > >> > > > a
> > >> > > > > > > > >> > > > > > few
> > >> > > > > > > > >> > > > > > >> > > reasons
> > >> > > > > > > > >> > > > > > >> > > > > to
> > >> > > > > > > > >> > > > > > >> > > > > > >>> favor
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > UUIDs. There are
> several
> > >> > cases
> > >> > > > > where
> > >> > > > > > > > topics
> > >> > > > > > > > >> > from
> > >> > > > > > > > >> > > > > > >> different
> > >> > > > > > > > >> > > > > > >> > > > > clusters
> > >> > > > > > > > >> > > > > > >> > > > > > >>> may
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > interact now and in
> the
> > >> > future.
> > >> > > > For
> > >> > > > > > > > >> example,
> > >> > > > > > > > >> > > > Mirror
> > >> > > > > > > > >> > > > > > >> Maker 2
> > >> > > > > > > > >> > > > > > >> > > may
> > >> > > > > > > > >> > > > > > >> > > > > > >>> benefit
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > from being able to
> > detect
> > >> > when
> > >> > > a
> > >> > > > > > > cluster
> > >> > > > > > > > >> being
> > >> > > > > > > > >> > > > > > mirrored
> > >> > > > > > > > >> > > > > > >> is
> > >> > > > > > > > >> > > > > > >> > > > > deleted
> > >> > > > > > > > >> > > > > > >> > > > > > >>> and
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > recreated and globally
> > >> unique
> > >> > > > > > > identifiers
> > >> > > > > > > > >> > would
> > >> > > > > > > > >> > > > make
> > >> > > > > > > > >> > > > > > >> > > resolving
> > >> > > > > > > > >> > > > > > >> > > > > > issues
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > easier than sequence
> IDs
> > >> > which
> > >> > > > may
> > >> > > > > > > > collide
> > >> > > > > > > > >> > > between
> > >> > > > > > > > >> > > > > > >> > clusters.
> > >> > > > > > > > >> > > > > > >> > > > > > KIP-405
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > (tiered storage) will
> > also
> > >> > > > benefit
> > >> > > > > > from
> > >> > > > > > > > >> > globally
> > >> > > > > > > > >> > > > > > unique
> > >> > > > > > > > >> > > > > > >> IDs
> > >> > > > > > > > >> > > > > > >> > > as
> > >> > > > > > > > >> > > > > > >> > > > > > shared
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > buckets may be used
> > >> between
> > >> > > > > clusters.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Globally unique IDs
> > would
> > >> > also
> > >> > > > make
> > >> > > > > > > > >> > > functionality
> > >> > > > > > > > >> > > > > like
> > >> > > > > > > > >> > > > > > >> > moving
> > >> > > > > > > > >> > > > > > >> > > > > > topics
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > between disparate
> > clusters
> > >> > > easier
> > >> > > > > in
> > >> > > > > > > the
> > >> > > > > > > > >> > future,
> > >> > > > > > > > >> > > > > > >> simplify
> > >> > > > > > > > >> > > > > > >> > any
> > >> > > > > > > > >> > > > > > >> > > > > > future
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > implementations of
> > backups
> > >> > and
> > >> > > > > > > restores,
> > >> > > > > > > > >> and
> > >> > > > > > > > >> > > more.
> > >> > > > > > > > >> > > > > In
> > >> > > > > > > > >> > > > > > >> > > general,
> > >> > > > > > > > >> > > > > > >> > > > > > >>> unique IDs
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > would ensure that the
> > >> source
> > >> > > > > cluster
> > >> > > > > > > > >> topics do
> > >> > > > > > > > >> > > not
> > >> > > > > > > > >> > > > > > >> conflict
> > >> > > > > > > > >> > > > > > >> > > > with
> > >> > > > > > > > >> > > > > > >> > > > > > the
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > destination cluster
> > >> topics.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > If we were to use
> > sequence
> > >> > ids,
> > >> > > > we
> > >> > > > > > > would
> > >> > > > > > > > >> need
> > >> > > > > > > > >> > > > > > >> sufficiently
> > >> > > > > > > > >> > > > > > >> > > > large
> > >> > > > > > > > >> > > > > > >> > > > > > >>> cluster
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > ids to be stored with
> > the
> > >> > topic
> > >> > > > > > > > >> identifiers or
> > >> > > > > > > > >> > > we
> > >> > > > > > > > >> > > > > run
> > >> > > > > > > > >> > > > > > >> the
> > >> > > > > > > > >> > > > > > >> > > risk
> > >> > > > > > > > >> > > > > > >> > > > of
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > collisions. This will
> > >> give up
> > >> > > any
> > >> > > > > > > > >> advantage in
> > >> > > > > > > > >> > > > > > >> compactness
> > >> > > > > > > > >> > > > > > >> > > that
> > >> > > > > > > > >> > > > > > >> > > > > > >>> sequence
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > numbers may bring.
> Given
> > >> > these
> > >> > > > > > > > advantages I
> > >> > > > > > > > >> > > think
> > >> > > > > > > > >> > > > it
> > >> > > > > > > > >> > > > > > >> makes
> > >> > > > > > > > >> > > > > > >> > > > sense
> > >> > > > > > > > >> > > > > > >> > > > > to
> > >> > > > > > > > >> > > > > > >> > > > > > >>> use
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > UUIDs.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Gokul: This is an
> > >> interesting
> > >> > > > idea,
> > >> > > > > > but
> > >> > > > > > > > >> this
> > >> > > > > > > > >> > is
> > >> > > > > > > > >> > > a
> > >> > > > > > > > >> > > > > > >> breaking
> > >> > > > > > > > >> > > > > > >> > > > > change.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> Out of
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > scope for now, but
> maybe
> > >> > worth
> > >> > > > > > > discussing
> > >> > > > > > > > >> in
> > >> > > > > > > > >> > the
> > >> > > > > > > > >> > > > > > future.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Hope this explains
> some
> > of
> > >> > the
> > >> > > > > > > decisions,
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Justine
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > On Fri, Sep 11, 2020
> at
> > >> 8:27
> > >> > AM
> > >> > > > > Gokul
> > >> > > > > > > > >> Ramanan
> > >> > > > > > > > >> > > > > > >> Subramanian <
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > gokul2411s@gmail.com>
> > >> wrote:
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Hi.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Thanks for the KIP.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Have you thought
> about
> > >> > > whether
> > >> > > > it
> > >> > > > > > > makes
> > >> > > > > > > > >> > sense
> > >> > > > > > > > >> > > to
> > >> > > > > > > > >> > > > > > >> support
> > >> > > > > > > > >> > > > > > >> > > > > > >>> authorizing a
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > principal for a
> topic
> > ID
> > >> > > rather
> > >> > > > > > than
> > >> > > > > > > a
> > >> > > > > > > > >> topic
> > >> > > > > > > > >> > > > name
> > >> > > > > > > > >> > > > > to
> > >> > > > > > > > >> > > > > > >> > > achieve
> > >> > > > > > > > >> > > > > > >> > > > > > >>> tighter
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > security?
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Or is the topic ID
> > >> > > > fundamentally
> > >> > > > > an
> > >> > > > > > > > >> internal
> > >> > > > > > > > >> > > > > detail
> > >> > > > > > > > >> > > > > > >> > similar
> > >> > > > > > > > >> > > > > > >> > > > to
> > >> > > > > > > > >> > > > > > >> > > > > > >>> epochs
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > used
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > in a bunch of other
> > >> places
> > >> > in
> > >> > > > > > Kafka?
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Thanks.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > On Fri, Sep 11, 2020
> > at
> > >> > 4:06
> > >> > > PM
> > >> > > > > > John
> > >> > > > > > > > >> > Roesler <
> > >> > > > > > > > >> > > > > > >> > > > > > vvcephei@apache.org>
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > wrote:
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Hello Justine,
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Thanks for the
> KIP!
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > I happen to have
> > been
> > >> > > > > confronted
> > >> > > > > > > > >> recently
> > >> > > > > > > > >> > > with
> > >> > > > > > > > >> > > > > the
> > >> > > > > > > > >> > > > > > >> need
> > >> > > > > > > > >> > > > > > >> > > to
> > >> > > > > > > > >> > > > > > >> > > > > keep
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > track of
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > a
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > large number of
> > >> topics as
> > >> > > > > > compactly
> > >> > > > > > > > as
> > >> > > > > > > > >> > > > > possible. I
> > >> > > > > > > > >> > > > > > >> was
> > >> > > > > > > > >> > > > > > >> > > > going
> > >> > > > > > > > >> > > > > > >> > > > > to
> > >> > > > > > > > >> > > > > > >> > > > > > >>> come
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > up
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > with some way to
> > >> > dictionary
> > >> > > > > > encode
> > >> > > > > > > > the
> > >> > > > > > > > >> > topic
> > >> > > > > > > > >> > > > > names
> > >> > > > > > > > >> > > > > > >> as
> > >> > > > > > > > >> > > > > > >> > > > > integers,
> > >> > > > > > > > >> > > > > > >> > > > > > >>> but
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > this
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > seems much better!
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Apologies if this
> > has
> > >> > been
> > >> > > > > raised
> > >> > > > > > > > >> before,
> > >> > > > > > > > >> > > but
> > >> > > > > > > > >> > > > > I’m
> > >> > > > > > > > >> > > > > > >> > > wondering
> > >> > > > > > > > >> > > > > > >> > > > > > >>> about the
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > choice of UUID vs
> > >> > sequence
> > >> > > > > number
> > >> > > > > > > for
> > >> > > > > > > > >> the
> > >> > > > > > > > >> > > ids.
> > >> > > > > > > > >> > > > > > >> > Typically,
> > >> > > > > > > > >> > > > > > >> > > > > I’ve
> > >> > > > > > > > >> > > > > > >> > > > > > >>> seen
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > UUIDs
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > in two situations:
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 1. When processes
> > >> need to
> > >> > > > > > generate
> > >> > > > > > > > >> > > > non-colliding
> > >> > > > > > > > >> > > > > > >> > > > identifiers
> > >> > > > > > > > >> > > > > > >> > > > > > >>> without
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > coordination.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 2. When the
> > identifier
> > >> > > needs
> > >> > > > to
> > >> > > > > > be
> > >> > > > > > > > >> > > > “universally
> > >> > > > > > > > >> > > > > > >> > unique”;
> > >> > > > > > > > >> > > > > > >> > > > > I.e.,
> > >> > > > > > > > >> > > > > > >> > > > > > >>> the
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > identifier needs
> to
> > >> > > > distinguish
> > >> > > > > > the
> > >> > > > > > > > >> entity
> > >> > > > > > > > >> > > > from
> > >> > > > > > > > >> > > > > > all
> > >> > > > > > > > >> > > > > > >> > other
> > >> > > > > > > > >> > > > > > >> > > > > > >>> entities
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > that
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > could ever exist.
> > >> This is
> > >> > > > > useful
> > >> > > > > > in
> > >> > > > > > > > >> cases
> > >> > > > > > > > >> > > > where
> > >> > > > > > > > >> > > > > > >> > entities
> > >> > > > > > > > >> > > > > > >> > > > from
> > >> > > > > > > > >> > > > > > >> > > > > > all
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > kinds
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > of
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > systems get mixed
> > >> > together,
> > >> > > > > such
> > >> > > > > > as
> > >> > > > > > > > >> when
> > >> > > > > > > > >> > > > dumping
> > >> > > > > > > > >> > > > > > >> logs
> > >> > > > > > > > >> > > > > > >> > > from
> > >> > > > > > > > >> > > > > > >> > > > > all
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > processes
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > in
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a company into a
> > >> common
> > >> > > > system.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Maybe I’m being
> > >> > > > short-sighted,
> > >> > > > > > but
> > >> > > > > > > it
> > >> > > > > > > > >> > > doesn’t
> > >> > > > > > > > >> > > > > seem
> > >> > > > > > > > >> > > > > > >> like
> > >> > > > > > > > >> > > > > > >> > > > > either
> > >> > > > > > > > >> > > > > > >> > > > > > >>> really
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > applies here. It
> > seems
> > >> > like
> > >> > > > the
> > >> > > > > > > > brokers
> > >> > > > > > > > >> > > could
> > >> > > > > > > > >> > > > > and
> > >> > > > > > > > >> > > > > > >> would
> > >> > > > > > > > >> > > > > > >> > > > > achieve
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > consensus
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > when creating a
> > topic
> > >> > > anyway,
> > >> > > > > > which
> > >> > > > > > > > is
> > >> > > > > > > > >> all
> > >> > > > > > > > >> > > > > that’s
> > >> > > > > > > > >> > > > > > >> > > required
> > >> > > > > > > > >> > > > > > >> > > > to
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > generate
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > non-colliding
> > sequence
> > >> > ids.
> > >> > > > For
> > >> > > > > > the
> > >> > > > > > > > >> > second,
> > >> > > > > > > > >> > > as
> > >> > > > > > > > >> > > > > you
> > >> > > > > > > > >> > > > > > >> > > mention,
> > >> > > > > > > > >> > > > > > >> > > > > we
> > >> > > > > > > > >> > > > > > >> > > > > > >>> could
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > assign
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a UUID to the
> > cluster
> > >> as
> > >> > a
> > >> > > > > whole,
> > >> > > > > > > > which
> > >> > > > > > > > >> > > would
> > >> > > > > > > > >> > > > > > render
> > >> > > > > > > > >> > > > > > >> > any
> > >> > > > > > > > >> > > > > > >> > > > > > resource
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > scoped
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > the broker
> > universally
> > >> > > unique
> > >> > > > > as
> > >> > > > > > > > well.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > The reason I
> mention
> > >> this
> > >> > > is
> > >> > > > > > that,
> > >> > > > > > > > >> > although
> > >> > > > > > > > >> > > a
> > >> > > > > > > > >> > > > > UUID
> > >> > > > > > > > >> > > > > > >> is
> > >> > > > > > > > >> > > > > > >> > way
> > >> > > > > > > > >> > > > > > >> > > > > more
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > compact
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > than topic names,
> > it’s
> > >> > > still
> > >> > > > 16
> > >> > > > > > > > bytes.
> > >> > > > > > > > >> In
> > >> > > > > > > > >> > > > > > contrast,
> > >> > > > > > > > >> > > > > > >> a
> > >> > > > > > > > >> > > > > > >> > > > 4-byte
> > >> > > > > > > > >> > > > > > >> > > > > > >>> integer
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > sequence id would
> > give
> > >> > us 4
> > >> > > > > > billion
> > >> > > > > > > > >> unique
> > >> > > > > > > > >> > > > > topics
> > >> > > > > > > > >> > > > > > >> per
> > >> > > > > > > > >> > > > > > >> > > > > cluster,
> > >> > > > > > > > >> > > > > > >> > > > > > >>> which
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > seems
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > like enough ;)
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Considering the
> > >> number of
> > >> > > > > > different
> > >> > > > > > > > >> times
> > >> > > > > > > > >> > > > these
> > >> > > > > > > > >> > > > > > >> topic
> > >> > > > > > > > >> > > > > > >> > > > > > >>> identifiers are
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > sent
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > over the wire or
> > >> stored
> > >> > in
> > >> > > > > > memory,
> > >> > > > > > > it
> > >> > > > > > > > >> > seems
> > >> > > > > > > > >> > > > like
> > >> > > > > > > > >> > > > > > it
> > >> > > > > > > > >> > > > > > >> > might
> > >> > > > > > > > >> > > > > > >> > > > be
> > >> > > > > > > > >> > > > > > >> > > > > > >>> worth
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > the
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > additional 4x
> space
> > >> > > savings.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > What do you think
> > >> about
> > >> > > this?
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Thanks,
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > John
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > On Fri, Sep 11,
> > 2020,
> > >> at
> > >> > > > 03:20,
> > >> > > > > > Tom
> > >> > > > > > > > >> > Bentley
> > >> > > > > > > > >> > > > > wrote:
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Hi Justine,
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > This looks like
> a
> > >> very
> > >> > > > > welcome
> > >> > > > > > > > >> > > improvement.
> > >> > > > > > > > >> > > > > > >> Thanks!
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Maybe I missed
> it,
> > >> but
> > >> > > the
> > >> > > > > KIP
> > >> > > > > > > > >> doesn't
> > >> > > > > > > > >> > > seem
> > >> > > > > > > > >> > > > to
> > >> > > > > > > > >> > > > > > >> > mention
> > >> > > > > > > > >> > > > > > >> > > > > > changing
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > DeleteTopicsRequest
> > >> to
> > >> > > > > identify
> > >> > > > > > > the
> > >> > > > > > > > >> > topic
> > >> > > > > > > > >> > > > > using
> > >> > > > > > > > >> > > > > > an
> > >> > > > > > > > >> > > > > > >> > id.
> > >> > > > > > > > >> > > > > > >> > > > > Maybe
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > that's out
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > of
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > scope, but
> > >> > > > > DeleteTopicsRequest
> > >> > > > > > is
> > >> > > > > > > > not
> > >> > > > > > > > >> > > listed
> > >> > > > > > > > >> > > > > > among
> > >> > > > > > > > >> > > > > > >> > the
> > >> > > > > > > > >> > > > > > >> > > > > Future
> > >> > > > > > > > >> > > > > > >> > > > > > >>> Work
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > APIs
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > either.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Kind regards,
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Tom
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > On Thu, Sep 10,
> > >> 2020 at
> > >> > > > 3:59
> > >> > > > > PM
> > >> > > > > > > > >> Satish
> > >> > > > > > > > >> > > > > Duggana <
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > >> satish.duggana@gmail.com
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > wrote:
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Thanks
> > >> Lucas/Justine
> > >> > > for
> > >> > > > > the
> > >> > > > > > > nice
> > >> > > > > > > > >> KIP.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > It has several
> > >> > benefits
> > >> > > > > which
> > >> > > > > > > > also
> > >> > > > > > > > >> > > include
> > >> > > > > > > > >> > > > > > >> > > simplifying
> > >> > > > > > > > >> > > > > > >> > > > > the
> > >> > > > > > > > >> > > > > > >> > > > > > >>> topic
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > deletion
> process
> > >> by
> > >> > > > > > controller
> > >> > > > > > > > and
> > >> > > > > > > > >> > logs
> > >> > > > > > > > >> > > > > > cleanup
> > >> > > > > > > > >> > > > > > >> by
> > >> > > > > > > > >> > > > > > >> > > > > brokers
> > >> > > > > > > > >> > > > > > >> > > > > > in
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > corner
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > cases.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Best,
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Satish.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > On Wed, Sep 9,
> > >> 2020
> > >> > at
> > >> > > > > 10:07
> > >> > > > > > PM
> > >> > > > > > > > >> > Justine
> > >> > > > > > > > >> > > > > > Olshan <
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > jolshan@confluent.io
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > wrote:
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Hello all,
> > it's
> > >> > been
> > >> > > > > > almost a
> > >> > > > > > > > >> year!
> > >> > > > > > > > >> > > I've
> > >> > > > > > > > >> > > > > > made
> > >> > > > > > > > >> > > > > > >> > some
> > >> > > > > > > > >> > > > > > >> > > > > > changes
> > >> > > > > > > > >> > > > > > >> > > > > > >>> to
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > this
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > KIP
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > and hope to
> > >> continue
> > >> > > the
> > >> > > > > > > > >> discussion.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > One of the
> > main
> > >> > > changes
> > >> > > > > > I've
> > >> > > > > > > > >> added
> > >> > > > > > > > >> > is
> > >> > > > > > > > >> > > > now
> > >> > > > > > > > >> > > > > > the
> > >> > > > > > > > >> > > > > > >> > > > metadata
> > >> > > > > > > > >> > > > > > >> > > > > > >>> response
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > will
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > include the
> > topic
> > >> ID
> > >> > > (as
> > >> > > > > > Colin
> > >> > > > > > > > >> > > suggested).
> > >> > > > > > > > >> > > > > > >> Clients
> > >> > > > > > > > >> > > > > > >> > > can
> > >> > > > > > > > >> > > > > > >> > > > > > >>> obtain the
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > topicID
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > of a given
> topic
> > >> > > through
> > >> > > > a
> > >> > > > > > > > >> > > > TopicDescription.
> > >> > > > > > > > >> > > > > > The
> > >> > > > > > > > >> > > > > > >> > > > topicId
> > >> > > > > > > > >> > > > > > >> > > > > > will
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > also be
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > included with
> > the
> > >> > > > > > > UpdateMetadata
> > >> > > > > > > > >> > > request.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Let me know
> > what
> > >> > you
> > >> > > > all
> > >> > > > > > > think.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Thank you,
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Justine
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > On
> 2019/09/13
> > >> > > 16:38:26,
> > >> > > > > > > "Colin
> > >> > > > > > > > >> > > McCabe" <
> > >> > > > > > > > >> > > > > > >> > > > > > cmccabe@apache.org
> > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > wrote:
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Hi Lucas,
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Thanks for
> > >> > tackling
> > >> > > > > this.
> > >> > > > > > > > >> Topic
> > >> > > > > > > > >> > IDs
> > >> > > > > > > > >> > > > > are a
> > >> > > > > > > > >> > > > > > >> > great
> > >> > > > > > > > >> > > > > > >> > > > > idea,
> > >> > > > > > > > >> > > > > > >> > > > > > >>> and
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > this
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > is
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > really good
> > >> writeup.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > For
> > >> > > > > > > /brokers/topics/[topic],
> > >> > > > > > > > >> the
> > >> > > > > > > > >> > > > schema
> > >> > > > > > > > >> > > > > > >> version
> > >> > > > > > > > >> > > > > > >> > > > > should
> > >> > > > > > > > >> > > > > > >> > > > > > be
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > bumped
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > version 3,
> > rather
> > >> > than
> > >> > > 2.
> > >> > > > > > > > KIP-455
> > >> > > > > > > > >> > > bumped
> > >> > > > > > > > >> > > > > the
> > >> > > > > > > > >> > > > > > >> > version
> > >> > > > > > > > >> > > > > > >> > > > of
> > >> > > > > > > > >> > > > > > >> > > > > > this
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > znode
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 2
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > already :)
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Given that
> > >> we're
> > >> > > > going
> > >> > > > > to
> > >> > > > > > > be
> > >> > > > > > > > >> > seeing
> > >> > > > > > > > >> > > > > these
> > >> > > > > > > > >> > > > > > >> > things
> > >> > > > > > > > >> > > > > > >> > > as
> > >> > > > > > > > >> > > > > > >> > > > > > >>> strings
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > as
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > lot
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > (in
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > logs, in
> > >> ZooKeeper,
> > >> > on
> > >> > > > the
> > >> > > > > > > > >> > command-line,
> > >> > > > > > > > >> > > > > > etc.),
> > >> > > > > > > > >> > > > > > >> > does
> > >> > > > > > > > >> > > > > > >> > > it
> > >> > > > > > > > >> > > > > > >> > > > > > make
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > sense to
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > use
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > base64 when
> > >> > converting
> > >> > > > them
> > >> > > > > > to
> > >> > > > > > > > >> > strings?
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Here is an
> > >> > example
> > >> > > of
> > >> > > > > the
> > >> > > > > > > hex
> > >> > > > > > > > >> > > > > > >> representation:
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >> > > > > > > > >> > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > And here
> is
> > an
> > >> > > > example
> > >> > > > > in
> > >> > > > > > > > >> base64.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >> > > > b8tRS7h4TJ2Vt43Dp85v2A
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > The base64
> > >> > version
> > >> > > > > saves
> > >> > > > > > 15
> > >> > > > > > > > >> > letters
> > >> > > > > > > > >> > > > (to
> > >> > > > > > > > >> > > > > be
> > >> > > > > > > > >> > > > > > >> > fair,
> > >> > > > > > > > >> > > > > > >> > > 4
> > >> > > > > > > > >> > > > > > >> > > > of
> > >> > > > > > > > >> > > > > > >> > > > > > >>> those
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > were
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > dashes that we
> > >> could
> > >> > > have
> > >> > > > > > > elided
> > >> > > > > > > > in
> > >> > > > > > > > >> > the
> > >> > > > > > > > >> > > > hex
> > >> > > > > > > > >> > > > > > >> > > > > > representation.)
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Another
> > thing
> > >> to
> > >> > > > > consider
> > >> > > > > > > is
> > >> > > > > > > > >> that
> > >> > > > > > > > >> > we
> > >> > > > > > > > >> > > > > > should
> > >> > > > > > > > >> > > > > > >> > > specify
> > >> > > > > > > > >> > > > > > >> > > > > > that
> > >> > > > > > > > >> > > > > > >> > > > > > >>> the
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > all-zeroes
> UUID
> > is
> > >> > not
> > >> > > a
> > >> > > > > > valid
> > >> > > > > > > > >> topic
> > >> > > > > > > > >> > > UUID.
> > >> > > > > > > > >> > > > > >  We
> > >> > > > > > > > >> > > > > > >> > can't
> > >> > > > > > > > >> > > > > > >> > > > use
> > >> > > > > > > > >> > > > > > >> > > > > > >>> null
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > for
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > this
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > because we
> can't
> > >> > pass a
> > >> > > > > null
> > >> > > > > > > UUID
> > >> > > > > > > > >> over
> > >> > > > > > > > >> > > the
> > >> > > > > > > > >> > > > > RPC
> > >> > > > > > > > >> > > > > > >> > > protocol
> > >> > > > > > > > >> > > > > > >> > > > > > >>> (there
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > is no
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > special
> pattern
> > >> for
> > >> > > null,
> > >> > > > > nor
> > >> > > > > > > do
> > >> > > > > > > > we
> > >> > > > > > > > >> > want
> > >> > > > > > > > >> > > > to
> > >> > > > > > > > >> > > > > > >> waste
> > >> > > > > > > > >> > > > > > >> > > space
> > >> > > > > > > > >> > > > > > >> > > > > > >>> reserving
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > such
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > pattern.)
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Maybe I
> > missed
> > >> > it,
> > >> > > > but
> > >> > > > > > did
> > >> > > > > > > > you
> > >> > > > > > > > >> > > > describe
> > >> > > > > > > > >> > > > > > >> > > "migration
> > >> > > > > > > > >> > > > > > >> > > > > > of...
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > existing
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > topic[s]
> without
> > >> > topic
> > >> > > > IDs"
> > >> > > > > > in
> > >> > > > > > > > >> detail
> > >> > > > > > > > >> > in
> > >> > > > > > > > >> > > > any
> > >> > > > > > > > >> > > > > > >> > section?
> > >> > > > > > > > >> > > > > > >> > > > It
> > >> > > > > > > > >> > > > > > >> > > > > > >>> seems
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > like
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > when
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > the new
> > controller
> > >> > > > becomes
> > >> > > > > > > > active,
> > >> > > > > > > > >> it
> > >> > > > > > > > >> > > > should
> > >> > > > > > > > >> > > > > > >> just
> > >> > > > > > > > >> > > > > > >> > > > > generate
> > >> > > > > > > > >> > > > > > >> > > > > > >>> random
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > UUIDs for
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > these, and
> write
> > >> the
> > >> > > > random
> > >> > > > > > > UUIDs
> > >> > > > > > > > >> back
> > >> > > > > > > > >> > > to
> > >> > > > > > > > >> > > > > > >> > ZooKeeper.
> > >> > > > > > > > >> > > > > > >> > > > It
> > >> > > > > > > > >> > > > > > >> > > > > > >>> would be
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > good
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > to
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > spell that
> out.
> > >> We
> > >> > > > should
> > >> > > > > > make
> > >> > > > > > > > it
> > >> > > > > > > > >> > clear
> > >> > > > > > > > >> > > > > that
> > >> > > > > > > > >> > > > > > >> this
> > >> > > > > > > > >> > > > > > >> > > > > happens
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > regardless
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > of
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > the
> inter-broker
> > >> > > protocol
> > >> > > > > > > version
> > >> > > > > > > > >> > (it's
> > >> > > > > > > > >> > > a
> > >> > > > > > > > >> > > > > > >> > compatible
> > >> > > > > > > > >> > > > > > >> > > > > > change).
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >> > > "LeaderAndIsrRequests
> > >> > > > > > > > >> including an
> > >> > > > > > > > >> > > > > > >> > > > is_every_partition
> > >> > > > > > > > >> > > > > > >> > > > > > >>> flag"
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > seems a
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > bit wordy.
> Can
> > we
> > >> > just
> > >> > > > > call
> > >> > > > > > > > these
> > >> > > > > > > > >> > "full
> > >> > > > > > > > >> > > > > > >> > > > > > >>> LeaderAndIsrRequests"?
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > Then
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > the
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > RPC field
> could
> > be
> > >> > > named
> > >> > > > > > > "full".
> > >> > > > > > > > >> > Also,
> > >> > > > > > > > >> > > it
> > >> > > > > > > > >> > > > > > would
> > >> > > > > > > > >> > > > > > >> > > > probably
> > >> > > > > > > > >> > > > > > >> > > > > > be
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > better
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > for the
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > RPC field to
> be
> > an
> > >> > enum
> > >> > > > of
> > >> > > > > {
> > >> > > > > > > > >> > > UNSPECIFIED,
> > >> > > > > > > > >> > > > > > >> > > INCREMENTAL,
> > >> > > > > > > > >> > > > > > >> > > > > FULL
> > >> > > > > > > > >> > > > > > >> > > > > > >>> }, so
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > that
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > we
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > can cleanly
> > handle
> > >> > old
> > >> > > > > > versions
> > >> > > > > > > > (by
> > >> > > > > > > > >> > > > treating
> > >> > > > > > > > >> > > > > > >> them
> > >> > > > > > > > >> > > > > > >> > as
> > >> > > > > > > > >> > > > > > >> > > > > > >>> UNSPECIFIED)
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > In the
> > >> > > > > > LeaderAndIsrRequest
> > >> > > > > > > > >> > section,
> > >> > > > > > > > >> > > > you
> > >> > > > > > > > >> > > > > > >> write
> > >> > > > > > > > >> > > > > > >> > "A
> > >> > > > > > > > >> > > > > > >> > > > > final
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > deletion
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > event
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > will be
> > secheduled
> > >> > for
> > >> > > X
> > >> > > > ms
> > >> > > > > > > after
> > >> > > > > > > > >> the
> > >> > > > > > > > >> > > > > > >> > > > LeaderAndIsrRequest
> > >> > > > > > > > >> > > > > > >> > > > > > was
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > first
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > received..."
> I
> > >> guess
> > >> > > > the X
> > >> > > > > > > was a
> > >> > > > > > > > >> > > > > placeholder
> > >> > > > > > > > >> > > > > > >> that
> > >> > > > > > > > >> > > > > > >> > > you
> > >> > > > > > > > >> > > > > > >> > > > > > >>> intended
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > replace
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > before
> posting?
> > :)
> > >> > In
> > >> > > > any
> > >> > > > > > > case,
> > >> > > > > > > > >> this
> > >> > > > > > > > >> > > > seems
> > >> > > > > > > > >> > > > > > like
> > >> > > > > > > > >> > > > > > >> > the
> > >> > > > > > > > >> > > > > > >> > > > kind
> > >> > > > > > > > >> > > > > > >> > > > > > of
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > thing
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > we'd
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > want a
> > >> configuration
> > >> > > for.
> > >> > > > > > > Let's
> > >> > > > > > > > >> > > describe
> > >> > > > > > > > >> > > > > that
> > >> > > > > > > > >> > > > > > >> > > > > > configuration
> > >> > > > > > > > >> > > > > > >> > > > > > >>> key
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > somewhere
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > in this KIP,
> > >> > including
> > >> > > > what
> > >> > > > > > its
> > >> > > > > > > > >> > default
> > >> > > > > > > > >> > > > > value
> > >> > > > > > > > >> > > > > > >> is.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > We should
> > >> > probably
> > >> > > > also
> > >> > > > > > > log a
> > >> > > > > > > > >> > bunch
> > >> > > > > > > > >> > > of
> > >> > > > > > > > >> > > > > > >> messages
> > >> > > > > > > > >> > > > > > >> > > at
> > >> > > > > > > > >> > > > > > >> > > > > WARN
> > >> > > > > > > > >> > > > > > >> > > > > > >>> level
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > when
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > something is
> > >> > scheduled
> > >> > > > for
> > >> > > > > > > > >> deletion,
> > >> > > > > > > > >> > as
> > >> > > > > > > > >> > > > > well.
> > >> > > > > > > > >> > > > > > >> > (Maybe
> > >> > > > > > > > >> > > > > > >> > > > > this
> > >> > > > > > > > >> > > > > > >> > > > > > >>> was
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > assumed, but
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > it would be
> good
> > >> to
> > >> > > > mention
> > >> > > > > > > it).
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > I feel
> like
> > >> there
> > >> > > > are a
> > >> > > > > > few
> > >> > > > > > > > >> > sections
> > >> > > > > > > > >> > > > > that
> > >> > > > > > > > >> > > > > > >> > should
> > >> > > > > > > > >> > > > > > >> > > be
> > >> > > > > > > > >> > > > > > >> > > > > > >>> moved to
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > "rejected
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > alternatives."
> > >> For
> > >> > > > > example,
> > >> > > > > > in
> > >> > > > > > > > the
> > >> > > > > > > > >> > > > > > DeleteTopics
> > >> > > > > > > > >> > > > > > >> > > > section,
> > >> > > > > > > > >> > > > > > >> > > > > > >>> since
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > we're
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > not
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > going to do
> > >> option 1
> > >> > or
> > >> > > > 2,
> > >> > > > > > > these
> > >> > > > > > > > >> > should
> > >> > > > > > > > >> > > be
> > >> > > > > > > > >> > > > > > moved
> > >> > > > > > > > >> > > > > > >> > into
> > >> > > > > > > > >> > > > > > >> > > > > > >>> "rejected
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > alternatives,"
> > >> > rather
> > >> > > > than
> > >> > > > > > > > >> appearing
> > >> > > > > > > > >> > > > > inline.
> > >> > > > > > > > >> > > > > > >> > > Another
> > >> > > > > > > > >> > > > > > >> > > > > case
> > >> > > > > > > > >> > > > > > >> > > > > > >>> is
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > the
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > "Should
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > we remove
> topic
> > >> name
> > >> > > from
> > >> > > > > the
> > >> > > > > > > > >> protocol
> > >> > > > > > > > >> > > > where
> > >> > > > > > > > >> > > > > > >> > > possible"
> > >> > > > > > > > >> > > > > > >> > > > > > >>> section.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > This
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > is
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > clearly
> > >> discussing a
> > >> > > > design
> > >> > > > > > > > >> > alternative
> > >> > > > > > > > >> > > > that
> > >> > > > > > > > >> > > > > > >> we're
> > >> > > > > > > > >> > > > > > >> > > not
> > >> > > > > > > > >> > > > > > >> > > > > > >>> proposing
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > implement:
> > >> removing
> > >> > the
> > >> > > > > topic
> > >> > > > > > > > name
> > >> > > > > > > > >> > from
> > >> > > > > > > > >> > > > > those
> > >> > > > > > > > >> > > > > > >> > > > protocols.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Is it
> really
> > >> > > > necessary
> > >> > > > > to
> > >> > > > > > > > have
> > >> > > > > > > > >> a
> > >> > > > > > > > >> > new
> > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> /admin/delete_topics_by_id
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > path
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > in ZooKeeper?
> > It
> > >> > seems
> > >> > > > > like
> > >> > > > > > we
> > >> > > > > > > > >> don't
> > >> > > > > > > > >> > > > really
> > >> > > > > > > > >> > > > > > >> need
> > >> > > > > > > > >> > > > > > >> > > this.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> Whenever
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > there is
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > a new
> > controller,
> > >> > we'll
> > >> > > > > send
> > >> > > > > > > out
> > >> > > > > > > > >> full
> > >> > > > > > > > >> > > > > > >> > > > > LeaderAndIsrRequests
> > >> > > > > > > > >> > > > > > >> > > > > > >>> which
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > will
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > trigger the
> > stale
> > >> > > topics
> > >> > > > to
> > >> > > > > > be
> > >> > > > > > > > >> cleaned
> > >> > > > > > > > >> > > up.
> > >> > > > > > > > >> > > > > >  The
> > >> > > > > > > > >> > > > > > >> > > active
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > controller
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > will
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > also send the
> > full
> > >> > > > > > > > >> LeaderAndIsrRequest
> > >> > > > > > > > >> > > to
> > >> > > > > > > > >> > > > > > >> brokers
> > >> > > > > > > > >> > > > > > >> > > that
> > >> > > > > > > > >> > > > > > >> > > > > are
> > >> > > > > > > > >> > > > > > >> > > > > > >>> just
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > starting
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > up.    So we
> > don't
> > >> > > really
> > >> > > > > > need
> > >> > > > > > > > this
> > >> > > > > > > > >> > kind
> > >> > > > > > > > >> > > > of
> > >> > > > > > > > >> > > > > > >> > two-phase
> > >> > > > > > > > >> > > > > > >> > > > > > commit
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > (send
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > out
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > >> StopReplicasRequest,
> > >> > > get
> > >> > > > > ACKs
> > >> > > > > > > > from
> > >> > > > > > > > >> all
> > >> > > > > > > > >> > > > > nodes,
> > >> > > > > > > > >> > > > > > >> > commit
> > >> > > > > > > > >> > > > > > >> > > by
> > >> > > > > > > > >> > > > > > >> > > > > > >>> removing
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > >> /admin/delete_topics
> > >> > > > node)
> > >> > > > > > any
> > >> > > > > > > > >> more.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > You
> mention
> > >> that
> > >> > > > > > > FetchRequest
> > >> > > > > > > > >> will
> > >> > > > > > > > >> > > now
> > >> > > > > > > > >> > > > > > >> include
> > >> > > > > > > > >> > > > > > >> > > UUID
> > >> > > > > > > > >> > > > > > >> > > > > to
> > >> > > > > > > > >> > > > > > >> > > > > > >>> avoid
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > issues
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > where requests
> > are
> > >> > made
> > >> > > > to
> > >> > > > > > > stale
> > >> > > > > > > > >> > > > partitions.
> > >> > > > > > > > >> > > > > > >> > > However,
> > >> > > > > > > > >> > > > > > >> > > > > > >>> adding a
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > UUID
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> MetadataRequest
> > is
> > >> > > listed
> > >> > > > > as
> > >> > > > > > > > future
> > >> > > > > > > > >> > > work,
> > >> > > > > > > > >> > > > > out
> > >> > > > > > > > >> > > > > > of
> > >> > > > > > > > >> > > > > > >> > > scope
> > >> > > > > > > > >> > > > > > >> > > > > for
> > >> > > > > > > > >> > > > > > >> > > > > > >>> this
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > KIP.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > How
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > will the
> client
> > >> learn
> > >> > > > what
> > >> > > > > > the
> > >> > > > > > > > >> topic
> > >> > > > > > > > >> > > UUID
> > >> > > > > > > > >> > > > > is,
> > >> > > > > > > > >> > > > > > if
> > >> > > > > > > > >> > > > > > >> > the
> > >> > > > > > > > >> > > > > > >> > > > > > metadata
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > response
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > doesn't
> include
> > >> that
> > >> > > > > > > information?
> > >> > > > > > > > >> It
> > >> > > > > > > > >> > > > seems
> > >> > > > > > > > >> > > > > > like
> > >> > > > > > > > >> > > > > > >> > > adding
> > >> > > > > > > > >> > > > > > >> > > > > the
> > >> > > > > > > > >> > > > > > >> > > > > > >>> UUID
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > to
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> MetadataResponse
> > >> > would
> > >> > > be
> > >> > > > > an
> > >> > > > > > > > >> > improvement
> > >> > > > > > > > >> > > > > here
> > >> > > > > > > > >> > > > > > >> that
> > >> > > > > > > > >> > > > > > >> > > > might
> > >> > > > > > > > >> > > > > > >> > > > > > not
> > >> > > > > > > > >> > > > > > >> > > > > > >>> be
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > too
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > hard to
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > make.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > best,
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Colin
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > On Mon,
> Sep
> > 9,
> > >> > > 2019,
> > >> > > > at
> > >> > > > > > > > 17:48,
> > >> > > > > > > > >> > > Ryanne
> > >> > > > > > > > >> > > > > > Dolan
> > >> > > > > > > > >> > > > > > >> > > wrote:
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > Lucas,
> > this
> > >> > would
> > >> > > > be
> > >> > > > > > > great.
> > >> > > > > > > > >> I've
> > >> > > > > > > > >> > > run
> > >> > > > > > > > >> > > > > > into
> > >> > > > > > > > >> > > > > > >> > > issues
> > >> > > > > > > > >> > > > > > >> > > > > with
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > topics
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > being
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> > resurrected
> > >> > > > > > accidentally,
> > >> > > > > > > > >> since
> > >> > > > > > > > >> > a
> > >> > > > > > > > >> > > > > client
> > >> > > > > > > > >> > > > > > >> > cannot
> > >> > > > > > > > >> > > > > > >> > > > > > easily
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > distinguish
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > between
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > a
> deleted
> > >> topic
> > >> > > > and a
> > >> > > > > > new
> > >> > > > > > > > >> topic
> > >> > > > > > > > >> > > with
> > >> > > > > > > > >> > > > > the
> > >> > > > > > > > >> > > > > > >> same
> > >> > > > > > > > >> > > > > > >> > > > name.
> > >> > > > > > > > >> > > > > > >> > > > > > I'd
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > need
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > the
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > ID
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> accessible
> > >> from
> > >> > > the
> > >> > > > > > > client
> > >> > > > > > > > to
> > >> > > > > > > > >> > > solve
> > >> > > > > > > > >> > > > > that
> > >> > > > > > > > >> > > > > > >> > issue,
> > >> > > > > > > > >> > > > > > >> > > > but
> > >> > > > > > > > >> > > > > > >> > > > > > >>> this
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > is a
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > good
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > first
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > step.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > Ryanne
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > On Wed,
> > Sep
> > >> 4,
> > >> > > 2019
> > >> > > > > at
> > >> > > > > > > 1:41
> > >> > > > > > > > >> PM
> > >> > > > > > > > >> > > Lucas
> > >> > > > > > > > >> > > > > > >> > > Bradstreet <
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> lucas@confluent.io>
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > wrote:
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Hi
> all,
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > I
> would
> > >> like
> > >> > to
> > >> > > > > kick
> > >> > > > > > > off
> > >> > > > > > > > >> > > > discussion
> > >> > > > > > > > >> > > > > of
> > >> > > > > > > > >> > > > > > >> > > KIP-516,
> > >> > > > > > > > >> > > > > > >> > > > > an
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > implementation
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > of topic
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > IDs
> for
> > >> > Kafka.
> > >> > > > > Topic
> > >> > > > > > > IDs
> > >> > > > > > > > >> aim
> > >> > > > > > > > >> > to
> > >> > > > > > > > >> > > > > solve
> > >> > > > > > > > >> > > > > > >> topic
> > >> > > > > > > > >> > > > > > >> > > > > > >>> uniqueness
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > problems in
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Kafka,
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > where
> > >> > referring
> > >> > > > to
> > >> > > > > a
> > >> > > > > > > > topic
> > >> > > > > > > > >> by
> > >> > > > > > > > >> > > name
> > >> > > > > > > > >> > > > > > >> alone is
> > >> > > > > > > > >> > > > > > >> > > > > > >>> insufficient.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Such
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > cases
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> include
> > >> when
> > >> > a
> > >> > > > > topic
> > >> > > > > > > has
> > >> > > > > > > > >> been
> > >> > > > > > > > >> > > > > deleted
> > >> > > > > > > > >> > > > > > >> and
> > >> > > > > > > > >> > > > > > >> > > > > recreated
> > >> > > > > > > > >> > > > > > >> > > > > > >>> with
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > the
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > same
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > name.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Unique
> > >> > > > identifiers
> > >> > > > > > will
> > >> > > > > > > > >> help
> > >> > > > > > > > >> > > > > simplify
> > >> > > > > > > > >> > > > > > >> and
> > >> > > > > > > > >> > > > > > >> > > > improve
> > >> > > > > > > > >> > > > > > >> > > > > > >>> Kafka's
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > topic
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > deletion
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> process,
> > >> as
> > >> > > well
> > >> > > > as
> > >> > > > > > > > prevent
> > >> > > > > > > > >> > > cases
> > >> > > > > > > > >> > > > > > where
> > >> > > > > > > > >> > > > > > >> > > brokers
> > >> > > > > > > > >> > > > > > >> > > > > may
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > incorrectly
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > interact
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > with
> > stale
> > >> > > > versions
> > >> > > > > > of
> > >> > > > > > > > >> topics.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > >> > > > > > > > >> > > > > > >> > > > > >
> > >> > > > > > > > >> > > > > > >> > > > >
> > >> > > > > > > > >> > > > > > >> > > >
> > >> > > > > > > > >> > > > > > >> > >
> > >> > > > > > > > >> > > > > > >> >
> > >> > > > > > > > >> > > > > > >>
> > >> > > > > > > > >> > > > > >
> > >> > > > > > > > >> > > > >
> > >> > > > > > > > >> > > >
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> >
> > >> > > > > > > > >>
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> Looking
> > >> > forward
> > >> > > > to
> > >> > > > > > your
> > >> > > > > > > > >> > > thoughts.
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Lucas
> > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > >> > > > > > > > >> > > > > > >> > > > > > >>> >
> > >> > > > > > > > >> > > > > > >> > > > > > >>>
> > >> > > > > > > > >> > > > > > >> > > > > > >>
> > >> > > > > > > > >> > > > > > >> > > > > >
> > >> > > > > > > > >> > > > > > >> > > > >
> > >> > > > > > > > >> > > > > > >> > > >
> > >> > > > > > > > >> > > > > > >> > >
> > >> > > > > > > > >> > > > > > >> >
> > >> > > > > > > > >> > > > > > >>
> > >> > > > > > > > >> > > > > > >
> > >> > > > > > > > >> > > > > >
> > >> > > > > > > > >> > > > >
> > >> > > > > > > > >> > > >
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> >
> > >> > > > > > > > >>
> > >> > > > > > > > >
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> >
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Jun Rao <ju...@confluent.io>]

Hi, Justine,

Thanks for the summary. Just a few minor comments blow.

30.  {"name": "id", "type": "string", "doc": "version id"}}: The doc should
say UUID. The issue seems unfixed.

40. Since UUID is public facing, could you include its definition?

41. StopReplicaResponse still includes the topic field.

42. "It is unnecessary to include the name of the topic in the following
Request/Response calls" It would be useful to include all modified requests
(e.g. produce) in the list below.

Thanks,

Jun


On Wed, Sep 30, 2020 at 10:17 AM Justine Olshan <jo...@confluent.io>
wrote:

> Hello again,
>
> I've taken some time to discuss some of the remaining points brought up by
> the previous emails offline. Here are some of the conclusions.
>
> 1. Directory Structure:
> There was some talk about whether the directory structure should be changed
> to replace all topic names with topic IDs.
> This will be a useful change, but will prevent downgrades. It will be best
> to wait until a major release, likely alongside KIP-500 changes that will
> prevent downgrades. I've updated the KIP to include this change with the
> note about migration and deprecation.
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-log.dirlayout
>
> 2. Partition Metadata file
> There was some disagreement about the inclusion of the partition metadata
> file.
> This file will be needed to persist the topic ID, especially while we still
> have the old directory structure. Even after the changes, this file can be
> useful for debugging and recovery.
> Creating a single mapping file from topic names to topic IDs was
> considered, but ultimately scrapped as it would not be as easy to maintain.
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PartitionMetadatafile
>
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PersistingTopicIDs
>
> 3. Produce Protocols
> After some further discussion, this replacing the topic name with topic ID
> in these protocols has been added to the KIP.
> This will cut down on the size of the protocol. Since changes to fetch are
> included, it does make sense to update these protocols.
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-Produce
>
> 4. KIP-500 Compatibility
> After some discussion with those involved with KIP-500, it seems best to
> use a sentinel topic ID for the metadata topic that is used before the
> first controller election.
> We had an issue where this metadata topic may not be assigned an ID before
> utilizing the Vote and Fetch protocols. It was decided to reserve a unique
> ID for this topic to be used until the controller could give the topic a
> unique ID.
> Having the topic keep the sentinel ID (not be reassigned to a unique ID)
> was considered, but it seemed like a good idea to leave the option open for
> the metadata topic to have a unique ID in cases where it would need to be
> differentiated from other clusters' metadata topics. (ex. tiered storage)
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-CompatibilitywithKIP-500
>
> I've also split up the KIP into sub-tasks on the JIRA. Hopefully this will
> provide a better idea about what tasks we have, and eventually provide a
> place to see what's done and what is left.
> If there is a task I am missing, please let me know!
> https://issues.apache.org/jira/browse/KAFKA-8872
>
> Of course, these decisions are not set in stone, and I would love to hear
> any feedback.
>
> Thanks,
> Justine
>
> On Mon, Sep 28, 2020 at 11:38 AM Justine Olshan <jo...@confluent.io>
> wrote:
>
> > Hello all,
> >
> > I just wanted to follow up on this discussion. Did we come to an
> > understanding about the directory structure?
> >
> > I think the biggest question here is what is acceptable to leave out due
> > to scope vs. what is considered to be too much tech debt.
> > This KIP is already pretty large with the number of changes, but it also
> > makes sense to do things correctly, so I'd love to hear everyone's
> thoughts.
> >
> > Thanks,
> > Justine
> >
> > On Fri, Sep 25, 2020 at 8:19 AM Lucas Bradstreet <lu...@confluent.io>
> > wrote:
> >
> >> Hi Ismael,
> >>
> >> If you do not store it in a metadata file or in the directory structure
> >> would you then
> >> require the LeaderAndIsrRequest following startup to give you some
> notion
> >> of
> >> topic name in memory? We would still need this information for the older
> >> protocols, but
> >> perhaps this is what's meant by tech debt.
> >>
> >> Once we're free of the old non-topicID requests then I think you
> wouldn't
> >> need to retain the topic name.
> >> I think the ability to easily look up topic names associated with
> >> partition
> >> directories would still be missed
> >> when diagnosing issues, though maybe it wouldn't be a deal breaker with
> >> the
> >> right tooling.
> >>
> >> Thanks,
> >>
> >> Lucas
> >>
> >> On Fri, Sep 25, 2020 at 7:55 AM Ismael Juma <is...@juma.me.uk> wrote:
> >>
> >> > Hi Lucas,
> >> >
> >> > Why would you include the name and id? I think you'd want to remove
> the
> >> > name from the directory name right? Jason's suggestion was that if you
> >> > remove the name from the directory, then why would you need the id
> name
> >> > mapping file?
> >> >
> >> > Ismael
> >> >
> >> > On Thu, Sep 24, 2020 at 4:24 PM Lucas Bradstreet <lu...@confluent.io>
> >> > wrote:
> >> >
> >> > > > 2. Part of the usage of the file is to have persistent storage of
> >> the
> >> > > topic
> >> > > ID and use it to compare with the ID supplied in the LeaderAndIsr
> >> > Request.
> >> > > There is some discussion in the KIP about changes to the directory
> >> > > structure, but I believe directory changes were considered to be out
> >> of
> >> > > scope when the KIP was written.
> >> > >
> >> > >
> >> > > Yeah, I was hoping to get a better understanding of why it was taken
> >> out
> >> > of
> >> > > scope. Perhaps Lucas Bradstreet might have more insight about the
> >> > decision.
> >> > > Basically my point is that we have to create additional
> infrastructure
> >> > here
> >> > > to support the name/id mapping, so I wanted to understand if we just
> >> > > consider this a sort of tech debt. It would be useful to cover how
> we
> >> > > handle the case when this file gets corrupted. Seems like we just
> >> have to
> >> > > trust that it matches whatever the controller tells us and rewrite
> it?
> >> > >
> >> > >
> >> > > Hi Jason, Justine,
> >> > >
> >> > > My thought process is that we will likely need the metadata file
> >> whether
> >> > we
> >> > > rename the directories or not.
> >> > > Linux supports filenames of up to 255 bytes and that would not be
> >> enough
> >> > to
> >> > > support a directory name
> >> > >  including both the name and topic ID. Given that fact, it seemed
> >> > > reasonable to add the metadata file
> >> > > and leave the directory rename until some time in the future
> (possibly
> >> > > never).
> >> > >
> >> > > If the file does get corrupted I think you're right. We would either
> >> have
> >> > > to trust it matches what the controller tells us
> >> > >  or error out and let an administrator resolve it by checking across
> >> > > replicas for consistency.
> >> > >
> >> > > Lucas
> >> > >
> >> > >
> >> > > On Thu, Sep 24, 2020 at 3:41 PM Jason Gustafson <jason@confluent.io
> >
> >> > > wrote:
> >> > >
> >> > > > Thanks Justine. Responses below:
> >> > > >
> >> > > > > 1. Yes, the directory will still be based on the topic names.
> >> > > > LeaderAndIsrRequest is one of the few requests that will still
> >> contain
> >> > > the
> >> > > > topic name. So I think we have this covered. Sorry for confusion.
> >> > > >
> >> > > > Ah, you're right. My eyes passed right over the field.
> >> > > >
> >> > > > > 2. Part of the usage of the file is to have persistent storage
> of
> >> the
> >> > > > topic
> >> > > > ID and use it to compare with the ID supplied in the LeaderAndIsr
> >> > > Request.
> >> > > > There is some discussion in the KIP about changes to the directory
> >> > > > structure, but I believe directory changes were considered to be
> >> out of
> >> > > > scope when the KIP was written.
> >> > > >
> >> > > > Yeah, I was hoping to get a better understanding of why it was
> taken
> >> > out
> >> > > of
> >> > > > scope. Perhaps Lucas Bradstreet might have more insight about the
> >> > > decision.
> >> > > > Basically my point is that we have to create additional
> >> infrastructure
> >> > > here
> >> > > > to support the name/id mapping, so I wanted to understand if we
> just
> >> > > > consider this a sort of tech debt. It would be useful to cover how
> >> we
> >> > > > handle the case when this file gets corrupted. Seems like we just
> >> have
> >> > to
> >> > > > trust that it matches whatever the controller tells us and rewrite
> >> it?
> >> > > >
> >> > > > > 3. I think this is a good point, but I again I wonder about the
> >> scope
> >> > > of
> >> > > > the KIP. Most of the changes mentioned in the KIP are for
> supporting
> >> > > topic
> >> > > > deletion and I believe that is why the produce request was listed
> >> under
> >> > > > future work.
> >> > > >
> >> > > > That's fair. I brought it up since `Fetch` is already included. If
> >> > we've
> >> > > > got `Metadata` and `Fetch`, seems we may as well do `Produce` and
> >> save
> >> > an
> >> > > > extra kip. No strong objection though if you want to leave it out.
> >> > > >
> >> > > >
> >> > > > -Jason
> >> > > >
> >> > > >
> >> > > > On Thu, Sep 24, 2020 at 3:26 PM Justine Olshan <
> >> jolshan@confluent.io>
> >> > > > wrote:
> >> > > >
> >> > > > > Hi Jason,
> >> > > > >
> >> > > > > Thanks for your comments.
> >> > > > >
> >> > > > > 1. Yes, the directory will still be based on the topic names.
> >> > > > > LeaderAndIsrRequest is one of the few requests that will still
> >> > contain
> >> > > > the
> >> > > > > topic name. So I think we have this covered. Sorry for
> confusion.
> >> > > > >
> >> > > > > 2. Part of the usage of the file is to have persistent storage
> of
> >> the
> >> > > > topic
> >> > > > > ID and use it to compare with the ID supplied in the
> LeaderAndIsr
> >> > > > Request.
> >> > > > > There is some discussion in the KIP about changes to the
> directory
> >> > > > > structure, but I believe directory changes were considered to be
> >> out
> >> > of
> >> > > > > scope when the KIP was written.
> >> > > > >
> >> > > > > 3. I think this is a good point, but I again I wonder about the
> >> scope
> >> > > of
> >> > > > > the KIP. Most of the changes mentioned in the KIP are for
> >> supporting
> >> > > > topic
> >> > > > > deletion and I believe that is why the produce request was
> listed
> >> > under
> >> > > > > future work.
> >> > > > >
> >> > > > > 4. This sounds like it might be a good solution, but I will need
> >> to
> >> > > > discuss
> >> > > > > more with KIP-500 folks to get the details right.
> >> > > > >
> >> > > > > Thanks,
> >> > > > > Justine
> >> > > > >
> >> > > > > On Thu, Sep 24, 2020 at 12:30 PM Jason Gustafson <
> >> jason@confluent.io
> >> > >
> >> > > > > wrote:
> >> > > > >
> >> > > > > > Hi Justine,
> >> > > > > >
> >> > > > > > Thanks for picking up this work. I have a few
> >> questions/comments:
> >> > > > > >
> >> > > > > > 1. It sounds like the directory structure is still going to be
> >> > based
> >> > > on
> >> > > > > > topic names. Do I have that right? One complication is that
> the
> >> > > > > > LeaderAndIsr request does not include the topic name any
> longer.
> >> > This
> >> > > > > means
> >> > > > > > that a replica must wait for the UpdateMetadata request to
> >> arrive
> >> > > with
> >> > > > > the
> >> > > > > > topic name mapping before it can create the directory. I
> wonder
> >> if
> >> > it
> >> > > > > would
> >> > > > > > be simpler to avoid assumptions on the ordering of
> >> UpdateMetadata
> >> > and
> >> > > > let
> >> > > > > > LeaderAndIsr include the topic name as well. Feels like we are
> >> not
> >> > > > saving
> >> > > > > > that much by excluding it.
> >> > > > > >
> >> > > > > > 2. On a related note, it seems that the reason we have the
> >> > partition
> >> > > > > > metadata file is because we are not changing the directory
> >> > structure.
> >> > > > We
> >> > > > > > need it so that we remember which directories map to which
> topic
> >> > id.
> >> > > I
> >> > > > > > think it would be useful to add some detail about why changing
> >> the
> >> > > > > > directory layout was rejected. Basically trying to understand
> >> how
> >> > > much
> >> > > > > > effort we are saving by skipping this step if we have to
> >> implement
> >> > > this
> >> > > > > > additional bookkeeping. One concern I have for example is that
> >> the
> >> > > > > > partition metadata file gets corrupt and then all bets are off
> >> as
> >> > far
> >> > > > as
> >> > > > > > topic consistency.
> >> > > > > >
> >> > > > > > 3. Is it worth adding support now for topic ids in the
> `Produce`
> >> > > > request
> >> > > > > > now? Overhead is mentioned as one of the motivations and the
> >> best
> >> > > APIs
> >> > > > to
> >> > > > > > get savings from are `Produce` and `Fetch`.
> >> > > > > >
> >> > > > > > 4. When it comes to bootstrapping the metadata topic with
> >> respect
> >> > to
> >> > > > > > KIP-500, one suggestion would be to reserve a sentinel topic
> ID
> >> > which
> >> > > > can
> >> > > > > > be used initially by new brokers. When the first controller is
> >> > > elected,
> >> > > > > it
> >> > > > > > can assign a topicId to the metadata topic. This is a bit
> >> similar
> >> > to
> >> > > > how
> >> > > > > we
> >> > > > > > were planning to generate the clusterId.
> >> > > > > >
> >> > > > > > Thanks,
> >> > > > > > Jason
> >> > > > > >
> >> > > > > >
> >> > > > > > On Thu, Sep 24, 2020 at 11:10 AM Jun Rao <ju...@confluent.io>
> >> wrote:
> >> > > > > >
> >> > > > > > > Hi, Justine,
> >> > > > > > >
> >> > > > > > > Thanks for the updated KIP. A few more comments below.
> >> > > > > > >
> >> > > > > > > 30.  {"name": "id", "type": "string", "doc": "version id"}}:
> >> The
> >> > > doc
> >> > > > > > should
> >> > > > > > > say UUID.
> >> > > > > > >
> >> > > > > > > 31. LeaderAndIsrResponse v5 and StopReplicaResponse v4 : It
> >> seems
> >> > > > there
> >> > > > > > is
> >> > > > > > > no need to add topic_id at partitions level.
> >> > > > > > >
> >> > > > > > > 32. Regarding partition metadata file. Perhaps the key can
> be
> >> a
> >> > > > single
> >> > > > > > > word, sth like the following.
> >> > > > > > > version: 0
> >> > > > > > > topic_id: 46bdb63f-9e8d-4a38-bf7b-ee4eb2a794e4
> >> > > > > > >
> >> > > > > > > 33. Another tricky thing that I realized is how to support
> the
> >> > > > metadata
> >> > > > > > > topic introduced in KIP-595. It's a bit tricky to assign a
> >> UUID
> >> > to
> >> > > > the
> >> > > > > > > metadata topic since we have a chicken-and-egg problem. The
> >> > > > controller
> >> > > > > > > needs to persist the UUID in the metadata topic in order to
> >> > assign
> >> > > > one
> >> > > > > > > successfully, but the metadata topic is needed to elect a
> >> > > controller
> >> > > > > > > (KIP-631). So, this probably needs a bit more thought.
> >> > > > > > >
> >> > > > > > > Jun
> >> > > > > > >
> >> > > > > > > On Thu, Sep 24, 2020 at 3:04 AM Ismael Juma <
> >> ismael@juma.me.uk>
> >> > > > wrote:
> >> > > > > > >
> >> > > > > > > > Also, can we provide more details on how the Partition
> >> Metadata
> >> > > > file
> >> > > > > > will
> >> > > > > > > > be used?
> >> > > > > > > >
> >> > > > > > > > Ismael
> >> > > > > > > >
> >> > > > > > > > On Thu, Sep 24, 2020 at 3:01 AM Ismael Juma <
> >> ismael@juma.me.uk
> >> > >
> >> > > > > wrote:
> >> > > > > > > >
> >> > > > > > > > > Hi Justine,
> >> > > > > > > > >
> >> > > > > > > > > I think we need to update the "Rejected Alternatives"
> >> section
> >> > > to
> >> > > > > take
> >> > > > > > > > into
> >> > > > > > > > > account that the proposal now removes the topic name
> from
> >> the
> >> > > > fetch
> >> > > > > > > > > request. Also, if we are removing it from the Fetch
> >> request,
> >> > > does
> >> > > > > it
> >> > > > > > > make
> >> > > > > > > > > sense not to remove it from similar requests like
> >> > > > > ListOffsetRequest?
> >> > > > > > > > >
> >> > > > > > > > > Ismael
> >> > > > > > > > >
> >> > > > > > > > > On Thu, Sep 24, 2020 at 2:46 AM David Jacot <
> >> > > djacot@confluent.io
> >> > > > >
> >> > > > > > > wrote:
> >> > > > > > > > >
> >> > > > > > > > >> Hi Justine,
> >> > > > > > > > >>
> >> > > > > > > > >> Thanks for the KIP. I finally had time to read it :).
> It
> >> is
> >> > a
> >> > > > > great
> >> > > > > > > > >> improvement.
> >> > > > > > > > >>
> >> > > > > > > > >> I have a few comments/questions:
> >> > > > > > > > >>
> >> > > > > > > > >> 1. It seems that the schema of the StopReplicaRequest
> is
> >> > > > slightly
> >> > > > > > > > >> outdated.
> >> > > > > > > > >> We
> >> > > > > > > > >> did some changes as part of KIP-570. V3 is already
> >> organized
> >> > > by
> >> > > > > > > topics.
> >> > > > > > > > >>
> >> > > > > > > > >> 2. I just want to make sure that I understand the
> >> > > reconciliation
> >> > > > > > > > >> logic correctly. When
> >> > > > > > > > >> an "INCREMENTAL" LeaderAndIsr Request is received, the
> >> > broker
> >> > > > will
> >> > > > > > > also
> >> > > > > > > > >> reconcile
> >> > > > > > > > >> when the local uuid does not match the uuid in the
> >> request,
> >> > > > right?
> >> > > > > > In
> >> > > > > > > > this
> >> > > > > > > > >> case, the
> >> > > > > > > > >> local log is staged for deletion.
> >> > > > > > > > >>
> >> > > > > > > > >> 3. In the documentation of the `
> >> delete.stale.topic.delay.ms
> >> > `
> >> > > > > > config,
> >> > > > > > > it
> >> > > > > > > > >> says "When a
> >> > > > > > > > >> FULL LeaderAndIsrRequest is received..." but I suppose
> it
> >> > > > applies
> >> > > > > to
> >> > > > > > > > both
> >> > > > > > > > >> types.
> >> > > > > > > > >>
> >> > > > > > > > >> Best,
> >> > > > > > > > >> David
> >> > > > > > > > >>
> >> > > > > > > > >> On Thu, Sep 24, 2020 at 1:40 AM Justine Olshan <
> >> > > > > > jolshan@confluent.io>
> >> > > > > > > > >> wrote:
> >> > > > > > > > >>
> >> > > > > > > > >> > Hi Jun,
> >> > > > > > > > >> >
> >> > > > > > > > >> > Thanks for the comments. I apologize for some of the
> >> typos
> >> > > and
> >> > > > > > > > >> confusion.
> >> > > > > > > > >> > I’ve updated the KIP to fix some of the issues you
> >> > > mentioned.
> >> > > > > > > > >> >
> >> > > > > > > > >> > 20.2 I’ve changed the type to String.
> >> > > > > > > > >> > 20.1/3 I’ve updated the TopicZNode to fix formatting
> >> and
> >> > > > reflect
> >> > > > > > the
> >> > > > > > > > >> latest
> >> > > > > > > > >> > version before this change.
> >> > > > > > > > >> >
> >> > > > > > > > >> > 21. You are correct and I’ve removed this line. I’ve
> >> also
> >> > > > added
> >> > > > > a
> >> > > > > > > line
> >> > > > > > > > >> > mentioning an IBP bump is necessary for migration
> >> > > > > > > > >> >
> >> > > > > > > > >> > 22. I think the wording was unclear but your summary
> is
> >> > what
> >> > > > was
> >> > > > > > > > >> intended
> >> > > > > > > > >> > by this line. I’ve updated to make this point more
> >> clear.
> >> > > > > “Remove
> >> > > > > > > > >> deleted
> >> > > > > > > > >> > topics from replicas by sending StopReplicaRequest V3
> >> > before
> >> > > > the
> >> > > > > > IBP
> >> > > > > > > > >> bump
> >> > > > > > > > >> > using the old logic, and using V4 and the new logic
> >> with
> >> > > topic
> >> > > > > IDs
> >> > > > > > > > after
> >> > > > > > > > >> > the IBP bump.”
> >> > > > > > > > >> >
> >> > > > > > > > >> > 23. I’ve removed the unspecified type from the KIP
> and
> >> > > mention
> >> > > > > > that
> >> > > > > > > > IBP
> >> > > > > > > > >> > will be used to handle this request. “IBP will be
> used
> >> to
> >> > > > > > determine
> >> > > > > > > > >> whether
> >> > > > > > > > >> > this new form of the request will be used. For older
> >> > > requests,
> >> > > > > we
> >> > > > > > > will
> >> > > > > > > > >> > ignore this field and default to previous behavior.”
> >> > > > > > > > >> >
> >> > > > > > > > >> > 24. I’ve fixed this typo.
> >> > > > > > > > >> >
> >> > > > > > > > >> > 25. I've added a topics at a higher level for
> >> > > > > LeaderAndIsrResponse
> >> > > > > > > v5,
> >> > > > > > > > >> > StopReplicaResponse v4. I've also changed
> >> > StopReplicaRequest
> >> > > > v4
> >> > > > > to
> >> > > > > > > > have
> >> > > > > > > > >> > topics at a higher level.
> >> > > > > > > > >> >
> >> > > > > > > > >> > 26. I’ve updated forgotten_topics_data--added the
> >> topic ID
> >> > > and
> >> > > > > > > removed
> >> > > > > > > > >> the
> >> > > > > > > > >> > topic name
> >> > > > > > > > >> >
> >> > > > > > > > >> > 27. I’ve decided on plain text, and I’ve added an
> >> example.
> >> > > > > > > > >> >
> >> > > > > > > > >> > 28. I’ve added this idea to future work.
> >> > > > > > > > >> >
> >> > > > > > > > >> > Thanks again for taking a look,
> >> > > > > > > > >> >
> >> > > > > > > > >> > Justine
> >> > > > > > > > >> >
> >> > > > > > > > >> > On Wed, Sep 23, 2020 at 10:28 AM Jun Rao <
> >> > jun@confluent.io>
> >> > > > > > wrote:
> >> > > > > > > > >> >
> >> > > > > > > > >> > > Hi, Justine,
> >> > > > > > > > >> > >
> >> > > > > > > > >> > > Thanks for the response. Made another pass. A few
> >> more
> >> > > > > comments
> >> > > > > > > > below.
> >> > > > > > > > >> > >
> >> > > > > > > > >> > > 20. znode schema:
> >> > > > > > > > >> > > 20.1 It seems that {"name": "version", "type":
> "int",
> >> > > "id":
> >> > > > > > > "UUID",
> >> > > > > > > > >> > "doc":
> >> > > > > > > > >> > > "version id"} should be {"name": "version", "type":
> >> > > "int"},
> >> > > > > > > {"name":
> >> > > > > > > > >> > "id",
> >> > > > > > > > >> > > "type": "UUID", "doc": "version id"}.
> >> > > > > > > > >> > > 20.2 The znode format is JSON which doesn't have
> UUID
> >> > > type.
> >> > > > So
> >> > > > > > the
> >> > > > > > > > >> type
> >> > > > > > > > >> > > probably should be string?
> >> > > > > > > > >> > > 20.3 Also, the existing format used seems outdated.
> >> It
> >> > > > should
> >> > > > > > have
> >> > > > > > > > the
> >> > > > > > > > >> > > following format.
> >> > > > > > > > >> > >     Json.encodeAsBytes(Map(
> >> > > > > > > > >> > >       "version" -> 2,
> >> > > > > > > > >> > >       "partitions" -> replicaAssignmentJson.asJava,
> >> > > > > > > > >> > >       "adding_replicas" ->
> >> > > > > addingReplicasAssignmentJson.asJava,
> >> > > > > > > > >> > >       "removing_replicas" ->
> >> > > > > > removingReplicasAssignmentJson.asJava
> >> > > > > > > > >> > >     ).asJava)
> >> > > > > > > > >> > >   }
> >> > > > > > > > >> > >
> >> > > > > > > > >> > > 21. Migration: The KIP says "The migration process
> >> can
> >> > > take
> >> > > > > > place
> >> > > > > > > > >> without
> >> > > > > > > > >> > > an inter-broker protocol bump, as the format stored
> >> in
> >> > > > > > > > >> > > /brokers/topics/[topic] will be compatible with
> older
> >> > > broker
> >> > > > > > > > >> versions."
> >> > > > > > > > >> > > However, since we are bumping up the version of
> >> > > inter-broker
> >> > > > > > > > >> requests, it
> >> > > > > > > > >> > > seems that we need to use IBP for migration.
> >> > > > > > > > >> > >
> >> > > > > > > > >> > > 22. The KIP says "Remove deleted topics from
> >> replicas by
> >> > > > > sending
> >> > > > > > > > >> > > StopReplicaRequest V3 for any topics which do not
> >> > contain
> >> > > a
> >> > > > > > topic
> >> > > > > > > > ID,
> >> > > > > > > > >> and
> >> > > > > > > > >> > > V4 for any topics which do contain a topic ID.".
> >> > However,
> >> > > if
> >> > > > > we
> >> > > > > > > use
> >> > > > > > > > >> IBP,
> >> > > > > > > > >> > it
> >> > > > > > > > >> > > seems that the controller will either send
> >> > > > StopReplicaRequest
> >> > > > > V3
> >> > > > > > > > >> > > or StopReplicaRequest V4, but never mixed V3 and V4
> >> for
> >> > > > > > different
> >> > > > > > > > >> topics.
> >> > > > > > > > >> > > Basically, before the IBP bump, V3 will be used.
> >> After
> >> > the
> >> > > > IBP
> >> > > > > > > bump,
> >> > > > > > > > >> > > topicId will be created and V4 will be used.
> >> > > > > > > > >> > >
> >> > > > > > > > >> > > 23. Given that we depend on IBP, do we still need
> "0
> >> > > > > > UNSPECIFIED"
> >> > > > > > > > >> > > in LeaderAndIsr?
> >> > > > > > > > >> > >
> >> > > > > > > > >> > > 24. LeaderAndIsrResponse v5 : It still has the
> topic
> >> > > field.
> >> > > > > > > > >> > >
> >> > > > > > > > >> > > 25. LeaderAndIsrResponse v5, StopReplicaResponse
> v4:
> >> > Could
> >> > > > we
> >> > > > > > use
> >> > > > > > > > this
> >> > > > > > > > >> > > opportunity to organize the response in 2 levels,
> >> first
> >> > by
> >> > > > > > topic,
> >> > > > > > > > >> then by
> >> > > > > > > > >> > > partition, as most other requests/responses?
> >> > > > > > > > >> > >
> >> > > > > > > > >> > > 26. FetchRequest v13 : Should forgotten_topics_data
> >> use
> >> > > > > topicId
> >> > > > > > > too?
> >> > > > > > > > >> > >
> >> > > > > > > > >> > > 27. "This file can either be plain text (key/value
> >> > pairs)
> >> > > or
> >> > > > > > > JSON."
> >> > > > > > > > >> Have
> >> > > > > > > > >> > we
> >> > > > > > > > >> > > decided which one to use? Also, it would be helpful
> >> to
> >> > > > provide
> >> > > > > > an
> >> > > > > > > > >> > example.
> >> > > > > > > > >> > >
> >> > > > > > > > >> > > 28. Future improvement: Another future improvement
> >> > > > opportunity
> >> > > > > > is
> >> > > > > > > to
> >> > > > > > > > >> use
> >> > > > > > > > >> > > topicId in GroupMetadataManager.offsetCommitKey in
> >> the
> >> > > > > > > offset_commit
> >> > > > > > > > >> > topic.
> >> > > > > > > > >> > > This may save some space.
> >> > > > > > > > >> > >
> >> > > > > > > > >> > > Thanks,
> >> > > > > > > > >> > >
> >> > > > > > > > >> > > Jun
> >> > > > > > > > >> > >
> >> > > > > > > > >> > > On Wed, Sep 23, 2020 at 8:50 AM Justine Olshan <
> >> > > > > > > > jolshan@confluent.io>
> >> > > > > > > > >> > > wrote:
> >> > > > > > > > >> > >
> >> > > > > > > > >> > > > Hi Tom,
> >> > > > > > > > >> > > >
> >> > > > > > > > >> > > > Thanks for the comment. I think this is a really
> >> good
> >> > > idea
> >> > > > > and
> >> > > > > > > it
> >> > > > > > > > >> has
> >> > > > > > > > >> > > been
> >> > > > > > > > >> > > > added to the KIP under the newly added tooling
> >> > section.
> >> > > > > > > > >> > > >
> >> > > > > > > > >> > > > Thanks again,
> >> > > > > > > > >> > > > Justine
> >> > > > > > > > >> > > >
> >> > > > > > > > >> > > > On Wed, Sep 23, 2020 at 3:17 AM Tom Bentley <
> >> > > > > > > tbentley@redhat.com>
> >> > > > > > > > >> > wrote:
> >> > > > > > > > >> > > >
> >> > > > > > > > >> > > > > Hi Justine,
> >> > > > > > > > >> > > > >
> >> > > > > > > > >> > > > > I know you started the vote thread, but on
> >> > re-reading
> >> > > > the
> >> > > > > > KIP
> >> > > > > > > I
> >> > > > > > > > >> > noticed
> >> > > > > > > > >> > > > > that although the topic id is included in the
> >> > > > > > MetadataResponse
> >> > > > > > > > >> it's
> >> > > > > > > > >> > not
> >> > > > > > > > >> > > > > surfaced in the output from `kafka-topics.sh
> >> > > > --describe`.
> >> > > > > > > Maybe
> >> > > > > > > > >> that
> >> > > > > > > > >> > > was
> >> > > > > > > > >> > > > > intentional because ids are intentionally not
> >> really
> >> > > > > > something
> >> > > > > > > > the
> >> > > > > > > > >> > user
> >> > > > > > > > >> > > > > should care deeply about, but it would also
> make
> >> > life
> >> > > > > harder
> >> > > > > > > for
> >> > > > > > > > >> > anyone
> >> > > > > > > > >> > > > > debugging Kafka and this would likely get worse
> >> the
> >> > > more
> >> > > > > > topic
> >> > > > > > > > ids
> >> > > > > > > > >> > got
> >> > > > > > > > >> > > > > rolled out across the protocols, clients etc.
> It
> >> > seems
> >> > > > > > likely
> >> > > > > > > > that
> >> > > > > > > > >> > > > > `kafka-topics.sh` will eventually need the
> >> ability
> >> > to
> >> > > > show
> >> > > > > > the
> >> > > > > > > > id
> >> > > > > > > > >> of
> >> > > > > > > > >> > a
> >> > > > > > > > >> > > > > topic and perhaps find a topic name given an
> id.
> >> Is
> >> > > > there
> >> > > > > > any
> >> > > > > > > > >> reason
> >> > > > > > > > >> > > not
> >> > > > > > > > >> > > > to
> >> > > > > > > > >> > > > > implement that in this KIP?
> >> > > > > > > > >> > > > >
> >> > > > > > > > >> > > > > Many thanks,
> >> > > > > > > > >> > > > >
> >> > > > > > > > >> > > > > Tom
> >> > > > > > > > >> > > > >
> >> > > > > > > > >> > > > > On Mon, Sep 21, 2020 at 9:54 PM Justine Olshan
> <
> >> > > > > > > > >> jolshan@confluent.io
> >> > > > > > > > >> > >
> >> > > > > > > > >> > > > > wrote:
> >> > > > > > > > >> > > > >
> >> > > > > > > > >> > > > > > Hi all,
> >> > > > > > > > >> > > > > >
> >> > > > > > > > >> > > > > > After thinking about it, I've decided to
> remove
> >> > the
> >> > > > > topic
> >> > > > > > > name
> >> > > > > > > > >> from
> >> > > > > > > > >> > > the
> >> > > > > > > > >> > > > > > Fetch Request and Response after all. Since
> >> there
> >> > > are
> >> > > > so
> >> > > > > > > many
> >> > > > > > > > of
> >> > > > > > > > >> > > these
> >> > > > > > > > >> > > > > > requests per second, it is worth removing the
> >> > extra
> >> > > > > > > > information.
> >> > > > > > > > >> > I've
> >> > > > > > > > >> > > > > > updated the KIP to reflect this change.
> >> > > > > > > > >> > > > > >
> >> > > > > > > > >> > > > > > Please let me know if there is anything else
> we
> >> > > should
> >> > > > > > > discuss
> >> > > > > > > > >> > before
> >> > > > > > > > >> > > > > > voting.
> >> > > > > > > > >> > > > > >
> >> > > > > > > > >> > > > > > Thank you,
> >> > > > > > > > >> > > > > > Justine
> >> > > > > > > > >> > > > > >
> >> > > > > > > > >> > > > > > On Fri, Sep 18, 2020 at 9:46 AM Justine
> Olshan
> >> <
> >> > > > > > > > >> > jolshan@confluent.io
> >> > > > > > > > >> > > >
> >> > > > > > > > >> > > > > > wrote:
> >> > > > > > > > >> > > > > >
> >> > > > > > > > >> > > > > > > Hi Jun,
> >> > > > > > > > >> > > > > > >
> >> > > > > > > > >> > > > > > > I see what you are saying. For now we can
> >> remove
> >> > > the
> >> > > > > > extra
> >> > > > > > > > >> > > > information.
> >> > > > > > > > >> > > > > > > I'll leave the option to add more fields to
> >> the
> >> > > file
> >> > > > > in
> >> > > > > > > the
> >> > > > > > > > >> > future.
> >> > > > > > > > >> > > > The
> >> > > > > > > > >> > > > > > KIP
> >> > > > > > > > >> > > > > > > has been updated to reflect this change.
> >> > > > > > > > >> > > > > > >
> >> > > > > > > > >> > > > > > > Thanks,
> >> > > > > > > > >> > > > > > > Justine
> >> > > > > > > > >> > > > > > >
> >> > > > > > > > >> > > > > > > On Fri, Sep 18, 2020 at 8:46 AM Jun Rao <
> >> > > > > > jun@confluent.io
> >> > > > > > > >
> >> > > > > > > > >> > wrote:
> >> > > > > > > > >> > > > > > >
> >> > > > > > > > >> > > > > > >> Hi, Justine,
> >> > > > > > > > >> > > > > > >>
> >> > > > > > > > >> > > > > > >> Thanks for the reply.
> >> > > > > > > > >> > > > > > >>
> >> > > > > > > > >> > > > > > >> 13. If the log directory is the source of
> >> > truth,
> >> > > it
> >> > > > > > means
> >> > > > > > > > >> that
> >> > > > > > > > >> > the
> >> > > > > > > > >> > > > > > >> redundant info in the metadata file will
> be
> >> > > > ignored.
> >> > > > > > Then
> >> > > > > > > > the
> >> > > > > > > > >> > > > question
> >> > > > > > > > >> > > > > > is
> >> > > > > > > > >> > > > > > >> why do we need to put the redundant info
> in
> >> the
> >> > > > > > metadata
> >> > > > > > > > file
> >> > > > > > > > >> > now?
> >> > > > > > > > >> > > > > > >>
> >> > > > > > > > >> > > > > > >> Thanks,
> >> > > > > > > > >> > > > > > >>
> >> > > > > > > > >> > > > > > >> Jun
> >> > > > > > > > >> > > > > > >>
> >> > > > > > > > >> > > > > > >> On Thu, Sep 17, 2020 at 5:07 PM Justine
> >> Olshan
> >> > <
> >> > > > > > > > >> > > > jolshan@confluent.io>
> >> > > > > > > > >> > > > > > >> wrote:
> >> > > > > > > > >> > > > > > >>
> >> > > > > > > > >> > > > > > >> > Hi Jun,
> >> > > > > > > > >> > > > > > >> > Thanks for the quick response!
> >> > > > > > > > >> > > > > > >> >
> >> > > > > > > > >> > > > > > >> > 12. I've decided to bump up the versions
> >> on
> >> > the
> >> > > > > > > requests
> >> > > > > > > > >> and
> >> > > > > > > > >> > > > updated
> >> > > > > > > > >> > > > > > the
> >> > > > > > > > >> > > > > > >> > KIP. I think it's good we thoroughly
> >> > discussed
> >> > > > the
> >> > > > > > > > options
> >> > > > > > > > >> > here,
> >> > > > > > > > >> > > > so
> >> > > > > > > > >> > > > > we
> >> > > > > > > > >> > > > > > >> know
> >> > > > > > > > >> > > > > > >> > we made a good choice. :)
> >> > > > > > > > >> > > > > > >> >
> >> > > > > > > > >> > > > > > >> > 13. This is an interesting situation. I
> >> think
> >> > > if
> >> > > > > this
> >> > > > > > > > does
> >> > > > > > > > >> > occur
> >> > > > > > > > >> > > > we
> >> > > > > > > > >> > > > > > >> should
> >> > > > > > > > >> > > > > > >> > give a warning. I agree that it's hard
> to
> >> > know
> >> > > > the
> >> > > > > > > source
> >> > > > > > > > >> of
> >> > > > > > > > >> > > truth
> >> > > > > > > > >> > > > > for
> >> > > > > > > > >> > > > > > >> sure
> >> > > > > > > > >> > > > > > >> > since the directory or the file could be
> >> > > manually
> >> > > > > > > > >> modified. I
> >> > > > > > > > >> > > > guess
> >> > > > > > > > >> > > > > > the
> >> > > > > > > > >> > > > > > >> > directory could be used as the source of
> >> > truth.
> >> > > > To
> >> > > > > be
> >> > > > > > > > >> honest,
> >> > > > > > > > >> > > I'm
> >> > > > > > > > >> > > > > not
> >> > > > > > > > >> > > > > > >> > really sure what happens in kafka when
> the
> >> > log
> >> > > > > > > directory
> >> > > > > > > > is
> >> > > > > > > > >> > > > renamed
> >> > > > > > > > >> > > > > > >> > manually in such a way. I'm also
> >> wondering if
> >> > > the
> >> > > > > > > > >> situation is
> >> > > > > > > > >> > > > > > >> recoverable
> >> > > > > > > > >> > > > > > >> > in this scenario.
> >> > > > > > > > >> > > > > > >> >
> >> > > > > > > > >> > > > > > >> > Thanks,
> >> > > > > > > > >> > > > > > >> > Justine
> >> > > > > > > > >> > > > > > >> >
> >> > > > > > > > >> > > > > > >> > On Thu, Sep 17, 2020 at 4:28 PM Jun Rao
> <
> >> > > > > > > > jun@confluent.io>
> >> > > > > > > > >> > > wrote:
> >> > > > > > > > >> > > > > > >> >
> >> > > > > > > > >> > > > > > >> > > Hi, Justine,
> >> > > > > > > > >> > > > > > >> > >
> >> > > > > > > > >> > > > > > >> > > Thanks for the reply.
> >> > > > > > > > >> > > > > > >> > >
> >> > > > > > > > >> > > > > > >> > > 12. I don't have a strong preference
> >> > either.
> >> > > > > > However,
> >> > > > > > > > if
> >> > > > > > > > >> we
> >> > > > > > > > >> > > need
> >> > > > > > > > >> > > > > IBP
> >> > > > > > > > >> > > > > > >> > > anyway, maybe it's easier to just bump
> >> up
> >> > the
> >> > > > > > version
> >> > > > > > > > for
> >> > > > > > > > >> > all
> >> > > > > > > > >> > > > > inter
> >> > > > > > > > >> > > > > > >> > broker
> >> > > > > > > > >> > > > > > >> > > requests and add the topic id field
> as a
> >> > > > regular
> >> > > > > > > > field. A
> >> > > > > > > > >> > > > regular
> >> > > > > > > > >> > > > > > >> field
> >> > > > > > > > >> > > > > > >> > is
> >> > > > > > > > >> > > > > > >> > > a bit more concise in wire transfer
> >> than a
> >> > > > > flexible
> >> > > > > > > > >> field.
> >> > > > > > > > >> > > > > > >> > >
> >> > > > > > > > >> > > > > > >> > > 13. The confusion that I was referring
> >> to
> >> > is
> >> > > > > > between
> >> > > > > > > > the
> >> > > > > > > > >> > topic
> >> > > > > > > > >> > > > > name
> >> > > > > > > > >> > > > > > >> and
> >> > > > > > > > >> > > > > > >> > > partition number between the log dir
> and
> >> > the
> >> > > > > > metadata
> >> > > > > > > > >> file.
> >> > > > > > > > >> > > For
> >> > > > > > > > >> > > > > > >> example,
> >> > > > > > > > >> > > > > > >> > if
> >> > > > > > > > >> > > > > > >> > > the log dir is topicA-1 and the
> metadata
> >> > file
> >> > > > in
> >> > > > > it
> >> > > > > > > has
> >> > > > > > > > >> > topicB
> >> > > > > > > > >> > > > and
> >> > > > > > > > >> > > > > > >> > > partition 0 (say due to a bug or
> manual
> >> > > > > > > modification),
> >> > > > > > > > >> which
> >> > > > > > > > >> > > one
> >> > > > > > > > >> > > > > do
> >> > > > > > > > >> > > > > > we
> >> > > > > > > > >> > > > > > >> > use
> >> > > > > > > > >> > > > > > >> > > as the source of truth?
> >> > > > > > > > >> > > > > > >> > >
> >> > > > > > > > >> > > > > > >> > > Jun
> >> > > > > > > > >> > > > > > >> > >
> >> > > > > > > > >> > > > > > >> > > On Thu, Sep 17, 2020 at 3:43 PM
> Justine
> >> > > Olshan
> >> > > > <
> >> > > > > > > > >> > > > > > jolshan@confluent.io>
> >> > > > > > > > >> > > > > > >> > > wrote:
> >> > > > > > > > >> > > > > > >> > >
> >> > > > > > > > >> > > > > > >> > > > Hi Jun,
> >> > > > > > > > >> > > > > > >> > > > Thanks for the comments.
> >> > > > > > > > >> > > > > > >> > > >
> >> > > > > > > > >> > > > > > >> > > > 12. I bumped the LeaderAndIsrRequest
> >> > > because
> >> > > > I
> >> > > > > > > > removed
> >> > > > > > > > >> the
> >> > > > > > > > >> > > > topic
> >> > > > > > > > >> > > > > > >> name
> >> > > > > > > > >> > > > > > >> > > field
> >> > > > > > > > >> > > > > > >> > > > in the response. It may be possible
> to
> >> > > avoid
> >> > > > > > > bumping
> >> > > > > > > > >> the
> >> > > > > > > > >> > > > version
> >> > > > > > > > >> > > > > > >> > without
> >> > > > > > > > >> > > > > > >> > > > that change, but I may be missing
> >> > > something.
> >> > > > > > > > >> > > > > > >> > > > I believe StopReplica is actually on
> >> > > version
> >> > > > 3
> >> > > > > > now,
> >> > > > > > > > but
> >> > > > > > > > >> > > > because
> >> > > > > > > > >> > > > > > >> > version 2
> >> > > > > > > > >> > > > > > >> > > > is flexible, I kept that listed as
> >> > version
> >> > > 2
> >> > > > on
> >> > > > > > the
> >> > > > > > > > KIP
> >> > > > > > > > >> > > page.
> >> > > > > > > > >> > > > > > >> However,
> >> > > > > > > > >> > > > > > >> > > you
> >> > > > > > > > >> > > > > > >> > > > may be right in that we may need to
> >> bump
> >> > > the
> >> > > > > > > version
> >> > > > > > > > on
> >> > > > > > > > >> > > > > > StopReplica
> >> > > > > > > > >> > > > > > >> to
> >> > > > > > > > >> > > > > > >> > > deal
> >> > > > > > > > >> > > > > > >> > > > with deletion differently as
> mentioned
> >> > > > above. I
> >> > > > > > > don't
> >> > > > > > > > >> know
> >> > > > > > > > >> > > if
> >> > > > > > > > >> > > > I
> >> > > > > > > > >> > > > > > >> have a
> >> > > > > > > > >> > > > > > >> > > big
> >> > > > > > > > >> > > > > > >> > > > preference over used tagged fields
> or
> >> > not.
> >> > > > > > > > >> > > > > > >> > > >
> >> > > > > > > > >> > > > > > >> > > > 13. I was thinking that in the case
> >> where
> >> > > the
> >> > > > > > file
> >> > > > > > > > and
> >> > > > > > > > >> the
> >> > > > > > > > >> > > > > request
> >> > > > > > > > >> > > > > > >> > topic
> >> > > > > > > > >> > > > > > >> > > > ids don't match, it means that the
> >> > broker's
> >> > > > > > > topic/the
> >> > > > > > > > >> one
> >> > > > > > > > >> > in
> >> > > > > > > > >> > > > the
> >> > > > > > > > >> > > > > > >> file
> >> > > > > > > > >> > > > > > >> > has
> >> > > > > > > > >> > > > > > >> > > > been deleted. In that case, we would
> >> need
> >> > > to
> >> > > > > > delete
> >> > > > > > > > the
> >> > > > > > > > >> > old
> >> > > > > > > > >> > > > > topic
> >> > > > > > > > >> > > > > > >> and
> >> > > > > > > > >> > > > > > >> > > start
> >> > > > > > > > >> > > > > > >> > > > receiving the new version. If the
> >> topic
> >> > > name
> >> > > > > were
> >> > > > > > > to
> >> > > > > > > > >> > change,
> >> > > > > > > > >> > > > but
> >> > > > > > > > >> > > > > > the
> >> > > > > > > > >> > > > > > >> > ids
> >> > > > > > > > >> > > > > > >> > > > still matched, the file would also
> >> need
> >> > to
> >> > > > > > update.
> >> > > > > > > > Am I
> >> > > > > > > > >> > > > missing
> >> > > > > > > > >> > > > > a
> >> > > > > > > > >> > > > > > >> case
> >> > > > > > > > >> > > > > > >> > > > where the file would be correct and
> >> not
> >> > the
> >> > > > > > > request?
> >> > > > > > > > >> > > > > > >> > > >
> >> > > > > > > > >> > > > > > >> > > > Thanks,
> >> > > > > > > > >> > > > > > >> > > > Justine
> >> > > > > > > > >> > > > > > >> > > >
> >> > > > > > > > >> > > > > > >> > > > On Thu, Sep 17, 2020 at 3:18 PM Jun
> >> Rao <
> >> > > > > > > > >> jun@confluent.io
> >> > > > > > > > >> > >
> >> > > > > > > > >> > > > > wrote:
> >> > > > > > > > >> > > > > > >> > > >
> >> > > > > > > > >> > > > > > >> > > > > Hi, Justine,
> >> > > > > > > > >> > > > > > >> > > > >
> >> > > > > > > > >> > > > > > >> > > > > Thanks for the reply. A couple of
> >> more
> >> > > > > comments
> >> > > > > > > > >> below.
> >> > > > > > > > >> > > > > > >> > > > >
> >> > > > > > > > >> > > > > > >> > > > > 12. ListOffset and OffsetForLeader
> >> > > > currently
> >> > > > > > > don't
> >> > > > > > > > >> > support
> >> > > > > > > > >> > > > > > >> flexible
> >> > > > > > > > >> > > > > > >> > > > fields.
> >> > > > > > > > >> > > > > > >> > > > > So, we have to bump up the version
> >> > number
> >> > > > and
> >> > > > > > use
> >> > > > > > > > >> IBP at
> >> > > > > > > > >> > > > least
> >> > > > > > > > >> > > > > > for
> >> > > > > > > > >> > > > > > >> > > these
> >> > > > > > > > >> > > > > > >> > > > > two requests. Note that it seems
> >> 2.7.0
> >> > > will
> >> > > > > > > require
> >> > > > > > > > >> IBP
> >> > > > > > > > >> > > > anyway
> >> > > > > > > > >> > > > > > >> > because
> >> > > > > > > > >> > > > > > >> > > of
> >> > > > > > > > >> > > > > > >> > > > > changes in KAFKA-10435. Also, it
> >> seems
> >> > > that
> >> > > > > the
> >> > > > > > > > >> version
> >> > > > > > > > >> > > for
> >> > > > > > > > >> > > > > > >> > > > > LeaderAndIsrRequest and
> StopReplica
> >> are
> >> > > > > bumped
> >> > > > > > > even
> >> > > > > > > > >> > though
> >> > > > > > > > >> > > > we
> >> > > > > > > > >> > > > > > only
> >> > > > > > > > >> > > > > > >> > > added
> >> > > > > > > > >> > > > > > >> > > > a
> >> > > > > > > > >> > > > > > >> > > > > tagged field. But since IBP is
> >> needed
> >> > > > anyway,
> >> > > > > > we
> >> > > > > > > > may
> >> > > > > > > > >> > want
> >> > > > > > > > >> > > to
> >> > > > > > > > >> > > > > > >> revisit
> >> > > > > > > > >> > > > > > >> > > the
> >> > > > > > > > >> > > > > > >> > > > > overall tagged field choice.
> >> > > > > > > > >> > > > > > >> > > > >
> >> > > > > > > > >> > > > > > >> > > > > 13. The only downside is the
> >> potential
> >> > > > > > confusion
> >> > > > > > > on
> >> > > > > > > > >> > which
> >> > > > > > > > >> > > > one
> >> > > > > > > > >> > > > > is
> >> > > > > > > > >> > > > > > >> the
> >> > > > > > > > >> > > > > > >> > > > source
> >> > > > > > > > >> > > > > > >> > > > > of truth if they don't match.
> >> Another
> >> > > > option
> >> > > > > is
> >> > > > > > > to
> >> > > > > > > > >> > include
> >> > > > > > > > >> > > > > those
> >> > > > > > > > >> > > > > > >> > fields
> >> > > > > > > > >> > > > > > >> > > > in
> >> > > > > > > > >> > > > > > >> > > > > the metadata file when we actually
> >> > change
> >> > > > the
> >> > > > > > > > >> directory
> >> > > > > > > > >> > > > > > structure.
> >> > > > > > > > >> > > > > > >> > > > >
> >> > > > > > > > >> > > > > > >> > > > > Thanks,
> >> > > > > > > > >> > > > > > >> > > > >
> >> > > > > > > > >> > > > > > >> > > > > Jun
> >> > > > > > > > >> > > > > > >> > > > >
> >> > > > > > > > >> > > > > > >> > > > > On Thu, Sep 17, 2020 at 2:01 PM
> >> Justine
> >> > > > > Olshan
> >> > > > > > <
> >> > > > > > > > >> > > > > > >> jolshan@confluent.io
> >> > > > > > > > >> > > > > > >> > >
> >> > > > > > > > >> > > > > > >> > > > > wrote:
> >> > > > > > > > >> > > > > > >> > > > >
> >> > > > > > > > >> > > > > > >> > > > > > Hello all,
> >> > > > > > > > >> > > > > > >> > > > > >
> >> > > > > > > > >> > > > > > >> > > > > > I've thought some more about
> >> removing
> >> > > the
> >> > > > > > topic
> >> > > > > > > > >> name
> >> > > > > > > > >> > > field
> >> > > > > > > > >> > > > > > from
> >> > > > > > > > >> > > > > > >> > some
> >> > > > > > > > >> > > > > > >> > > of
> >> > > > > > > > >> > > > > > >> > > > > the
> >> > > > > > > > >> > > > > > >> > > > > > requests. On closer inspection
> of
> >> the
> >> > > > > > > > >> > > requests/responses,
> >> > > > > > > > >> > > > it
> >> > > > > > > > >> > > > > > >> seems
> >> > > > > > > > >> > > > > > >> > > that
> >> > > > > > > > >> > > > > > >> > > > > the
> >> > > > > > > > >> > > > > > >> > > > > > internal changes would be much
> >> larger
> >> > > > than
> >> > > > > I
> >> > > > > > > > >> expected.
> >> > > > > > > > >> > > > Some
> >> > > > > > > > >> > > > > > >> > protocols
> >> > > > > > > > >> > > > > > >> > > > > > involve clients, so they would
> >> > require
> >> > > > > > changes
> >> > > > > > > > too.
> >> > > > > > > > >> > I'm
> >> > > > > > > > >> > > > > > thinking
> >> > > > > > > > >> > > > > > >> > that
> >> > > > > > > > >> > > > > > >> > > > for
> >> > > > > > > > >> > > > > > >> > > > > > now, removing the topic name
> from
> >> > these
> >> > > > > > > requests
> >> > > > > > > > >> and
> >> > > > > > > > >> > > > > responses
> >> > > > > > > > >> > > > > > >> are
> >> > > > > > > > >> > > > > > >> > > out
> >> > > > > > > > >> > > > > > >> > > > of
> >> > > > > > > > >> > > > > > >> > > > > > scope.
> >> > > > > > > > >> > > > > > >> > > > > >
> >> > > > > > > > >> > > > > > >> > > > > > I have decided to just keep the
> >> > change
> >> > > > > > > > >> > > > LeaderAndIsrResponse
> >> > > > > > > > >> > > > > to
> >> > > > > > > > >> > > > > > >> > remove
> >> > > > > > > > >> > > > > > >> > > > the
> >> > > > > > > > >> > > > > > >> > > > > > topic name, and have updated the
> >> KIP
> >> > to
> >> > > > > > reflect
> >> > > > > > > > >> this
> >> > > > > > > > >> > > > > change. I
> >> > > > > > > > >> > > > > > >> have
> >> > > > > > > > >> > > > > > >> > > > also
> >> > > > > > > > >> > > > > > >> > > > > > mentioned the other requests and
> >> > > > responses
> >> > > > > in
> >> > > > > > > > >> future
> >> > > > > > > > >> > > work.
> >> > > > > > > > >> > > > > > >> > > > > >
> >> > > > > > > > >> > > > > > >> > > > > > I'm hoping to start the voting
> >> > process
> >> > > > > soon,
> >> > > > > > so
> >> > > > > > > > >> let me
> >> > > > > > > > >> > > > know
> >> > > > > > > > >> > > > > if
> >> > > > > > > > >> > > > > > >> > there
> >> > > > > > > > >> > > > > > >> > > is
> >> > > > > > > > >> > > > > > >> > > > > > anything else we should discuss.
> >> > > > > > > > >> > > > > > >> > > > > >
> >> > > > > > > > >> > > > > > >> > > > > > Thank you,
> >> > > > > > > > >> > > > > > >> > > > > > Justine
> >> > > > > > > > >> > > > > > >> > > > > >
> >> > > > > > > > >> > > > > > >> > > > > > On Tue, Sep 15, 2020 at 3:57 PM
> >> > Justine
> >> > > > > > Olshan
> >> > > > > > > <
> >> > > > > > > > >> > > > > > >> > jolshan@confluent.io
> >> > > > > > > > >> > > > > > >> > > >
> >> > > > > > > > >> > > > > > >> > > > > > wrote:
> >> > > > > > > > >> > > > > > >> > > > > >
> >> > > > > > > > >> > > > > > >> > > > > > > Hello again,
> >> > > > > > > > >> > > > > > >> > > > > > > To follow up on some of the
> >> other
> >> > > > > comments:
> >> > > > > > > > >> > > > > > >> > > > > > >
> >> > > > > > > > >> > > > > > >> > > > > > > 10/11) We can remove the topic
> >> name
> >> > > > from
> >> > > > > > > these
> >> > > > > > > > >> > > > > > >> > requests/responses,
> >> > > > > > > > >> > > > > > >> > > > and
> >> > > > > > > > >> > > > > > >> > > > > > > that means we will just have
> to
> >> > make
> >> > > a
> >> > > > > few
> >> > > > > > > > >> internal
> >> > > > > > > > >> > > > > changes
> >> > > > > > > > >> > > > > > to
> >> > > > > > > > >> > > > > > >> > make
> >> > > > > > > > >> > > > > > >> > > > > > > partitions accessible by topic
> >> id
> >> > and
> >> > > > > > > > partition.
> >> > > > > > > > >> I
> >> > > > > > > > >> > can
> >> > > > > > > > >> > > > > > update
> >> > > > > > > > >> > > > > > >> the
> >> > > > > > > > >> > > > > > >> > > KIP
> >> > > > > > > > >> > > > > > >> > > > > to
> >> > > > > > > > >> > > > > > >> > > > > > > remove them unless anyone
> thinks
> >> > they
> >> > > > > > should
> >> > > > > > > > >> stay.
> >> > > > > > > > >> > > > > > >> > > > > > >
> >> > > > > > > > >> > > > > > >> > > > > > > 12) Addressed in the previous
> >> > email.
> >> > > > I've
> >> > > > > > > > updated
> >> > > > > > > > >> > the
> >> > > > > > > > >> > > > KIP
> >> > > > > > > > >> > > > > to
> >> > > > > > > > >> > > > > > >> > > include
> >> > > > > > > > >> > > > > > >> > > > > > > tagged fields for the requests
> >> and
> >> > > > > > responses.
> >> > > > > > > > >> (More
> >> > > > > > > > >> > on
> >> > > > > > > > >> > > > > that
> >> > > > > > > > >> > > > > > >> > below)
> >> > > > > > > > >> > > > > > >> > > > > > >
> >> > > > > > > > >> > > > > > >> > > > > > > 13) I think part of the idea
> for
> >> > > > > including
> >> > > > > > > this
> >> > > > > > > > >> > > > > information
> >> > > > > > > > >> > > > > > >> is to
> >> > > > > > > > >> > > > > > >> > > > > prepare
> >> > > > > > > > >> > > > > > >> > > > > > > for future changes. Perhaps
> the
> >> > > > directory
> >> > > > > > > > >> structure
> >> > > > > > > > >> > > > might
> >> > > > > > > > >> > > > > > >> change
> >> > > > > > > > >> > > > > > >> > > from
> >> > > > > > > > >> > > > > > >> > > > > > > topicName_partitionNumber to
> >> > > something
> >> > > > > like
> >> > > > > > > > >> > > > > > >> > > topicID_partitionNumber.
> >> > > > > > > > >> > > > > > >> > > > > Then
> >> > > > > > > > >> > > > > > >> > > > > > > it would be useful to have the
> >> > topic
> >> > > > name
> >> > > > > > in
> >> > > > > > > > the
> >> > > > > > > > >> > file
> >> > > > > > > > >> > > > > since
> >> > > > > > > > >> > > > > > it
> >> > > > > > > > >> > > > > > >> > > would
> >> > > > > > > > >> > > > > > >> > > > > not
> >> > > > > > > > >> > > > > > >> > > > > > be
> >> > > > > > > > >> > > > > > >> > > > > > > in the directory structure.
> >> > > Supporting
> >> > > > > > topic
> >> > > > > > > > >> renames
> >> > > > > > > > >> > > > might
> >> > > > > > > > >> > > > > > be
> >> > > > > > > > >> > > > > > >> > > easier
> >> > > > > > > > >> > > > > > >> > > > if
> >> > > > > > > > >> > > > > > >> > > > > > the
> >> > > > > > > > >> > > > > > >> > > > > > > other fields are included.
> Would
> >> > > there
> >> > > > be
> >> > > > > > any
> >> > > > > > > > >> > > downsides
> >> > > > > > > > >> > > > to
> >> > > > > > > > >> > > > > > >> > > including
> >> > > > > > > > >> > > > > > >> > > > > this
> >> > > > > > > > >> > > > > > >> > > > > > > information?
> >> > > > > > > > >> > > > > > >> > > > > > >
> >> > > > > > > > >> > > > > > >> > > > > > > 14)  Yes, we would need to
> copy
> >> the
> >> > > > > > partition
> >> > > > > > > > >> > metadata
> >> > > > > > > > >> > > > > file
> >> > > > > > > > >> > > > > > in
> >> > > > > > > > >> > > > > > >> > this
> >> > > > > > > > >> > > > > > >> > > > > > > process. I've updated the KIP
> to
> >> > > > include
> >> > > > > > > this.
> >> > > > > > > > >> > > > > > >> > > > > > >
> >> > > > > > > > >> > > > > > >> > > > > > > 15) I believe Lucas meant v1
> >> and v2
> >> > > > here.
> >> > > > > > He
> >> > > > > > > > was
> >> > > > > > > > >> > > > referring
> >> > > > > > > > >> > > > > > to
> >> > > > > > > > >> > > > > > >> how
> >> > > > > > > > >> > > > > > >> > > the
> >> > > > > > > > >> > > > > > >> > > > > > > requests would fall under
> >> different
> >> > > IBP
> >> > > > > and
> >> > > > > > > > meant
> >> > > > > > > > >> > that
> >> > > > > > > > >> > > > > older
> >> > > > > > > > >> > > > > > >> > > brokers
> >> > > > > > > > >> > > > > > >> > > > > > would
> >> > > > > > > > >> > > > > > >> > > > > > > have to use the older version
> of
> >> > the
> >> > > > > > request
> >> > > > > > > > and
> >> > > > > > > > >> the
> >> > > > > > > > >> > > > > > existing
> >> > > > > > > > >> > > > > > >> > topic
> >> > > > > > > > >> > > > > > >> > > > > > > deletion process. At first, it
> >> > seemed
> >> > > > > like
> >> > > > > > > > tagged
> >> > > > > > > > >> > > fields
> >> > > > > > > > >> > > > > > would
> >> > > > > > > > >> > > > > > >> > > > resolve
> >> > > > > > > > >> > > > > > >> > > > > > > the IBP issue. However, we may
> >> need
> >> > > IBP
> >> > > > > for
> >> > > > > > > > this
> >> > > > > > > > >> > > request
> >> > > > > > > > >> > > > > > after
> >> > > > > > > > >> > > > > > >> > all
> >> > > > > > > > >> > > > > > >> > > > > since
> >> > > > > > > > >> > > > > > >> > > > > > > the controller handles the
> topic
> >> > > > deletion
> >> > > > > > > > >> > differently
> >> > > > > > > > >> > > > > > >> depending
> >> > > > > > > > >> > > > > > >> > on
> >> > > > > > > > >> > > > > > >> > > > the
> >> > > > > > > > >> > > > > > >> > > > > > IBP
> >> > > > > > > > >> > > > > > >> > > > > > > version. In an older version,
> we
> >> > > can't
> >> > > > > just
> >> > > > > > > > send
> >> > > > > > > > >> a
> >> > > > > > > > >> > > > > > StopReplica
> >> > > > > > > > >> > > > > > >> > > delete
> >> > > > > > > > >> > > > > > >> > > > > the
> >> > > > > > > > >> > > > > > >> > > > > > > topic immediately like we'd
> >> want to
> >> > > for
> >> > > > > > this
> >> > > > > > > > KIP.
> >> > > > > > > > >> > > > > > >> > > > > > >
> >> > > > > > > > >> > > > > > >> > > > > > > This makes me wonder if we
> want
> >> > > tagged
> >> > > > > > fields
> >> > > > > > > > on
> >> > > > > > > > >> all
> >> > > > > > > > >> > > the
> >> > > > > > > > >> > > > > > >> requests
> >> > > > > > > > >> > > > > > >> > > > after
> >> > > > > > > > >> > > > > > >> > > > > > > all. Let me know your
> thoughts!
> >> > > > > > > > >> > > > > > >> > > > > > >
> >> > > > > > > > >> > > > > > >> > > > > > > Justine
> >> > > > > > > > >> > > > > > >> > > > > > >
> >> > > > > > > > >> > > > > > >> > > > > > > On Tue, Sep 15, 2020 at 1:03
> PM
> >> > > Justine
> >> > > > > > > Olshan
> >> > > > > > > > <
> >> > > > > > > > >> > > > > > >> > > jolshan@confluent.io
> >> > > > > > > > >> > > > > > >> > > > >
> >> > > > > > > > >> > > > > > >> > > > > > > wrote:
> >> > > > > > > > >> > > > > > >> > > > > > >
> >> > > > > > > > >> > > > > > >> > > > > > >> Hi all,
> >> > > > > > > > >> > > > > > >> > > > > > >> Jun brought up a good point
> in
> >> his
> >> > > > last
> >> > > > > > > email
> >> > > > > > > > >> about
> >> > > > > > > > >> > > > > tagged
> >> > > > > > > > >> > > > > > >> > fields,
> >> > > > > > > > >> > > > > > >> > > > and
> >> > > > > > > > >> > > > > > >> > > > > > >> I've updated the KIP to
> reflect
> >> > that
> >> > > > the
> >> > > > > > > > >> changes to
> >> > > > > > > > >> > > > > > requests
> >> > > > > > > > >> > > > > > >> and
> >> > > > > > > > >> > > > > > >> > > > > > responses
> >> > > > > > > > >> > > > > > >> > > > > > >> will be in the form of tagged
> >> > fields
> >> > > > to
> >> > > > > > > avoid
> >> > > > > > > > >> > > changing
> >> > > > > > > > >> > > > > IBP.
> >> > > > > > > > >> > > > > > >> > > > > > >>
> >> > > > > > > > >> > > > > > >> > > > > > >> Jun: I plan on sending a
> >> followup
> >> > > > email
> >> > > > > to
> >> > > > > > > > >> address
> >> > > > > > > > >> > > some
> >> > > > > > > > >> > > > > of
> >> > > > > > > > >> > > > > > >> the
> >> > > > > > > > >> > > > > > >> > > other
> >> > > > > > > > >> > > > > > >> > > > > > >> points.
> >> > > > > > > > >> > > > > > >> > > > > > >>
> >> > > > > > > > >> > > > > > >> > > > > > >> Thanks,
> >> > > > > > > > >> > > > > > >> > > > > > >> Justine
> >> > > > > > > > >> > > > > > >> > > > > > >>
> >> > > > > > > > >> > > > > > >> > > > > > >> On Mon, Sep 14, 2020 at 4:25
> PM
> >> > Jun
> >> > > > Rao
> >> > > > > <
> >> > > > > > > > >> > > > > jun@confluent.io>
> >> > > > > > > > >> > > > > > >> > wrote:
> >> > > > > > > > >> > > > > > >> > > > > > >>
> >> > > > > > > > >> > > > > > >> > > > > > >>> Hi, Justine,
> >> > > > > > > > >> > > > > > >> > > > > > >>>
> >> > > > > > > > >> > > > > > >> > > > > > >>> Thanks for the updated KIP.
> A
> >> few
> >> > > > > > comments
> >> > > > > > > > >> below.
> >> > > > > > > > >> > > > > > >> > > > > > >>>
> >> > > > > > > > >> > > > > > >> > > > > > >>> 10. LeaderAndIsr Response:
> Do
> >> we
> >> > > need
> >> > > > > the
> >> > > > > > > > topic
> >> > > > > > > > >> > > name?
> >> > > > > > > > >> > > > > > >> > > > > > >>>
> >> > > > > > > > >> > > > > > >> > > > > > >>> 11. For the changed
> >> > > request/response,
> >> > > > > > other
> >> > > > > > > > >> than
> >> > > > > > > > >> > > > > > >> LeaderAndIsr,
> >> > > > > > > > >> > > > > > >> > > > > > >>> UpdateMetadata, Metadata, do
> >> we
> >> > > need
> >> > > > to
> >> > > > > > > > include
> >> > > > > > > > >> > the
> >> > > > > > > > >> > > > > topic
> >> > > > > > > > >> > > > > > >> name?
> >> > > > > > > > >> > > > > > >> > > > > > >>>
> >> > > > > > > > >> > > > > > >> > > > > > >>> 12. It seems that upgrades
> >> don't
> >> > > > > require
> >> > > > > > > IBP.
> >> > > > > > > > >> Does
> >> > > > > > > > >> > > > that
> >> > > > > > > > >> > > > > > mean
> >> > > > > > > > >> > > > > > >> > the
> >> > > > > > > > >> > > > > > >> > > > new
> >> > > > > > > > >> > > > > > >> > > > > > >>> fields
> >> > > > > > > > >> > > > > > >> > > > > > >>> in all the request/response
> >> are
> >> > > added
> >> > > > > as
> >> > > > > > > > tagged
> >> > > > > > > > >> > > fields
> >> > > > > > > > >> > > > > > >> without
> >> > > > > > > > >> > > > > > >> > > > > bumping
> >> > > > > > > > >> > > > > > >> > > > > > up
> >> > > > > > > > >> > > > > > >> > > > > > >>> the request version? It
> would
> >> be
> >> > > > useful
> >> > > > > > to
> >> > > > > > > > make
> >> > > > > > > > >> > that
> >> > > > > > > > >> > > > > > clear.
> >> > > > > > > > >> > > > > > >> > > > > > >>>
> >> > > > > > > > >> > > > > > >> > > > > > >>> 13. Partition Metadata file:
> >> Do
> >> > we
> >> > > > need
> >> > > > > > to
> >> > > > > > > > >> include
> >> > > > > > > > >> > > the
> >> > > > > > > > >> > > > > > topic
> >> > > > > > > > >> > > > > > >> > name
> >> > > > > > > > >> > > > > > >> > > > and
> >> > > > > > > > >> > > > > > >> > > > > > the
> >> > > > > > > > >> > > > > > >> > > > > > >>> partition id since they are
> >> > implied
> >> > > > in
> >> > > > > > the
> >> > > > > > > > >> > directory
> >> > > > > > > > >> > > > > name?
> >> > > > > > > > >> > > > > > >> > > > > > >>>
> >> > > > > > > > >> > > > > > >> > > > > > >>> 14. In the JBOD mode, we
> >> support
> >> > > > > moving a
> >> > > > > > > > >> > > partition's
> >> > > > > > > > >> > > > > data
> >> > > > > > > > >> > > > > > >> from
> >> > > > > > > > >> > > > > > >> > > one
> >> > > > > > > > >> > > > > > >> > > > > > disk
> >> > > > > > > > >> > > > > > >> > > > > > >>> to
> >> > > > > > > > >> > > > > > >> > > > > > >>> another. Will the new
> >> partition
> >> > > > > metadata
> >> > > > > > > file
> >> > > > > > > > >> be
> >> > > > > > > > >> > > > copied
> >> > > > > > > > >> > > > > > >> during
> >> > > > > > > > >> > > > > > >> > > that
> >> > > > > > > > >> > > > > > >> > > > > > >>> process?
> >> > > > > > > > >> > > > > > >> > > > > > >>>
> >> > > > > > > > >> > > > > > >> > > > > > >>> 15. The KIP says "Remove
> >> deleted
> >> > > > topics
> >> > > > > > > from
> >> > > > > > > > >> > > replicas
> >> > > > > > > > >> > > > by
> >> > > > > > > > >> > > > > > >> > sending
> >> > > > > > > > >> > > > > > >> > > > > > >>> StopReplicaRequest V2 for
> any
> >> > > topics
> >> > > > > > which
> >> > > > > > > do
> >> > > > > > > > >> not
> >> > > > > > > > >> > > > > contain
> >> > > > > > > > >> > > > > > a
> >> > > > > > > > >> > > > > > >> > topic
> >> > > > > > > > >> > > > > > >> > > > ID,
> >> > > > > > > > >> > > > > > >> > > > > > and
> >> > > > > > > > >> > > > > > >> > > > > > >>> V3 for any topics which do
> >> > contain
> >> > > a
> >> > > > > > topic
> >> > > > > > > > >> ID.".
> >> > > > > > > > >> > > > > However,
> >> > > > > > > > >> > > > > > it
> >> > > > > > > > >> > > > > > >> > > seems
> >> > > > > > > > >> > > > > > >> > > > > the
> >> > > > > > > > >> > > > > > >> > > > > > >>> updated controller will
> create
> >> > all
> >> > > > > > missing
> >> > > > > > > > >> topic
> >> > > > > > > > >> > IDs
> >> > > > > > > > >> > > > > first
> >> > > > > > > > >> > > > > > >> > before
> >> > > > > > > > >> > > > > > >> > > > > doing
> >> > > > > > > > >> > > > > > >> > > > > > >>> other actions. So, is
> >> > > > > StopReplicaRequest
> >> > > > > > V2
> >> > > > > > > > >> > needed?
> >> > > > > > > > >> > > > > > >> > > > > > >>>
> >> > > > > > > > >> > > > > > >> > > > > > >>> Jun
> >> > > > > > > > >> > > > > > >> > > > > > >>>
> >> > > > > > > > >> > > > > > >> > > > > > >>> On Fri, Sep 11, 2020 at
> 10:31
> >> AM
> >> > > John
> >> > > > > > > > Roesler <
> >> > > > > > > > >> > > > > > >> > > vvcephei@apache.org
> >> > > > > > > > >> > > > > > >> > > > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> wrote:
> >> > > > > > > > >> > > > > > >> > > > > > >>>
> >> > > > > > > > >> > > > > > >> > > > > > >>> > Thanks, Justine!
> >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > Your response seems
> >> compelling
> >> > to
> >> > > > me.
> >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > -John
> >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > On Fri, 2020-09-11 at
> 10:17
> >> > > -0700,
> >> > > > > > > Justine
> >> > > > > > > > >> > Olshan
> >> > > > > > > > >> > > > > wrote:
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > Hello all,
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > Thanks for continuing
> the
> >> > > > > > discussion! I
> >> > > > > > > > >> have a
> >> > > > > > > > >> > > few
> >> > > > > > > > >> > > > > > >> > responses
> >> > > > > > > > >> > > > > > >> > > to
> >> > > > > > > > >> > > > > > >> > > > > > your
> >> > > > > > > > >> > > > > > >> > > > > > >>> > points.
> >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > Tom: You are correct in
> >> that
> >> > > this
> >> > > > > KIP
> >> > > > > > > has
> >> > > > > > > > >> not
> >> > > > > > > > >> > > > > > mentioned
> >> > > > > > > > >> > > > > > >> the
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > DeleteTopicsRequest. I
> >> think
> >> > > that
> >> > > > > > this
> >> > > > > > > > >> would
> >> > > > > > > > >> > be
> >> > > > > > > > >> > > > out
> >> > > > > > > > >> > > > > of
> >> > > > > > > > >> > > > > > >> > scope
> >> > > > > > > > >> > > > > > >> > > > for
> >> > > > > > > > >> > > > > > >> > > > > > >>> now, but
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > may be something worth
> >> adding
> >> > > in
> >> > > > > the
> >> > > > > > > > >> future.
> >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > John: We did consider
> >> > sequence
> >> > > > ids,
> >> > > > > > but
> >> > > > > > > > >> there
> >> > > > > > > > >> > > are
> >> > > > > > > > >> > > > a
> >> > > > > > > > >> > > > > > few
> >> > > > > > > > >> > > > > > >> > > reasons
> >> > > > > > > > >> > > > > > >> > > > > to
> >> > > > > > > > >> > > > > > >> > > > > > >>> favor
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > UUIDs. There are several
> >> > cases
> >> > > > > where
> >> > > > > > > > topics
> >> > > > > > > > >> > from
> >> > > > > > > > >> > > > > > >> different
> >> > > > > > > > >> > > > > > >> > > > > clusters
> >> > > > > > > > >> > > > > > >> > > > > > >>> may
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > interact now and in the
> >> > future.
> >> > > > For
> >> > > > > > > > >> example,
> >> > > > > > > > >> > > > Mirror
> >> > > > > > > > >> > > > > > >> Maker 2
> >> > > > > > > > >> > > > > > >> > > may
> >> > > > > > > > >> > > > > > >> > > > > > >>> benefit
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > from being able to
> detect
> >> > when
> >> > > a
> >> > > > > > > cluster
> >> > > > > > > > >> being
> >> > > > > > > > >> > > > > > mirrored
> >> > > > > > > > >> > > > > > >> is
> >> > > > > > > > >> > > > > > >> > > > > deleted
> >> > > > > > > > >> > > > > > >> > > > > > >>> and
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > recreated and globally
> >> unique
> >> > > > > > > identifiers
> >> > > > > > > > >> > would
> >> > > > > > > > >> > > > make
> >> > > > > > > > >> > > > > > >> > > resolving
> >> > > > > > > > >> > > > > > >> > > > > > issues
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > easier than sequence IDs
> >> > which
> >> > > > may
> >> > > > > > > > collide
> >> > > > > > > > >> > > between
> >> > > > > > > > >> > > > > > >> > clusters.
> >> > > > > > > > >> > > > > > >> > > > > > KIP-405
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > (tiered storage) will
> also
> >> > > > benefit
> >> > > > > > from
> >> > > > > > > > >> > globally
> >> > > > > > > > >> > > > > > unique
> >> > > > > > > > >> > > > > > >> IDs
> >> > > > > > > > >> > > > > > >> > > as
> >> > > > > > > > >> > > > > > >> > > > > > shared
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > buckets may be used
> >> between
> >> > > > > clusters.
> >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > Globally unique IDs
> would
> >> > also
> >> > > > make
> >> > > > > > > > >> > > functionality
> >> > > > > > > > >> > > > > like
> >> > > > > > > > >> > > > > > >> > moving
> >> > > > > > > > >> > > > > > >> > > > > > topics
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > between disparate
> clusters
> >> > > easier
> >> > > > > in
> >> > > > > > > the
> >> > > > > > > > >> > future,
> >> > > > > > > > >> > > > > > >> simplify
> >> > > > > > > > >> > > > > > >> > any
> >> > > > > > > > >> > > > > > >> > > > > > future
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > implementations of
> backups
> >> > and
> >> > > > > > > restores,
> >> > > > > > > > >> and
> >> > > > > > > > >> > > more.
> >> > > > > > > > >> > > > > In
> >> > > > > > > > >> > > > > > >> > > general,
> >> > > > > > > > >> > > > > > >> > > > > > >>> unique IDs
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > would ensure that the
> >> source
> >> > > > > cluster
> >> > > > > > > > >> topics do
> >> > > > > > > > >> > > not
> >> > > > > > > > >> > > > > > >> conflict
> >> > > > > > > > >> > > > > > >> > > > with
> >> > > > > > > > >> > > > > > >> > > > > > the
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > destination cluster
> >> topics.
> >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > If we were to use
> sequence
> >> > ids,
> >> > > > we
> >> > > > > > > would
> >> > > > > > > > >> need
> >> > > > > > > > >> > > > > > >> sufficiently
> >> > > > > > > > >> > > > > > >> > > > large
> >> > > > > > > > >> > > > > > >> > > > > > >>> cluster
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > ids to be stored with
> the
> >> > topic
> >> > > > > > > > >> identifiers or
> >> > > > > > > > >> > > we
> >> > > > > > > > >> > > > > run
> >> > > > > > > > >> > > > > > >> the
> >> > > > > > > > >> > > > > > >> > > risk
> >> > > > > > > > >> > > > > > >> > > > of
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > collisions. This will
> >> give up
> >> > > any
> >> > > > > > > > >> advantage in
> >> > > > > > > > >> > > > > > >> compactness
> >> > > > > > > > >> > > > > > >> > > that
> >> > > > > > > > >> > > > > > >> > > > > > >>> sequence
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > numbers may bring. Given
> >> > these
> >> > > > > > > > advantages I
> >> > > > > > > > >> > > think
> >> > > > > > > > >> > > > it
> >> > > > > > > > >> > > > > > >> makes
> >> > > > > > > > >> > > > > > >> > > > sense
> >> > > > > > > > >> > > > > > >> > > > > to
> >> > > > > > > > >> > > > > > >> > > > > > >>> use
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > UUIDs.
> >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > Gokul: This is an
> >> interesting
> >> > > > idea,
> >> > > > > > but
> >> > > > > > > > >> this
> >> > > > > > > > >> > is
> >> > > > > > > > >> > > a
> >> > > > > > > > >> > > > > > >> breaking
> >> > > > > > > > >> > > > > > >> > > > > change.
> >> > > > > > > > >> > > > > > >> > > > > > >>> Out of
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > scope for now, but maybe
> >> > worth
> >> > > > > > > discussing
> >> > > > > > > > >> in
> >> > > > > > > > >> > the
> >> > > > > > > > >> > > > > > future.
> >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > Hope this explains some
> of
> >> > the
> >> > > > > > > decisions,
> >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > Justine
> >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > On Fri, Sep 11, 2020 at
> >> 8:27
> >> > AM
> >> > > > > Gokul
> >> > > > > > > > >> Ramanan
> >> > > > > > > > >> > > > > > >> Subramanian <
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > gokul2411s@gmail.com>
> >> wrote:
> >> > > > > > > > >> > > > > > >> > > > > > >>> > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Hi.
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Thanks for the KIP.
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Have you thought about
> >> > > whether
> >> > > > it
> >> > > > > > > makes
> >> > > > > > > > >> > sense
> >> > > > > > > > >> > > to
> >> > > > > > > > >> > > > > > >> support
> >> > > > > > > > >> > > > > > >> > > > > > >>> authorizing a
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > principal for a topic
> ID
> >> > > rather
> >> > > > > > than
> >> > > > > > > a
> >> > > > > > > > >> topic
> >> > > > > > > > >> > > > name
> >> > > > > > > > >> > > > > to
> >> > > > > > > > >> > > > > > >> > > achieve
> >> > > > > > > > >> > > > > > >> > > > > > >>> tighter
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > security?
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Or is the topic ID
> >> > > > fundamentally
> >> > > > > an
> >> > > > > > > > >> internal
> >> > > > > > > > >> > > > > detail
> >> > > > > > > > >> > > > > > >> > similar
> >> > > > > > > > >> > > > > > >> > > > to
> >> > > > > > > > >> > > > > > >> > > > > > >>> epochs
> >> > > > > > > > >> > > > > > >> > > > > > >>> > used
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > in a bunch of other
> >> places
> >> > in
> >> > > > > > Kafka?
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Thanks.
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > On Fri, Sep 11, 2020
> at
> >> > 4:06
> >> > > PM
> >> > > > > > John
> >> > > > > > > > >> > Roesler <
> >> > > > > > > > >> > > > > > >> > > > > > vvcephei@apache.org>
> >> > > > > > > > >> > > > > > >> > > > > > >>> > wrote:
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Hello Justine,
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Thanks for the KIP!
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > I happen to have
> been
> >> > > > > confronted
> >> > > > > > > > >> recently
> >> > > > > > > > >> > > with
> >> > > > > > > > >> > > > > the
> >> > > > > > > > >> > > > > > >> need
> >> > > > > > > > >> > > > > > >> > > to
> >> > > > > > > > >> > > > > > >> > > > > keep
> >> > > > > > > > >> > > > > > >> > > > > > >>> > track of
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > a
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > large number of
> >> topics as
> >> > > > > > compactly
> >> > > > > > > > as
> >> > > > > > > > >> > > > > possible. I
> >> > > > > > > > >> > > > > > >> was
> >> > > > > > > > >> > > > > > >> > > > going
> >> > > > > > > > >> > > > > > >> > > > > to
> >> > > > > > > > >> > > > > > >> > > > > > >>> come
> >> > > > > > > > >> > > > > > >> > > > > > >>> > up
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > with some way to
> >> > dictionary
> >> > > > > > encode
> >> > > > > > > > the
> >> > > > > > > > >> > topic
> >> > > > > > > > >> > > > > names
> >> > > > > > > > >> > > > > > >> as
> >> > > > > > > > >> > > > > > >> > > > > integers,
> >> > > > > > > > >> > > > > > >> > > > > > >>> but
> >> > > > > > > > >> > > > > > >> > > > > > >>> > this
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > seems much better!
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Apologies if this
> has
> >> > been
> >> > > > > raised
> >> > > > > > > > >> before,
> >> > > > > > > > >> > > but
> >> > > > > > > > >> > > > > I’m
> >> > > > > > > > >> > > > > > >> > > wondering
> >> > > > > > > > >> > > > > > >> > > > > > >>> about the
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > choice of UUID vs
> >> > sequence
> >> > > > > number
> >> > > > > > > for
> >> > > > > > > > >> the
> >> > > > > > > > >> > > ids.
> >> > > > > > > > >> > > > > > >> > Typically,
> >> > > > > > > > >> > > > > > >> > > > > I’ve
> >> > > > > > > > >> > > > > > >> > > > > > >>> seen
> >> > > > > > > > >> > > > > > >> > > > > > >>> > UUIDs
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > in two situations:
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 1. When processes
> >> need to
> >> > > > > > generate
> >> > > > > > > > >> > > > non-colliding
> >> > > > > > > > >> > > > > > >> > > > identifiers
> >> > > > > > > > >> > > > > > >> > > > > > >>> without
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > coordination.
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 2. When the
> identifier
> >> > > needs
> >> > > > to
> >> > > > > > be
> >> > > > > > > > >> > > > “universally
> >> > > > > > > > >> > > > > > >> > unique”;
> >> > > > > > > > >> > > > > > >> > > > > I.e.,
> >> > > > > > > > >> > > > > > >> > > > > > >>> the
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > identifier needs to
> >> > > > distinguish
> >> > > > > > the
> >> > > > > > > > >> entity
> >> > > > > > > > >> > > > from
> >> > > > > > > > >> > > > > > all
> >> > > > > > > > >> > > > > > >> > other
> >> > > > > > > > >> > > > > > >> > > > > > >>> entities
> >> > > > > > > > >> > > > > > >> > > > > > >>> > that
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > could ever exist.
> >> This is
> >> > > > > useful
> >> > > > > > in
> >> > > > > > > > >> cases
> >> > > > > > > > >> > > > where
> >> > > > > > > > >> > > > > > >> > entities
> >> > > > > > > > >> > > > > > >> > > > from
> >> > > > > > > > >> > > > > > >> > > > > > all
> >> > > > > > > > >> > > > > > >> > > > > > >>> > kinds
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > of
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > systems get mixed
> >> > together,
> >> > > > > such
> >> > > > > > as
> >> > > > > > > > >> when
> >> > > > > > > > >> > > > dumping
> >> > > > > > > > >> > > > > > >> logs
> >> > > > > > > > >> > > > > > >> > > from
> >> > > > > > > > >> > > > > > >> > > > > all
> >> > > > > > > > >> > > > > > >> > > > > > >>> > processes
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > in
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a company into a
> >> common
> >> > > > system.
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Maybe I’m being
> >> > > > short-sighted,
> >> > > > > > but
> >> > > > > > > it
> >> > > > > > > > >> > > doesn’t
> >> > > > > > > > >> > > > > seem
> >> > > > > > > > >> > > > > > >> like
> >> > > > > > > > >> > > > > > >> > > > > either
> >> > > > > > > > >> > > > > > >> > > > > > >>> really
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > applies here. It
> seems
> >> > like
> >> > > > the
> >> > > > > > > > brokers
> >> > > > > > > > >> > > could
> >> > > > > > > > >> > > > > and
> >> > > > > > > > >> > > > > > >> would
> >> > > > > > > > >> > > > > > >> > > > > achieve
> >> > > > > > > > >> > > > > > >> > > > > > >>> > consensus
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > when creating a
> topic
> >> > > anyway,
> >> > > > > > which
> >> > > > > > > > is
> >> > > > > > > > >> all
> >> > > > > > > > >> > > > > that’s
> >> > > > > > > > >> > > > > > >> > > required
> >> > > > > > > > >> > > > > > >> > > > to
> >> > > > > > > > >> > > > > > >> > > > > > >>> > generate
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > non-colliding
> sequence
> >> > ids.
> >> > > > For
> >> > > > > > the
> >> > > > > > > > >> > second,
> >> > > > > > > > >> > > as
> >> > > > > > > > >> > > > > you
> >> > > > > > > > >> > > > > > >> > > mention,
> >> > > > > > > > >> > > > > > >> > > > > we
> >> > > > > > > > >> > > > > > >> > > > > > >>> could
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > assign
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a UUID to the
> cluster
> >> as
> >> > a
> >> > > > > whole,
> >> > > > > > > > which
> >> > > > > > > > >> > > would
> >> > > > > > > > >> > > > > > render
> >> > > > > > > > >> > > > > > >> > any
> >> > > > > > > > >> > > > > > >> > > > > > resource
> >> > > > > > > > >> > > > > > >> > > > > > >>> > scoped
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > the broker
> universally
> >> > > unique
> >> > > > > as
> >> > > > > > > > well.
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > The reason I mention
> >> this
> >> > > is
> >> > > > > > that,
> >> > > > > > > > >> > although
> >> > > > > > > > >> > > a
> >> > > > > > > > >> > > > > UUID
> >> > > > > > > > >> > > > > > >> is
> >> > > > > > > > >> > > > > > >> > way
> >> > > > > > > > >> > > > > > >> > > > > more
> >> > > > > > > > >> > > > > > >> > > > > > >>> > compact
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > than topic names,
> it’s
> >> > > still
> >> > > > 16
> >> > > > > > > > bytes.
> >> > > > > > > > >> In
> >> > > > > > > > >> > > > > > contrast,
> >> > > > > > > > >> > > > > > >> a
> >> > > > > > > > >> > > > > > >> > > > 4-byte
> >> > > > > > > > >> > > > > > >> > > > > > >>> integer
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > sequence id would
> give
> >> > us 4
> >> > > > > > billion
> >> > > > > > > > >> unique
> >> > > > > > > > >> > > > > topics
> >> > > > > > > > >> > > > > > >> per
> >> > > > > > > > >> > > > > > >> > > > > cluster,
> >> > > > > > > > >> > > > > > >> > > > > > >>> which
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > seems
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > like enough ;)
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Considering the
> >> number of
> >> > > > > > different
> >> > > > > > > > >> times
> >> > > > > > > > >> > > > these
> >> > > > > > > > >> > > > > > >> topic
> >> > > > > > > > >> > > > > > >> > > > > > >>> identifiers are
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > sent
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > over the wire or
> >> stored
> >> > in
> >> > > > > > memory,
> >> > > > > > > it
> >> > > > > > > > >> > seems
> >> > > > > > > > >> > > > like
> >> > > > > > > > >> > > > > > it
> >> > > > > > > > >> > > > > > >> > might
> >> > > > > > > > >> > > > > > >> > > > be
> >> > > > > > > > >> > > > > > >> > > > > > >>> worth
> >> > > > > > > > >> > > > > > >> > > > > > >>> > the
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > additional 4x space
> >> > > savings.
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > What do you think
> >> about
> >> > > this?
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Thanks,
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > John
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > On Fri, Sep 11,
> 2020,
> >> at
> >> > > > 03:20,
> >> > > > > > Tom
> >> > > > > > > > >> > Bentley
> >> > > > > > > > >> > > > > wrote:
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Hi Justine,
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > This looks like a
> >> very
> >> > > > > welcome
> >> > > > > > > > >> > > improvement.
> >> > > > > > > > >> > > > > > >> Thanks!
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Maybe I missed it,
> >> but
> >> > > the
> >> > > > > KIP
> >> > > > > > > > >> doesn't
> >> > > > > > > > >> > > seem
> >> > > > > > > > >> > > > to
> >> > > > > > > > >> > > > > > >> > mention
> >> > > > > > > > >> > > > > > >> > > > > > changing
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> DeleteTopicsRequest
> >> to
> >> > > > > identify
> >> > > > > > > the
> >> > > > > > > > >> > topic
> >> > > > > > > > >> > > > > using
> >> > > > > > > > >> > > > > > an
> >> > > > > > > > >> > > > > > >> > id.
> >> > > > > > > > >> > > > > > >> > > > > Maybe
> >> > > > > > > > >> > > > > > >> > > > > > >>> > that's out
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > of
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > scope, but
> >> > > > > DeleteTopicsRequest
> >> > > > > > is
> >> > > > > > > > not
> >> > > > > > > > >> > > listed
> >> > > > > > > > >> > > > > > among
> >> > > > > > > > >> > > > > > >> > the
> >> > > > > > > > >> > > > > > >> > > > > Future
> >> > > > > > > > >> > > > > > >> > > > > > >>> Work
> >> > > > > > > > >> > > > > > >> > > > > > >>> > APIs
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > either.
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Kind regards,
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Tom
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > On Thu, Sep 10,
> >> 2020 at
> >> > > > 3:59
> >> > > > > PM
> >> > > > > > > > >> Satish
> >> > > > > > > > >> > > > > Duggana <
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >> satish.duggana@gmail.com
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > wrote:
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Thanks
> >> Lucas/Justine
> >> > > for
> >> > > > > the
> >> > > > > > > nice
> >> > > > > > > > >> KIP.
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > It has several
> >> > benefits
> >> > > > > which
> >> > > > > > > > also
> >> > > > > > > > >> > > include
> >> > > > > > > > >> > > > > > >> > > simplifying
> >> > > > > > > > >> > > > > > >> > > > > the
> >> > > > > > > > >> > > > > > >> > > > > > >>> topic
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > deletion process
> >> by
> >> > > > > > controller
> >> > > > > > > > and
> >> > > > > > > > >> > logs
> >> > > > > > > > >> > > > > > cleanup
> >> > > > > > > > >> > > > > > >> by
> >> > > > > > > > >> > > > > > >> > > > > brokers
> >> > > > > > > > >> > > > > > >> > > > > > in
> >> > > > > > > > >> > > > > > >> > > > > > >>> > corner
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > cases.
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Best,
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Satish.
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > On Wed, Sep 9,
> >> 2020
> >> > at
> >> > > > > 10:07
> >> > > > > > PM
> >> > > > > > > > >> > Justine
> >> > > > > > > > >> > > > > > Olshan <
> >> > > > > > > > >> > > > > > >> > > > > > >>> > jolshan@confluent.io
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > wrote:
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Hello all,
> it's
> >> > been
> >> > > > > > almost a
> >> > > > > > > > >> year!
> >> > > > > > > > >> > > I've
> >> > > > > > > > >> > > > > > made
> >> > > > > > > > >> > > > > > >> > some
> >> > > > > > > > >> > > > > > >> > > > > > changes
> >> > > > > > > > >> > > > > > >> > > > > > >>> to
> >> > > > > > > > >> > > > > > >> > > > > > >>> > this
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > KIP
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > and hope to
> >> continue
> >> > > the
> >> > > > > > > > >> discussion.
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > One of the
> main
> >> > > changes
> >> > > > > > I've
> >> > > > > > > > >> added
> >> > > > > > > > >> > is
> >> > > > > > > > >> > > > now
> >> > > > > > > > >> > > > > > the
> >> > > > > > > > >> > > > > > >> > > > metadata
> >> > > > > > > > >> > > > > > >> > > > > > >>> response
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > will
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > include the
> topic
> >> ID
> >> > > (as
> >> > > > > > Colin
> >> > > > > > > > >> > > suggested).
> >> > > > > > > > >> > > > > > >> Clients
> >> > > > > > > > >> > > > > > >> > > can
> >> > > > > > > > >> > > > > > >> > > > > > >>> obtain the
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > topicID
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > of a given topic
> >> > > through
> >> > > > a
> >> > > > > > > > >> > > > TopicDescription.
> >> > > > > > > > >> > > > > > The
> >> > > > > > > > >> > > > > > >> > > > topicId
> >> > > > > > > > >> > > > > > >> > > > > > will
> >> > > > > > > > >> > > > > > >> > > > > > >>> > also be
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > included with
> the
> >> > > > > > > UpdateMetadata
> >> > > > > > > > >> > > request.
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Let me know
> what
> >> > you
> >> > > > all
> >> > > > > > > think.
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Thank you,
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Justine
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > On 2019/09/13
> >> > > 16:38:26,
> >> > > > > > > "Colin
> >> > > > > > > > >> > > McCabe" <
> >> > > > > > > > >> > > > > > >> > > > > > cmccabe@apache.org
> >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > wrote:
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Hi Lucas,
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Thanks for
> >> > tackling
> >> > > > > this.
> >> > > > > > > > >> Topic
> >> > > > > > > > >> > IDs
> >> > > > > > > > >> > > > > are a
> >> > > > > > > > >> > > > > > >> > great
> >> > > > > > > > >> > > > > > >> > > > > idea,
> >> > > > > > > > >> > > > > > >> > > > > > >>> and
> >> > > > > > > > >> > > > > > >> > > > > > >>> > this
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > is
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > really good
> >> writeup.
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > For
> >> > > > > > > /brokers/topics/[topic],
> >> > > > > > > > >> the
> >> > > > > > > > >> > > > schema
> >> > > > > > > > >> > > > > > >> version
> >> > > > > > > > >> > > > > > >> > > > > should
> >> > > > > > > > >> > > > > > >> > > > > > be
> >> > > > > > > > >> > > > > > >> > > > > > >>> > bumped
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > version 3,
> rather
> >> > than
> >> > > 2.
> >> > > > > > > > KIP-455
> >> > > > > > > > >> > > bumped
> >> > > > > > > > >> > > > > the
> >> > > > > > > > >> > > > > > >> > version
> >> > > > > > > > >> > > > > > >> > > > of
> >> > > > > > > > >> > > > > > >> > > > > > this
> >> > > > > > > > >> > > > > > >> > > > > > >>> > znode
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 2
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > already :)
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Given that
> >> we're
> >> > > > going
> >> > > > > to
> >> > > > > > > be
> >> > > > > > > > >> > seeing
> >> > > > > > > > >> > > > > these
> >> > > > > > > > >> > > > > > >> > things
> >> > > > > > > > >> > > > > > >> > > as
> >> > > > > > > > >> > > > > > >> > > > > > >>> strings
> >> > > > > > > > >> > > > > > >> > > > > > >>> > as
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > lot
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > (in
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > logs, in
> >> ZooKeeper,
> >> > on
> >> > > > the
> >> > > > > > > > >> > command-line,
> >> > > > > > > > >> > > > > > etc.),
> >> > > > > > > > >> > > > > > >> > does
> >> > > > > > > > >> > > > > > >> > > it
> >> > > > > > > > >> > > > > > >> > > > > > make
> >> > > > > > > > >> > > > > > >> > > > > > >>> > sense to
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > use
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > base64 when
> >> > converting
> >> > > > them
> >> > > > > > to
> >> > > > > > > > >> > strings?
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Here is an
> >> > example
> >> > > of
> >> > > > > the
> >> > > > > > > hex
> >> > > > > > > > >> > > > > > >> representation:
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >> > > > > > > > >> > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > And here is
> an
> >> > > > example
> >> > > > > in
> >> > > > > > > > >> base64.
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >> > > > b8tRS7h4TJ2Vt43Dp85v2A
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > The base64
> >> > version
> >> > > > > saves
> >> > > > > > 15
> >> > > > > > > > >> > letters
> >> > > > > > > > >> > > > (to
> >> > > > > > > > >> > > > > be
> >> > > > > > > > >> > > > > > >> > fair,
> >> > > > > > > > >> > > > > > >> > > 4
> >> > > > > > > > >> > > > > > >> > > > of
> >> > > > > > > > >> > > > > > >> > > > > > >>> those
> >> > > > > > > > >> > > > > > >> > > > > > >>> > were
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > dashes that we
> >> could
> >> > > have
> >> > > > > > > elided
> >> > > > > > > > in
> >> > > > > > > > >> > the
> >> > > > > > > > >> > > > hex
> >> > > > > > > > >> > > > > > >> > > > > > representation.)
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Another
> thing
> >> to
> >> > > > > consider
> >> > > > > > > is
> >> > > > > > > > >> that
> >> > > > > > > > >> > we
> >> > > > > > > > >> > > > > > should
> >> > > > > > > > >> > > > > > >> > > specify
> >> > > > > > > > >> > > > > > >> > > > > > that
> >> > > > > > > > >> > > > > > >> > > > > > >>> the
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > all-zeroes UUID
> is
> >> > not
> >> > > a
> >> > > > > > valid
> >> > > > > > > > >> topic
> >> > > > > > > > >> > > UUID.
> >> > > > > > > > >> > > > > >  We
> >> > > > > > > > >> > > > > > >> > can't
> >> > > > > > > > >> > > > > > >> > > > use
> >> > > > > > > > >> > > > > > >> > > > > > >>> null
> >> > > > > > > > >> > > > > > >> > > > > > >>> > for
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > this
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > because we can't
> >> > pass a
> >> > > > > null
> >> > > > > > > UUID
> >> > > > > > > > >> over
> >> > > > > > > > >> > > the
> >> > > > > > > > >> > > > > RPC
> >> > > > > > > > >> > > > > > >> > > protocol
> >> > > > > > > > >> > > > > > >> > > > > > >>> (there
> >> > > > > > > > >> > > > > > >> > > > > > >>> > is no
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > special pattern
> >> for
> >> > > null,
> >> > > > > nor
> >> > > > > > > do
> >> > > > > > > > we
> >> > > > > > > > >> > want
> >> > > > > > > > >> > > > to
> >> > > > > > > > >> > > > > > >> waste
> >> > > > > > > > >> > > > > > >> > > space
> >> > > > > > > > >> > > > > > >> > > > > > >>> reserving
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > such
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > pattern.)
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Maybe I
> missed
> >> > it,
> >> > > > but
> >> > > > > > did
> >> > > > > > > > you
> >> > > > > > > > >> > > > describe
> >> > > > > > > > >> > > > > > >> > > "migration
> >> > > > > > > > >> > > > > > >> > > > > > of...
> >> > > > > > > > >> > > > > > >> > > > > > >>> > existing
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > topic[s] without
> >> > topic
> >> > > > IDs"
> >> > > > > > in
> >> > > > > > > > >> detail
> >> > > > > > > > >> > in
> >> > > > > > > > >> > > > any
> >> > > > > > > > >> > > > > > >> > section?
> >> > > > > > > > >> > > > > > >> > > > It
> >> > > > > > > > >> > > > > > >> > > > > > >>> seems
> >> > > > > > > > >> > > > > > >> > > > > > >>> > like
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > when
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > the new
> controller
> >> > > > becomes
> >> > > > > > > > active,
> >> > > > > > > > >> it
> >> > > > > > > > >> > > > should
> >> > > > > > > > >> > > > > > >> just
> >> > > > > > > > >> > > > > > >> > > > > generate
> >> > > > > > > > >> > > > > > >> > > > > > >>> random
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > UUIDs for
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > these, and write
> >> the
> >> > > > random
> >> > > > > > > UUIDs
> >> > > > > > > > >> back
> >> > > > > > > > >> > > to
> >> > > > > > > > >> > > > > > >> > ZooKeeper.
> >> > > > > > > > >> > > > > > >> > > > It
> >> > > > > > > > >> > > > > > >> > > > > > >>> would be
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > good
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > to
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > spell that out.
> >> We
> >> > > > should
> >> > > > > > make
> >> > > > > > > > it
> >> > > > > > > > >> > clear
> >> > > > > > > > >> > > > > that
> >> > > > > > > > >> > > > > > >> this
> >> > > > > > > > >> > > > > > >> > > > > happens
> >> > > > > > > > >> > > > > > >> > > > > > >>> > regardless
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > of
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > the inter-broker
> >> > > protocol
> >> > > > > > > version
> >> > > > > > > > >> > (it's
> >> > > > > > > > >> > > a
> >> > > > > > > > >> > > > > > >> > compatible
> >> > > > > > > > >> > > > > > >> > > > > > change).
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >> > > "LeaderAndIsrRequests
> >> > > > > > > > >> including an
> >> > > > > > > > >> > > > > > >> > > > is_every_partition
> >> > > > > > > > >> > > > > > >> > > > > > >>> flag"
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > seems a
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > bit wordy.  Can
> we
> >> > just
> >> > > > > call
> >> > > > > > > > these
> >> > > > > > > > >> > "full
> >> > > > > > > > >> > > > > > >> > > > > > >>> LeaderAndIsrRequests"?
> >> > > > > > > > >> > > > > > >> > > > > > >>> > Then
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > the
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > RPC field could
> be
> >> > > named
> >> > > > > > > "full".
> >> > > > > > > > >> > Also,
> >> > > > > > > > >> > > it
> >> > > > > > > > >> > > > > > would
> >> > > > > > > > >> > > > > > >> > > > probably
> >> > > > > > > > >> > > > > > >> > > > > > be
> >> > > > > > > > >> > > > > > >> > > > > > >>> > better
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > for the
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > RPC field to be
> an
> >> > enum
> >> > > > of
> >> > > > > {
> >> > > > > > > > >> > > UNSPECIFIED,
> >> > > > > > > > >> > > > > > >> > > INCREMENTAL,
> >> > > > > > > > >> > > > > > >> > > > > FULL
> >> > > > > > > > >> > > > > > >> > > > > > >>> }, so
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > that
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > we
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > can cleanly
> handle
> >> > old
> >> > > > > > versions
> >> > > > > > > > (by
> >> > > > > > > > >> > > > treating
> >> > > > > > > > >> > > > > > >> them
> >> > > > > > > > >> > > > > > >> > as
> >> > > > > > > > >> > > > > > >> > > > > > >>> UNSPECIFIED)
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > In the
> >> > > > > > LeaderAndIsrRequest
> >> > > > > > > > >> > section,
> >> > > > > > > > >> > > > you
> >> > > > > > > > >> > > > > > >> write
> >> > > > > > > > >> > > > > > >> > "A
> >> > > > > > > > >> > > > > > >> > > > > final
> >> > > > > > > > >> > > > > > >> > > > > > >>> > deletion
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > event
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > will be
> secheduled
> >> > for
> >> > > X
> >> > > > ms
> >> > > > > > > after
> >> > > > > > > > >> the
> >> > > > > > > > >> > > > > > >> > > > LeaderAndIsrRequest
> >> > > > > > > > >> > > > > > >> > > > > > was
> >> > > > > > > > >> > > > > > >> > > > > > >>> > first
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > received..."  I
> >> guess
> >> > > > the X
> >> > > > > > > was a
> >> > > > > > > > >> > > > > placeholder
> >> > > > > > > > >> > > > > > >> that
> >> > > > > > > > >> > > > > > >> > > you
> >> > > > > > > > >> > > > > > >> > > > > > >>> intended
> >> > > > > > > > >> > > > > > >> > > > > > >>> > to
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > replace
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > before posting?
> :)
> >> > In
> >> > > > any
> >> > > > > > > case,
> >> > > > > > > > >> this
> >> > > > > > > > >> > > > seems
> >> > > > > > > > >> > > > > > like
> >> > > > > > > > >> > > > > > >> > the
> >> > > > > > > > >> > > > > > >> > > > kind
> >> > > > > > > > >> > > > > > >> > > > > > of
> >> > > > > > > > >> > > > > > >> > > > > > >>> > thing
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > we'd
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > want a
> >> configuration
> >> > > for.
> >> > > > > > > Let's
> >> > > > > > > > >> > > describe
> >> > > > > > > > >> > > > > that
> >> > > > > > > > >> > > > > > >> > > > > > configuration
> >> > > > > > > > >> > > > > > >> > > > > > >>> key
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > somewhere
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > in this KIP,
> >> > including
> >> > > > what
> >> > > > > > its
> >> > > > > > > > >> > default
> >> > > > > > > > >> > > > > value
> >> > > > > > > > >> > > > > > >> is.
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > We should
> >> > probably
> >> > > > also
> >> > > > > > > log a
> >> > > > > > > > >> > bunch
> >> > > > > > > > >> > > of
> >> > > > > > > > >> > > > > > >> messages
> >> > > > > > > > >> > > > > > >> > > at
> >> > > > > > > > >> > > > > > >> > > > > WARN
> >> > > > > > > > >> > > > > > >> > > > > > >>> level
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > when
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > something is
> >> > scheduled
> >> > > > for
> >> > > > > > > > >> deletion,
> >> > > > > > > > >> > as
> >> > > > > > > > >> > > > > well.
> >> > > > > > > > >> > > > > > >> > (Maybe
> >> > > > > > > > >> > > > > > >> > > > > this
> >> > > > > > > > >> > > > > > >> > > > > > >>> was
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > assumed, but
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > it would be good
> >> to
> >> > > > mention
> >> > > > > > > it).
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > I feel like
> >> there
> >> > > > are a
> >> > > > > > few
> >> > > > > > > > >> > sections
> >> > > > > > > > >> > > > > that
> >> > > > > > > > >> > > > > > >> > should
> >> > > > > > > > >> > > > > > >> > > be
> >> > > > > > > > >> > > > > > >> > > > > > >>> moved to
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > "rejected
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > alternatives."
> >> For
> >> > > > > example,
> >> > > > > > in
> >> > > > > > > > the
> >> > > > > > > > >> > > > > > DeleteTopics
> >> > > > > > > > >> > > > > > >> > > > section,
> >> > > > > > > > >> > > > > > >> > > > > > >>> since
> >> > > > > > > > >> > > > > > >> > > > > > >>> > we're
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > not
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > going to do
> >> option 1
> >> > or
> >> > > > 2,
> >> > > > > > > these
> >> > > > > > > > >> > should
> >> > > > > > > > >> > > be
> >> > > > > > > > >> > > > > > moved
> >> > > > > > > > >> > > > > > >> > into
> >> > > > > > > > >> > > > > > >> > > > > > >>> "rejected
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > alternatives,"
> >> > rather
> >> > > > than
> >> > > > > > > > >> appearing
> >> > > > > > > > >> > > > > inline.
> >> > > > > > > > >> > > > > > >> > > Another
> >> > > > > > > > >> > > > > > >> > > > > case
> >> > > > > > > > >> > > > > > >> > > > > > >>> is
> >> > > > > > > > >> > > > > > >> > > > > > >>> > the
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > "Should
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > we remove topic
> >> name
> >> > > from
> >> > > > > the
> >> > > > > > > > >> protocol
> >> > > > > > > > >> > > > where
> >> > > > > > > > >> > > > > > >> > > possible"
> >> > > > > > > > >> > > > > > >> > > > > > >>> section.
> >> > > > > > > > >> > > > > > >> > > > > > >>> > This
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > is
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > clearly
> >> discussing a
> >> > > > design
> >> > > > > > > > >> > alternative
> >> > > > > > > > >> > > > that
> >> > > > > > > > >> > > > > > >> we're
> >> > > > > > > > >> > > > > > >> > > not
> >> > > > > > > > >> > > > > > >> > > > > > >>> proposing
> >> > > > > > > > >> > > > > > >> > > > > > >>> > to
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > implement:
> >> removing
> >> > the
> >> > > > > topic
> >> > > > > > > > name
> >> > > > > > > > >> > from
> >> > > > > > > > >> > > > > those
> >> > > > > > > > >> > > > > > >> > > > protocols.
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Is it really
> >> > > > necessary
> >> > > > > to
> >> > > > > > > > have
> >> > > > > > > > >> a
> >> > > > > > > > >> > new
> >> > > > > > > > >> > > > > > >> > > > > > >>> > /admin/delete_topics_by_id
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > path
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > in ZooKeeper?
> It
> >> > seems
> >> > > > > like
> >> > > > > > we
> >> > > > > > > > >> don't
> >> > > > > > > > >> > > > really
> >> > > > > > > > >> > > > > > >> need
> >> > > > > > > > >> > > > > > >> > > this.
> >> > > > > > > > >> > > > > > >> > > > > > >>> Whenever
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > there is
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > a new
> controller,
> >> > we'll
> >> > > > > send
> >> > > > > > > out
> >> > > > > > > > >> full
> >> > > > > > > > >> > > > > > >> > > > > LeaderAndIsrRequests
> >> > > > > > > > >> > > > > > >> > > > > > >>> which
> >> > > > > > > > >> > > > > > >> > > > > > >>> > will
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > trigger the
> stale
> >> > > topics
> >> > > > to
> >> > > > > > be
> >> > > > > > > > >> cleaned
> >> > > > > > > > >> > > up.
> >> > > > > > > > >> > > > > >  The
> >> > > > > > > > >> > > > > > >> > > active
> >> > > > > > > > >> > > > > > >> > > > > > >>> > controller
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > will
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > also send the
> full
> >> > > > > > > > >> LeaderAndIsrRequest
> >> > > > > > > > >> > > to
> >> > > > > > > > >> > > > > > >> brokers
> >> > > > > > > > >> > > > > > >> > > that
> >> > > > > > > > >> > > > > > >> > > > > are
> >> > > > > > > > >> > > > > > >> > > > > > >>> just
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > starting
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > up.    So we
> don't
> >> > > really
> >> > > > > > need
> >> > > > > > > > this
> >> > > > > > > > >> > kind
> >> > > > > > > > >> > > > of
> >> > > > > > > > >> > > > > > >> > two-phase
> >> > > > > > > > >> > > > > > >> > > > > > commit
> >> > > > > > > > >> > > > > > >> > > > > > >>> > (send
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > out
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> >> StopReplicasRequest,
> >> > > get
> >> > > > > ACKs
> >> > > > > > > > from
> >> > > > > > > > >> all
> >> > > > > > > > >> > > > > nodes,
> >> > > > > > > > >> > > > > > >> > commit
> >> > > > > > > > >> > > > > > >> > > by
> >> > > > > > > > >> > > > > > >> > > > > > >>> removing
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> >> /admin/delete_topics
> >> > > > node)
> >> > > > > > any
> >> > > > > > > > >> more.
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > You mention
> >> that
> >> > > > > > > FetchRequest
> >> > > > > > > > >> will
> >> > > > > > > > >> > > now
> >> > > > > > > > >> > > > > > >> include
> >> > > > > > > > >> > > > > > >> > > UUID
> >> > > > > > > > >> > > > > > >> > > > > to
> >> > > > > > > > >> > > > > > >> > > > > > >>> avoid
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > issues
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > where requests
> are
> >> > made
> >> > > > to
> >> > > > > > > stale
> >> > > > > > > > >> > > > partitions.
> >> > > > > > > > >> > > > > > >> > > However,
> >> > > > > > > > >> > > > > > >> > > > > > >>> adding a
> >> > > > > > > > >> > > > > > >> > > > > > >>> > UUID
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > MetadataRequest
> is
> >> > > listed
> >> > > > > as
> >> > > > > > > > future
> >> > > > > > > > >> > > work,
> >> > > > > > > > >> > > > > out
> >> > > > > > > > >> > > > > > of
> >> > > > > > > > >> > > > > > >> > > scope
> >> > > > > > > > >> > > > > > >> > > > > for
> >> > > > > > > > >> > > > > > >> > > > > > >>> this
> >> > > > > > > > >> > > > > > >> > > > > > >>> > KIP.
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > How
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > will the client
> >> learn
> >> > > > what
> >> > > > > > the
> >> > > > > > > > >> topic
> >> > > > > > > > >> > > UUID
> >> > > > > > > > >> > > > > is,
> >> > > > > > > > >> > > > > > if
> >> > > > > > > > >> > > > > > >> > the
> >> > > > > > > > >> > > > > > >> > > > > > metadata
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > response
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > doesn't include
> >> that
> >> > > > > > > information?
> >> > > > > > > > >> It
> >> > > > > > > > >> > > > seems
> >> > > > > > > > >> > > > > > like
> >> > > > > > > > >> > > > > > >> > > adding
> >> > > > > > > > >> > > > > > >> > > > > the
> >> > > > > > > > >> > > > > > >> > > > > > >>> UUID
> >> > > > > > > > >> > > > > > >> > > > > > >>> > to
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > MetadataResponse
> >> > would
> >> > > be
> >> > > > > an
> >> > > > > > > > >> > improvement
> >> > > > > > > > >> > > > > here
> >> > > > > > > > >> > > > > > >> that
> >> > > > > > > > >> > > > > > >> > > > might
> >> > > > > > > > >> > > > > > >> > > > > > not
> >> > > > > > > > >> > > > > > >> > > > > > >>> be
> >> > > > > > > > >> > > > > > >> > > > > > >>> > too
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > hard to
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > make.
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > best,
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Colin
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > On Mon, Sep
> 9,
> >> > > 2019,
> >> > > > at
> >> > > > > > > > 17:48,
> >> > > > > > > > >> > > Ryanne
> >> > > > > > > > >> > > > > > Dolan
> >> > > > > > > > >> > > > > > >> > > wrote:
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > Lucas,
> this
> >> > would
> >> > > > be
> >> > > > > > > great.
> >> > > > > > > > >> I've
> >> > > > > > > > >> > > run
> >> > > > > > > > >> > > > > > into
> >> > > > > > > > >> > > > > > >> > > issues
> >> > > > > > > > >> > > > > > >> > > > > with
> >> > > > > > > > >> > > > > > >> > > > > > >>> > topics
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > being
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> resurrected
> >> > > > > > accidentally,
> >> > > > > > > > >> since
> >> > > > > > > > >> > a
> >> > > > > > > > >> > > > > client
> >> > > > > > > > >> > > > > > >> > cannot
> >> > > > > > > > >> > > > > > >> > > > > > easily
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > distinguish
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > between
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > a deleted
> >> topic
> >> > > > and a
> >> > > > > > new
> >> > > > > > > > >> topic
> >> > > > > > > > >> > > with
> >> > > > > > > > >> > > > > the
> >> > > > > > > > >> > > > > > >> same
> >> > > > > > > > >> > > > > > >> > > > name.
> >> > > > > > > > >> > > > > > >> > > > > > I'd
> >> > > > > > > > >> > > > > > >> > > > > > >>> > need
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > the
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > ID
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > accessible
> >> from
> >> > > the
> >> > > > > > > client
> >> > > > > > > > to
> >> > > > > > > > >> > > solve
> >> > > > > > > > >> > > > > that
> >> > > > > > > > >> > > > > > >> > issue,
> >> > > > > > > > >> > > > > > >> > > > but
> >> > > > > > > > >> > > > > > >> > > > > > >>> this
> >> > > > > > > > >> > > > > > >> > > > > > >>> > is a
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > good
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > first
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > step.
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > Ryanne
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > On Wed,
> Sep
> >> 4,
> >> > > 2019
> >> > > > > at
> >> > > > > > > 1:41
> >> > > > > > > > >> PM
> >> > > > > > > > >> > > Lucas
> >> > > > > > > > >> > > > > > >> > > Bradstreet <
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > lucas@confluent.io>
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > wrote:
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Hi all,
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > I would
> >> like
> >> > to
> >> > > > > kick
> >> > > > > > > off
> >> > > > > > > > >> > > > discussion
> >> > > > > > > > >> > > > > of
> >> > > > > > > > >> > > > > > >> > > KIP-516,
> >> > > > > > > > >> > > > > > >> > > > > an
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > implementation
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > of topic
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > IDs for
> >> > Kafka.
> >> > > > > Topic
> >> > > > > > > IDs
> >> > > > > > > > >> aim
> >> > > > > > > > >> > to
> >> > > > > > > > >> > > > > solve
> >> > > > > > > > >> > > > > > >> topic
> >> > > > > > > > >> > > > > > >> > > > > > >>> uniqueness
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > problems in
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Kafka,
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > where
> >> > referring
> >> > > > to
> >> > > > > a
> >> > > > > > > > topic
> >> > > > > > > > >> by
> >> > > > > > > > >> > > name
> >> > > > > > > > >> > > > > > >> alone is
> >> > > > > > > > >> > > > > > >> > > > > > >>> insufficient.
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Such
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > cases
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > include
> >> when
> >> > a
> >> > > > > topic
> >> > > > > > > has
> >> > > > > > > > >> been
> >> > > > > > > > >> > > > > deleted
> >> > > > > > > > >> > > > > > >> and
> >> > > > > > > > >> > > > > > >> > > > > recreated
> >> > > > > > > > >> > > > > > >> > > > > > >>> with
> >> > > > > > > > >> > > > > > >> > > > > > >>> > the
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > same
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > name.
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Unique
> >> > > > identifiers
> >> > > > > > will
> >> > > > > > > > >> help
> >> > > > > > > > >> > > > > simplify
> >> > > > > > > > >> > > > > > >> and
> >> > > > > > > > >> > > > > > >> > > > improve
> >> > > > > > > > >> > > > > > >> > > > > > >>> Kafka's
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > topic
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > deletion
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > process,
> >> as
> >> > > well
> >> > > > as
> >> > > > > > > > prevent
> >> > > > > > > > >> > > cases
> >> > > > > > > > >> > > > > > where
> >> > > > > > > > >> > > > > > >> > > brokers
> >> > > > > > > > >> > > > > > >> > > > > may
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > incorrectly
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > interact
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > with
> stale
> >> > > > versions
> >> > > > > > of
> >> > > > > > > > >> topics.
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >> > > > > > > > >> > > > > > >> > > > > > >>>
> >> > > > > > > > >> > > > > > >> > > > > >
> >> > > > > > > > >> > > > > > >> > > > >
> >> > > > > > > > >> > > > > > >> > > >
> >> > > > > > > > >> > > > > > >> > >
> >> > > > > > > > >> > > > > > >> >
> >> > > > > > > > >> > > > > > >>
> >> > > > > > > > >> > > > > >
> >> > > > > > > > >> > > > >
> >> > > > > > > > >> > > >
> >> > > > > > > > >> > >
> >> > > > > > > > >> >
> >> > > > > > > > >>
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Looking
> >> > forward
> >> > > > to
> >> > > > > > your
> >> > > > > > > > >> > > thoughts.
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Lucas
> >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >> > > > > > > > >> > > > > > >> > > > > > >>> >
> >> > > > > > > > >> > > > > > >> > > > > > >>>
> >> > > > > > > > >> > > > > > >> > > > > > >>
> >> > > > > > > > >> > > > > > >> > > > > >
> >> > > > > > > > >> > > > > > >> > > > >
> >> > > > > > > > >> > > > > > >> > > >
> >> > > > > > > > >> > > > > > >> > >
> >> > > > > > > > >> > > > > > >> >
> >> > > > > > > > >> > > > > > >>
> >> > > > > > > > >> > > > > > >
> >> > > > > > > > >> > > > > >
> >> > > > > > > > >> > > > >
> >> > > > > > > > >> > > >
> >> > > > > > > > >> > >
> >> > > > > > > > >> >
> >> > > > > > > > >>
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Justine Olshan <jo...@confluent.io>]

Hello again,

I've taken some time to discuss some of the remaining points brought up by
the previous emails offline. Here are some of the conclusions.

1. Directory Structure:
There was some talk about whether the directory structure should be changed
to replace all topic names with topic IDs.
This will be a useful change, but will prevent downgrades. It will be best
to wait until a major release, likely alongside KIP-500 changes that will
prevent downgrades. I've updated the KIP to include this change with the
note about migration and deprecation.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-log.dirlayout

2. Partition Metadata file
There was some disagreement about the inclusion of the partition metadata
file.
This file will be needed to persist the topic ID, especially while we still
have the old directory structure. Even after the changes, this file can be
useful for debugging and recovery.
Creating a single mapping file from topic names to topic IDs was
considered, but ultimately scrapped as it would not be as easy to maintain.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PartitionMetadatafile

https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PersistingTopicIDs

3. Produce Protocols
After some further discussion, this replacing the topic name with topic ID
in these protocols has been added to the KIP.
This will cut down on the size of the protocol. Since changes to fetch are
included, it does make sense to update these protocols.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-Produce

4. KIP-500 Compatibility
After some discussion with those involved with KIP-500, it seems best to
use a sentinel topic ID for the metadata topic that is used before the
first controller election.
We had an issue where this metadata topic may not be assigned an ID before
utilizing the Vote and Fetch protocols. It was decided to reserve a unique
ID for this topic to be used until the controller could give the topic a
unique ID.
Having the topic keep the sentinel ID (not be reassigned to a unique ID)
was considered, but it seemed like a good idea to leave the option open for
the metadata topic to have a unique ID in cases where it would need to be
differentiated from other clusters' metadata topics. (ex. tiered storage)
https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-CompatibilitywithKIP-500

I've also split up the KIP into sub-tasks on the JIRA. Hopefully this will
provide a better idea about what tasks we have, and eventually provide a
place to see what's done and what is left.
If there is a task I am missing, please let me know!
https://issues.apache.org/jira/browse/KAFKA-8872

Of course, these decisions are not set in stone, and I would love to hear
any feedback.

Thanks,
Justine

On Mon, Sep 28, 2020 at 11:38 AM Justine Olshan <jo...@confluent.io>
wrote:

> Hello all,
>
> I just wanted to follow up on this discussion. Did we come to an
> understanding about the directory structure?
>
> I think the biggest question here is what is acceptable to leave out due
> to scope vs. what is considered to be too much tech debt.
> This KIP is already pretty large with the number of changes, but it also
> makes sense to do things correctly, so I'd love to hear everyone's thoughts.
>
> Thanks,
> Justine
>
> On Fri, Sep 25, 2020 at 8:19 AM Lucas Bradstreet <lu...@confluent.io>
> wrote:
>
>> Hi Ismael,
>>
>> If you do not store it in a metadata file or in the directory structure
>> would you then
>> require the LeaderAndIsrRequest following startup to give you some notion
>> of
>> topic name in memory? We would still need this information for the older
>> protocols, but
>> perhaps this is what's meant by tech debt.
>>
>> Once we're free of the old non-topicID requests then I think you wouldn't
>> need to retain the topic name.
>> I think the ability to easily look up topic names associated with
>> partition
>> directories would still be missed
>> when diagnosing issues, though maybe it wouldn't be a deal breaker with
>> the
>> right tooling.
>>
>> Thanks,
>>
>> Lucas
>>
>> On Fri, Sep 25, 2020 at 7:55 AM Ismael Juma <is...@juma.me.uk> wrote:
>>
>> > Hi Lucas,
>> >
>> > Why would you include the name and id? I think you'd want to remove the
>> > name from the directory name right? Jason's suggestion was that if you
>> > remove the name from the directory, then why would you need the id name
>> > mapping file?
>> >
>> > Ismael
>> >
>> > On Thu, Sep 24, 2020 at 4:24 PM Lucas Bradstreet <lu...@confluent.io>
>> > wrote:
>> >
>> > > > 2. Part of the usage of the file is to have persistent storage of
>> the
>> > > topic
>> > > ID and use it to compare with the ID supplied in the LeaderAndIsr
>> > Request.
>> > > There is some discussion in the KIP about changes to the directory
>> > > structure, but I believe directory changes were considered to be out
>> of
>> > > scope when the KIP was written.
>> > >
>> > >
>> > > Yeah, I was hoping to get a better understanding of why it was taken
>> out
>> > of
>> > > scope. Perhaps Lucas Bradstreet might have more insight about the
>> > decision.
>> > > Basically my point is that we have to create additional infrastructure
>> > here
>> > > to support the name/id mapping, so I wanted to understand if we just
>> > > consider this a sort of tech debt. It would be useful to cover how we
>> > > handle the case when this file gets corrupted. Seems like we just
>> have to
>> > > trust that it matches whatever the controller tells us and rewrite it?
>> > >
>> > >
>> > > Hi Jason, Justine,
>> > >
>> > > My thought process is that we will likely need the metadata file
>> whether
>> > we
>> > > rename the directories or not.
>> > > Linux supports filenames of up to 255 bytes and that would not be
>> enough
>> > to
>> > > support a directory name
>> > >  including both the name and topic ID. Given that fact, it seemed
>> > > reasonable to add the metadata file
>> > > and leave the directory rename until some time in the future (possibly
>> > > never).
>> > >
>> > > If the file does get corrupted I think you're right. We would either
>> have
>> > > to trust it matches what the controller tells us
>> > >  or error out and let an administrator resolve it by checking across
>> > > replicas for consistency.
>> > >
>> > > Lucas
>> > >
>> > >
>> > > On Thu, Sep 24, 2020 at 3:41 PM Jason Gustafson <ja...@confluent.io>
>> > > wrote:
>> > >
>> > > > Thanks Justine. Responses below:
>> > > >
>> > > > > 1. Yes, the directory will still be based on the topic names.
>> > > > LeaderAndIsrRequest is one of the few requests that will still
>> contain
>> > > the
>> > > > topic name. So I think we have this covered. Sorry for confusion.
>> > > >
>> > > > Ah, you're right. My eyes passed right over the field.
>> > > >
>> > > > > 2. Part of the usage of the file is to have persistent storage of
>> the
>> > > > topic
>> > > > ID and use it to compare with the ID supplied in the LeaderAndIsr
>> > > Request.
>> > > > There is some discussion in the KIP about changes to the directory
>> > > > structure, but I believe directory changes were considered to be
>> out of
>> > > > scope when the KIP was written.
>> > > >
>> > > > Yeah, I was hoping to get a better understanding of why it was taken
>> > out
>> > > of
>> > > > scope. Perhaps Lucas Bradstreet might have more insight about the
>> > > decision.
>> > > > Basically my point is that we have to create additional
>> infrastructure
>> > > here
>> > > > to support the name/id mapping, so I wanted to understand if we just
>> > > > consider this a sort of tech debt. It would be useful to cover how
>> we
>> > > > handle the case when this file gets corrupted. Seems like we just
>> have
>> > to
>> > > > trust that it matches whatever the controller tells us and rewrite
>> it?
>> > > >
>> > > > > 3. I think this is a good point, but I again I wonder about the
>> scope
>> > > of
>> > > > the KIP. Most of the changes mentioned in the KIP are for supporting
>> > > topic
>> > > > deletion and I believe that is why the produce request was listed
>> under
>> > > > future work.
>> > > >
>> > > > That's fair. I brought it up since `Fetch` is already included. If
>> > we've
>> > > > got `Metadata` and `Fetch`, seems we may as well do `Produce` and
>> save
>> > an
>> > > > extra kip. No strong objection though if you want to leave it out.
>> > > >
>> > > >
>> > > > -Jason
>> > > >
>> > > >
>> > > > On Thu, Sep 24, 2020 at 3:26 PM Justine Olshan <
>> jolshan@confluent.io>
>> > > > wrote:
>> > > >
>> > > > > Hi Jason,
>> > > > >
>> > > > > Thanks for your comments.
>> > > > >
>> > > > > 1. Yes, the directory will still be based on the topic names.
>> > > > > LeaderAndIsrRequest is one of the few requests that will still
>> > contain
>> > > > the
>> > > > > topic name. So I think we have this covered. Sorry for confusion.
>> > > > >
>> > > > > 2. Part of the usage of the file is to have persistent storage of
>> the
>> > > > topic
>> > > > > ID and use it to compare with the ID supplied in the LeaderAndIsr
>> > > > Request.
>> > > > > There is some discussion in the KIP about changes to the directory
>> > > > > structure, but I believe directory changes were considered to be
>> out
>> > of
>> > > > > scope when the KIP was written.
>> > > > >
>> > > > > 3. I think this is a good point, but I again I wonder about the
>> scope
>> > > of
>> > > > > the KIP. Most of the changes mentioned in the KIP are for
>> supporting
>> > > > topic
>> > > > > deletion and I believe that is why the produce request was listed
>> > under
>> > > > > future work.
>> > > > >
>> > > > > 4. This sounds like it might be a good solution, but I will need
>> to
>> > > > discuss
>> > > > > more with KIP-500 folks to get the details right.
>> > > > >
>> > > > > Thanks,
>> > > > > Justine
>> > > > >
>> > > > > On Thu, Sep 24, 2020 at 12:30 PM Jason Gustafson <
>> jason@confluent.io
>> > >
>> > > > > wrote:
>> > > > >
>> > > > > > Hi Justine,
>> > > > > >
>> > > > > > Thanks for picking up this work. I have a few
>> questions/comments:
>> > > > > >
>> > > > > > 1. It sounds like the directory structure is still going to be
>> > based
>> > > on
>> > > > > > topic names. Do I have that right? One complication is that the
>> > > > > > LeaderAndIsr request does not include the topic name any longer.
>> > This
>> > > > > means
>> > > > > > that a replica must wait for the UpdateMetadata request to
>> arrive
>> > > with
>> > > > > the
>> > > > > > topic name mapping before it can create the directory. I wonder
>> if
>> > it
>> > > > > would
>> > > > > > be simpler to avoid assumptions on the ordering of
>> UpdateMetadata
>> > and
>> > > > let
>> > > > > > LeaderAndIsr include the topic name as well. Feels like we are
>> not
>> > > > saving
>> > > > > > that much by excluding it.
>> > > > > >
>> > > > > > 2. On a related note, it seems that the reason we have the
>> > partition
>> > > > > > metadata file is because we are not changing the directory
>> > structure.
>> > > > We
>> > > > > > need it so that we remember which directories map to which topic
>> > id.
>> > > I
>> > > > > > think it would be useful to add some detail about why changing
>> the
>> > > > > > directory layout was rejected. Basically trying to understand
>> how
>> > > much
>> > > > > > effort we are saving by skipping this step if we have to
>> implement
>> > > this
>> > > > > > additional bookkeeping. One concern I have for example is that
>> the
>> > > > > > partition metadata file gets corrupt and then all bets are off
>> as
>> > far
>> > > > as
>> > > > > > topic consistency.
>> > > > > >
>> > > > > > 3. Is it worth adding support now for topic ids in the `Produce`
>> > > > request
>> > > > > > now? Overhead is mentioned as one of the motivations and the
>> best
>> > > APIs
>> > > > to
>> > > > > > get savings from are `Produce` and `Fetch`.
>> > > > > >
>> > > > > > 4. When it comes to bootstrapping the metadata topic with
>> respect
>> > to
>> > > > > > KIP-500, one suggestion would be to reserve a sentinel topic ID
>> > which
>> > > > can
>> > > > > > be used initially by new brokers. When the first controller is
>> > > elected,
>> > > > > it
>> > > > > > can assign a topicId to the metadata topic. This is a bit
>> similar
>> > to
>> > > > how
>> > > > > we
>> > > > > > were planning to generate the clusterId.
>> > > > > >
>> > > > > > Thanks,
>> > > > > > Jason
>> > > > > >
>> > > > > >
>> > > > > > On Thu, Sep 24, 2020 at 11:10 AM Jun Rao <ju...@confluent.io>
>> wrote:
>> > > > > >
>> > > > > > > Hi, Justine,
>> > > > > > >
>> > > > > > > Thanks for the updated KIP. A few more comments below.
>> > > > > > >
>> > > > > > > 30.  {"name": "id", "type": "string", "doc": "version id"}}:
>> The
>> > > doc
>> > > > > > should
>> > > > > > > say UUID.
>> > > > > > >
>> > > > > > > 31. LeaderAndIsrResponse v5 and StopReplicaResponse v4 : It
>> seems
>> > > > there
>> > > > > > is
>> > > > > > > no need to add topic_id at partitions level.
>> > > > > > >
>> > > > > > > 32. Regarding partition metadata file. Perhaps the key can be
>> a
>> > > > single
>> > > > > > > word, sth like the following.
>> > > > > > > version: 0
>> > > > > > > topic_id: 46bdb63f-9e8d-4a38-bf7b-ee4eb2a794e4
>> > > > > > >
>> > > > > > > 33. Another tricky thing that I realized is how to support the
>> > > > metadata
>> > > > > > > topic introduced in KIP-595. It's a bit tricky to assign a
>> UUID
>> > to
>> > > > the
>> > > > > > > metadata topic since we have a chicken-and-egg problem. The
>> > > > controller
>> > > > > > > needs to persist the UUID in the metadata topic in order to
>> > assign
>> > > > one
>> > > > > > > successfully, but the metadata topic is needed to elect a
>> > > controller
>> > > > > > > (KIP-631). So, this probably needs a bit more thought.
>> > > > > > >
>> > > > > > > Jun
>> > > > > > >
>> > > > > > > On Thu, Sep 24, 2020 at 3:04 AM Ismael Juma <
>> ismael@juma.me.uk>
>> > > > wrote:
>> > > > > > >
>> > > > > > > > Also, can we provide more details on how the Partition
>> Metadata
>> > > > file
>> > > > > > will
>> > > > > > > > be used?
>> > > > > > > >
>> > > > > > > > Ismael
>> > > > > > > >
>> > > > > > > > On Thu, Sep 24, 2020 at 3:01 AM Ismael Juma <
>> ismael@juma.me.uk
>> > >
>> > > > > wrote:
>> > > > > > > >
>> > > > > > > > > Hi Justine,
>> > > > > > > > >
>> > > > > > > > > I think we need to update the "Rejected Alternatives"
>> section
>> > > to
>> > > > > take
>> > > > > > > > into
>> > > > > > > > > account that the proposal now removes the topic name from
>> the
>> > > > fetch
>> > > > > > > > > request. Also, if we are removing it from the Fetch
>> request,
>> > > does
>> > > > > it
>> > > > > > > make
>> > > > > > > > > sense not to remove it from similar requests like
>> > > > > ListOffsetRequest?
>> > > > > > > > >
>> > > > > > > > > Ismael
>> > > > > > > > >
>> > > > > > > > > On Thu, Sep 24, 2020 at 2:46 AM David Jacot <
>> > > djacot@confluent.io
>> > > > >
>> > > > > > > wrote:
>> > > > > > > > >
>> > > > > > > > >> Hi Justine,
>> > > > > > > > >>
>> > > > > > > > >> Thanks for the KIP. I finally had time to read it :). It
>> is
>> > a
>> > > > > great
>> > > > > > > > >> improvement.
>> > > > > > > > >>
>> > > > > > > > >> I have a few comments/questions:
>> > > > > > > > >>
>> > > > > > > > >> 1. It seems that the schema of the StopReplicaRequest is
>> > > > slightly
>> > > > > > > > >> outdated.
>> > > > > > > > >> We
>> > > > > > > > >> did some changes as part of KIP-570. V3 is already
>> organized
>> > > by
>> > > > > > > topics.
>> > > > > > > > >>
>> > > > > > > > >> 2. I just want to make sure that I understand the
>> > > reconciliation
>> > > > > > > > >> logic correctly. When
>> > > > > > > > >> an "INCREMENTAL" LeaderAndIsr Request is received, the
>> > broker
>> > > > will
>> > > > > > > also
>> > > > > > > > >> reconcile
>> > > > > > > > >> when the local uuid does not match the uuid in the
>> request,
>> > > > right?
>> > > > > > In
>> > > > > > > > this
>> > > > > > > > >> case, the
>> > > > > > > > >> local log is staged for deletion.
>> > > > > > > > >>
>> > > > > > > > >> 3. In the documentation of the `
>> delete.stale.topic.delay.ms
>> > `
>> > > > > > config,
>> > > > > > > it
>> > > > > > > > >> says "When a
>> > > > > > > > >> FULL LeaderAndIsrRequest is received..." but I suppose it
>> > > > applies
>> > > > > to
>> > > > > > > > both
>> > > > > > > > >> types.
>> > > > > > > > >>
>> > > > > > > > >> Best,
>> > > > > > > > >> David
>> > > > > > > > >>
>> > > > > > > > >> On Thu, Sep 24, 2020 at 1:40 AM Justine Olshan <
>> > > > > > jolshan@confluent.io>
>> > > > > > > > >> wrote:
>> > > > > > > > >>
>> > > > > > > > >> > Hi Jun,
>> > > > > > > > >> >
>> > > > > > > > >> > Thanks for the comments. I apologize for some of the
>> typos
>> > > and
>> > > > > > > > >> confusion.
>> > > > > > > > >> > I’ve updated the KIP to fix some of the issues you
>> > > mentioned.
>> > > > > > > > >> >
>> > > > > > > > >> > 20.2 I’ve changed the type to String.
>> > > > > > > > >> > 20.1/3 I’ve updated the TopicZNode to fix formatting
>> and
>> > > > reflect
>> > > > > > the
>> > > > > > > > >> latest
>> > > > > > > > >> > version before this change.
>> > > > > > > > >> >
>> > > > > > > > >> > 21. You are correct and I’ve removed this line. I’ve
>> also
>> > > > added
>> > > > > a
>> > > > > > > line
>> > > > > > > > >> > mentioning an IBP bump is necessary for migration
>> > > > > > > > >> >
>> > > > > > > > >> > 22. I think the wording was unclear but your summary is
>> > what
>> > > > was
>> > > > > > > > >> intended
>> > > > > > > > >> > by this line. I’ve updated to make this point more
>> clear.
>> > > > > “Remove
>> > > > > > > > >> deleted
>> > > > > > > > >> > topics from replicas by sending StopReplicaRequest V3
>> > before
>> > > > the
>> > > > > > IBP
>> > > > > > > > >> bump
>> > > > > > > > >> > using the old logic, and using V4 and the new logic
>> with
>> > > topic
>> > > > > IDs
>> > > > > > > > after
>> > > > > > > > >> > the IBP bump.”
>> > > > > > > > >> >
>> > > > > > > > >> > 23. I’ve removed the unspecified type from the KIP and
>> > > mention
>> > > > > > that
>> > > > > > > > IBP
>> > > > > > > > >> > will be used to handle this request. “IBP will be used
>> to
>> > > > > > determine
>> > > > > > > > >> whether
>> > > > > > > > >> > this new form of the request will be used. For older
>> > > requests,
>> > > > > we
>> > > > > > > will
>> > > > > > > > >> > ignore this field and default to previous behavior.”
>> > > > > > > > >> >
>> > > > > > > > >> > 24. I’ve fixed this typo.
>> > > > > > > > >> >
>> > > > > > > > >> > 25. I've added a topics at a higher level for
>> > > > > LeaderAndIsrResponse
>> > > > > > > v5,
>> > > > > > > > >> > StopReplicaResponse v4. I've also changed
>> > StopReplicaRequest
>> > > > v4
>> > > > > to
>> > > > > > > > have
>> > > > > > > > >> > topics at a higher level.
>> > > > > > > > >> >
>> > > > > > > > >> > 26. I’ve updated forgotten_topics_data--added the
>> topic ID
>> > > and
>> > > > > > > removed
>> > > > > > > > >> the
>> > > > > > > > >> > topic name
>> > > > > > > > >> >
>> > > > > > > > >> > 27. I’ve decided on plain text, and I’ve added an
>> example.
>> > > > > > > > >> >
>> > > > > > > > >> > 28. I’ve added this idea to future work.
>> > > > > > > > >> >
>> > > > > > > > >> > Thanks again for taking a look,
>> > > > > > > > >> >
>> > > > > > > > >> > Justine
>> > > > > > > > >> >
>> > > > > > > > >> > On Wed, Sep 23, 2020 at 10:28 AM Jun Rao <
>> > jun@confluent.io>
>> > > > > > wrote:
>> > > > > > > > >> >
>> > > > > > > > >> > > Hi, Justine,
>> > > > > > > > >> > >
>> > > > > > > > >> > > Thanks for the response. Made another pass. A few
>> more
>> > > > > comments
>> > > > > > > > below.
>> > > > > > > > >> > >
>> > > > > > > > >> > > 20. znode schema:
>> > > > > > > > >> > > 20.1 It seems that {"name": "version", "type": "int",
>> > > "id":
>> > > > > > > "UUID",
>> > > > > > > > >> > "doc":
>> > > > > > > > >> > > "version id"} should be {"name": "version", "type":
>> > > "int"},
>> > > > > > > {"name":
>> > > > > > > > >> > "id",
>> > > > > > > > >> > > "type": "UUID", "doc": "version id"}.
>> > > > > > > > >> > > 20.2 The znode format is JSON which doesn't have UUID
>> > > type.
>> > > > So
>> > > > > > the
>> > > > > > > > >> type
>> > > > > > > > >> > > probably should be string?
>> > > > > > > > >> > > 20.3 Also, the existing format used seems outdated.
>> It
>> > > > should
>> > > > > > have
>> > > > > > > > the
>> > > > > > > > >> > > following format.
>> > > > > > > > >> > >     Json.encodeAsBytes(Map(
>> > > > > > > > >> > >       "version" -> 2,
>> > > > > > > > >> > >       "partitions" -> replicaAssignmentJson.asJava,
>> > > > > > > > >> > >       "adding_replicas" ->
>> > > > > addingReplicasAssignmentJson.asJava,
>> > > > > > > > >> > >       "removing_replicas" ->
>> > > > > > removingReplicasAssignmentJson.asJava
>> > > > > > > > >> > >     ).asJava)
>> > > > > > > > >> > >   }
>> > > > > > > > >> > >
>> > > > > > > > >> > > 21. Migration: The KIP says "The migration process
>> can
>> > > take
>> > > > > > place
>> > > > > > > > >> without
>> > > > > > > > >> > > an inter-broker protocol bump, as the format stored
>> in
>> > > > > > > > >> > > /brokers/topics/[topic] will be compatible with older
>> > > broker
>> > > > > > > > >> versions."
>> > > > > > > > >> > > However, since we are bumping up the version of
>> > > inter-broker
>> > > > > > > > >> requests, it
>> > > > > > > > >> > > seems that we need to use IBP for migration.
>> > > > > > > > >> > >
>> > > > > > > > >> > > 22. The KIP says "Remove deleted topics from
>> replicas by
>> > > > > sending
>> > > > > > > > >> > > StopReplicaRequest V3 for any topics which do not
>> > contain
>> > > a
>> > > > > > topic
>> > > > > > > > ID,
>> > > > > > > > >> and
>> > > > > > > > >> > > V4 for any topics which do contain a topic ID.".
>> > However,
>> > > if
>> > > > > we
>> > > > > > > use
>> > > > > > > > >> IBP,
>> > > > > > > > >> > it
>> > > > > > > > >> > > seems that the controller will either send
>> > > > StopReplicaRequest
>> > > > > V3
>> > > > > > > > >> > > or StopReplicaRequest V4, but never mixed V3 and V4
>> for
>> > > > > > different
>> > > > > > > > >> topics.
>> > > > > > > > >> > > Basically, before the IBP bump, V3 will be used.
>> After
>> > the
>> > > > IBP
>> > > > > > > bump,
>> > > > > > > > >> > > topicId will be created and V4 will be used.
>> > > > > > > > >> > >
>> > > > > > > > >> > > 23. Given that we depend on IBP, do we still need "0
>> > > > > > UNSPECIFIED"
>> > > > > > > > >> > > in LeaderAndIsr?
>> > > > > > > > >> > >
>> > > > > > > > >> > > 24. LeaderAndIsrResponse v5 : It still has the topic
>> > > field.
>> > > > > > > > >> > >
>> > > > > > > > >> > > 25. LeaderAndIsrResponse v5, StopReplicaResponse v4:
>> > Could
>> > > > we
>> > > > > > use
>> > > > > > > > this
>> > > > > > > > >> > > opportunity to organize the response in 2 levels,
>> first
>> > by
>> > > > > > topic,
>> > > > > > > > >> then by
>> > > > > > > > >> > > partition, as most other requests/responses?
>> > > > > > > > >> > >
>> > > > > > > > >> > > 26. FetchRequest v13 : Should forgotten_topics_data
>> use
>> > > > > topicId
>> > > > > > > too?
>> > > > > > > > >> > >
>> > > > > > > > >> > > 27. "This file can either be plain text (key/value
>> > pairs)
>> > > or
>> > > > > > > JSON."
>> > > > > > > > >> Have
>> > > > > > > > >> > we
>> > > > > > > > >> > > decided which one to use? Also, it would be helpful
>> to
>> > > > provide
>> > > > > > an
>> > > > > > > > >> > example.
>> > > > > > > > >> > >
>> > > > > > > > >> > > 28. Future improvement: Another future improvement
>> > > > opportunity
>> > > > > > is
>> > > > > > > to
>> > > > > > > > >> use
>> > > > > > > > >> > > topicId in GroupMetadataManager.offsetCommitKey in
>> the
>> > > > > > > offset_commit
>> > > > > > > > >> > topic.
>> > > > > > > > >> > > This may save some space.
>> > > > > > > > >> > >
>> > > > > > > > >> > > Thanks,
>> > > > > > > > >> > >
>> > > > > > > > >> > > Jun
>> > > > > > > > >> > >
>> > > > > > > > >> > > On Wed, Sep 23, 2020 at 8:50 AM Justine Olshan <
>> > > > > > > > jolshan@confluent.io>
>> > > > > > > > >> > > wrote:
>> > > > > > > > >> > >
>> > > > > > > > >> > > > Hi Tom,
>> > > > > > > > >> > > >
>> > > > > > > > >> > > > Thanks for the comment. I think this is a really
>> good
>> > > idea
>> > > > > and
>> > > > > > > it
>> > > > > > > > >> has
>> > > > > > > > >> > > been
>> > > > > > > > >> > > > added to the KIP under the newly added tooling
>> > section.
>> > > > > > > > >> > > >
>> > > > > > > > >> > > > Thanks again,
>> > > > > > > > >> > > > Justine
>> > > > > > > > >> > > >
>> > > > > > > > >> > > > On Wed, Sep 23, 2020 at 3:17 AM Tom Bentley <
>> > > > > > > tbentley@redhat.com>
>> > > > > > > > >> > wrote:
>> > > > > > > > >> > > >
>> > > > > > > > >> > > > > Hi Justine,
>> > > > > > > > >> > > > >
>> > > > > > > > >> > > > > I know you started the vote thread, but on
>> > re-reading
>> > > > the
>> > > > > > KIP
>> > > > > > > I
>> > > > > > > > >> > noticed
>> > > > > > > > >> > > > > that although the topic id is included in the
>> > > > > > MetadataResponse
>> > > > > > > > >> it's
>> > > > > > > > >> > not
>> > > > > > > > >> > > > > surfaced in the output from `kafka-topics.sh
>> > > > --describe`.
>> > > > > > > Maybe
>> > > > > > > > >> that
>> > > > > > > > >> > > was
>> > > > > > > > >> > > > > intentional because ids are intentionally not
>> really
>> > > > > > something
>> > > > > > > > the
>> > > > > > > > >> > user
>> > > > > > > > >> > > > > should care deeply about, but it would also make
>> > life
>> > > > > harder
>> > > > > > > for
>> > > > > > > > >> > anyone
>> > > > > > > > >> > > > > debugging Kafka and this would likely get worse
>> the
>> > > more
>> > > > > > topic
>> > > > > > > > ids
>> > > > > > > > >> > got
>> > > > > > > > >> > > > > rolled out across the protocols, clients etc. It
>> > seems
>> > > > > > likely
>> > > > > > > > that
>> > > > > > > > >> > > > > `kafka-topics.sh` will eventually need the
>> ability
>> > to
>> > > > show
>> > > > > > the
>> > > > > > > > id
>> > > > > > > > >> of
>> > > > > > > > >> > a
>> > > > > > > > >> > > > > topic and perhaps find a topic name given an id.
>> Is
>> > > > there
>> > > > > > any
>> > > > > > > > >> reason
>> > > > > > > > >> > > not
>> > > > > > > > >> > > > to
>> > > > > > > > >> > > > > implement that in this KIP?
>> > > > > > > > >> > > > >
>> > > > > > > > >> > > > > Many thanks,
>> > > > > > > > >> > > > >
>> > > > > > > > >> > > > > Tom
>> > > > > > > > >> > > > >
>> > > > > > > > >> > > > > On Mon, Sep 21, 2020 at 9:54 PM Justine Olshan <
>> > > > > > > > >> jolshan@confluent.io
>> > > > > > > > >> > >
>> > > > > > > > >> > > > > wrote:
>> > > > > > > > >> > > > >
>> > > > > > > > >> > > > > > Hi all,
>> > > > > > > > >> > > > > >
>> > > > > > > > >> > > > > > After thinking about it, I've decided to remove
>> > the
>> > > > > topic
>> > > > > > > name
>> > > > > > > > >> from
>> > > > > > > > >> > > the
>> > > > > > > > >> > > > > > Fetch Request and Response after all. Since
>> there
>> > > are
>> > > > so
>> > > > > > > many
>> > > > > > > > of
>> > > > > > > > >> > > these
>> > > > > > > > >> > > > > > requests per second, it is worth removing the
>> > extra
>> > > > > > > > information.
>> > > > > > > > >> > I've
>> > > > > > > > >> > > > > > updated the KIP to reflect this change.
>> > > > > > > > >> > > > > >
>> > > > > > > > >> > > > > > Please let me know if there is anything else we
>> > > should
>> > > > > > > discuss
>> > > > > > > > >> > before
>> > > > > > > > >> > > > > > voting.
>> > > > > > > > >> > > > > >
>> > > > > > > > >> > > > > > Thank you,
>> > > > > > > > >> > > > > > Justine
>> > > > > > > > >> > > > > >
>> > > > > > > > >> > > > > > On Fri, Sep 18, 2020 at 9:46 AM Justine Olshan
>> <
>> > > > > > > > >> > jolshan@confluent.io
>> > > > > > > > >> > > >
>> > > > > > > > >> > > > > > wrote:
>> > > > > > > > >> > > > > >
>> > > > > > > > >> > > > > > > Hi Jun,
>> > > > > > > > >> > > > > > >
>> > > > > > > > >> > > > > > > I see what you are saying. For now we can
>> remove
>> > > the
>> > > > > > extra
>> > > > > > > > >> > > > information.
>> > > > > > > > >> > > > > > > I'll leave the option to add more fields to
>> the
>> > > file
>> > > > > in
>> > > > > > > the
>> > > > > > > > >> > future.
>> > > > > > > > >> > > > The
>> > > > > > > > >> > > > > > KIP
>> > > > > > > > >> > > > > > > has been updated to reflect this change.
>> > > > > > > > >> > > > > > >
>> > > > > > > > >> > > > > > > Thanks,
>> > > > > > > > >> > > > > > > Justine
>> > > > > > > > >> > > > > > >
>> > > > > > > > >> > > > > > > On Fri, Sep 18, 2020 at 8:46 AM Jun Rao <
>> > > > > > jun@confluent.io
>> > > > > > > >
>> > > > > > > > >> > wrote:
>> > > > > > > > >> > > > > > >
>> > > > > > > > >> > > > > > >> Hi, Justine,
>> > > > > > > > >> > > > > > >>
>> > > > > > > > >> > > > > > >> Thanks for the reply.
>> > > > > > > > >> > > > > > >>
>> > > > > > > > >> > > > > > >> 13. If the log directory is the source of
>> > truth,
>> > > it
>> > > > > > means
>> > > > > > > > >> that
>> > > > > > > > >> > the
>> > > > > > > > >> > > > > > >> redundant info in the metadata file will be
>> > > > ignored.
>> > > > > > Then
>> > > > > > > > the
>> > > > > > > > >> > > > question
>> > > > > > > > >> > > > > > is
>> > > > > > > > >> > > > > > >> why do we need to put the redundant info in
>> the
>> > > > > > metadata
>> > > > > > > > file
>> > > > > > > > >> > now?
>> > > > > > > > >> > > > > > >>
>> > > > > > > > >> > > > > > >> Thanks,
>> > > > > > > > >> > > > > > >>
>> > > > > > > > >> > > > > > >> Jun
>> > > > > > > > >> > > > > > >>
>> > > > > > > > >> > > > > > >> On Thu, Sep 17, 2020 at 5:07 PM Justine
>> Olshan
>> > <
>> > > > > > > > >> > > > jolshan@confluent.io>
>> > > > > > > > >> > > > > > >> wrote:
>> > > > > > > > >> > > > > > >>
>> > > > > > > > >> > > > > > >> > Hi Jun,
>> > > > > > > > >> > > > > > >> > Thanks for the quick response!
>> > > > > > > > >> > > > > > >> >
>> > > > > > > > >> > > > > > >> > 12. I've decided to bump up the versions
>> on
>> > the
>> > > > > > > requests
>> > > > > > > > >> and
>> > > > > > > > >> > > > updated
>> > > > > > > > >> > > > > > the
>> > > > > > > > >> > > > > > >> > KIP. I think it's good we thoroughly
>> > discussed
>> > > > the
>> > > > > > > > options
>> > > > > > > > >> > here,
>> > > > > > > > >> > > > so
>> > > > > > > > >> > > > > we
>> > > > > > > > >> > > > > > >> know
>> > > > > > > > >> > > > > > >> > we made a good choice. :)
>> > > > > > > > >> > > > > > >> >
>> > > > > > > > >> > > > > > >> > 13. This is an interesting situation. I
>> think
>> > > if
>> > > > > this
>> > > > > > > > does
>> > > > > > > > >> > occur
>> > > > > > > > >> > > > we
>> > > > > > > > >> > > > > > >> should
>> > > > > > > > >> > > > > > >> > give a warning. I agree that it's hard to
>> > know
>> > > > the
>> > > > > > > source
>> > > > > > > > >> of
>> > > > > > > > >> > > truth
>> > > > > > > > >> > > > > for
>> > > > > > > > >> > > > > > >> sure
>> > > > > > > > >> > > > > > >> > since the directory or the file could be
>> > > manually
>> > > > > > > > >> modified. I
>> > > > > > > > >> > > > guess
>> > > > > > > > >> > > > > > the
>> > > > > > > > >> > > > > > >> > directory could be used as the source of
>> > truth.
>> > > > To
>> > > > > be
>> > > > > > > > >> honest,
>> > > > > > > > >> > > I'm
>> > > > > > > > >> > > > > not
>> > > > > > > > >> > > > > > >> > really sure what happens in kafka when the
>> > log
>> > > > > > > directory
>> > > > > > > > is
>> > > > > > > > >> > > > renamed
>> > > > > > > > >> > > > > > >> > manually in such a way. I'm also
>> wondering if
>> > > the
>> > > > > > > > >> situation is
>> > > > > > > > >> > > > > > >> recoverable
>> > > > > > > > >> > > > > > >> > in this scenario.
>> > > > > > > > >> > > > > > >> >
>> > > > > > > > >> > > > > > >> > Thanks,
>> > > > > > > > >> > > > > > >> > Justine
>> > > > > > > > >> > > > > > >> >
>> > > > > > > > >> > > > > > >> > On Thu, Sep 17, 2020 at 4:28 PM Jun Rao <
>> > > > > > > > jun@confluent.io>
>> > > > > > > > >> > > wrote:
>> > > > > > > > >> > > > > > >> >
>> > > > > > > > >> > > > > > >> > > Hi, Justine,
>> > > > > > > > >> > > > > > >> > >
>> > > > > > > > >> > > > > > >> > > Thanks for the reply.
>> > > > > > > > >> > > > > > >> > >
>> > > > > > > > >> > > > > > >> > > 12. I don't have a strong preference
>> > either.
>> > > > > > However,
>> > > > > > > > if
>> > > > > > > > >> we
>> > > > > > > > >> > > need
>> > > > > > > > >> > > > > IBP
>> > > > > > > > >> > > > > > >> > > anyway, maybe it's easier to just bump
>> up
>> > the
>> > > > > > version
>> > > > > > > > for
>> > > > > > > > >> > all
>> > > > > > > > >> > > > > inter
>> > > > > > > > >> > > > > > >> > broker
>> > > > > > > > >> > > > > > >> > > requests and add the topic id field as a
>> > > > regular
>> > > > > > > > field. A
>> > > > > > > > >> > > > regular
>> > > > > > > > >> > > > > > >> field
>> > > > > > > > >> > > > > > >> > is
>> > > > > > > > >> > > > > > >> > > a bit more concise in wire transfer
>> than a
>> > > > > flexible
>> > > > > > > > >> field.
>> > > > > > > > >> > > > > > >> > >
>> > > > > > > > >> > > > > > >> > > 13. The confusion that I was referring
>> to
>> > is
>> > > > > > between
>> > > > > > > > the
>> > > > > > > > >> > topic
>> > > > > > > > >> > > > > name
>> > > > > > > > >> > > > > > >> and
>> > > > > > > > >> > > > > > >> > > partition number between the log dir and
>> > the
>> > > > > > metadata
>> > > > > > > > >> file.
>> > > > > > > > >> > > For
>> > > > > > > > >> > > > > > >> example,
>> > > > > > > > >> > > > > > >> > if
>> > > > > > > > >> > > > > > >> > > the log dir is topicA-1 and the metadata
>> > file
>> > > > in
>> > > > > it
>> > > > > > > has
>> > > > > > > > >> > topicB
>> > > > > > > > >> > > > and
>> > > > > > > > >> > > > > > >> > > partition 0 (say due to a bug or manual
>> > > > > > > modification),
>> > > > > > > > >> which
>> > > > > > > > >> > > one
>> > > > > > > > >> > > > > do
>> > > > > > > > >> > > > > > we
>> > > > > > > > >> > > > > > >> > use
>> > > > > > > > >> > > > > > >> > > as the source of truth?
>> > > > > > > > >> > > > > > >> > >
>> > > > > > > > >> > > > > > >> > > Jun
>> > > > > > > > >> > > > > > >> > >
>> > > > > > > > >> > > > > > >> > > On Thu, Sep 17, 2020 at 3:43 PM Justine
>> > > Olshan
>> > > > <
>> > > > > > > > >> > > > > > jolshan@confluent.io>
>> > > > > > > > >> > > > > > >> > > wrote:
>> > > > > > > > >> > > > > > >> > >
>> > > > > > > > >> > > > > > >> > > > Hi Jun,
>> > > > > > > > >> > > > > > >> > > > Thanks for the comments.
>> > > > > > > > >> > > > > > >> > > >
>> > > > > > > > >> > > > > > >> > > > 12. I bumped the LeaderAndIsrRequest
>> > > because
>> > > > I
>> > > > > > > > removed
>> > > > > > > > >> the
>> > > > > > > > >> > > > topic
>> > > > > > > > >> > > > > > >> name
>> > > > > > > > >> > > > > > >> > > field
>> > > > > > > > >> > > > > > >> > > > in the response. It may be possible to
>> > > avoid
>> > > > > > > bumping
>> > > > > > > > >> the
>> > > > > > > > >> > > > version
>> > > > > > > > >> > > > > > >> > without
>> > > > > > > > >> > > > > > >> > > > that change, but I may be missing
>> > > something.
>> > > > > > > > >> > > > > > >> > > > I believe StopReplica is actually on
>> > > version
>> > > > 3
>> > > > > > now,
>> > > > > > > > but
>> > > > > > > > >> > > > because
>> > > > > > > > >> > > > > > >> > version 2
>> > > > > > > > >> > > > > > >> > > > is flexible, I kept that listed as
>> > version
>> > > 2
>> > > > on
>> > > > > > the
>> > > > > > > > KIP
>> > > > > > > > >> > > page.
>> > > > > > > > >> > > > > > >> However,
>> > > > > > > > >> > > > > > >> > > you
>> > > > > > > > >> > > > > > >> > > > may be right in that we may need to
>> bump
>> > > the
>> > > > > > > version
>> > > > > > > > on
>> > > > > > > > >> > > > > > StopReplica
>> > > > > > > > >> > > > > > >> to
>> > > > > > > > >> > > > > > >> > > deal
>> > > > > > > > >> > > > > > >> > > > with deletion differently as mentioned
>> > > > above. I
>> > > > > > > don't
>> > > > > > > > >> know
>> > > > > > > > >> > > if
>> > > > > > > > >> > > > I
>> > > > > > > > >> > > > > > >> have a
>> > > > > > > > >> > > > > > >> > > big
>> > > > > > > > >> > > > > > >> > > > preference over used tagged fields or
>> > not.
>> > > > > > > > >> > > > > > >> > > >
>> > > > > > > > >> > > > > > >> > > > 13. I was thinking that in the case
>> where
>> > > the
>> > > > > > file
>> > > > > > > > and
>> > > > > > > > >> the
>> > > > > > > > >> > > > > request
>> > > > > > > > >> > > > > > >> > topic
>> > > > > > > > >> > > > > > >> > > > ids don't match, it means that the
>> > broker's
>> > > > > > > topic/the
>> > > > > > > > >> one
>> > > > > > > > >> > in
>> > > > > > > > >> > > > the
>> > > > > > > > >> > > > > > >> file
>> > > > > > > > >> > > > > > >> > has
>> > > > > > > > >> > > > > > >> > > > been deleted. In that case, we would
>> need
>> > > to
>> > > > > > delete
>> > > > > > > > the
>> > > > > > > > >> > old
>> > > > > > > > >> > > > > topic
>> > > > > > > > >> > > > > > >> and
>> > > > > > > > >> > > > > > >> > > start
>> > > > > > > > >> > > > > > >> > > > receiving the new version. If the
>> topic
>> > > name
>> > > > > were
>> > > > > > > to
>> > > > > > > > >> > change,
>> > > > > > > > >> > > > but
>> > > > > > > > >> > > > > > the
>> > > > > > > > >> > > > > > >> > ids
>> > > > > > > > >> > > > > > >> > > > still matched, the file would also
>> need
>> > to
>> > > > > > update.
>> > > > > > > > Am I
>> > > > > > > > >> > > > missing
>> > > > > > > > >> > > > > a
>> > > > > > > > >> > > > > > >> case
>> > > > > > > > >> > > > > > >> > > > where the file would be correct and
>> not
>> > the
>> > > > > > > request?
>> > > > > > > > >> > > > > > >> > > >
>> > > > > > > > >> > > > > > >> > > > Thanks,
>> > > > > > > > >> > > > > > >> > > > Justine
>> > > > > > > > >> > > > > > >> > > >
>> > > > > > > > >> > > > > > >> > > > On Thu, Sep 17, 2020 at 3:18 PM Jun
>> Rao <
>> > > > > > > > >> jun@confluent.io
>> > > > > > > > >> > >
>> > > > > > > > >> > > > > wrote:
>> > > > > > > > >> > > > > > >> > > >
>> > > > > > > > >> > > > > > >> > > > > Hi, Justine,
>> > > > > > > > >> > > > > > >> > > > >
>> > > > > > > > >> > > > > > >> > > > > Thanks for the reply. A couple of
>> more
>> > > > > comments
>> > > > > > > > >> below.
>> > > > > > > > >> > > > > > >> > > > >
>> > > > > > > > >> > > > > > >> > > > > 12. ListOffset and OffsetForLeader
>> > > > currently
>> > > > > > > don't
>> > > > > > > > >> > support
>> > > > > > > > >> > > > > > >> flexible
>> > > > > > > > >> > > > > > >> > > > fields.
>> > > > > > > > >> > > > > > >> > > > > So, we have to bump up the version
>> > number
>> > > > and
>> > > > > > use
>> > > > > > > > >> IBP at
>> > > > > > > > >> > > > least
>> > > > > > > > >> > > > > > for
>> > > > > > > > >> > > > > > >> > > these
>> > > > > > > > >> > > > > > >> > > > > two requests. Note that it seems
>> 2.7.0
>> > > will
>> > > > > > > require
>> > > > > > > > >> IBP
>> > > > > > > > >> > > > anyway
>> > > > > > > > >> > > > > > >> > because
>> > > > > > > > >> > > > > > >> > > of
>> > > > > > > > >> > > > > > >> > > > > changes in KAFKA-10435. Also, it
>> seems
>> > > that
>> > > > > the
>> > > > > > > > >> version
>> > > > > > > > >> > > for
>> > > > > > > > >> > > > > > >> > > > > LeaderAndIsrRequest and StopReplica
>> are
>> > > > > bumped
>> > > > > > > even
>> > > > > > > > >> > though
>> > > > > > > > >> > > > we
>> > > > > > > > >> > > > > > only
>> > > > > > > > >> > > > > > >> > > added
>> > > > > > > > >> > > > > > >> > > > a
>> > > > > > > > >> > > > > > >> > > > > tagged field. But since IBP is
>> needed
>> > > > anyway,
>> > > > > > we
>> > > > > > > > may
>> > > > > > > > >> > want
>> > > > > > > > >> > > to
>> > > > > > > > >> > > > > > >> revisit
>> > > > > > > > >> > > > > > >> > > the
>> > > > > > > > >> > > > > > >> > > > > overall tagged field choice.
>> > > > > > > > >> > > > > > >> > > > >
>> > > > > > > > >> > > > > > >> > > > > 13. The only downside is the
>> potential
>> > > > > > confusion
>> > > > > > > on
>> > > > > > > > >> > which
>> > > > > > > > >> > > > one
>> > > > > > > > >> > > > > is
>> > > > > > > > >> > > > > > >> the
>> > > > > > > > >> > > > > > >> > > > source
>> > > > > > > > >> > > > > > >> > > > > of truth if they don't match.
>> Another
>> > > > option
>> > > > > is
>> > > > > > > to
>> > > > > > > > >> > include
>> > > > > > > > >> > > > > those
>> > > > > > > > >> > > > > > >> > fields
>> > > > > > > > >> > > > > > >> > > > in
>> > > > > > > > >> > > > > > >> > > > > the metadata file when we actually
>> > change
>> > > > the
>> > > > > > > > >> directory
>> > > > > > > > >> > > > > > structure.
>> > > > > > > > >> > > > > > >> > > > >
>> > > > > > > > >> > > > > > >> > > > > Thanks,
>> > > > > > > > >> > > > > > >> > > > >
>> > > > > > > > >> > > > > > >> > > > > Jun
>> > > > > > > > >> > > > > > >> > > > >
>> > > > > > > > >> > > > > > >> > > > > On Thu, Sep 17, 2020 at 2:01 PM
>> Justine
>> > > > > Olshan
>> > > > > > <
>> > > > > > > > >> > > > > > >> jolshan@confluent.io
>> > > > > > > > >> > > > > > >> > >
>> > > > > > > > >> > > > > > >> > > > > wrote:
>> > > > > > > > >> > > > > > >> > > > >
>> > > > > > > > >> > > > > > >> > > > > > Hello all,
>> > > > > > > > >> > > > > > >> > > > > >
>> > > > > > > > >> > > > > > >> > > > > > I've thought some more about
>> removing
>> > > the
>> > > > > > topic
>> > > > > > > > >> name
>> > > > > > > > >> > > field
>> > > > > > > > >> > > > > > from
>> > > > > > > > >> > > > > > >> > some
>> > > > > > > > >> > > > > > >> > > of
>> > > > > > > > >> > > > > > >> > > > > the
>> > > > > > > > >> > > > > > >> > > > > > requests. On closer inspection of
>> the
>> > > > > > > > >> > > requests/responses,
>> > > > > > > > >> > > > it
>> > > > > > > > >> > > > > > >> seems
>> > > > > > > > >> > > > > > >> > > that
>> > > > > > > > >> > > > > > >> > > > > the
>> > > > > > > > >> > > > > > >> > > > > > internal changes would be much
>> larger
>> > > > than
>> > > > > I
>> > > > > > > > >> expected.
>> > > > > > > > >> > > > Some
>> > > > > > > > >> > > > > > >> > protocols
>> > > > > > > > >> > > > > > >> > > > > > involve clients, so they would
>> > require
>> > > > > > changes
>> > > > > > > > too.
>> > > > > > > > >> > I'm
>> > > > > > > > >> > > > > > thinking
>> > > > > > > > >> > > > > > >> > that
>> > > > > > > > >> > > > > > >> > > > for
>> > > > > > > > >> > > > > > >> > > > > > now, removing the topic name from
>> > these
>> > > > > > > requests
>> > > > > > > > >> and
>> > > > > > > > >> > > > > responses
>> > > > > > > > >> > > > > > >> are
>> > > > > > > > >> > > > > > >> > > out
>> > > > > > > > >> > > > > > >> > > > of
>> > > > > > > > >> > > > > > >> > > > > > scope.
>> > > > > > > > >> > > > > > >> > > > > >
>> > > > > > > > >> > > > > > >> > > > > > I have decided to just keep the
>> > change
>> > > > > > > > >> > > > LeaderAndIsrResponse
>> > > > > > > > >> > > > > to
>> > > > > > > > >> > > > > > >> > remove
>> > > > > > > > >> > > > > > >> > > > the
>> > > > > > > > >> > > > > > >> > > > > > topic name, and have updated the
>> KIP
>> > to
>> > > > > > reflect
>> > > > > > > > >> this
>> > > > > > > > >> > > > > change. I
>> > > > > > > > >> > > > > > >> have
>> > > > > > > > >> > > > > > >> > > > also
>> > > > > > > > >> > > > > > >> > > > > > mentioned the other requests and
>> > > > responses
>> > > > > in
>> > > > > > > > >> future
>> > > > > > > > >> > > work.
>> > > > > > > > >> > > > > > >> > > > > >
>> > > > > > > > >> > > > > > >> > > > > > I'm hoping to start the voting
>> > process
>> > > > > soon,
>> > > > > > so
>> > > > > > > > >> let me
>> > > > > > > > >> > > > know
>> > > > > > > > >> > > > > if
>> > > > > > > > >> > > > > > >> > there
>> > > > > > > > >> > > > > > >> > > is
>> > > > > > > > >> > > > > > >> > > > > > anything else we should discuss.
>> > > > > > > > >> > > > > > >> > > > > >
>> > > > > > > > >> > > > > > >> > > > > > Thank you,
>> > > > > > > > >> > > > > > >> > > > > > Justine
>> > > > > > > > >> > > > > > >> > > > > >
>> > > > > > > > >> > > > > > >> > > > > > On Tue, Sep 15, 2020 at 3:57 PM
>> > Justine
>> > > > > > Olshan
>> > > > > > > <
>> > > > > > > > >> > > > > > >> > jolshan@confluent.io
>> > > > > > > > >> > > > > > >> > > >
>> > > > > > > > >> > > > > > >> > > > > > wrote:
>> > > > > > > > >> > > > > > >> > > > > >
>> > > > > > > > >> > > > > > >> > > > > > > Hello again,
>> > > > > > > > >> > > > > > >> > > > > > > To follow up on some of the
>> other
>> > > > > comments:
>> > > > > > > > >> > > > > > >> > > > > > >
>> > > > > > > > >> > > > > > >> > > > > > > 10/11) We can remove the topic
>> name
>> > > > from
>> > > > > > > these
>> > > > > > > > >> > > > > > >> > requests/responses,
>> > > > > > > > >> > > > > > >> > > > and
>> > > > > > > > >> > > > > > >> > > > > > > that means we will just have to
>> > make
>> > > a
>> > > > > few
>> > > > > > > > >> internal
>> > > > > > > > >> > > > > changes
>> > > > > > > > >> > > > > > to
>> > > > > > > > >> > > > > > >> > make
>> > > > > > > > >> > > > > > >> > > > > > > partitions accessible by topic
>> id
>> > and
>> > > > > > > > partition.
>> > > > > > > > >> I
>> > > > > > > > >> > can
>> > > > > > > > >> > > > > > update
>> > > > > > > > >> > > > > > >> the
>> > > > > > > > >> > > > > > >> > > KIP
>> > > > > > > > >> > > > > > >> > > > > to
>> > > > > > > > >> > > > > > >> > > > > > > remove them unless anyone thinks
>> > they
>> > > > > > should
>> > > > > > > > >> stay.
>> > > > > > > > >> > > > > > >> > > > > > >
>> > > > > > > > >> > > > > > >> > > > > > > 12) Addressed in the previous
>> > email.
>> > > > I've
>> > > > > > > > updated
>> > > > > > > > >> > the
>> > > > > > > > >> > > > KIP
>> > > > > > > > >> > > > > to
>> > > > > > > > >> > > > > > >> > > include
>> > > > > > > > >> > > > > > >> > > > > > > tagged fields for the requests
>> and
>> > > > > > responses.
>> > > > > > > > >> (More
>> > > > > > > > >> > on
>> > > > > > > > >> > > > > that
>> > > > > > > > >> > > > > > >> > below)
>> > > > > > > > >> > > > > > >> > > > > > >
>> > > > > > > > >> > > > > > >> > > > > > > 13) I think part of the idea for
>> > > > > including
>> > > > > > > this
>> > > > > > > > >> > > > > information
>> > > > > > > > >> > > > > > >> is to
>> > > > > > > > >> > > > > > >> > > > > prepare
>> > > > > > > > >> > > > > > >> > > > > > > for future changes. Perhaps the
>> > > > directory
>> > > > > > > > >> structure
>> > > > > > > > >> > > > might
>> > > > > > > > >> > > > > > >> change
>> > > > > > > > >> > > > > > >> > > from
>> > > > > > > > >> > > > > > >> > > > > > > topicName_partitionNumber to
>> > > something
>> > > > > like
>> > > > > > > > >> > > > > > >> > > topicID_partitionNumber.
>> > > > > > > > >> > > > > > >> > > > > Then
>> > > > > > > > >> > > > > > >> > > > > > > it would be useful to have the
>> > topic
>> > > > name
>> > > > > > in
>> > > > > > > > the
>> > > > > > > > >> > file
>> > > > > > > > >> > > > > since
>> > > > > > > > >> > > > > > it
>> > > > > > > > >> > > > > > >> > > would
>> > > > > > > > >> > > > > > >> > > > > not
>> > > > > > > > >> > > > > > >> > > > > > be
>> > > > > > > > >> > > > > > >> > > > > > > in the directory structure.
>> > > Supporting
>> > > > > > topic
>> > > > > > > > >> renames
>> > > > > > > > >> > > > might
>> > > > > > > > >> > > > > > be
>> > > > > > > > >> > > > > > >> > > easier
>> > > > > > > > >> > > > > > >> > > > if
>> > > > > > > > >> > > > > > >> > > > > > the
>> > > > > > > > >> > > > > > >> > > > > > > other fields are included. Would
>> > > there
>> > > > be
>> > > > > > any
>> > > > > > > > >> > > downsides
>> > > > > > > > >> > > > to
>> > > > > > > > >> > > > > > >> > > including
>> > > > > > > > >> > > > > > >> > > > > this
>> > > > > > > > >> > > > > > >> > > > > > > information?
>> > > > > > > > >> > > > > > >> > > > > > >
>> > > > > > > > >> > > > > > >> > > > > > > 14)  Yes, we would need to copy
>> the
>> > > > > > partition
>> > > > > > > > >> > metadata
>> > > > > > > > >> > > > > file
>> > > > > > > > >> > > > > > in
>> > > > > > > > >> > > > > > >> > this
>> > > > > > > > >> > > > > > >> > > > > > > process. I've updated the KIP to
>> > > > include
>> > > > > > > this.
>> > > > > > > > >> > > > > > >> > > > > > >
>> > > > > > > > >> > > > > > >> > > > > > > 15) I believe Lucas meant v1
>> and v2
>> > > > here.
>> > > > > > He
>> > > > > > > > was
>> > > > > > > > >> > > > referring
>> > > > > > > > >> > > > > > to
>> > > > > > > > >> > > > > > >> how
>> > > > > > > > >> > > > > > >> > > the
>> > > > > > > > >> > > > > > >> > > > > > > requests would fall under
>> different
>> > > IBP
>> > > > > and
>> > > > > > > > meant
>> > > > > > > > >> > that
>> > > > > > > > >> > > > > older
>> > > > > > > > >> > > > > > >> > > brokers
>> > > > > > > > >> > > > > > >> > > > > > would
>> > > > > > > > >> > > > > > >> > > > > > > have to use the older version of
>> > the
>> > > > > > request
>> > > > > > > > and
>> > > > > > > > >> the
>> > > > > > > > >> > > > > > existing
>> > > > > > > > >> > > > > > >> > topic
>> > > > > > > > >> > > > > > >> > > > > > > deletion process. At first, it
>> > seemed
>> > > > > like
>> > > > > > > > tagged
>> > > > > > > > >> > > fields
>> > > > > > > > >> > > > > > would
>> > > > > > > > >> > > > > > >> > > > resolve
>> > > > > > > > >> > > > > > >> > > > > > > the IBP issue. However, we may
>> need
>> > > IBP
>> > > > > for
>> > > > > > > > this
>> > > > > > > > >> > > request
>> > > > > > > > >> > > > > > after
>> > > > > > > > >> > > > > > >> > all
>> > > > > > > > >> > > > > > >> > > > > since
>> > > > > > > > >> > > > > > >> > > > > > > the controller handles the topic
>> > > > deletion
>> > > > > > > > >> > differently
>> > > > > > > > >> > > > > > >> depending
>> > > > > > > > >> > > > > > >> > on
>> > > > > > > > >> > > > > > >> > > > the
>> > > > > > > > >> > > > > > >> > > > > > IBP
>> > > > > > > > >> > > > > > >> > > > > > > version. In an older version, we
>> > > can't
>> > > > > just
>> > > > > > > > send
>> > > > > > > > >> a
>> > > > > > > > >> > > > > > StopReplica
>> > > > > > > > >> > > > > > >> > > delete
>> > > > > > > > >> > > > > > >> > > > > the
>> > > > > > > > >> > > > > > >> > > > > > > topic immediately like we'd
>> want to
>> > > for
>> > > > > > this
>> > > > > > > > KIP.
>> > > > > > > > >> > > > > > >> > > > > > >
>> > > > > > > > >> > > > > > >> > > > > > > This makes me wonder if we want
>> > > tagged
>> > > > > > fields
>> > > > > > > > on
>> > > > > > > > >> all
>> > > > > > > > >> > > the
>> > > > > > > > >> > > > > > >> requests
>> > > > > > > > >> > > > > > >> > > > after
>> > > > > > > > >> > > > > > >> > > > > > > all. Let me know your thoughts!
>> > > > > > > > >> > > > > > >> > > > > > >
>> > > > > > > > >> > > > > > >> > > > > > > Justine
>> > > > > > > > >> > > > > > >> > > > > > >
>> > > > > > > > >> > > > > > >> > > > > > > On Tue, Sep 15, 2020 at 1:03 PM
>> > > Justine
>> > > > > > > Olshan
>> > > > > > > > <
>> > > > > > > > >> > > > > > >> > > jolshan@confluent.io
>> > > > > > > > >> > > > > > >> > > > >
>> > > > > > > > >> > > > > > >> > > > > > > wrote:
>> > > > > > > > >> > > > > > >> > > > > > >
>> > > > > > > > >> > > > > > >> > > > > > >> Hi all,
>> > > > > > > > >> > > > > > >> > > > > > >> Jun brought up a good point in
>> his
>> > > > last
>> > > > > > > email
>> > > > > > > > >> about
>> > > > > > > > >> > > > > tagged
>> > > > > > > > >> > > > > > >> > fields,
>> > > > > > > > >> > > > > > >> > > > and
>> > > > > > > > >> > > > > > >> > > > > > >> I've updated the KIP to reflect
>> > that
>> > > > the
>> > > > > > > > >> changes to
>> > > > > > > > >> > > > > > requests
>> > > > > > > > >> > > > > > >> and
>> > > > > > > > >> > > > > > >> > > > > > responses
>> > > > > > > > >> > > > > > >> > > > > > >> will be in the form of tagged
>> > fields
>> > > > to
>> > > > > > > avoid
>> > > > > > > > >> > > changing
>> > > > > > > > >> > > > > IBP.
>> > > > > > > > >> > > > > > >> > > > > > >>
>> > > > > > > > >> > > > > > >> > > > > > >> Jun: I plan on sending a
>> followup
>> > > > email
>> > > > > to
>> > > > > > > > >> address
>> > > > > > > > >> > > some
>> > > > > > > > >> > > > > of
>> > > > > > > > >> > > > > > >> the
>> > > > > > > > >> > > > > > >> > > other
>> > > > > > > > >> > > > > > >> > > > > > >> points.
>> > > > > > > > >> > > > > > >> > > > > > >>
>> > > > > > > > >> > > > > > >> > > > > > >> Thanks,
>> > > > > > > > >> > > > > > >> > > > > > >> Justine
>> > > > > > > > >> > > > > > >> > > > > > >>
>> > > > > > > > >> > > > > > >> > > > > > >> On Mon, Sep 14, 2020 at 4:25 PM
>> > Jun
>> > > > Rao
>> > > > > <
>> > > > > > > > >> > > > > jun@confluent.io>
>> > > > > > > > >> > > > > > >> > wrote:
>> > > > > > > > >> > > > > > >> > > > > > >>
>> > > > > > > > >> > > > > > >> > > > > > >>> Hi, Justine,
>> > > > > > > > >> > > > > > >> > > > > > >>>
>> > > > > > > > >> > > > > > >> > > > > > >>> Thanks for the updated KIP. A
>> few
>> > > > > > comments
>> > > > > > > > >> below.
>> > > > > > > > >> > > > > > >> > > > > > >>>
>> > > > > > > > >> > > > > > >> > > > > > >>> 10. LeaderAndIsr Response: Do
>> we
>> > > need
>> > > > > the
>> > > > > > > > topic
>> > > > > > > > >> > > name?
>> > > > > > > > >> > > > > > >> > > > > > >>>
>> > > > > > > > >> > > > > > >> > > > > > >>> 11. For the changed
>> > > request/response,
>> > > > > > other
>> > > > > > > > >> than
>> > > > > > > > >> > > > > > >> LeaderAndIsr,
>> > > > > > > > >> > > > > > >> > > > > > >>> UpdateMetadata, Metadata, do
>> we
>> > > need
>> > > > to
>> > > > > > > > include
>> > > > > > > > >> > the
>> > > > > > > > >> > > > > topic
>> > > > > > > > >> > > > > > >> name?
>> > > > > > > > >> > > > > > >> > > > > > >>>
>> > > > > > > > >> > > > > > >> > > > > > >>> 12. It seems that upgrades
>> don't
>> > > > > require
>> > > > > > > IBP.
>> > > > > > > > >> Does
>> > > > > > > > >> > > > that
>> > > > > > > > >> > > > > > mean
>> > > > > > > > >> > > > > > >> > the
>> > > > > > > > >> > > > > > >> > > > new
>> > > > > > > > >> > > > > > >> > > > > > >>> fields
>> > > > > > > > >> > > > > > >> > > > > > >>> in all the request/response
>> are
>> > > added
>> > > > > as
>> > > > > > > > tagged
>> > > > > > > > >> > > fields
>> > > > > > > > >> > > > > > >> without
>> > > > > > > > >> > > > > > >> > > > > bumping
>> > > > > > > > >> > > > > > >> > > > > > up
>> > > > > > > > >> > > > > > >> > > > > > >>> the request version? It would
>> be
>> > > > useful
>> > > > > > to
>> > > > > > > > make
>> > > > > > > > >> > that
>> > > > > > > > >> > > > > > clear.
>> > > > > > > > >> > > > > > >> > > > > > >>>
>> > > > > > > > >> > > > > > >> > > > > > >>> 13. Partition Metadata file:
>> Do
>> > we
>> > > > need
>> > > > > > to
>> > > > > > > > >> include
>> > > > > > > > >> > > the
>> > > > > > > > >> > > > > > topic
>> > > > > > > > >> > > > > > >> > name
>> > > > > > > > >> > > > > > >> > > > and
>> > > > > > > > >> > > > > > >> > > > > > the
>> > > > > > > > >> > > > > > >> > > > > > >>> partition id since they are
>> > implied
>> > > > in
>> > > > > > the
>> > > > > > > > >> > directory
>> > > > > > > > >> > > > > name?
>> > > > > > > > >> > > > > > >> > > > > > >>>
>> > > > > > > > >> > > > > > >> > > > > > >>> 14. In the JBOD mode, we
>> support
>> > > > > moving a
>> > > > > > > > >> > > partition's
>> > > > > > > > >> > > > > data
>> > > > > > > > >> > > > > > >> from
>> > > > > > > > >> > > > > > >> > > one
>> > > > > > > > >> > > > > > >> > > > > > disk
>> > > > > > > > >> > > > > > >> > > > > > >>> to
>> > > > > > > > >> > > > > > >> > > > > > >>> another. Will the new
>> partition
>> > > > > metadata
>> > > > > > > file
>> > > > > > > > >> be
>> > > > > > > > >> > > > copied
>> > > > > > > > >> > > > > > >> during
>> > > > > > > > >> > > > > > >> > > that
>> > > > > > > > >> > > > > > >> > > > > > >>> process?
>> > > > > > > > >> > > > > > >> > > > > > >>>
>> > > > > > > > >> > > > > > >> > > > > > >>> 15. The KIP says "Remove
>> deleted
>> > > > topics
>> > > > > > > from
>> > > > > > > > >> > > replicas
>> > > > > > > > >> > > > by
>> > > > > > > > >> > > > > > >> > sending
>> > > > > > > > >> > > > > > >> > > > > > >>> StopReplicaRequest V2 for any
>> > > topics
>> > > > > > which
>> > > > > > > do
>> > > > > > > > >> not
>> > > > > > > > >> > > > > contain
>> > > > > > > > >> > > > > > a
>> > > > > > > > >> > > > > > >> > topic
>> > > > > > > > >> > > > > > >> > > > ID,
>> > > > > > > > >> > > > > > >> > > > > > and
>> > > > > > > > >> > > > > > >> > > > > > >>> V3 for any topics which do
>> > contain
>> > > a
>> > > > > > topic
>> > > > > > > > >> ID.".
>> > > > > > > > >> > > > > However,
>> > > > > > > > >> > > > > > it
>> > > > > > > > >> > > > > > >> > > seems
>> > > > > > > > >> > > > > > >> > > > > the
>> > > > > > > > >> > > > > > >> > > > > > >>> updated controller will create
>> > all
>> > > > > > missing
>> > > > > > > > >> topic
>> > > > > > > > >> > IDs
>> > > > > > > > >> > > > > first
>> > > > > > > > >> > > > > > >> > before
>> > > > > > > > >> > > > > > >> > > > > doing
>> > > > > > > > >> > > > > > >> > > > > > >>> other actions. So, is
>> > > > > StopReplicaRequest
>> > > > > > V2
>> > > > > > > > >> > needed?
>> > > > > > > > >> > > > > > >> > > > > > >>>
>> > > > > > > > >> > > > > > >> > > > > > >>> Jun
>> > > > > > > > >> > > > > > >> > > > > > >>>
>> > > > > > > > >> > > > > > >> > > > > > >>> On Fri, Sep 11, 2020 at 10:31
>> AM
>> > > John
>> > > > > > > > Roesler <
>> > > > > > > > >> > > > > > >> > > vvcephei@apache.org
>> > > > > > > > >> > > > > > >> > > > >
>> > > > > > > > >> > > > > > >> > > > > > >>> wrote:
>> > > > > > > > >> > > > > > >> > > > > > >>>
>> > > > > > > > >> > > > > > >> > > > > > >>> > Thanks, Justine!
>> > > > > > > > >> > > > > > >> > > > > > >>> >
>> > > > > > > > >> > > > > > >> > > > > > >>> > Your response seems
>> compelling
>> > to
>> > > > me.
>> > > > > > > > >> > > > > > >> > > > > > >>> >
>> > > > > > > > >> > > > > > >> > > > > > >>> > -John
>> > > > > > > > >> > > > > > >> > > > > > >>> >
>> > > > > > > > >> > > > > > >> > > > > > >>> > On Fri, 2020-09-11 at 10:17
>> > > -0700,
>> > > > > > > Justine
>> > > > > > > > >> > Olshan
>> > > > > > > > >> > > > > wrote:
>> > > > > > > > >> > > > > > >> > > > > > >>> > > Hello all,
>> > > > > > > > >> > > > > > >> > > > > > >>> > > Thanks for continuing the
>> > > > > > discussion! I
>> > > > > > > > >> have a
>> > > > > > > > >> > > few
>> > > > > > > > >> > > > > > >> > responses
>> > > > > > > > >> > > > > > >> > > to
>> > > > > > > > >> > > > > > >> > > > > > your
>> > > > > > > > >> > > > > > >> > > > > > >>> > points.
>> > > > > > > > >> > > > > > >> > > > > > >>> > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > Tom: You are correct in
>> that
>> > > this
>> > > > > KIP
>> > > > > > > has
>> > > > > > > > >> not
>> > > > > > > > >> > > > > > mentioned
>> > > > > > > > >> > > > > > >> the
>> > > > > > > > >> > > > > > >> > > > > > >>> > > DeleteTopicsRequest. I
>> think
>> > > that
>> > > > > > this
>> > > > > > > > >> would
>> > > > > > > > >> > be
>> > > > > > > > >> > > > out
>> > > > > > > > >> > > > > of
>> > > > > > > > >> > > > > > >> > scope
>> > > > > > > > >> > > > > > >> > > > for
>> > > > > > > > >> > > > > > >> > > > > > >>> now, but
>> > > > > > > > >> > > > > > >> > > > > > >>> > > may be something worth
>> adding
>> > > in
>> > > > > the
>> > > > > > > > >> future.
>> > > > > > > > >> > > > > > >> > > > > > >>> > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > John: We did consider
>> > sequence
>> > > > ids,
>> > > > > > but
>> > > > > > > > >> there
>> > > > > > > > >> > > are
>> > > > > > > > >> > > > a
>> > > > > > > > >> > > > > > few
>> > > > > > > > >> > > > > > >> > > reasons
>> > > > > > > > >> > > > > > >> > > > > to
>> > > > > > > > >> > > > > > >> > > > > > >>> favor
>> > > > > > > > >> > > > > > >> > > > > > >>> > > UUIDs. There are several
>> > cases
>> > > > > where
>> > > > > > > > topics
>> > > > > > > > >> > from
>> > > > > > > > >> > > > > > >> different
>> > > > > > > > >> > > > > > >> > > > > clusters
>> > > > > > > > >> > > > > > >> > > > > > >>> may
>> > > > > > > > >> > > > > > >> > > > > > >>> > > interact now and in the
>> > future.
>> > > > For
>> > > > > > > > >> example,
>> > > > > > > > >> > > > Mirror
>> > > > > > > > >> > > > > > >> Maker 2
>> > > > > > > > >> > > > > > >> > > may
>> > > > > > > > >> > > > > > >> > > > > > >>> benefit
>> > > > > > > > >> > > > > > >> > > > > > >>> > > from being able to detect
>> > when
>> > > a
>> > > > > > > cluster
>> > > > > > > > >> being
>> > > > > > > > >> > > > > > mirrored
>> > > > > > > > >> > > > > > >> is
>> > > > > > > > >> > > > > > >> > > > > deleted
>> > > > > > > > >> > > > > > >> > > > > > >>> and
>> > > > > > > > >> > > > > > >> > > > > > >>> > > recreated and globally
>> unique
>> > > > > > > identifiers
>> > > > > > > > >> > would
>> > > > > > > > >> > > > make
>> > > > > > > > >> > > > > > >> > > resolving
>> > > > > > > > >> > > > > > >> > > > > > issues
>> > > > > > > > >> > > > > > >> > > > > > >>> > > easier than sequence IDs
>> > which
>> > > > may
>> > > > > > > > collide
>> > > > > > > > >> > > between
>> > > > > > > > >> > > > > > >> > clusters.
>> > > > > > > > >> > > > > > >> > > > > > KIP-405
>> > > > > > > > >> > > > > > >> > > > > > >>> > > (tiered storage) will also
>> > > > benefit
>> > > > > > from
>> > > > > > > > >> > globally
>> > > > > > > > >> > > > > > unique
>> > > > > > > > >> > > > > > >> IDs
>> > > > > > > > >> > > > > > >> > > as
>> > > > > > > > >> > > > > > >> > > > > > shared
>> > > > > > > > >> > > > > > >> > > > > > >>> > > buckets may be used
>> between
>> > > > > clusters.
>> > > > > > > > >> > > > > > >> > > > > > >>> > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > Globally unique IDs would
>> > also
>> > > > make
>> > > > > > > > >> > > functionality
>> > > > > > > > >> > > > > like
>> > > > > > > > >> > > > > > >> > moving
>> > > > > > > > >> > > > > > >> > > > > > topics
>> > > > > > > > >> > > > > > >> > > > > > >>> > > between disparate clusters
>> > > easier
>> > > > > in
>> > > > > > > the
>> > > > > > > > >> > future,
>> > > > > > > > >> > > > > > >> simplify
>> > > > > > > > >> > > > > > >> > any
>> > > > > > > > >> > > > > > >> > > > > > future
>> > > > > > > > >> > > > > > >> > > > > > >>> > > implementations of backups
>> > and
>> > > > > > > restores,
>> > > > > > > > >> and
>> > > > > > > > >> > > more.
>> > > > > > > > >> > > > > In
>> > > > > > > > >> > > > > > >> > > general,
>> > > > > > > > >> > > > > > >> > > > > > >>> unique IDs
>> > > > > > > > >> > > > > > >> > > > > > >>> > > would ensure that the
>> source
>> > > > > cluster
>> > > > > > > > >> topics do
>> > > > > > > > >> > > not
>> > > > > > > > >> > > > > > >> conflict
>> > > > > > > > >> > > > > > >> > > > with
>> > > > > > > > >> > > > > > >> > > > > > the
>> > > > > > > > >> > > > > > >> > > > > > >>> > > destination cluster
>> topics.
>> > > > > > > > >> > > > > > >> > > > > > >>> > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > If we were to use sequence
>> > ids,
>> > > > we
>> > > > > > > would
>> > > > > > > > >> need
>> > > > > > > > >> > > > > > >> sufficiently
>> > > > > > > > >> > > > > > >> > > > large
>> > > > > > > > >> > > > > > >> > > > > > >>> cluster
>> > > > > > > > >> > > > > > >> > > > > > >>> > > ids to be stored with the
>> > topic
>> > > > > > > > >> identifiers or
>> > > > > > > > >> > > we
>> > > > > > > > >> > > > > run
>> > > > > > > > >> > > > > > >> the
>> > > > > > > > >> > > > > > >> > > risk
>> > > > > > > > >> > > > > > >> > > > of
>> > > > > > > > >> > > > > > >> > > > > > >>> > > collisions. This will
>> give up
>> > > any
>> > > > > > > > >> advantage in
>> > > > > > > > >> > > > > > >> compactness
>> > > > > > > > >> > > > > > >> > > that
>> > > > > > > > >> > > > > > >> > > > > > >>> sequence
>> > > > > > > > >> > > > > > >> > > > > > >>> > > numbers may bring. Given
>> > these
>> > > > > > > > advantages I
>> > > > > > > > >> > > think
>> > > > > > > > >> > > > it
>> > > > > > > > >> > > > > > >> makes
>> > > > > > > > >> > > > > > >> > > > sense
>> > > > > > > > >> > > > > > >> > > > > to
>> > > > > > > > >> > > > > > >> > > > > > >>> use
>> > > > > > > > >> > > > > > >> > > > > > >>> > > UUIDs.
>> > > > > > > > >> > > > > > >> > > > > > >>> > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > Gokul: This is an
>> interesting
>> > > > idea,
>> > > > > > but
>> > > > > > > > >> this
>> > > > > > > > >> > is
>> > > > > > > > >> > > a
>> > > > > > > > >> > > > > > >> breaking
>> > > > > > > > >> > > > > > >> > > > > change.
>> > > > > > > > >> > > > > > >> > > > > > >>> Out of
>> > > > > > > > >> > > > > > >> > > > > > >>> > > scope for now, but maybe
>> > worth
>> > > > > > > discussing
>> > > > > > > > >> in
>> > > > > > > > >> > the
>> > > > > > > > >> > > > > > future.
>> > > > > > > > >> > > > > > >> > > > > > >>> > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > Hope this explains some of
>> > the
>> > > > > > > decisions,
>> > > > > > > > >> > > > > > >> > > > > > >>> > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > Justine
>> > > > > > > > >> > > > > > >> > > > > > >>> > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > On Fri, Sep 11, 2020 at
>> 8:27
>> > AM
>> > > > > Gokul
>> > > > > > > > >> Ramanan
>> > > > > > > > >> > > > > > >> Subramanian <
>> > > > > > > > >> > > > > > >> > > > > > >>> > > gokul2411s@gmail.com>
>> wrote:
>> > > > > > > > >> > > > > > >> > > > > > >>> > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > Hi.
>> > > > > > > > >> > > > > > >> > > > > > >>> > > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > Thanks for the KIP.
>> > > > > > > > >> > > > > > >> > > > > > >>> > > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > Have you thought about
>> > > whether
>> > > > it
>> > > > > > > makes
>> > > > > > > > >> > sense
>> > > > > > > > >> > > to
>> > > > > > > > >> > > > > > >> support
>> > > > > > > > >> > > > > > >> > > > > > >>> authorizing a
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > principal for a topic ID
>> > > rather
>> > > > > > than
>> > > > > > > a
>> > > > > > > > >> topic
>> > > > > > > > >> > > > name
>> > > > > > > > >> > > > > to
>> > > > > > > > >> > > > > > >> > > achieve
>> > > > > > > > >> > > > > > >> > > > > > >>> tighter
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > security?
>> > > > > > > > >> > > > > > >> > > > > > >>> > > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > Or is the topic ID
>> > > > fundamentally
>> > > > > an
>> > > > > > > > >> internal
>> > > > > > > > >> > > > > detail
>> > > > > > > > >> > > > > > >> > similar
>> > > > > > > > >> > > > > > >> > > > to
>> > > > > > > > >> > > > > > >> > > > > > >>> epochs
>> > > > > > > > >> > > > > > >> > > > > > >>> > used
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > in a bunch of other
>> places
>> > in
>> > > > > > Kafka?
>> > > > > > > > >> > > > > > >> > > > > > >>> > > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > Thanks.
>> > > > > > > > >> > > > > > >> > > > > > >>> > > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > On Fri, Sep 11, 2020 at
>> > 4:06
>> > > PM
>> > > > > > John
>> > > > > > > > >> > Roesler <
>> > > > > > > > >> > > > > > >> > > > > > vvcephei@apache.org>
>> > > > > > > > >> > > > > > >> > > > > > >>> > wrote:
>> > > > > > > > >> > > > > > >> > > > > > >>> > > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Hello Justine,
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Thanks for the KIP!
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > I happen to have been
>> > > > > confronted
>> > > > > > > > >> recently
>> > > > > > > > >> > > with
>> > > > > > > > >> > > > > the
>> > > > > > > > >> > > > > > >> need
>> > > > > > > > >> > > > > > >> > > to
>> > > > > > > > >> > > > > > >> > > > > keep
>> > > > > > > > >> > > > > > >> > > > > > >>> > track of
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > a
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > large number of
>> topics as
>> > > > > > compactly
>> > > > > > > > as
>> > > > > > > > >> > > > > possible. I
>> > > > > > > > >> > > > > > >> was
>> > > > > > > > >> > > > > > >> > > > going
>> > > > > > > > >> > > > > > >> > > > > to
>> > > > > > > > >> > > > > > >> > > > > > >>> come
>> > > > > > > > >> > > > > > >> > > > > > >>> > up
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > with some way to
>> > dictionary
>> > > > > > encode
>> > > > > > > > the
>> > > > > > > > >> > topic
>> > > > > > > > >> > > > > names
>> > > > > > > > >> > > > > > >> as
>> > > > > > > > >> > > > > > >> > > > > integers,
>> > > > > > > > >> > > > > > >> > > > > > >>> but
>> > > > > > > > >> > > > > > >> > > > > > >>> > this
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > seems much better!
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Apologies if this has
>> > been
>> > > > > raised
>> > > > > > > > >> before,
>> > > > > > > > >> > > but
>> > > > > > > > >> > > > > I’m
>> > > > > > > > >> > > > > > >> > > wondering
>> > > > > > > > >> > > > > > >> > > > > > >>> about the
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > choice of UUID vs
>> > sequence
>> > > > > number
>> > > > > > > for
>> > > > > > > > >> the
>> > > > > > > > >> > > ids.
>> > > > > > > > >> > > > > > >> > Typically,
>> > > > > > > > >> > > > > > >> > > > > I’ve
>> > > > > > > > >> > > > > > >> > > > > > >>> seen
>> > > > > > > > >> > > > > > >> > > > > > >>> > UUIDs
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > in two situations:
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 1. When processes
>> need to
>> > > > > > generate
>> > > > > > > > >> > > > non-colliding
>> > > > > > > > >> > > > > > >> > > > identifiers
>> > > > > > > > >> > > > > > >> > > > > > >>> without
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > coordination.
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 2. When the identifier
>> > > needs
>> > > > to
>> > > > > > be
>> > > > > > > > >> > > > “universally
>> > > > > > > > >> > > > > > >> > unique”;
>> > > > > > > > >> > > > > > >> > > > > I.e.,
>> > > > > > > > >> > > > > > >> > > > > > >>> the
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > identifier needs to
>> > > > distinguish
>> > > > > > the
>> > > > > > > > >> entity
>> > > > > > > > >> > > > from
>> > > > > > > > >> > > > > > all
>> > > > > > > > >> > > > > > >> > other
>> > > > > > > > >> > > > > > >> > > > > > >>> entities
>> > > > > > > > >> > > > > > >> > > > > > >>> > that
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > could ever exist.
>> This is
>> > > > > useful
>> > > > > > in
>> > > > > > > > >> cases
>> > > > > > > > >> > > > where
>> > > > > > > > >> > > > > > >> > entities
>> > > > > > > > >> > > > > > >> > > > from
>> > > > > > > > >> > > > > > >> > > > > > all
>> > > > > > > > >> > > > > > >> > > > > > >>> > kinds
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > of
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > systems get mixed
>> > together,
>> > > > > such
>> > > > > > as
>> > > > > > > > >> when
>> > > > > > > > >> > > > dumping
>> > > > > > > > >> > > > > > >> logs
>> > > > > > > > >> > > > > > >> > > from
>> > > > > > > > >> > > > > > >> > > > > all
>> > > > > > > > >> > > > > > >> > > > > > >>> > processes
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > in
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a company into a
>> common
>> > > > system.
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Maybe I’m being
>> > > > short-sighted,
>> > > > > > but
>> > > > > > > it
>> > > > > > > > >> > > doesn’t
>> > > > > > > > >> > > > > seem
>> > > > > > > > >> > > > > > >> like
>> > > > > > > > >> > > > > > >> > > > > either
>> > > > > > > > >> > > > > > >> > > > > > >>> really
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > applies here. It seems
>> > like
>> > > > the
>> > > > > > > > brokers
>> > > > > > > > >> > > could
>> > > > > > > > >> > > > > and
>> > > > > > > > >> > > > > > >> would
>> > > > > > > > >> > > > > > >> > > > > achieve
>> > > > > > > > >> > > > > > >> > > > > > >>> > consensus
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > when creating a topic
>> > > anyway,
>> > > > > > which
>> > > > > > > > is
>> > > > > > > > >> all
>> > > > > > > > >> > > > > that’s
>> > > > > > > > >> > > > > > >> > > required
>> > > > > > > > >> > > > > > >> > > > to
>> > > > > > > > >> > > > > > >> > > > > > >>> > generate
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > non-colliding sequence
>> > ids.
>> > > > For
>> > > > > > the
>> > > > > > > > >> > second,
>> > > > > > > > >> > > as
>> > > > > > > > >> > > > > you
>> > > > > > > > >> > > > > > >> > > mention,
>> > > > > > > > >> > > > > > >> > > > > we
>> > > > > > > > >> > > > > > >> > > > > > >>> could
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > assign
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a UUID to the cluster
>> as
>> > a
>> > > > > whole,
>> > > > > > > > which
>> > > > > > > > >> > > would
>> > > > > > > > >> > > > > > render
>> > > > > > > > >> > > > > > >> > any
>> > > > > > > > >> > > > > > >> > > > > > resource
>> > > > > > > > >> > > > > > >> > > > > > >>> > scoped
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > the broker universally
>> > > unique
>> > > > > as
>> > > > > > > > well.
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > The reason I mention
>> this
>> > > is
>> > > > > > that,
>> > > > > > > > >> > although
>> > > > > > > > >> > > a
>> > > > > > > > >> > > > > UUID
>> > > > > > > > >> > > > > > >> is
>> > > > > > > > >> > > > > > >> > way
>> > > > > > > > >> > > > > > >> > > > > more
>> > > > > > > > >> > > > > > >> > > > > > >>> > compact
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > than topic names, it’s
>> > > still
>> > > > 16
>> > > > > > > > bytes.
>> > > > > > > > >> In
>> > > > > > > > >> > > > > > contrast,
>> > > > > > > > >> > > > > > >> a
>> > > > > > > > >> > > > > > >> > > > 4-byte
>> > > > > > > > >> > > > > > >> > > > > > >>> integer
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > sequence id would give
>> > us 4
>> > > > > > billion
>> > > > > > > > >> unique
>> > > > > > > > >> > > > > topics
>> > > > > > > > >> > > > > > >> per
>> > > > > > > > >> > > > > > >> > > > > cluster,
>> > > > > > > > >> > > > > > >> > > > > > >>> which
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > seems
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > like enough ;)
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Considering the
>> number of
>> > > > > > different
>> > > > > > > > >> times
>> > > > > > > > >> > > > these
>> > > > > > > > >> > > > > > >> topic
>> > > > > > > > >> > > > > > >> > > > > > >>> identifiers are
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > sent
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > over the wire or
>> stored
>> > in
>> > > > > > memory,
>> > > > > > > it
>> > > > > > > > >> > seems
>> > > > > > > > >> > > > like
>> > > > > > > > >> > > > > > it
>> > > > > > > > >> > > > > > >> > might
>> > > > > > > > >> > > > > > >> > > > be
>> > > > > > > > >> > > > > > >> > > > > > >>> worth
>> > > > > > > > >> > > > > > >> > > > > > >>> > the
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > additional 4x space
>> > > savings.
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > What do you think
>> about
>> > > this?
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Thanks,
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > John
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > On Fri, Sep 11, 2020,
>> at
>> > > > 03:20,
>> > > > > > Tom
>> > > > > > > > >> > Bentley
>> > > > > > > > >> > > > > wrote:
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Hi Justine,
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > This looks like a
>> very
>> > > > > welcome
>> > > > > > > > >> > > improvement.
>> > > > > > > > >> > > > > > >> Thanks!
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Maybe I missed it,
>> but
>> > > the
>> > > > > KIP
>> > > > > > > > >> doesn't
>> > > > > > > > >> > > seem
>> > > > > > > > >> > > > to
>> > > > > > > > >> > > > > > >> > mention
>> > > > > > > > >> > > > > > >> > > > > > changing
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > DeleteTopicsRequest
>> to
>> > > > > identify
>> > > > > > > the
>> > > > > > > > >> > topic
>> > > > > > > > >> > > > > using
>> > > > > > > > >> > > > > > an
>> > > > > > > > >> > > > > > >> > id.
>> > > > > > > > >> > > > > > >> > > > > Maybe
>> > > > > > > > >> > > > > > >> > > > > > >>> > that's out
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > of
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > scope, but
>> > > > > DeleteTopicsRequest
>> > > > > > is
>> > > > > > > > not
>> > > > > > > > >> > > listed
>> > > > > > > > >> > > > > > among
>> > > > > > > > >> > > > > > >> > the
>> > > > > > > > >> > > > > > >> > > > > Future
>> > > > > > > > >> > > > > > >> > > > > > >>> Work
>> > > > > > > > >> > > > > > >> > > > > > >>> > APIs
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > either.
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Kind regards,
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Tom
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > On Thu, Sep 10,
>> 2020 at
>> > > > 3:59
>> > > > > PM
>> > > > > > > > >> Satish
>> > > > > > > > >> > > > > Duggana <
>> > > > > > > > >> > > > > > >> > > > > > >>> > > >
>> satish.duggana@gmail.com
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > wrote:
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Thanks
>> Lucas/Justine
>> > > for
>> > > > > the
>> > > > > > > nice
>> > > > > > > > >> KIP.
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > It has several
>> > benefits
>> > > > > which
>> > > > > > > > also
>> > > > > > > > >> > > include
>> > > > > > > > >> > > > > > >> > > simplifying
>> > > > > > > > >> > > > > > >> > > > > the
>> > > > > > > > >> > > > > > >> > > > > > >>> topic
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > deletion process
>> by
>> > > > > > controller
>> > > > > > > > and
>> > > > > > > > >> > logs
>> > > > > > > > >> > > > > > cleanup
>> > > > > > > > >> > > > > > >> by
>> > > > > > > > >> > > > > > >> > > > > brokers
>> > > > > > > > >> > > > > > >> > > > > > in
>> > > > > > > > >> > > > > > >> > > > > > >>> > corner
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > cases.
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Best,
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Satish.
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > On Wed, Sep 9,
>> 2020
>> > at
>> > > > > 10:07
>> > > > > > PM
>> > > > > > > > >> > Justine
>> > > > > > > > >> > > > > > Olshan <
>> > > > > > > > >> > > > > > >> > > > > > >>> > jolshan@confluent.io
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > wrote:
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Hello all, it's
>> > been
>> > > > > > almost a
>> > > > > > > > >> year!
>> > > > > > > > >> > > I've
>> > > > > > > > >> > > > > > made
>> > > > > > > > >> > > > > > >> > some
>> > > > > > > > >> > > > > > >> > > > > > changes
>> > > > > > > > >> > > > > > >> > > > > > >>> to
>> > > > > > > > >> > > > > > >> > > > > > >>> > this
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > KIP
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > and hope to
>> continue
>> > > the
>> > > > > > > > >> discussion.
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > One of the main
>> > > changes
>> > > > > > I've
>> > > > > > > > >> added
>> > > > > > > > >> > is
>> > > > > > > > >> > > > now
>> > > > > > > > >> > > > > > the
>> > > > > > > > >> > > > > > >> > > > metadata
>> > > > > > > > >> > > > > > >> > > > > > >>> response
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > will
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > include the topic
>> ID
>> > > (as
>> > > > > > Colin
>> > > > > > > > >> > > suggested).
>> > > > > > > > >> > > > > > >> Clients
>> > > > > > > > >> > > > > > >> > > can
>> > > > > > > > >> > > > > > >> > > > > > >>> obtain the
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > topicID
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > of a given topic
>> > > through
>> > > > a
>> > > > > > > > >> > > > TopicDescription.
>> > > > > > > > >> > > > > > The
>> > > > > > > > >> > > > > > >> > > > topicId
>> > > > > > > > >> > > > > > >> > > > > > will
>> > > > > > > > >> > > > > > >> > > > > > >>> > also be
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > included with the
>> > > > > > > UpdateMetadata
>> > > > > > > > >> > > request.
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Let me know what
>> > you
>> > > > all
>> > > > > > > think.
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Thank you,
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Justine
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > On 2019/09/13
>> > > 16:38:26,
>> > > > > > > "Colin
>> > > > > > > > >> > > McCabe" <
>> > > > > > > > >> > > > > > >> > > > > > cmccabe@apache.org
>> > > > > > > > >> > > > > > >> > > > > > >>> >
>> > > > > > > > >> > > > > > >> > > > > > >>> > wrote:
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Hi Lucas,
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Thanks for
>> > tackling
>> > > > > this.
>> > > > > > > > >> Topic
>> > > > > > > > >> > IDs
>> > > > > > > > >> > > > > are a
>> > > > > > > > >> > > > > > >> > great
>> > > > > > > > >> > > > > > >> > > > > idea,
>> > > > > > > > >> > > > > > >> > > > > > >>> and
>> > > > > > > > >> > > > > > >> > > > > > >>> > this
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > is
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > really good
>> writeup.
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > For
>> > > > > > > /brokers/topics/[topic],
>> > > > > > > > >> the
>> > > > > > > > >> > > > schema
>> > > > > > > > >> > > > > > >> version
>> > > > > > > > >> > > > > > >> > > > > should
>> > > > > > > > >> > > > > > >> > > > > > be
>> > > > > > > > >> > > > > > >> > > > > > >>> > bumped
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > version 3, rather
>> > than
>> > > 2.
>> > > > > > > > KIP-455
>> > > > > > > > >> > > bumped
>> > > > > > > > >> > > > > the
>> > > > > > > > >> > > > > > >> > version
>> > > > > > > > >> > > > > > >> > > > of
>> > > > > > > > >> > > > > > >> > > > > > this
>> > > > > > > > >> > > > > > >> > > > > > >>> > znode
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 2
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > already :)
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Given that
>> we're
>> > > > going
>> > > > > to
>> > > > > > > be
>> > > > > > > > >> > seeing
>> > > > > > > > >> > > > > these
>> > > > > > > > >> > > > > > >> > things
>> > > > > > > > >> > > > > > >> > > as
>> > > > > > > > >> > > > > > >> > > > > > >>> strings
>> > > > > > > > >> > > > > > >> > > > > > >>> > as
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > lot
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > (in
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > logs, in
>> ZooKeeper,
>> > on
>> > > > the
>> > > > > > > > >> > command-line,
>> > > > > > > > >> > > > > > etc.),
>> > > > > > > > >> > > > > > >> > does
>> > > > > > > > >> > > > > > >> > > it
>> > > > > > > > >> > > > > > >> > > > > > make
>> > > > > > > > >> > > > > > >> > > > > > >>> > sense to
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > use
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > base64 when
>> > converting
>> > > > them
>> > > > > > to
>> > > > > > > > >> > strings?
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Here is an
>> > example
>> > > of
>> > > > > the
>> > > > > > > hex
>> > > > > > > > >> > > > > > >> representation:
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
>> > > > > > > > >> > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > And here is an
>> > > > example
>> > > > > in
>> > > > > > > > >> base64.
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
>> > > > b8tRS7h4TJ2Vt43Dp85v2A
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > The base64
>> > version
>> > > > > saves
>> > > > > > 15
>> > > > > > > > >> > letters
>> > > > > > > > >> > > > (to
>> > > > > > > > >> > > > > be
>> > > > > > > > >> > > > > > >> > fair,
>> > > > > > > > >> > > > > > >> > > 4
>> > > > > > > > >> > > > > > >> > > > of
>> > > > > > > > >> > > > > > >> > > > > > >>> those
>> > > > > > > > >> > > > > > >> > > > > > >>> > were
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > dashes that we
>> could
>> > > have
>> > > > > > > elided
>> > > > > > > > in
>> > > > > > > > >> > the
>> > > > > > > > >> > > > hex
>> > > > > > > > >> > > > > > >> > > > > > representation.)
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Another thing
>> to
>> > > > > consider
>> > > > > > > is
>> > > > > > > > >> that
>> > > > > > > > >> > we
>> > > > > > > > >> > > > > > should
>> > > > > > > > >> > > > > > >> > > specify
>> > > > > > > > >> > > > > > >> > > > > > that
>> > > > > > > > >> > > > > > >> > > > > > >>> the
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > all-zeroes UUID is
>> > not
>> > > a
>> > > > > > valid
>> > > > > > > > >> topic
>> > > > > > > > >> > > UUID.
>> > > > > > > > >> > > > > >  We
>> > > > > > > > >> > > > > > >> > can't
>> > > > > > > > >> > > > > > >> > > > use
>> > > > > > > > >> > > > > > >> > > > > > >>> null
>> > > > > > > > >> > > > > > >> > > > > > >>> > for
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > this
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > because we can't
>> > pass a
>> > > > > null
>> > > > > > > UUID
>> > > > > > > > >> over
>> > > > > > > > >> > > the
>> > > > > > > > >> > > > > RPC
>> > > > > > > > >> > > > > > >> > > protocol
>> > > > > > > > >> > > > > > >> > > > > > >>> (there
>> > > > > > > > >> > > > > > >> > > > > > >>> > is no
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > special pattern
>> for
>> > > null,
>> > > > > nor
>> > > > > > > do
>> > > > > > > > we
>> > > > > > > > >> > want
>> > > > > > > > >> > > > to
>> > > > > > > > >> > > > > > >> waste
>> > > > > > > > >> > > > > > >> > > space
>> > > > > > > > >> > > > > > >> > > > > > >>> reserving
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > such
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > pattern.)
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Maybe I missed
>> > it,
>> > > > but
>> > > > > > did
>> > > > > > > > you
>> > > > > > > > >> > > > describe
>> > > > > > > > >> > > > > > >> > > "migration
>> > > > > > > > >> > > > > > >> > > > > > of...
>> > > > > > > > >> > > > > > >> > > > > > >>> > existing
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > topic[s] without
>> > topic
>> > > > IDs"
>> > > > > > in
>> > > > > > > > >> detail
>> > > > > > > > >> > in
>> > > > > > > > >> > > > any
>> > > > > > > > >> > > > > > >> > section?
>> > > > > > > > >> > > > > > >> > > > It
>> > > > > > > > >> > > > > > >> > > > > > >>> seems
>> > > > > > > > >> > > > > > >> > > > > > >>> > like
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > when
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > the new controller
>> > > > becomes
>> > > > > > > > active,
>> > > > > > > > >> it
>> > > > > > > > >> > > > should
>> > > > > > > > >> > > > > > >> just
>> > > > > > > > >> > > > > > >> > > > > generate
>> > > > > > > > >> > > > > > >> > > > > > >>> random
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > UUIDs for
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > these, and write
>> the
>> > > > random
>> > > > > > > UUIDs
>> > > > > > > > >> back
>> > > > > > > > >> > > to
>> > > > > > > > >> > > > > > >> > ZooKeeper.
>> > > > > > > > >> > > > > > >> > > > It
>> > > > > > > > >> > > > > > >> > > > > > >>> would be
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > good
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > to
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > spell that out.
>> We
>> > > > should
>> > > > > > make
>> > > > > > > > it
>> > > > > > > > >> > clear
>> > > > > > > > >> > > > > that
>> > > > > > > > >> > > > > > >> this
>> > > > > > > > >> > > > > > >> > > > > happens
>> > > > > > > > >> > > > > > >> > > > > > >>> > regardless
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > of
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > the inter-broker
>> > > protocol
>> > > > > > > version
>> > > > > > > > >> > (it's
>> > > > > > > > >> > > a
>> > > > > > > > >> > > > > > >> > compatible
>> > > > > > > > >> > > > > > >> > > > > > change).
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
>> > > "LeaderAndIsrRequests
>> > > > > > > > >> including an
>> > > > > > > > >> > > > > > >> > > > is_every_partition
>> > > > > > > > >> > > > > > >> > > > > > >>> flag"
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > seems a
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > bit wordy.  Can we
>> > just
>> > > > > call
>> > > > > > > > these
>> > > > > > > > >> > "full
>> > > > > > > > >> > > > > > >> > > > > > >>> LeaderAndIsrRequests"?
>> > > > > > > > >> > > > > > >> > > > > > >>> > Then
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > the
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > RPC field could be
>> > > named
>> > > > > > > "full".
>> > > > > > > > >> > Also,
>> > > > > > > > >> > > it
>> > > > > > > > >> > > > > > would
>> > > > > > > > >> > > > > > >> > > > probably
>> > > > > > > > >> > > > > > >> > > > > > be
>> > > > > > > > >> > > > > > >> > > > > > >>> > better
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > for the
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > RPC field to be an
>> > enum
>> > > > of
>> > > > > {
>> > > > > > > > >> > > UNSPECIFIED,
>> > > > > > > > >> > > > > > >> > > INCREMENTAL,
>> > > > > > > > >> > > > > > >> > > > > FULL
>> > > > > > > > >> > > > > > >> > > > > > >>> }, so
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > that
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > we
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > can cleanly handle
>> > old
>> > > > > > versions
>> > > > > > > > (by
>> > > > > > > > >> > > > treating
>> > > > > > > > >> > > > > > >> them
>> > > > > > > > >> > > > > > >> > as
>> > > > > > > > >> > > > > > >> > > > > > >>> UNSPECIFIED)
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > In the
>> > > > > > LeaderAndIsrRequest
>> > > > > > > > >> > section,
>> > > > > > > > >> > > > you
>> > > > > > > > >> > > > > > >> write
>> > > > > > > > >> > > > > > >> > "A
>> > > > > > > > >> > > > > > >> > > > > final
>> > > > > > > > >> > > > > > >> > > > > > >>> > deletion
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > event
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > will be secheduled
>> > for
>> > > X
>> > > > ms
>> > > > > > > after
>> > > > > > > > >> the
>> > > > > > > > >> > > > > > >> > > > LeaderAndIsrRequest
>> > > > > > > > >> > > > > > >> > > > > > was
>> > > > > > > > >> > > > > > >> > > > > > >>> > first
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > received..."  I
>> guess
>> > > > the X
>> > > > > > > was a
>> > > > > > > > >> > > > > placeholder
>> > > > > > > > >> > > > > > >> that
>> > > > > > > > >> > > > > > >> > > you
>> > > > > > > > >> > > > > > >> > > > > > >>> intended
>> > > > > > > > >> > > > > > >> > > > > > >>> > to
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > replace
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > before posting? :)
>> > In
>> > > > any
>> > > > > > > case,
>> > > > > > > > >> this
>> > > > > > > > >> > > > seems
>> > > > > > > > >> > > > > > like
>> > > > > > > > >> > > > > > >> > the
>> > > > > > > > >> > > > > > >> > > > kind
>> > > > > > > > >> > > > > > >> > > > > > of
>> > > > > > > > >> > > > > > >> > > > > > >>> > thing
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > we'd
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > want a
>> configuration
>> > > for.
>> > > > > > > Let's
>> > > > > > > > >> > > describe
>> > > > > > > > >> > > > > that
>> > > > > > > > >> > > > > > >> > > > > > configuration
>> > > > > > > > >> > > > > > >> > > > > > >>> key
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > somewhere
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > in this KIP,
>> > including
>> > > > what
>> > > > > > its
>> > > > > > > > >> > default
>> > > > > > > > >> > > > > value
>> > > > > > > > >> > > > > > >> is.
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > We should
>> > probably
>> > > > also
>> > > > > > > log a
>> > > > > > > > >> > bunch
>> > > > > > > > >> > > of
>> > > > > > > > >> > > > > > >> messages
>> > > > > > > > >> > > > > > >> > > at
>> > > > > > > > >> > > > > > >> > > > > WARN
>> > > > > > > > >> > > > > > >> > > > > > >>> level
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > when
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > something is
>> > scheduled
>> > > > for
>> > > > > > > > >> deletion,
>> > > > > > > > >> > as
>> > > > > > > > >> > > > > well.
>> > > > > > > > >> > > > > > >> > (Maybe
>> > > > > > > > >> > > > > > >> > > > > this
>> > > > > > > > >> > > > > > >> > > > > > >>> was
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > assumed, but
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > it would be good
>> to
>> > > > mention
>> > > > > > > it).
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > I feel like
>> there
>> > > > are a
>> > > > > > few
>> > > > > > > > >> > sections
>> > > > > > > > >> > > > > that
>> > > > > > > > >> > > > > > >> > should
>> > > > > > > > >> > > > > > >> > > be
>> > > > > > > > >> > > > > > >> > > > > > >>> moved to
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > "rejected
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > alternatives."
>> For
>> > > > > example,
>> > > > > > in
>> > > > > > > > the
>> > > > > > > > >> > > > > > DeleteTopics
>> > > > > > > > >> > > > > > >> > > > section,
>> > > > > > > > >> > > > > > >> > > > > > >>> since
>> > > > > > > > >> > > > > > >> > > > > > >>> > we're
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > not
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > going to do
>> option 1
>> > or
>> > > > 2,
>> > > > > > > these
>> > > > > > > > >> > should
>> > > > > > > > >> > > be
>> > > > > > > > >> > > > > > moved
>> > > > > > > > >> > > > > > >> > into
>> > > > > > > > >> > > > > > >> > > > > > >>> "rejected
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > alternatives,"
>> > rather
>> > > > than
>> > > > > > > > >> appearing
>> > > > > > > > >> > > > > inline.
>> > > > > > > > >> > > > > > >> > > Another
>> > > > > > > > >> > > > > > >> > > > > case
>> > > > > > > > >> > > > > > >> > > > > > >>> is
>> > > > > > > > >> > > > > > >> > > > > > >>> > the
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > "Should
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > we remove topic
>> name
>> > > from
>> > > > > the
>> > > > > > > > >> protocol
>> > > > > > > > >> > > > where
>> > > > > > > > >> > > > > > >> > > possible"
>> > > > > > > > >> > > > > > >> > > > > > >>> section.
>> > > > > > > > >> > > > > > >> > > > > > >>> > This
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > is
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > clearly
>> discussing a
>> > > > design
>> > > > > > > > >> > alternative
>> > > > > > > > >> > > > that
>> > > > > > > > >> > > > > > >> we're
>> > > > > > > > >> > > > > > >> > > not
>> > > > > > > > >> > > > > > >> > > > > > >>> proposing
>> > > > > > > > >> > > > > > >> > > > > > >>> > to
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > implement:
>> removing
>> > the
>> > > > > topic
>> > > > > > > > name
>> > > > > > > > >> > from
>> > > > > > > > >> > > > > those
>> > > > > > > > >> > > > > > >> > > > protocols.
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Is it really
>> > > > necessary
>> > > > > to
>> > > > > > > > have
>> > > > > > > > >> a
>> > > > > > > > >> > new
>> > > > > > > > >> > > > > > >> > > > > > >>> > /admin/delete_topics_by_id
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > path
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > in ZooKeeper?  It
>> > seems
>> > > > > like
>> > > > > > we
>> > > > > > > > >> don't
>> > > > > > > > >> > > > really
>> > > > > > > > >> > > > > > >> need
>> > > > > > > > >> > > > > > >> > > this.
>> > > > > > > > >> > > > > > >> > > > > > >>> Whenever
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > there is
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > a new controller,
>> > we'll
>> > > > > send
>> > > > > > > out
>> > > > > > > > >> full
>> > > > > > > > >> > > > > > >> > > > > LeaderAndIsrRequests
>> > > > > > > > >> > > > > > >> > > > > > >>> which
>> > > > > > > > >> > > > > > >> > > > > > >>> > will
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > trigger the stale
>> > > topics
>> > > > to
>> > > > > > be
>> > > > > > > > >> cleaned
>> > > > > > > > >> > > up.
>> > > > > > > > >> > > > > >  The
>> > > > > > > > >> > > > > > >> > > active
>> > > > > > > > >> > > > > > >> > > > > > >>> > controller
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > will
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > also send the full
>> > > > > > > > >> LeaderAndIsrRequest
>> > > > > > > > >> > > to
>> > > > > > > > >> > > > > > >> brokers
>> > > > > > > > >> > > > > > >> > > that
>> > > > > > > > >> > > > > > >> > > > > are
>> > > > > > > > >> > > > > > >> > > > > > >>> just
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > starting
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > up.    So we don't
>> > > really
>> > > > > > need
>> > > > > > > > this
>> > > > > > > > >> > kind
>> > > > > > > > >> > > > of
>> > > > > > > > >> > > > > > >> > two-phase
>> > > > > > > > >> > > > > > >> > > > > > commit
>> > > > > > > > >> > > > > > >> > > > > > >>> > (send
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > out
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
>> StopReplicasRequest,
>> > > get
>> > > > > ACKs
>> > > > > > > > from
>> > > > > > > > >> all
>> > > > > > > > >> > > > > nodes,
>> > > > > > > > >> > > > > > >> > commit
>> > > > > > > > >> > > > > > >> > > by
>> > > > > > > > >> > > > > > >> > > > > > >>> removing
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
>> /admin/delete_topics
>> > > > node)
>> > > > > > any
>> > > > > > > > >> more.
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > You mention
>> that
>> > > > > > > FetchRequest
>> > > > > > > > >> will
>> > > > > > > > >> > > now
>> > > > > > > > >> > > > > > >> include
>> > > > > > > > >> > > > > > >> > > UUID
>> > > > > > > > >> > > > > > >> > > > > to
>> > > > > > > > >> > > > > > >> > > > > > >>> avoid
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > issues
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > where requests are
>> > made
>> > > > to
>> > > > > > > stale
>> > > > > > > > >> > > > partitions.
>> > > > > > > > >> > > > > > >> > > However,
>> > > > > > > > >> > > > > > >> > > > > > >>> adding a
>> > > > > > > > >> > > > > > >> > > > > > >>> > UUID
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > MetadataRequest is
>> > > listed
>> > > > > as
>> > > > > > > > future
>> > > > > > > > >> > > work,
>> > > > > > > > >> > > > > out
>> > > > > > > > >> > > > > > of
>> > > > > > > > >> > > > > > >> > > scope
>> > > > > > > > >> > > > > > >> > > > > for
>> > > > > > > > >> > > > > > >> > > > > > >>> this
>> > > > > > > > >> > > > > > >> > > > > > >>> > KIP.
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > How
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > will the client
>> learn
>> > > > what
>> > > > > > the
>> > > > > > > > >> topic
>> > > > > > > > >> > > UUID
>> > > > > > > > >> > > > > is,
>> > > > > > > > >> > > > > > if
>> > > > > > > > >> > > > > > >> > the
>> > > > > > > > >> > > > > > >> > > > > > metadata
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > response
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > doesn't include
>> that
>> > > > > > > information?
>> > > > > > > > >> It
>> > > > > > > > >> > > > seems
>> > > > > > > > >> > > > > > like
>> > > > > > > > >> > > > > > >> > > adding
>> > > > > > > > >> > > > > > >> > > > > the
>> > > > > > > > >> > > > > > >> > > > > > >>> UUID
>> > > > > > > > >> > > > > > >> > > > > > >>> > to
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > MetadataResponse
>> > would
>> > > be
>> > > > > an
>> > > > > > > > >> > improvement
>> > > > > > > > >> > > > > here
>> > > > > > > > >> > > > > > >> that
>> > > > > > > > >> > > > > > >> > > > might
>> > > > > > > > >> > > > > > >> > > > > > not
>> > > > > > > > >> > > > > > >> > > > > > >>> be
>> > > > > > > > >> > > > > > >> > > > > > >>> > too
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > hard to
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > make.
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > best,
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Colin
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > On Mon, Sep 9,
>> > > 2019,
>> > > > at
>> > > > > > > > 17:48,
>> > > > > > > > >> > > Ryanne
>> > > > > > > > >> > > > > > Dolan
>> > > > > > > > >> > > > > > >> > > wrote:
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > Lucas, this
>> > would
>> > > > be
>> > > > > > > great.
>> > > > > > > > >> I've
>> > > > > > > > >> > > run
>> > > > > > > > >> > > > > > into
>> > > > > > > > >> > > > > > >> > > issues
>> > > > > > > > >> > > > > > >> > > > > with
>> > > > > > > > >> > > > > > >> > > > > > >>> > topics
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > being
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > resurrected
>> > > > > > accidentally,
>> > > > > > > > >> since
>> > > > > > > > >> > a
>> > > > > > > > >> > > > > client
>> > > > > > > > >> > > > > > >> > cannot
>> > > > > > > > >> > > > > > >> > > > > > easily
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > distinguish
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > between
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > a deleted
>> topic
>> > > > and a
>> > > > > > new
>> > > > > > > > >> topic
>> > > > > > > > >> > > with
>> > > > > > > > >> > > > > the
>> > > > > > > > >> > > > > > >> same
>> > > > > > > > >> > > > > > >> > > > name.
>> > > > > > > > >> > > > > > >> > > > > > I'd
>> > > > > > > > >> > > > > > >> > > > > > >>> > need
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > the
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > ID
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > accessible
>> from
>> > > the
>> > > > > > > client
>> > > > > > > > to
>> > > > > > > > >> > > solve
>> > > > > > > > >> > > > > that
>> > > > > > > > >> > > > > > >> > issue,
>> > > > > > > > >> > > > > > >> > > > but
>> > > > > > > > >> > > > > > >> > > > > > >>> this
>> > > > > > > > >> > > > > > >> > > > > > >>> > is a
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > good
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > first
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > step.
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > Ryanne
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > On Wed, Sep
>> 4,
>> > > 2019
>> > > > > at
>> > > > > > > 1:41
>> > > > > > > > >> PM
>> > > > > > > > >> > > Lucas
>> > > > > > > > >> > > > > > >> > > Bradstreet <
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > lucas@confluent.io>
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > wrote:
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Hi all,
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > I would
>> like
>> > to
>> > > > > kick
>> > > > > > > off
>> > > > > > > > >> > > > discussion
>> > > > > > > > >> > > > > of
>> > > > > > > > >> > > > > > >> > > KIP-516,
>> > > > > > > > >> > > > > > >> > > > > an
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > implementation
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > of topic
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > IDs for
>> > Kafka.
>> > > > > Topic
>> > > > > > > IDs
>> > > > > > > > >> aim
>> > > > > > > > >> > to
>> > > > > > > > >> > > > > solve
>> > > > > > > > >> > > > > > >> topic
>> > > > > > > > >> > > > > > >> > > > > > >>> uniqueness
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > problems in
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Kafka,
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > where
>> > referring
>> > > > to
>> > > > > a
>> > > > > > > > topic
>> > > > > > > > >> by
>> > > > > > > > >> > > name
>> > > > > > > > >> > > > > > >> alone is
>> > > > > > > > >> > > > > > >> > > > > > >>> insufficient.
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > Such
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > cases
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > include
>> when
>> > a
>> > > > > topic
>> > > > > > > has
>> > > > > > > > >> been
>> > > > > > > > >> > > > > deleted
>> > > > > > > > >> > > > > > >> and
>> > > > > > > > >> > > > > > >> > > > > recreated
>> > > > > > > > >> > > > > > >> > > > > > >>> with
>> > > > > > > > >> > > > > > >> > > > > > >>> > the
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > same
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > name.
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Unique
>> > > > identifiers
>> > > > > > will
>> > > > > > > > >> help
>> > > > > > > > >> > > > > simplify
>> > > > > > > > >> > > > > > >> and
>> > > > > > > > >> > > > > > >> > > > improve
>> > > > > > > > >> > > > > > >> > > > > > >>> Kafka's
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > topic
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > deletion
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > process,
>> as
>> > > well
>> > > > as
>> > > > > > > > prevent
>> > > > > > > > >> > > cases
>> > > > > > > > >> > > > > > where
>> > > > > > > > >> > > > > > >> > > brokers
>> > > > > > > > >> > > > > > >> > > > > may
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > incorrectly
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > interact
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > with stale
>> > > > versions
>> > > > > > of
>> > > > > > > > >> topics.
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > >
>> > > > > > > > >> > > > > > >> > > > > > >>> >
>> > > > > > > > >> > > > > > >> > > > > > >>>
>> > > > > > > > >> > > > > > >> > > > > >
>> > > > > > > > >> > > > > > >> > > > >
>> > > > > > > > >> > > > > > >> > > >
>> > > > > > > > >> > > > > > >> > >
>> > > > > > > > >> > > > > > >> >
>> > > > > > > > >> > > > > > >>
>> > > > > > > > >> > > > > >
>> > > > > > > > >> > > > >
>> > > > > > > > >> > > >
>> > > > > > > > >> > >
>> > > > > > > > >> >
>> > > > > > > > >>
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Looking
>> > forward
>> > > > to
>> > > > > > your
>> > > > > > > > >> > > thoughts.
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Lucas
>> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
>> > > > > > > > >> > > > > > >> > > > > > >>> >
>> > > > > > > > >> > > > > > >> > > > > > >>> >
>> > > > > > > > >> > > > > > >> > > > > > >>>
>> > > > > > > > >> > > > > > >> > > > > > >>
>> > > > > > > > >> > > > > > >> > > > > >
>> > > > > > > > >> > > > > > >> > > > >
>> > > > > > > > >> > > > > > >> > > >
>> > > > > > > > >> > > > > > >> > >
>> > > > > > > > >> > > > > > >> >
>> > > > > > > > >> > > > > > >>
>> > > > > > > > >> > > > > > >
>> > > > > > > > >> > > > > >
>> > > > > > > > >> > > > >
>> > > > > > > > >> > > >
>> > > > > > > > >> > >
>> > > > > > > > >> >
>> > > > > > > > >>
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Justine Olshan <jo...@confluent.io>]

Hello all,

I just wanted to follow up on this discussion. Did we come to an
understanding about the directory structure?

I think the biggest question here is what is acceptable to leave out due to
scope vs. what is considered to be too much tech debt.
This KIP is already pretty large with the number of changes, but it also
makes sense to do things correctly, so I'd love to hear everyone's thoughts.

Thanks,
Justine

On Fri, Sep 25, 2020 at 8:19 AM Lucas Bradstreet <lu...@confluent.io> wrote:

> Hi Ismael,
>
> If you do not store it in a metadata file or in the directory structure
> would you then
> require the LeaderAndIsrRequest following startup to give you some notion
> of
> topic name in memory? We would still need this information for the older
> protocols, but
> perhaps this is what's meant by tech debt.
>
> Once we're free of the old non-topicID requests then I think you wouldn't
> need to retain the topic name.
> I think the ability to easily look up topic names associated with partition
> directories would still be missed
> when diagnosing issues, though maybe it wouldn't be a deal breaker with the
> right tooling.
>
> Thanks,
>
> Lucas
>
> On Fri, Sep 25, 2020 at 7:55 AM Ismael Juma <is...@juma.me.uk> wrote:
>
> > Hi Lucas,
> >
> > Why would you include the name and id? I think you'd want to remove the
> > name from the directory name right? Jason's suggestion was that if you
> > remove the name from the directory, then why would you need the id name
> > mapping file?
> >
> > Ismael
> >
> > On Thu, Sep 24, 2020 at 4:24 PM Lucas Bradstreet <lu...@confluent.io>
> > wrote:
> >
> > > > 2. Part of the usage of the file is to have persistent storage of the
> > > topic
> > > ID and use it to compare with the ID supplied in the LeaderAndIsr
> > Request.
> > > There is some discussion in the KIP about changes to the directory
> > > structure, but I believe directory changes were considered to be out of
> > > scope when the KIP was written.
> > >
> > >
> > > Yeah, I was hoping to get a better understanding of why it was taken
> out
> > of
> > > scope. Perhaps Lucas Bradstreet might have more insight about the
> > decision.
> > > Basically my point is that we have to create additional infrastructure
> > here
> > > to support the name/id mapping, so I wanted to understand if we just
> > > consider this a sort of tech debt. It would be useful to cover how we
> > > handle the case when this file gets corrupted. Seems like we just have
> to
> > > trust that it matches whatever the controller tells us and rewrite it?
> > >
> > >
> > > Hi Jason, Justine,
> > >
> > > My thought process is that we will likely need the metadata file
> whether
> > we
> > > rename the directories or not.
> > > Linux supports filenames of up to 255 bytes and that would not be
> enough
> > to
> > > support a directory name
> > >  including both the name and topic ID. Given that fact, it seemed
> > > reasonable to add the metadata file
> > > and leave the directory rename until some time in the future (possibly
> > > never).
> > >
> > > If the file does get corrupted I think you're right. We would either
> have
> > > to trust it matches what the controller tells us
> > >  or error out and let an administrator resolve it by checking across
> > > replicas for consistency.
> > >
> > > Lucas
> > >
> > >
> > > On Thu, Sep 24, 2020 at 3:41 PM Jason Gustafson <ja...@confluent.io>
> > > wrote:
> > >
> > > > Thanks Justine. Responses below:
> > > >
> > > > > 1. Yes, the directory will still be based on the topic names.
> > > > LeaderAndIsrRequest is one of the few requests that will still
> contain
> > > the
> > > > topic name. So I think we have this covered. Sorry for confusion.
> > > >
> > > > Ah, you're right. My eyes passed right over the field.
> > > >
> > > > > 2. Part of the usage of the file is to have persistent storage of
> the
> > > > topic
> > > > ID and use it to compare with the ID supplied in the LeaderAndIsr
> > > Request.
> > > > There is some discussion in the KIP about changes to the directory
> > > > structure, but I believe directory changes were considered to be out
> of
> > > > scope when the KIP was written.
> > > >
> > > > Yeah, I was hoping to get a better understanding of why it was taken
> > out
> > > of
> > > > scope. Perhaps Lucas Bradstreet might have more insight about the
> > > decision.
> > > > Basically my point is that we have to create additional
> infrastructure
> > > here
> > > > to support the name/id mapping, so I wanted to understand if we just
> > > > consider this a sort of tech debt. It would be useful to cover how we
> > > > handle the case when this file gets corrupted. Seems like we just
> have
> > to
> > > > trust that it matches whatever the controller tells us and rewrite
> it?
> > > >
> > > > > 3. I think this is a good point, but I again I wonder about the
> scope
> > > of
> > > > the KIP. Most of the changes mentioned in the KIP are for supporting
> > > topic
> > > > deletion and I believe that is why the produce request was listed
> under
> > > > future work.
> > > >
> > > > That's fair. I brought it up since `Fetch` is already included. If
> > we've
> > > > got `Metadata` and `Fetch`, seems we may as well do `Produce` and
> save
> > an
> > > > extra kip. No strong objection though if you want to leave it out.
> > > >
> > > >
> > > > -Jason
> > > >
> > > >
> > > > On Thu, Sep 24, 2020 at 3:26 PM Justine Olshan <jolshan@confluent.io
> >
> > > > wrote:
> > > >
> > > > > Hi Jason,
> > > > >
> > > > > Thanks for your comments.
> > > > >
> > > > > 1. Yes, the directory will still be based on the topic names.
> > > > > LeaderAndIsrRequest is one of the few requests that will still
> > contain
> > > > the
> > > > > topic name. So I think we have this covered. Sorry for confusion.
> > > > >
> > > > > 2. Part of the usage of the file is to have persistent storage of
> the
> > > > topic
> > > > > ID and use it to compare with the ID supplied in the LeaderAndIsr
> > > > Request.
> > > > > There is some discussion in the KIP about changes to the directory
> > > > > structure, but I believe directory changes were considered to be
> out
> > of
> > > > > scope when the KIP was written.
> > > > >
> > > > > 3. I think this is a good point, but I again I wonder about the
> scope
> > > of
> > > > > the KIP. Most of the changes mentioned in the KIP are for
> supporting
> > > > topic
> > > > > deletion and I believe that is why the produce request was listed
> > under
> > > > > future work.
> > > > >
> > > > > 4. This sounds like it might be a good solution, but I will need to
> > > > discuss
> > > > > more with KIP-500 folks to get the details right.
> > > > >
> > > > > Thanks,
> > > > > Justine
> > > > >
> > > > > On Thu, Sep 24, 2020 at 12:30 PM Jason Gustafson <
> jason@confluent.io
> > >
> > > > > wrote:
> > > > >
> > > > > > Hi Justine,
> > > > > >
> > > > > > Thanks for picking up this work. I have a few questions/comments:
> > > > > >
> > > > > > 1. It sounds like the directory structure is still going to be
> > based
> > > on
> > > > > > topic names. Do I have that right? One complication is that the
> > > > > > LeaderAndIsr request does not include the topic name any longer.
> > This
> > > > > means
> > > > > > that a replica must wait for the UpdateMetadata request to arrive
> > > with
> > > > > the
> > > > > > topic name mapping before it can create the directory. I wonder
> if
> > it
> > > > > would
> > > > > > be simpler to avoid assumptions on the ordering of UpdateMetadata
> > and
> > > > let
> > > > > > LeaderAndIsr include the topic name as well. Feels like we are
> not
> > > > saving
> > > > > > that much by excluding it.
> > > > > >
> > > > > > 2. On a related note, it seems that the reason we have the
> > partition
> > > > > > metadata file is because we are not changing the directory
> > structure.
> > > > We
> > > > > > need it so that we remember which directories map to which topic
> > id.
> > > I
> > > > > > think it would be useful to add some detail about why changing
> the
> > > > > > directory layout was rejected. Basically trying to understand how
> > > much
> > > > > > effort we are saving by skipping this step if we have to
> implement
> > > this
> > > > > > additional bookkeeping. One concern I have for example is that
> the
> > > > > > partition metadata file gets corrupt and then all bets are off as
> > far
> > > > as
> > > > > > topic consistency.
> > > > > >
> > > > > > 3. Is it worth adding support now for topic ids in the `Produce`
> > > > request
> > > > > > now? Overhead is mentioned as one of the motivations and the best
> > > APIs
> > > > to
> > > > > > get savings from are `Produce` and `Fetch`.
> > > > > >
> > > > > > 4. When it comes to bootstrapping the metadata topic with respect
> > to
> > > > > > KIP-500, one suggestion would be to reserve a sentinel topic ID
> > which
> > > > can
> > > > > > be used initially by new brokers. When the first controller is
> > > elected,
> > > > > it
> > > > > > can assign a topicId to the metadata topic. This is a bit similar
> > to
> > > > how
> > > > > we
> > > > > > were planning to generate the clusterId.
> > > > > >
> > > > > > Thanks,
> > > > > > Jason
> > > > > >
> > > > > >
> > > > > > On Thu, Sep 24, 2020 at 11:10 AM Jun Rao <ju...@confluent.io>
> wrote:
> > > > > >
> > > > > > > Hi, Justine,
> > > > > > >
> > > > > > > Thanks for the updated KIP. A few more comments below.
> > > > > > >
> > > > > > > 30.  {"name": "id", "type": "string", "doc": "version id"}}:
> The
> > > doc
> > > > > > should
> > > > > > > say UUID.
> > > > > > >
> > > > > > > 31. LeaderAndIsrResponse v5 and StopReplicaResponse v4 : It
> seems
> > > > there
> > > > > > is
> > > > > > > no need to add topic_id at partitions level.
> > > > > > >
> > > > > > > 32. Regarding partition metadata file. Perhaps the key can be a
> > > > single
> > > > > > > word, sth like the following.
> > > > > > > version: 0
> > > > > > > topic_id: 46bdb63f-9e8d-4a38-bf7b-ee4eb2a794e4
> > > > > > >
> > > > > > > 33. Another tricky thing that I realized is how to support the
> > > > metadata
> > > > > > > topic introduced in KIP-595. It's a bit tricky to assign a UUID
> > to
> > > > the
> > > > > > > metadata topic since we have a chicken-and-egg problem. The
> > > > controller
> > > > > > > needs to persist the UUID in the metadata topic in order to
> > assign
> > > > one
> > > > > > > successfully, but the metadata topic is needed to elect a
> > > controller
> > > > > > > (KIP-631). So, this probably needs a bit more thought.
> > > > > > >
> > > > > > > Jun
> > > > > > >
> > > > > > > On Thu, Sep 24, 2020 at 3:04 AM Ismael Juma <ismael@juma.me.uk
> >
> > > > wrote:
> > > > > > >
> > > > > > > > Also, can we provide more details on how the Partition
> Metadata
> > > > file
> > > > > > will
> > > > > > > > be used?
> > > > > > > >
> > > > > > > > Ismael
> > > > > > > >
> > > > > > > > On Thu, Sep 24, 2020 at 3:01 AM Ismael Juma <
> ismael@juma.me.uk
> > >
> > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi Justine,
> > > > > > > > >
> > > > > > > > > I think we need to update the "Rejected Alternatives"
> section
> > > to
> > > > > take
> > > > > > > > into
> > > > > > > > > account that the proposal now removes the topic name from
> the
> > > > fetch
> > > > > > > > > request. Also, if we are removing it from the Fetch
> request,
> > > does
> > > > > it
> > > > > > > make
> > > > > > > > > sense not to remove it from similar requests like
> > > > > ListOffsetRequest?
> > > > > > > > >
> > > > > > > > > Ismael
> > > > > > > > >
> > > > > > > > > On Thu, Sep 24, 2020 at 2:46 AM David Jacot <
> > > djacot@confluent.io
> > > > >
> > > > > > > wrote:
> > > > > > > > >
> > > > > > > > >> Hi Justine,
> > > > > > > > >>
> > > > > > > > >> Thanks for the KIP. I finally had time to read it :). It
> is
> > a
> > > > > great
> > > > > > > > >> improvement.
> > > > > > > > >>
> > > > > > > > >> I have a few comments/questions:
> > > > > > > > >>
> > > > > > > > >> 1. It seems that the schema of the StopReplicaRequest is
> > > > slightly
> > > > > > > > >> outdated.
> > > > > > > > >> We
> > > > > > > > >> did some changes as part of KIP-570. V3 is already
> organized
> > > by
> > > > > > > topics.
> > > > > > > > >>
> > > > > > > > >> 2. I just want to make sure that I understand the
> > > reconciliation
> > > > > > > > >> logic correctly. When
> > > > > > > > >> an "INCREMENTAL" LeaderAndIsr Request is received, the
> > broker
> > > > will
> > > > > > > also
> > > > > > > > >> reconcile
> > > > > > > > >> when the local uuid does not match the uuid in the
> request,
> > > > right?
> > > > > > In
> > > > > > > > this
> > > > > > > > >> case, the
> > > > > > > > >> local log is staged for deletion.
> > > > > > > > >>
> > > > > > > > >> 3. In the documentation of the `
> delete.stale.topic.delay.ms
> > `
> > > > > > config,
> > > > > > > it
> > > > > > > > >> says "When a
> > > > > > > > >> FULL LeaderAndIsrRequest is received..." but I suppose it
> > > > applies
> > > > > to
> > > > > > > > both
> > > > > > > > >> types.
> > > > > > > > >>
> > > > > > > > >> Best,
> > > > > > > > >> David
> > > > > > > > >>
> > > > > > > > >> On Thu, Sep 24, 2020 at 1:40 AM Justine Olshan <
> > > > > > jolshan@confluent.io>
> > > > > > > > >> wrote:
> > > > > > > > >>
> > > > > > > > >> > Hi Jun,
> > > > > > > > >> >
> > > > > > > > >> > Thanks for the comments. I apologize for some of the
> typos
> > > and
> > > > > > > > >> confusion.
> > > > > > > > >> > I’ve updated the KIP to fix some of the issues you
> > > mentioned.
> > > > > > > > >> >
> > > > > > > > >> > 20.2 I’ve changed the type to String.
> > > > > > > > >> > 20.1/3 I’ve updated the TopicZNode to fix formatting and
> > > > reflect
> > > > > > the
> > > > > > > > >> latest
> > > > > > > > >> > version before this change.
> > > > > > > > >> >
> > > > > > > > >> > 21. You are correct and I’ve removed this line. I’ve
> also
> > > > added
> > > > > a
> > > > > > > line
> > > > > > > > >> > mentioning an IBP bump is necessary for migration
> > > > > > > > >> >
> > > > > > > > >> > 22. I think the wording was unclear but your summary is
> > what
> > > > was
> > > > > > > > >> intended
> > > > > > > > >> > by this line. I’ve updated to make this point more
> clear.
> > > > > “Remove
> > > > > > > > >> deleted
> > > > > > > > >> > topics from replicas by sending StopReplicaRequest V3
> > before
> > > > the
> > > > > > IBP
> > > > > > > > >> bump
> > > > > > > > >> > using the old logic, and using V4 and the new logic with
> > > topic
> > > > > IDs
> > > > > > > > after
> > > > > > > > >> > the IBP bump.”
> > > > > > > > >> >
> > > > > > > > >> > 23. I’ve removed the unspecified type from the KIP and
> > > mention
> > > > > > that
> > > > > > > > IBP
> > > > > > > > >> > will be used to handle this request. “IBP will be used
> to
> > > > > > determine
> > > > > > > > >> whether
> > > > > > > > >> > this new form of the request will be used. For older
> > > requests,
> > > > > we
> > > > > > > will
> > > > > > > > >> > ignore this field and default to previous behavior.”
> > > > > > > > >> >
> > > > > > > > >> > 24. I’ve fixed this typo.
> > > > > > > > >> >
> > > > > > > > >> > 25. I've added a topics at a higher level for
> > > > > LeaderAndIsrResponse
> > > > > > > v5,
> > > > > > > > >> > StopReplicaResponse v4. I've also changed
> > StopReplicaRequest
> > > > v4
> > > > > to
> > > > > > > > have
> > > > > > > > >> > topics at a higher level.
> > > > > > > > >> >
> > > > > > > > >> > 26. I’ve updated forgotten_topics_data--added the topic
> ID
> > > and
> > > > > > > removed
> > > > > > > > >> the
> > > > > > > > >> > topic name
> > > > > > > > >> >
> > > > > > > > >> > 27. I’ve decided on plain text, and I’ve added an
> example.
> > > > > > > > >> >
> > > > > > > > >> > 28. I’ve added this idea to future work.
> > > > > > > > >> >
> > > > > > > > >> > Thanks again for taking a look,
> > > > > > > > >> >
> > > > > > > > >> > Justine
> > > > > > > > >> >
> > > > > > > > >> > On Wed, Sep 23, 2020 at 10:28 AM Jun Rao <
> > jun@confluent.io>
> > > > > > wrote:
> > > > > > > > >> >
> > > > > > > > >> > > Hi, Justine,
> > > > > > > > >> > >
> > > > > > > > >> > > Thanks for the response. Made another pass. A few more
> > > > > comments
> > > > > > > > below.
> > > > > > > > >> > >
> > > > > > > > >> > > 20. znode schema:
> > > > > > > > >> > > 20.1 It seems that {"name": "version", "type": "int",
> > > "id":
> > > > > > > "UUID",
> > > > > > > > >> > "doc":
> > > > > > > > >> > > "version id"} should be {"name": "version", "type":
> > > "int"},
> > > > > > > {"name":
> > > > > > > > >> > "id",
> > > > > > > > >> > > "type": "UUID", "doc": "version id"}.
> > > > > > > > >> > > 20.2 The znode format is JSON which doesn't have UUID
> > > type.
> > > > So
> > > > > > the
> > > > > > > > >> type
> > > > > > > > >> > > probably should be string?
> > > > > > > > >> > > 20.3 Also, the existing format used seems outdated. It
> > > > should
> > > > > > have
> > > > > > > > the
> > > > > > > > >> > > following format.
> > > > > > > > >> > >     Json.encodeAsBytes(Map(
> > > > > > > > >> > >       "version" -> 2,
> > > > > > > > >> > >       "partitions" -> replicaAssignmentJson.asJava,
> > > > > > > > >> > >       "adding_replicas" ->
> > > > > addingReplicasAssignmentJson.asJava,
> > > > > > > > >> > >       "removing_replicas" ->
> > > > > > removingReplicasAssignmentJson.asJava
> > > > > > > > >> > >     ).asJava)
> > > > > > > > >> > >   }
> > > > > > > > >> > >
> > > > > > > > >> > > 21. Migration: The KIP says "The migration process can
> > > take
> > > > > > place
> > > > > > > > >> without
> > > > > > > > >> > > an inter-broker protocol bump, as the format stored in
> > > > > > > > >> > > /brokers/topics/[topic] will be compatible with older
> > > broker
> > > > > > > > >> versions."
> > > > > > > > >> > > However, since we are bumping up the version of
> > > inter-broker
> > > > > > > > >> requests, it
> > > > > > > > >> > > seems that we need to use IBP for migration.
> > > > > > > > >> > >
> > > > > > > > >> > > 22. The KIP says "Remove deleted topics from replicas
> by
> > > > > sending
> > > > > > > > >> > > StopReplicaRequest V3 for any topics which do not
> > contain
> > > a
> > > > > > topic
> > > > > > > > ID,
> > > > > > > > >> and
> > > > > > > > >> > > V4 for any topics which do contain a topic ID.".
> > However,
> > > if
> > > > > we
> > > > > > > use
> > > > > > > > >> IBP,
> > > > > > > > >> > it
> > > > > > > > >> > > seems that the controller will either send
> > > > StopReplicaRequest
> > > > > V3
> > > > > > > > >> > > or StopReplicaRequest V4, but never mixed V3 and V4
> for
> > > > > > different
> > > > > > > > >> topics.
> > > > > > > > >> > > Basically, before the IBP bump, V3 will be used. After
> > the
> > > > IBP
> > > > > > > bump,
> > > > > > > > >> > > topicId will be created and V4 will be used.
> > > > > > > > >> > >
> > > > > > > > >> > > 23. Given that we depend on IBP, do we still need "0
> > > > > > UNSPECIFIED"
> > > > > > > > >> > > in LeaderAndIsr?
> > > > > > > > >> > >
> > > > > > > > >> > > 24. LeaderAndIsrResponse v5 : It still has the topic
> > > field.
> > > > > > > > >> > >
> > > > > > > > >> > > 25. LeaderAndIsrResponse v5, StopReplicaResponse v4:
> > Could
> > > > we
> > > > > > use
> > > > > > > > this
> > > > > > > > >> > > opportunity to organize the response in 2 levels,
> first
> > by
> > > > > > topic,
> > > > > > > > >> then by
> > > > > > > > >> > > partition, as most other requests/responses?
> > > > > > > > >> > >
> > > > > > > > >> > > 26. FetchRequest v13 : Should forgotten_topics_data
> use
> > > > > topicId
> > > > > > > too?
> > > > > > > > >> > >
> > > > > > > > >> > > 27. "This file can either be plain text (key/value
> > pairs)
> > > or
> > > > > > > JSON."
> > > > > > > > >> Have
> > > > > > > > >> > we
> > > > > > > > >> > > decided which one to use? Also, it would be helpful to
> > > > provide
> > > > > > an
> > > > > > > > >> > example.
> > > > > > > > >> > >
> > > > > > > > >> > > 28. Future improvement: Another future improvement
> > > > opportunity
> > > > > > is
> > > > > > > to
> > > > > > > > >> use
> > > > > > > > >> > > topicId in GroupMetadataManager.offsetCommitKey in the
> > > > > > > offset_commit
> > > > > > > > >> > topic.
> > > > > > > > >> > > This may save some space.
> > > > > > > > >> > >
> > > > > > > > >> > > Thanks,
> > > > > > > > >> > >
> > > > > > > > >> > > Jun
> > > > > > > > >> > >
> > > > > > > > >> > > On Wed, Sep 23, 2020 at 8:50 AM Justine Olshan <
> > > > > > > > jolshan@confluent.io>
> > > > > > > > >> > > wrote:
> > > > > > > > >> > >
> > > > > > > > >> > > > Hi Tom,
> > > > > > > > >> > > >
> > > > > > > > >> > > > Thanks for the comment. I think this is a really
> good
> > > idea
> > > > > and
> > > > > > > it
> > > > > > > > >> has
> > > > > > > > >> > > been
> > > > > > > > >> > > > added to the KIP under the newly added tooling
> > section.
> > > > > > > > >> > > >
> > > > > > > > >> > > > Thanks again,
> > > > > > > > >> > > > Justine
> > > > > > > > >> > > >
> > > > > > > > >> > > > On Wed, Sep 23, 2020 at 3:17 AM Tom Bentley <
> > > > > > > tbentley@redhat.com>
> > > > > > > > >> > wrote:
> > > > > > > > >> > > >
> > > > > > > > >> > > > > Hi Justine,
> > > > > > > > >> > > > >
> > > > > > > > >> > > > > I know you started the vote thread, but on
> > re-reading
> > > > the
> > > > > > KIP
> > > > > > > I
> > > > > > > > >> > noticed
> > > > > > > > >> > > > > that although the topic id is included in the
> > > > > > MetadataResponse
> > > > > > > > >> it's
> > > > > > > > >> > not
> > > > > > > > >> > > > > surfaced in the output from `kafka-topics.sh
> > > > --describe`.
> > > > > > > Maybe
> > > > > > > > >> that
> > > > > > > > >> > > was
> > > > > > > > >> > > > > intentional because ids are intentionally not
> really
> > > > > > something
> > > > > > > > the
> > > > > > > > >> > user
> > > > > > > > >> > > > > should care deeply about, but it would also make
> > life
> > > > > harder
> > > > > > > for
> > > > > > > > >> > anyone
> > > > > > > > >> > > > > debugging Kafka and this would likely get worse
> the
> > > more
> > > > > > topic
> > > > > > > > ids
> > > > > > > > >> > got
> > > > > > > > >> > > > > rolled out across the protocols, clients etc. It
> > seems
> > > > > > likely
> > > > > > > > that
> > > > > > > > >> > > > > `kafka-topics.sh` will eventually need the ability
> > to
> > > > show
> > > > > > the
> > > > > > > > id
> > > > > > > > >> of
> > > > > > > > >> > a
> > > > > > > > >> > > > > topic and perhaps find a topic name given an id.
> Is
> > > > there
> > > > > > any
> > > > > > > > >> reason
> > > > > > > > >> > > not
> > > > > > > > >> > > > to
> > > > > > > > >> > > > > implement that in this KIP?
> > > > > > > > >> > > > >
> > > > > > > > >> > > > > Many thanks,
> > > > > > > > >> > > > >
> > > > > > > > >> > > > > Tom
> > > > > > > > >> > > > >
> > > > > > > > >> > > > > On Mon, Sep 21, 2020 at 9:54 PM Justine Olshan <
> > > > > > > > >> jolshan@confluent.io
> > > > > > > > >> > >
> > > > > > > > >> > > > > wrote:
> > > > > > > > >> > > > >
> > > > > > > > >> > > > > > Hi all,
> > > > > > > > >> > > > > >
> > > > > > > > >> > > > > > After thinking about it, I've decided to remove
> > the
> > > > > topic
> > > > > > > name
> > > > > > > > >> from
> > > > > > > > >> > > the
> > > > > > > > >> > > > > > Fetch Request and Response after all. Since
> there
> > > are
> > > > so
> > > > > > > many
> > > > > > > > of
> > > > > > > > >> > > these
> > > > > > > > >> > > > > > requests per second, it is worth removing the
> > extra
> > > > > > > > information.
> > > > > > > > >> > I've
> > > > > > > > >> > > > > > updated the KIP to reflect this change.
> > > > > > > > >> > > > > >
> > > > > > > > >> > > > > > Please let me know if there is anything else we
> > > should
> > > > > > > discuss
> > > > > > > > >> > before
> > > > > > > > >> > > > > > voting.
> > > > > > > > >> > > > > >
> > > > > > > > >> > > > > > Thank you,
> > > > > > > > >> > > > > > Justine
> > > > > > > > >> > > > > >
> > > > > > > > >> > > > > > On Fri, Sep 18, 2020 at 9:46 AM Justine Olshan <
> > > > > > > > >> > jolshan@confluent.io
> > > > > > > > >> > > >
> > > > > > > > >> > > > > > wrote:
> > > > > > > > >> > > > > >
> > > > > > > > >> > > > > > > Hi Jun,
> > > > > > > > >> > > > > > >
> > > > > > > > >> > > > > > > I see what you are saying. For now we can
> remove
> > > the
> > > > > > extra
> > > > > > > > >> > > > information.
> > > > > > > > >> > > > > > > I'll leave the option to add more fields to
> the
> > > file
> > > > > in
> > > > > > > the
> > > > > > > > >> > future.
> > > > > > > > >> > > > The
> > > > > > > > >> > > > > > KIP
> > > > > > > > >> > > > > > > has been updated to reflect this change.
> > > > > > > > >> > > > > > >
> > > > > > > > >> > > > > > > Thanks,
> > > > > > > > >> > > > > > > Justine
> > > > > > > > >> > > > > > >
> > > > > > > > >> > > > > > > On Fri, Sep 18, 2020 at 8:46 AM Jun Rao <
> > > > > > jun@confluent.io
> > > > > > > >
> > > > > > > > >> > wrote:
> > > > > > > > >> > > > > > >
> > > > > > > > >> > > > > > >> Hi, Justine,
> > > > > > > > >> > > > > > >>
> > > > > > > > >> > > > > > >> Thanks for the reply.
> > > > > > > > >> > > > > > >>
> > > > > > > > >> > > > > > >> 13. If the log directory is the source of
> > truth,
> > > it
> > > > > > means
> > > > > > > > >> that
> > > > > > > > >> > the
> > > > > > > > >> > > > > > >> redundant info in the metadata file will be
> > > > ignored.
> > > > > > Then
> > > > > > > > the
> > > > > > > > >> > > > question
> > > > > > > > >> > > > > > is
> > > > > > > > >> > > > > > >> why do we need to put the redundant info in
> the
> > > > > > metadata
> > > > > > > > file
> > > > > > > > >> > now?
> > > > > > > > >> > > > > > >>
> > > > > > > > >> > > > > > >> Thanks,
> > > > > > > > >> > > > > > >>
> > > > > > > > >> > > > > > >> Jun
> > > > > > > > >> > > > > > >>
> > > > > > > > >> > > > > > >> On Thu, Sep 17, 2020 at 5:07 PM Justine
> Olshan
> > <
> > > > > > > > >> > > > jolshan@confluent.io>
> > > > > > > > >> > > > > > >> wrote:
> > > > > > > > >> > > > > > >>
> > > > > > > > >> > > > > > >> > Hi Jun,
> > > > > > > > >> > > > > > >> > Thanks for the quick response!
> > > > > > > > >> > > > > > >> >
> > > > > > > > >> > > > > > >> > 12. I've decided to bump up the versions on
> > the
> > > > > > > requests
> > > > > > > > >> and
> > > > > > > > >> > > > updated
> > > > > > > > >> > > > > > the
> > > > > > > > >> > > > > > >> > KIP. I think it's good we thoroughly
> > discussed
> > > > the
> > > > > > > > options
> > > > > > > > >> > here,
> > > > > > > > >> > > > so
> > > > > > > > >> > > > > we
> > > > > > > > >> > > > > > >> know
> > > > > > > > >> > > > > > >> > we made a good choice. :)
> > > > > > > > >> > > > > > >> >
> > > > > > > > >> > > > > > >> > 13. This is an interesting situation. I
> think
> > > if
> > > > > this
> > > > > > > > does
> > > > > > > > >> > occur
> > > > > > > > >> > > > we
> > > > > > > > >> > > > > > >> should
> > > > > > > > >> > > > > > >> > give a warning. I agree that it's hard to
> > know
> > > > the
> > > > > > > source
> > > > > > > > >> of
> > > > > > > > >> > > truth
> > > > > > > > >> > > > > for
> > > > > > > > >> > > > > > >> sure
> > > > > > > > >> > > > > > >> > since the directory or the file could be
> > > manually
> > > > > > > > >> modified. I
> > > > > > > > >> > > > guess
> > > > > > > > >> > > > > > the
> > > > > > > > >> > > > > > >> > directory could be used as the source of
> > truth.
> > > > To
> > > > > be
> > > > > > > > >> honest,
> > > > > > > > >> > > I'm
> > > > > > > > >> > > > > not
> > > > > > > > >> > > > > > >> > really sure what happens in kafka when the
> > log
> > > > > > > directory
> > > > > > > > is
> > > > > > > > >> > > > renamed
> > > > > > > > >> > > > > > >> > manually in such a way. I'm also wondering
> if
> > > the
> > > > > > > > >> situation is
> > > > > > > > >> > > > > > >> recoverable
> > > > > > > > >> > > > > > >> > in this scenario.
> > > > > > > > >> > > > > > >> >
> > > > > > > > >> > > > > > >> > Thanks,
> > > > > > > > >> > > > > > >> > Justine
> > > > > > > > >> > > > > > >> >
> > > > > > > > >> > > > > > >> > On Thu, Sep 17, 2020 at 4:28 PM Jun Rao <
> > > > > > > > jun@confluent.io>
> > > > > > > > >> > > wrote:
> > > > > > > > >> > > > > > >> >
> > > > > > > > >> > > > > > >> > > Hi, Justine,
> > > > > > > > >> > > > > > >> > >
> > > > > > > > >> > > > > > >> > > Thanks for the reply.
> > > > > > > > >> > > > > > >> > >
> > > > > > > > >> > > > > > >> > > 12. I don't have a strong preference
> > either.
> > > > > > However,
> > > > > > > > if
> > > > > > > > >> we
> > > > > > > > >> > > need
> > > > > > > > >> > > > > IBP
> > > > > > > > >> > > > > > >> > > anyway, maybe it's easier to just bump up
> > the
> > > > > > version
> > > > > > > > for
> > > > > > > > >> > all
> > > > > > > > >> > > > > inter
> > > > > > > > >> > > > > > >> > broker
> > > > > > > > >> > > > > > >> > > requests and add the topic id field as a
> > > > regular
> > > > > > > > field. A
> > > > > > > > >> > > > regular
> > > > > > > > >> > > > > > >> field
> > > > > > > > >> > > > > > >> > is
> > > > > > > > >> > > > > > >> > > a bit more concise in wire transfer than
> a
> > > > > flexible
> > > > > > > > >> field.
> > > > > > > > >> > > > > > >> > >
> > > > > > > > >> > > > > > >> > > 13. The confusion that I was referring to
> > is
> > > > > > between
> > > > > > > > the
> > > > > > > > >> > topic
> > > > > > > > >> > > > > name
> > > > > > > > >> > > > > > >> and
> > > > > > > > >> > > > > > >> > > partition number between the log dir and
> > the
> > > > > > metadata
> > > > > > > > >> file.
> > > > > > > > >> > > For
> > > > > > > > >> > > > > > >> example,
> > > > > > > > >> > > > > > >> > if
> > > > > > > > >> > > > > > >> > > the log dir is topicA-1 and the metadata
> > file
> > > > in
> > > > > it
> > > > > > > has
> > > > > > > > >> > topicB
> > > > > > > > >> > > > and
> > > > > > > > >> > > > > > >> > > partition 0 (say due to a bug or manual
> > > > > > > modification),
> > > > > > > > >> which
> > > > > > > > >> > > one
> > > > > > > > >> > > > > do
> > > > > > > > >> > > > > > we
> > > > > > > > >> > > > > > >> > use
> > > > > > > > >> > > > > > >> > > as the source of truth?
> > > > > > > > >> > > > > > >> > >
> > > > > > > > >> > > > > > >> > > Jun
> > > > > > > > >> > > > > > >> > >
> > > > > > > > >> > > > > > >> > > On Thu, Sep 17, 2020 at 3:43 PM Justine
> > > Olshan
> > > > <
> > > > > > > > >> > > > > > jolshan@confluent.io>
> > > > > > > > >> > > > > > >> > > wrote:
> > > > > > > > >> > > > > > >> > >
> > > > > > > > >> > > > > > >> > > > Hi Jun,
> > > > > > > > >> > > > > > >> > > > Thanks for the comments.
> > > > > > > > >> > > > > > >> > > >
> > > > > > > > >> > > > > > >> > > > 12. I bumped the LeaderAndIsrRequest
> > > because
> > > > I
> > > > > > > > removed
> > > > > > > > >> the
> > > > > > > > >> > > > topic
> > > > > > > > >> > > > > > >> name
> > > > > > > > >> > > > > > >> > > field
> > > > > > > > >> > > > > > >> > > > in the response. It may be possible to
> > > avoid
> > > > > > > bumping
> > > > > > > > >> the
> > > > > > > > >> > > > version
> > > > > > > > >> > > > > > >> > without
> > > > > > > > >> > > > > > >> > > > that change, but I may be missing
> > > something.
> > > > > > > > >> > > > > > >> > > > I believe StopReplica is actually on
> > > version
> > > > 3
> > > > > > now,
> > > > > > > > but
> > > > > > > > >> > > > because
> > > > > > > > >> > > > > > >> > version 2
> > > > > > > > >> > > > > > >> > > > is flexible, I kept that listed as
> > version
> > > 2
> > > > on
> > > > > > the
> > > > > > > > KIP
> > > > > > > > >> > > page.
> > > > > > > > >> > > > > > >> However,
> > > > > > > > >> > > > > > >> > > you
> > > > > > > > >> > > > > > >> > > > may be right in that we may need to
> bump
> > > the
> > > > > > > version
> > > > > > > > on
> > > > > > > > >> > > > > > StopReplica
> > > > > > > > >> > > > > > >> to
> > > > > > > > >> > > > > > >> > > deal
> > > > > > > > >> > > > > > >> > > > with deletion differently as mentioned
> > > > above. I
> > > > > > > don't
> > > > > > > > >> know
> > > > > > > > >> > > if
> > > > > > > > >> > > > I
> > > > > > > > >> > > > > > >> have a
> > > > > > > > >> > > > > > >> > > big
> > > > > > > > >> > > > > > >> > > > preference over used tagged fields or
> > not.
> > > > > > > > >> > > > > > >> > > >
> > > > > > > > >> > > > > > >> > > > 13. I was thinking that in the case
> where
> > > the
> > > > > > file
> > > > > > > > and
> > > > > > > > >> the
> > > > > > > > >> > > > > request
> > > > > > > > >> > > > > > >> > topic
> > > > > > > > >> > > > > > >> > > > ids don't match, it means that the
> > broker's
> > > > > > > topic/the
> > > > > > > > >> one
> > > > > > > > >> > in
> > > > > > > > >> > > > the
> > > > > > > > >> > > > > > >> file
> > > > > > > > >> > > > > > >> > has
> > > > > > > > >> > > > > > >> > > > been deleted. In that case, we would
> need
> > > to
> > > > > > delete
> > > > > > > > the
> > > > > > > > >> > old
> > > > > > > > >> > > > > topic
> > > > > > > > >> > > > > > >> and
> > > > > > > > >> > > > > > >> > > start
> > > > > > > > >> > > > > > >> > > > receiving the new version. If the topic
> > > name
> > > > > were
> > > > > > > to
> > > > > > > > >> > change,
> > > > > > > > >> > > > but
> > > > > > > > >> > > > > > the
> > > > > > > > >> > > > > > >> > ids
> > > > > > > > >> > > > > > >> > > > still matched, the file would also need
> > to
> > > > > > update.
> > > > > > > > Am I
> > > > > > > > >> > > > missing
> > > > > > > > >> > > > > a
> > > > > > > > >> > > > > > >> case
> > > > > > > > >> > > > > > >> > > > where the file would be correct and not
> > the
> > > > > > > request?
> > > > > > > > >> > > > > > >> > > >
> > > > > > > > >> > > > > > >> > > > Thanks,
> > > > > > > > >> > > > > > >> > > > Justine
> > > > > > > > >> > > > > > >> > > >
> > > > > > > > >> > > > > > >> > > > On Thu, Sep 17, 2020 at 3:18 PM Jun
> Rao <
> > > > > > > > >> jun@confluent.io
> > > > > > > > >> > >
> > > > > > > > >> > > > > wrote:
> > > > > > > > >> > > > > > >> > > >
> > > > > > > > >> > > > > > >> > > > > Hi, Justine,
> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > >> > > > > > >> > > > > Thanks for the reply. A couple of
> more
> > > > > comments
> > > > > > > > >> below.
> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > >> > > > > > >> > > > > 12. ListOffset and OffsetForLeader
> > > > currently
> > > > > > > don't
> > > > > > > > >> > support
> > > > > > > > >> > > > > > >> flexible
> > > > > > > > >> > > > > > >> > > > fields.
> > > > > > > > >> > > > > > >> > > > > So, we have to bump up the version
> > number
> > > > and
> > > > > > use
> > > > > > > > >> IBP at
> > > > > > > > >> > > > least
> > > > > > > > >> > > > > > for
> > > > > > > > >> > > > > > >> > > these
> > > > > > > > >> > > > > > >> > > > > two requests. Note that it seems
> 2.7.0
> > > will
> > > > > > > require
> > > > > > > > >> IBP
> > > > > > > > >> > > > anyway
> > > > > > > > >> > > > > > >> > because
> > > > > > > > >> > > > > > >> > > of
> > > > > > > > >> > > > > > >> > > > > changes in KAFKA-10435. Also, it
> seems
> > > that
> > > > > the
> > > > > > > > >> version
> > > > > > > > >> > > for
> > > > > > > > >> > > > > > >> > > > > LeaderAndIsrRequest and StopReplica
> are
> > > > > bumped
> > > > > > > even
> > > > > > > > >> > though
> > > > > > > > >> > > > we
> > > > > > > > >> > > > > > only
> > > > > > > > >> > > > > > >> > > added
> > > > > > > > >> > > > > > >> > > > a
> > > > > > > > >> > > > > > >> > > > > tagged field. But since IBP is needed
> > > > anyway,
> > > > > > we
> > > > > > > > may
> > > > > > > > >> > want
> > > > > > > > >> > > to
> > > > > > > > >> > > > > > >> revisit
> > > > > > > > >> > > > > > >> > > the
> > > > > > > > >> > > > > > >> > > > > overall tagged field choice.
> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > >> > > > > > >> > > > > 13. The only downside is the
> potential
> > > > > > confusion
> > > > > > > on
> > > > > > > > >> > which
> > > > > > > > >> > > > one
> > > > > > > > >> > > > > is
> > > > > > > > >> > > > > > >> the
> > > > > > > > >> > > > > > >> > > > source
> > > > > > > > >> > > > > > >> > > > > of truth if they don't match. Another
> > > > option
> > > > > is
> > > > > > > to
> > > > > > > > >> > include
> > > > > > > > >> > > > > those
> > > > > > > > >> > > > > > >> > fields
> > > > > > > > >> > > > > > >> > > > in
> > > > > > > > >> > > > > > >> > > > > the metadata file when we actually
> > change
> > > > the
> > > > > > > > >> directory
> > > > > > > > >> > > > > > structure.
> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > >> > > > > > >> > > > > Thanks,
> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > >> > > > > > >> > > > > Jun
> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > >> > > > > > >> > > > > On Thu, Sep 17, 2020 at 2:01 PM
> Justine
> > > > > Olshan
> > > > > > <
> > > > > > > > >> > > > > > >> jolshan@confluent.io
> > > > > > > > >> > > > > > >> > >
> > > > > > > > >> > > > > > >> > > > > wrote:
> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > >> > > > > > >> > > > > > Hello all,
> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > >> > > > > > >> > > > > > I've thought some more about
> removing
> > > the
> > > > > > topic
> > > > > > > > >> name
> > > > > > > > >> > > field
> > > > > > > > >> > > > > > from
> > > > > > > > >> > > > > > >> > some
> > > > > > > > >> > > > > > >> > > of
> > > > > > > > >> > > > > > >> > > > > the
> > > > > > > > >> > > > > > >> > > > > > requests. On closer inspection of
> the
> > > > > > > > >> > > requests/responses,
> > > > > > > > >> > > > it
> > > > > > > > >> > > > > > >> seems
> > > > > > > > >> > > > > > >> > > that
> > > > > > > > >> > > > > > >> > > > > the
> > > > > > > > >> > > > > > >> > > > > > internal changes would be much
> larger
> > > > than
> > > > > I
> > > > > > > > >> expected.
> > > > > > > > >> > > > Some
> > > > > > > > >> > > > > > >> > protocols
> > > > > > > > >> > > > > > >> > > > > > involve clients, so they would
> > require
> > > > > > changes
> > > > > > > > too.
> > > > > > > > >> > I'm
> > > > > > > > >> > > > > > thinking
> > > > > > > > >> > > > > > >> > that
> > > > > > > > >> > > > > > >> > > > for
> > > > > > > > >> > > > > > >> > > > > > now, removing the topic name from
> > these
> > > > > > > requests
> > > > > > > > >> and
> > > > > > > > >> > > > > responses
> > > > > > > > >> > > > > > >> are
> > > > > > > > >> > > > > > >> > > out
> > > > > > > > >> > > > > > >> > > > of
> > > > > > > > >> > > > > > >> > > > > > scope.
> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > >> > > > > > >> > > > > > I have decided to just keep the
> > change
> > > > > > > > >> > > > LeaderAndIsrResponse
> > > > > > > > >> > > > > to
> > > > > > > > >> > > > > > >> > remove
> > > > > > > > >> > > > > > >> > > > the
> > > > > > > > >> > > > > > >> > > > > > topic name, and have updated the
> KIP
> > to
> > > > > > reflect
> > > > > > > > >> this
> > > > > > > > >> > > > > change. I
> > > > > > > > >> > > > > > >> have
> > > > > > > > >> > > > > > >> > > > also
> > > > > > > > >> > > > > > >> > > > > > mentioned the other requests and
> > > > responses
> > > > > in
> > > > > > > > >> future
> > > > > > > > >> > > work.
> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > >> > > > > > >> > > > > > I'm hoping to start the voting
> > process
> > > > > soon,
> > > > > > so
> > > > > > > > >> let me
> > > > > > > > >> > > > know
> > > > > > > > >> > > > > if
> > > > > > > > >> > > > > > >> > there
> > > > > > > > >> > > > > > >> > > is
> > > > > > > > >> > > > > > >> > > > > > anything else we should discuss.
> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > >> > > > > > >> > > > > > Thank you,
> > > > > > > > >> > > > > > >> > > > > > Justine
> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > >> > > > > > >> > > > > > On Tue, Sep 15, 2020 at 3:57 PM
> > Justine
> > > > > > Olshan
> > > > > > > <
> > > > > > > > >> > > > > > >> > jolshan@confluent.io
> > > > > > > > >> > > > > > >> > > >
> > > > > > > > >> > > > > > >> > > > > > wrote:
> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > >> > > > > > >> > > > > > > Hello again,
> > > > > > > > >> > > > > > >> > > > > > > To follow up on some of the other
> > > > > comments:
> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > >> > > > > > >> > > > > > > 10/11) We can remove the topic
> name
> > > > from
> > > > > > > these
> > > > > > > > >> > > > > > >> > requests/responses,
> > > > > > > > >> > > > > > >> > > > and
> > > > > > > > >> > > > > > >> > > > > > > that means we will just have to
> > make
> > > a
> > > > > few
> > > > > > > > >> internal
> > > > > > > > >> > > > > changes
> > > > > > > > >> > > > > > to
> > > > > > > > >> > > > > > >> > make
> > > > > > > > >> > > > > > >> > > > > > > partitions accessible by topic id
> > and
> > > > > > > > partition.
> > > > > > > > >> I
> > > > > > > > >> > can
> > > > > > > > >> > > > > > update
> > > > > > > > >> > > > > > >> the
> > > > > > > > >> > > > > > >> > > KIP
> > > > > > > > >> > > > > > >> > > > > to
> > > > > > > > >> > > > > > >> > > > > > > remove them unless anyone thinks
> > they
> > > > > > should
> > > > > > > > >> stay.
> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > >> > > > > > >> > > > > > > 12) Addressed in the previous
> > email.
> > > > I've
> > > > > > > > updated
> > > > > > > > >> > the
> > > > > > > > >> > > > KIP
> > > > > > > > >> > > > > to
> > > > > > > > >> > > > > > >> > > include
> > > > > > > > >> > > > > > >> > > > > > > tagged fields for the requests
> and
> > > > > > responses.
> > > > > > > > >> (More
> > > > > > > > >> > on
> > > > > > > > >> > > > > that
> > > > > > > > >> > > > > > >> > below)
> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > >> > > > > > >> > > > > > > 13) I think part of the idea for
> > > > > including
> > > > > > > this
> > > > > > > > >> > > > > information
> > > > > > > > >> > > > > > >> is to
> > > > > > > > >> > > > > > >> > > > > prepare
> > > > > > > > >> > > > > > >> > > > > > > for future changes. Perhaps the
> > > > directory
> > > > > > > > >> structure
> > > > > > > > >> > > > might
> > > > > > > > >> > > > > > >> change
> > > > > > > > >> > > > > > >> > > from
> > > > > > > > >> > > > > > >> > > > > > > topicName_partitionNumber to
> > > something
> > > > > like
> > > > > > > > >> > > > > > >> > > topicID_partitionNumber.
> > > > > > > > >> > > > > > >> > > > > Then
> > > > > > > > >> > > > > > >> > > > > > > it would be useful to have the
> > topic
> > > > name
> > > > > > in
> > > > > > > > the
> > > > > > > > >> > file
> > > > > > > > >> > > > > since
> > > > > > > > >> > > > > > it
> > > > > > > > >> > > > > > >> > > would
> > > > > > > > >> > > > > > >> > > > > not
> > > > > > > > >> > > > > > >> > > > > > be
> > > > > > > > >> > > > > > >> > > > > > > in the directory structure.
> > > Supporting
> > > > > > topic
> > > > > > > > >> renames
> > > > > > > > >> > > > might
> > > > > > > > >> > > > > > be
> > > > > > > > >> > > > > > >> > > easier
> > > > > > > > >> > > > > > >> > > > if
> > > > > > > > >> > > > > > >> > > > > > the
> > > > > > > > >> > > > > > >> > > > > > > other fields are included. Would
> > > there
> > > > be
> > > > > > any
> > > > > > > > >> > > downsides
> > > > > > > > >> > > > to
> > > > > > > > >> > > > > > >> > > including
> > > > > > > > >> > > > > > >> > > > > this
> > > > > > > > >> > > > > > >> > > > > > > information?
> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > >> > > > > > >> > > > > > > 14)  Yes, we would need to copy
> the
> > > > > > partition
> > > > > > > > >> > metadata
> > > > > > > > >> > > > > file
> > > > > > > > >> > > > > > in
> > > > > > > > >> > > > > > >> > this
> > > > > > > > >> > > > > > >> > > > > > > process. I've updated the KIP to
> > > > include
> > > > > > > this.
> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > >> > > > > > >> > > > > > > 15) I believe Lucas meant v1 and
> v2
> > > > here.
> > > > > > He
> > > > > > > > was
> > > > > > > > >> > > > referring
> > > > > > > > >> > > > > > to
> > > > > > > > >> > > > > > >> how
> > > > > > > > >> > > > > > >> > > the
> > > > > > > > >> > > > > > >> > > > > > > requests would fall under
> different
> > > IBP
> > > > > and
> > > > > > > > meant
> > > > > > > > >> > that
> > > > > > > > >> > > > > older
> > > > > > > > >> > > > > > >> > > brokers
> > > > > > > > >> > > > > > >> > > > > > would
> > > > > > > > >> > > > > > >> > > > > > > have to use the older version of
> > the
> > > > > > request
> > > > > > > > and
> > > > > > > > >> the
> > > > > > > > >> > > > > > existing
> > > > > > > > >> > > > > > >> > topic
> > > > > > > > >> > > > > > >> > > > > > > deletion process. At first, it
> > seemed
> > > > > like
> > > > > > > > tagged
> > > > > > > > >> > > fields
> > > > > > > > >> > > > > > would
> > > > > > > > >> > > > > > >> > > > resolve
> > > > > > > > >> > > > > > >> > > > > > > the IBP issue. However, we may
> need
> > > IBP
> > > > > for
> > > > > > > > this
> > > > > > > > >> > > request
> > > > > > > > >> > > > > > after
> > > > > > > > >> > > > > > >> > all
> > > > > > > > >> > > > > > >> > > > > since
> > > > > > > > >> > > > > > >> > > > > > > the controller handles the topic
> > > > deletion
> > > > > > > > >> > differently
> > > > > > > > >> > > > > > >> depending
> > > > > > > > >> > > > > > >> > on
> > > > > > > > >> > > > > > >> > > > the
> > > > > > > > >> > > > > > >> > > > > > IBP
> > > > > > > > >> > > > > > >> > > > > > > version. In an older version, we
> > > can't
> > > > > just
> > > > > > > > send
> > > > > > > > >> a
> > > > > > > > >> > > > > > StopReplica
> > > > > > > > >> > > > > > >> > > delete
> > > > > > > > >> > > > > > >> > > > > the
> > > > > > > > >> > > > > > >> > > > > > > topic immediately like we'd want
> to
> > > for
> > > > > > this
> > > > > > > > KIP.
> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > >> > > > > > >> > > > > > > This makes me wonder if we want
> > > tagged
> > > > > > fields
> > > > > > > > on
> > > > > > > > >> all
> > > > > > > > >> > > the
> > > > > > > > >> > > > > > >> requests
> > > > > > > > >> > > > > > >> > > > after
> > > > > > > > >> > > > > > >> > > > > > > all. Let me know your thoughts!
> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > >> > > > > > >> > > > > > > Justine
> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > >> > > > > > >> > > > > > > On Tue, Sep 15, 2020 at 1:03 PM
> > > Justine
> > > > > > > Olshan
> > > > > > > > <
> > > > > > > > >> > > > > > >> > > jolshan@confluent.io
> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > >> > > > > > >> > > > > > > wrote:
> > > > > > > > >> > > > > > >> > > > > > >
> > > > > > > > >> > > > > > >> > > > > > >> Hi all,
> > > > > > > > >> > > > > > >> > > > > > >> Jun brought up a good point in
> his
> > > > last
> > > > > > > email
> > > > > > > > >> about
> > > > > > > > >> > > > > tagged
> > > > > > > > >> > > > > > >> > fields,
> > > > > > > > >> > > > > > >> > > > and
> > > > > > > > >> > > > > > >> > > > > > >> I've updated the KIP to reflect
> > that
> > > > the
> > > > > > > > >> changes to
> > > > > > > > >> > > > > > requests
> > > > > > > > >> > > > > > >> and
> > > > > > > > >> > > > > > >> > > > > > responses
> > > > > > > > >> > > > > > >> > > > > > >> will be in the form of tagged
> > fields
> > > > to
> > > > > > > avoid
> > > > > > > > >> > > changing
> > > > > > > > >> > > > > IBP.
> > > > > > > > >> > > > > > >> > > > > > >>
> > > > > > > > >> > > > > > >> > > > > > >> Jun: I plan on sending a
> followup
> > > > email
> > > > > to
> > > > > > > > >> address
> > > > > > > > >> > > some
> > > > > > > > >> > > > > of
> > > > > > > > >> > > > > > >> the
> > > > > > > > >> > > > > > >> > > other
> > > > > > > > >> > > > > > >> > > > > > >> points.
> > > > > > > > >> > > > > > >> > > > > > >>
> > > > > > > > >> > > > > > >> > > > > > >> Thanks,
> > > > > > > > >> > > > > > >> > > > > > >> Justine
> > > > > > > > >> > > > > > >> > > > > > >>
> > > > > > > > >> > > > > > >> > > > > > >> On Mon, Sep 14, 2020 at 4:25 PM
> > Jun
> > > > Rao
> > > > > <
> > > > > > > > >> > > > > jun@confluent.io>
> > > > > > > > >> > > > > > >> > wrote:
> > > > > > > > >> > > > > > >> > > > > > >>
> > > > > > > > >> > > > > > >> > > > > > >>> Hi, Justine,
> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > >> > > > > > >> > > > > > >>> Thanks for the updated KIP. A
> few
> > > > > > comments
> > > > > > > > >> below.
> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > >> > > > > > >> > > > > > >>> 10. LeaderAndIsr Response: Do
> we
> > > need
> > > > > the
> > > > > > > > topic
> > > > > > > > >> > > name?
> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > >> > > > > > >> > > > > > >>> 11. For the changed
> > > request/response,
> > > > > > other
> > > > > > > > >> than
> > > > > > > > >> > > > > > >> LeaderAndIsr,
> > > > > > > > >> > > > > > >> > > > > > >>> UpdateMetadata, Metadata, do we
> > > need
> > > > to
> > > > > > > > include
> > > > > > > > >> > the
> > > > > > > > >> > > > > topic
> > > > > > > > >> > > > > > >> name?
> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > >> > > > > > >> > > > > > >>> 12. It seems that upgrades
> don't
> > > > > require
> > > > > > > IBP.
> > > > > > > > >> Does
> > > > > > > > >> > > > that
> > > > > > > > >> > > > > > mean
> > > > > > > > >> > > > > > >> > the
> > > > > > > > >> > > > > > >> > > > new
> > > > > > > > >> > > > > > >> > > > > > >>> fields
> > > > > > > > >> > > > > > >> > > > > > >>> in all the request/response are
> > > added
> > > > > as
> > > > > > > > tagged
> > > > > > > > >> > > fields
> > > > > > > > >> > > > > > >> without
> > > > > > > > >> > > > > > >> > > > > bumping
> > > > > > > > >> > > > > > >> > > > > > up
> > > > > > > > >> > > > > > >> > > > > > >>> the request version? It would
> be
> > > > useful
> > > > > > to
> > > > > > > > make
> > > > > > > > >> > that
> > > > > > > > >> > > > > > clear.
> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > >> > > > > > >> > > > > > >>> 13. Partition Metadata file: Do
> > we
> > > > need
> > > > > > to
> > > > > > > > >> include
> > > > > > > > >> > > the
> > > > > > > > >> > > > > > topic
> > > > > > > > >> > > > > > >> > name
> > > > > > > > >> > > > > > >> > > > and
> > > > > > > > >> > > > > > >> > > > > > the
> > > > > > > > >> > > > > > >> > > > > > >>> partition id since they are
> > implied
> > > > in
> > > > > > the
> > > > > > > > >> > directory
> > > > > > > > >> > > > > name?
> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > >> > > > > > >> > > > > > >>> 14. In the JBOD mode, we
> support
> > > > > moving a
> > > > > > > > >> > > partition's
> > > > > > > > >> > > > > data
> > > > > > > > >> > > > > > >> from
> > > > > > > > >> > > > > > >> > > one
> > > > > > > > >> > > > > > >> > > > > > disk
> > > > > > > > >> > > > > > >> > > > > > >>> to
> > > > > > > > >> > > > > > >> > > > > > >>> another. Will the new partition
> > > > > metadata
> > > > > > > file
> > > > > > > > >> be
> > > > > > > > >> > > > copied
> > > > > > > > >> > > > > > >> during
> > > > > > > > >> > > > > > >> > > that
> > > > > > > > >> > > > > > >> > > > > > >>> process?
> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > >> > > > > > >> > > > > > >>> 15. The KIP says "Remove
> deleted
> > > > topics
> > > > > > > from
> > > > > > > > >> > > replicas
> > > > > > > > >> > > > by
> > > > > > > > >> > > > > > >> > sending
> > > > > > > > >> > > > > > >> > > > > > >>> StopReplicaRequest V2 for any
> > > topics
> > > > > > which
> > > > > > > do
> > > > > > > > >> not
> > > > > > > > >> > > > > contain
> > > > > > > > >> > > > > > a
> > > > > > > > >> > > > > > >> > topic
> > > > > > > > >> > > > > > >> > > > ID,
> > > > > > > > >> > > > > > >> > > > > > and
> > > > > > > > >> > > > > > >> > > > > > >>> V3 for any topics which do
> > contain
> > > a
> > > > > > topic
> > > > > > > > >> ID.".
> > > > > > > > >> > > > > However,
> > > > > > > > >> > > > > > it
> > > > > > > > >> > > > > > >> > > seems
> > > > > > > > >> > > > > > >> > > > > the
> > > > > > > > >> > > > > > >> > > > > > >>> updated controller will create
> > all
> > > > > > missing
> > > > > > > > >> topic
> > > > > > > > >> > IDs
> > > > > > > > >> > > > > first
> > > > > > > > >> > > > > > >> > before
> > > > > > > > >> > > > > > >> > > > > doing
> > > > > > > > >> > > > > > >> > > > > > >>> other actions. So, is
> > > > > StopReplicaRequest
> > > > > > V2
> > > > > > > > >> > needed?
> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > >> > > > > > >> > > > > > >>> Jun
> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > >> > > > > > >> > > > > > >>> On Fri, Sep 11, 2020 at 10:31
> AM
> > > John
> > > > > > > > Roesler <
> > > > > > > > >> > > > > > >> > > vvcephei@apache.org
> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > >> > > > > > >> > > > > > >>> wrote:
> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > >> > > > > > >> > > > > > >>> > Thanks, Justine!
> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > > >> > > > > > >> > > > > > >>> > Your response seems
> compelling
> > to
> > > > me.
> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > > >> > > > > > >> > > > > > >>> > -John
> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > > >> > > > > > >> > > > > > >>> > On Fri, 2020-09-11 at 10:17
> > > -0700,
> > > > > > > Justine
> > > > > > > > >> > Olshan
> > > > > > > > >> > > > > wrote:
> > > > > > > > >> > > > > > >> > > > > > >>> > > Hello all,
> > > > > > > > >> > > > > > >> > > > > > >>> > > Thanks for continuing the
> > > > > > discussion! I
> > > > > > > > >> have a
> > > > > > > > >> > > few
> > > > > > > > >> > > > > > >> > responses
> > > > > > > > >> > > > > > >> > > to
> > > > > > > > >> > > > > > >> > > > > > your
> > > > > > > > >> > > > > > >> > > > > > >>> > points.
> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > Tom: You are correct in
> that
> > > this
> > > > > KIP
> > > > > > > has
> > > > > > > > >> not
> > > > > > > > >> > > > > > mentioned
> > > > > > > > >> > > > > > >> the
> > > > > > > > >> > > > > > >> > > > > > >>> > > DeleteTopicsRequest. I
> think
> > > that
> > > > > > this
> > > > > > > > >> would
> > > > > > > > >> > be
> > > > > > > > >> > > > out
> > > > > > > > >> > > > > of
> > > > > > > > >> > > > > > >> > scope
> > > > > > > > >> > > > > > >> > > > for
> > > > > > > > >> > > > > > >> > > > > > >>> now, but
> > > > > > > > >> > > > > > >> > > > > > >>> > > may be something worth
> adding
> > > in
> > > > > the
> > > > > > > > >> future.
> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > John: We did consider
> > sequence
> > > > ids,
> > > > > > but
> > > > > > > > >> there
> > > > > > > > >> > > are
> > > > > > > > >> > > > a
> > > > > > > > >> > > > > > few
> > > > > > > > >> > > > > > >> > > reasons
> > > > > > > > >> > > > > > >> > > > > to
> > > > > > > > >> > > > > > >> > > > > > >>> favor
> > > > > > > > >> > > > > > >> > > > > > >>> > > UUIDs. There are several
> > cases
> > > > > where
> > > > > > > > topics
> > > > > > > > >> > from
> > > > > > > > >> > > > > > >> different
> > > > > > > > >> > > > > > >> > > > > clusters
> > > > > > > > >> > > > > > >> > > > > > >>> may
> > > > > > > > >> > > > > > >> > > > > > >>> > > interact now and in the
> > future.
> > > > For
> > > > > > > > >> example,
> > > > > > > > >> > > > Mirror
> > > > > > > > >> > > > > > >> Maker 2
> > > > > > > > >> > > > > > >> > > may
> > > > > > > > >> > > > > > >> > > > > > >>> benefit
> > > > > > > > >> > > > > > >> > > > > > >>> > > from being able to detect
> > when
> > > a
> > > > > > > cluster
> > > > > > > > >> being
> > > > > > > > >> > > > > > mirrored
> > > > > > > > >> > > > > > >> is
> > > > > > > > >> > > > > > >> > > > > deleted
> > > > > > > > >> > > > > > >> > > > > > >>> and
> > > > > > > > >> > > > > > >> > > > > > >>> > > recreated and globally
> unique
> > > > > > > identifiers
> > > > > > > > >> > would
> > > > > > > > >> > > > make
> > > > > > > > >> > > > > > >> > > resolving
> > > > > > > > >> > > > > > >> > > > > > issues
> > > > > > > > >> > > > > > >> > > > > > >>> > > easier than sequence IDs
> > which
> > > > may
> > > > > > > > collide
> > > > > > > > >> > > between
> > > > > > > > >> > > > > > >> > clusters.
> > > > > > > > >> > > > > > >> > > > > > KIP-405
> > > > > > > > >> > > > > > >> > > > > > >>> > > (tiered storage) will also
> > > > benefit
> > > > > > from
> > > > > > > > >> > globally
> > > > > > > > >> > > > > > unique
> > > > > > > > >> > > > > > >> IDs
> > > > > > > > >> > > > > > >> > > as
> > > > > > > > >> > > > > > >> > > > > > shared
> > > > > > > > >> > > > > > >> > > > > > >>> > > buckets may be used between
> > > > > clusters.
> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > Globally unique IDs would
> > also
> > > > make
> > > > > > > > >> > > functionality
> > > > > > > > >> > > > > like
> > > > > > > > >> > > > > > >> > moving
> > > > > > > > >> > > > > > >> > > > > > topics
> > > > > > > > >> > > > > > >> > > > > > >>> > > between disparate clusters
> > > easier
> > > > > in
> > > > > > > the
> > > > > > > > >> > future,
> > > > > > > > >> > > > > > >> simplify
> > > > > > > > >> > > > > > >> > any
> > > > > > > > >> > > > > > >> > > > > > future
> > > > > > > > >> > > > > > >> > > > > > >>> > > implementations of backups
> > and
> > > > > > > restores,
> > > > > > > > >> and
> > > > > > > > >> > > more.
> > > > > > > > >> > > > > In
> > > > > > > > >> > > > > > >> > > general,
> > > > > > > > >> > > > > > >> > > > > > >>> unique IDs
> > > > > > > > >> > > > > > >> > > > > > >>> > > would ensure that the
> source
> > > > > cluster
> > > > > > > > >> topics do
> > > > > > > > >> > > not
> > > > > > > > >> > > > > > >> conflict
> > > > > > > > >> > > > > > >> > > > with
> > > > > > > > >> > > > > > >> > > > > > the
> > > > > > > > >> > > > > > >> > > > > > >>> > > destination cluster topics.
> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > If we were to use sequence
> > ids,
> > > > we
> > > > > > > would
> > > > > > > > >> need
> > > > > > > > >> > > > > > >> sufficiently
> > > > > > > > >> > > > > > >> > > > large
> > > > > > > > >> > > > > > >> > > > > > >>> cluster
> > > > > > > > >> > > > > > >> > > > > > >>> > > ids to be stored with the
> > topic
> > > > > > > > >> identifiers or
> > > > > > > > >> > > we
> > > > > > > > >> > > > > run
> > > > > > > > >> > > > > > >> the
> > > > > > > > >> > > > > > >> > > risk
> > > > > > > > >> > > > > > >> > > > of
> > > > > > > > >> > > > > > >> > > > > > >>> > > collisions. This will give
> up
> > > any
> > > > > > > > >> advantage in
> > > > > > > > >> > > > > > >> compactness
> > > > > > > > >> > > > > > >> > > that
> > > > > > > > >> > > > > > >> > > > > > >>> sequence
> > > > > > > > >> > > > > > >> > > > > > >>> > > numbers may bring. Given
> > these
> > > > > > > > advantages I
> > > > > > > > >> > > think
> > > > > > > > >> > > > it
> > > > > > > > >> > > > > > >> makes
> > > > > > > > >> > > > > > >> > > > sense
> > > > > > > > >> > > > > > >> > > > > to
> > > > > > > > >> > > > > > >> > > > > > >>> use
> > > > > > > > >> > > > > > >> > > > > > >>> > > UUIDs.
> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > Gokul: This is an
> interesting
> > > > idea,
> > > > > > but
> > > > > > > > >> this
> > > > > > > > >> > is
> > > > > > > > >> > > a
> > > > > > > > >> > > > > > >> breaking
> > > > > > > > >> > > > > > >> > > > > change.
> > > > > > > > >> > > > > > >> > > > > > >>> Out of
> > > > > > > > >> > > > > > >> > > > > > >>> > > scope for now, but maybe
> > worth
> > > > > > > discussing
> > > > > > > > >> in
> > > > > > > > >> > the
> > > > > > > > >> > > > > > future.
> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > Hope this explains some of
> > the
> > > > > > > decisions,
> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > Justine
> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > On Fri, Sep 11, 2020 at
> 8:27
> > AM
> > > > > Gokul
> > > > > > > > >> Ramanan
> > > > > > > > >> > > > > > >> Subramanian <
> > > > > > > > >> > > > > > >> > > > > > >>> > > gokul2411s@gmail.com>
> wrote:
> > > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > > Hi.
> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > > Thanks for the KIP.
> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > > Have you thought about
> > > whether
> > > > it
> > > > > > > makes
> > > > > > > > >> > sense
> > > > > > > > >> > > to
> > > > > > > > >> > > > > > >> support
> > > > > > > > >> > > > > > >> > > > > > >>> authorizing a
> > > > > > > > >> > > > > > >> > > > > > >>> > > > principal for a topic ID
> > > rather
> > > > > > than
> > > > > > > a
> > > > > > > > >> topic
> > > > > > > > >> > > > name
> > > > > > > > >> > > > > to
> > > > > > > > >> > > > > > >> > > achieve
> > > > > > > > >> > > > > > >> > > > > > >>> tighter
> > > > > > > > >> > > > > > >> > > > > > >>> > > > security?
> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > > Or is the topic ID
> > > > fundamentally
> > > > > an
> > > > > > > > >> internal
> > > > > > > > >> > > > > detail
> > > > > > > > >> > > > > > >> > similar
> > > > > > > > >> > > > > > >> > > > to
> > > > > > > > >> > > > > > >> > > > > > >>> epochs
> > > > > > > > >> > > > > > >> > > > > > >>> > used
> > > > > > > > >> > > > > > >> > > > > > >>> > > > in a bunch of other
> places
> > in
> > > > > > Kafka?
> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > > Thanks.
> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > > On Fri, Sep 11, 2020 at
> > 4:06
> > > PM
> > > > > > John
> > > > > > > > >> > Roesler <
> > > > > > > > >> > > > > > >> > > > > > vvcephei@apache.org>
> > > > > > > > >> > > > > > >> > > > > > >>> > wrote:
> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Hello Justine,
> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Thanks for the KIP!
> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > I happen to have been
> > > > > confronted
> > > > > > > > >> recently
> > > > > > > > >> > > with
> > > > > > > > >> > > > > the
> > > > > > > > >> > > > > > >> need
> > > > > > > > >> > > > > > >> > > to
> > > > > > > > >> > > > > > >> > > > > keep
> > > > > > > > >> > > > > > >> > > > > > >>> > track of
> > > > > > > > >> > > > > > >> > > > > > >>> > > > a
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > large number of topics
> as
> > > > > > compactly
> > > > > > > > as
> > > > > > > > >> > > > > possible. I
> > > > > > > > >> > > > > > >> was
> > > > > > > > >> > > > > > >> > > > going
> > > > > > > > >> > > > > > >> > > > > to
> > > > > > > > >> > > > > > >> > > > > > >>> come
> > > > > > > > >> > > > > > >> > > > > > >>> > up
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > with some way to
> > dictionary
> > > > > > encode
> > > > > > > > the
> > > > > > > > >> > topic
> > > > > > > > >> > > > > names
> > > > > > > > >> > > > > > >> as
> > > > > > > > >> > > > > > >> > > > > integers,
> > > > > > > > >> > > > > > >> > > > > > >>> but
> > > > > > > > >> > > > > > >> > > > > > >>> > this
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > seems much better!
> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Apologies if this has
> > been
> > > > > raised
> > > > > > > > >> before,
> > > > > > > > >> > > but
> > > > > > > > >> > > > > I’m
> > > > > > > > >> > > > > > >> > > wondering
> > > > > > > > >> > > > > > >> > > > > > >>> about the
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > choice of UUID vs
> > sequence
> > > > > number
> > > > > > > for
> > > > > > > > >> the
> > > > > > > > >> > > ids.
> > > > > > > > >> > > > > > >> > Typically,
> > > > > > > > >> > > > > > >> > > > > I’ve
> > > > > > > > >> > > > > > >> > > > > > >>> seen
> > > > > > > > >> > > > > > >> > > > > > >>> > UUIDs
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > in two situations:
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 1. When processes need
> to
> > > > > > generate
> > > > > > > > >> > > > non-colliding
> > > > > > > > >> > > > > > >> > > > identifiers
> > > > > > > > >> > > > > > >> > > > > > >>> without
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > coordination.
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 2. When the identifier
> > > needs
> > > > to
> > > > > > be
> > > > > > > > >> > > > “universally
> > > > > > > > >> > > > > > >> > unique”;
> > > > > > > > >> > > > > > >> > > > > I.e.,
> > > > > > > > >> > > > > > >> > > > > > >>> the
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > identifier needs to
> > > > distinguish
> > > > > > the
> > > > > > > > >> entity
> > > > > > > > >> > > > from
> > > > > > > > >> > > > > > all
> > > > > > > > >> > > > > > >> > other
> > > > > > > > >> > > > > > >> > > > > > >>> entities
> > > > > > > > >> > > > > > >> > > > > > >>> > that
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > could ever exist. This
> is
> > > > > useful
> > > > > > in
> > > > > > > > >> cases
> > > > > > > > >> > > > where
> > > > > > > > >> > > > > > >> > entities
> > > > > > > > >> > > > > > >> > > > from
> > > > > > > > >> > > > > > >> > > > > > all
> > > > > > > > >> > > > > > >> > > > > > >>> > kinds
> > > > > > > > >> > > > > > >> > > > > > >>> > > > of
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > systems get mixed
> > together,
> > > > > such
> > > > > > as
> > > > > > > > >> when
> > > > > > > > >> > > > dumping
> > > > > > > > >> > > > > > >> logs
> > > > > > > > >> > > > > > >> > > from
> > > > > > > > >> > > > > > >> > > > > all
> > > > > > > > >> > > > > > >> > > > > > >>> > processes
> > > > > > > > >> > > > > > >> > > > > > >>> > > > in
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a company into a common
> > > > system.
> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Maybe I’m being
> > > > short-sighted,
> > > > > > but
> > > > > > > it
> > > > > > > > >> > > doesn’t
> > > > > > > > >> > > > > seem
> > > > > > > > >> > > > > > >> like
> > > > > > > > >> > > > > > >> > > > > either
> > > > > > > > >> > > > > > >> > > > > > >>> really
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > applies here. It seems
> > like
> > > > the
> > > > > > > > brokers
> > > > > > > > >> > > could
> > > > > > > > >> > > > > and
> > > > > > > > >> > > > > > >> would
> > > > > > > > >> > > > > > >> > > > > achieve
> > > > > > > > >> > > > > > >> > > > > > >>> > consensus
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > when creating a topic
> > > anyway,
> > > > > > which
> > > > > > > > is
> > > > > > > > >> all
> > > > > > > > >> > > > > that’s
> > > > > > > > >> > > > > > >> > > required
> > > > > > > > >> > > > > > >> > > > to
> > > > > > > > >> > > > > > >> > > > > > >>> > generate
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > non-colliding sequence
> > ids.
> > > > For
> > > > > > the
> > > > > > > > >> > second,
> > > > > > > > >> > > as
> > > > > > > > >> > > > > you
> > > > > > > > >> > > > > > >> > > mention,
> > > > > > > > >> > > > > > >> > > > > we
> > > > > > > > >> > > > > > >> > > > > > >>> could
> > > > > > > > >> > > > > > >> > > > > > >>> > > > assign
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a UUID to the cluster
> as
> > a
> > > > > whole,
> > > > > > > > which
> > > > > > > > >> > > would
> > > > > > > > >> > > > > > render
> > > > > > > > >> > > > > > >> > any
> > > > > > > > >> > > > > > >> > > > > > resource
> > > > > > > > >> > > > > > >> > > > > > >>> > scoped
> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > the broker universally
> > > unique
> > > > > as
> > > > > > > > well.
> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > The reason I mention
> this
> > > is
> > > > > > that,
> > > > > > > > >> > although
> > > > > > > > >> > > a
> > > > > > > > >> > > > > UUID
> > > > > > > > >> > > > > > >> is
> > > > > > > > >> > > > > > >> > way
> > > > > > > > >> > > > > > >> > > > > more
> > > > > > > > >> > > > > > >> > > > > > >>> > compact
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > than topic names, it’s
> > > still
> > > > 16
> > > > > > > > bytes.
> > > > > > > > >> In
> > > > > > > > >> > > > > > contrast,
> > > > > > > > >> > > > > > >> a
> > > > > > > > >> > > > > > >> > > > 4-byte
> > > > > > > > >> > > > > > >> > > > > > >>> integer
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > sequence id would give
> > us 4
> > > > > > billion
> > > > > > > > >> unique
> > > > > > > > >> > > > > topics
> > > > > > > > >> > > > > > >> per
> > > > > > > > >> > > > > > >> > > > > cluster,
> > > > > > > > >> > > > > > >> > > > > > >>> which
> > > > > > > > >> > > > > > >> > > > > > >>> > > > seems
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > like enough ;)
> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Considering the number
> of
> > > > > > different
> > > > > > > > >> times
> > > > > > > > >> > > > these
> > > > > > > > >> > > > > > >> topic
> > > > > > > > >> > > > > > >> > > > > > >>> identifiers are
> > > > > > > > >> > > > > > >> > > > > > >>> > > > sent
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > over the wire or stored
> > in
> > > > > > memory,
> > > > > > > it
> > > > > > > > >> > seems
> > > > > > > > >> > > > like
> > > > > > > > >> > > > > > it
> > > > > > > > >> > > > > > >> > might
> > > > > > > > >> > > > > > >> > > > be
> > > > > > > > >> > > > > > >> > > > > > >>> worth
> > > > > > > > >> > > > > > >> > > > > > >>> > the
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > additional 4x space
> > > savings.
> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > What do you think about
> > > this?
> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Thanks,
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > John
> > > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > On Fri, Sep 11, 2020,
> at
> > > > 03:20,
> > > > > > Tom
> > > > > > > > >> > Bentley
> > > > > > > > >> > > > > wrote:
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Hi Justine,
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > This looks like a
> very
> > > > > welcome
> > > > > > > > >> > > improvement.
> > > > > > > > >> > > > > > >> Thanks!
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Maybe I missed it,
> but
> > > the
> > > > > KIP
> > > > > > > > >> doesn't
> > > > > > > > >> > > seem
> > > > > > > > >> > > > to
> > > > > > > > >> > > > > > >> > mention
> > > > > > > > >> > > > > > >> > > > > > changing
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > DeleteTopicsRequest
> to
> > > > > identify
> > > > > > > the
> > > > > > > > >> > topic
> > > > > > > > >> > > > > using
> > > > > > > > >> > > > > > an
> > > > > > > > >> > > > > > >> > id.
> > > > > > > > >> > > > > > >> > > > > Maybe
> > > > > > > > >> > > > > > >> > > > > > >>> > that's out
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > of
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > scope, but
> > > > > DeleteTopicsRequest
> > > > > > is
> > > > > > > > not
> > > > > > > > >> > > listed
> > > > > > > > >> > > > > > among
> > > > > > > > >> > > > > > >> > the
> > > > > > > > >> > > > > > >> > > > > Future
> > > > > > > > >> > > > > > >> > > > > > >>> Work
> > > > > > > > >> > > > > > >> > > > > > >>> > APIs
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > either.
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Kind regards,
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Tom
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > On Thu, Sep 10, 2020
> at
> > > > 3:59
> > > > > PM
> > > > > > > > >> Satish
> > > > > > > > >> > > > > Duggana <
> > > > > > > > >> > > > > > >> > > > > > >>> > > > satish.duggana@gmail.com
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > wrote:
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Thanks
> Lucas/Justine
> > > for
> > > > > the
> > > > > > > nice
> > > > > > > > >> KIP.
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > It has several
> > benefits
> > > > > which
> > > > > > > > also
> > > > > > > > >> > > include
> > > > > > > > >> > > > > > >> > > simplifying
> > > > > > > > >> > > > > > >> > > > > the
> > > > > > > > >> > > > > > >> > > > > > >>> topic
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > deletion process by
> > > > > > controller
> > > > > > > > and
> > > > > > > > >> > logs
> > > > > > > > >> > > > > > cleanup
> > > > > > > > >> > > > > > >> by
> > > > > > > > >> > > > > > >> > > > > brokers
> > > > > > > > >> > > > > > >> > > > > > in
> > > > > > > > >> > > > > > >> > > > > > >>> > corner
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > cases.
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Best,
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Satish.
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > On Wed, Sep 9, 2020
> > at
> > > > > 10:07
> > > > > > PM
> > > > > > > > >> > Justine
> > > > > > > > >> > > > > > Olshan <
> > > > > > > > >> > > > > > >> > > > > > >>> > jolshan@confluent.io
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > wrote:
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Hello all, it's
> > been
> > > > > > almost a
> > > > > > > > >> year!
> > > > > > > > >> > > I've
> > > > > > > > >> > > > > > made
> > > > > > > > >> > > > > > >> > some
> > > > > > > > >> > > > > > >> > > > > > changes
> > > > > > > > >> > > > > > >> > > > > > >>> to
> > > > > > > > >> > > > > > >> > > > > > >>> > this
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > KIP
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > and hope to
> continue
> > > the
> > > > > > > > >> discussion.
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > One of the main
> > > changes
> > > > > > I've
> > > > > > > > >> added
> > > > > > > > >> > is
> > > > > > > > >> > > > now
> > > > > > > > >> > > > > > the
> > > > > > > > >> > > > > > >> > > > metadata
> > > > > > > > >> > > > > > >> > > > > > >>> response
> > > > > > > > >> > > > > > >> > > > > > >>> > > > will
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > include the topic
> ID
> > > (as
> > > > > > Colin
> > > > > > > > >> > > suggested).
> > > > > > > > >> > > > > > >> Clients
> > > > > > > > >> > > > > > >> > > can
> > > > > > > > >> > > > > > >> > > > > > >>> obtain the
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > topicID
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > of a given topic
> > > through
> > > > a
> > > > > > > > >> > > > TopicDescription.
> > > > > > > > >> > > > > > The
> > > > > > > > >> > > > > > >> > > > topicId
> > > > > > > > >> > > > > > >> > > > > > will
> > > > > > > > >> > > > > > >> > > > > > >>> > also be
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > included with the
> > > > > > > UpdateMetadata
> > > > > > > > >> > > request.
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Let me know what
> > you
> > > > all
> > > > > > > think.
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Thank you,
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Justine
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > On 2019/09/13
> > > 16:38:26,
> > > > > > > "Colin
> > > > > > > > >> > > McCabe" <
> > > > > > > > >> > > > > > >> > > > > > cmccabe@apache.org
> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > > >> > > > > > >> > > > > > >>> > wrote:
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Hi Lucas,
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Thanks for
> > tackling
> > > > > this.
> > > > > > > > >> Topic
> > > > > > > > >> > IDs
> > > > > > > > >> > > > > are a
> > > > > > > > >> > > > > > >> > great
> > > > > > > > >> > > > > > >> > > > > idea,
> > > > > > > > >> > > > > > >> > > > > > >>> and
> > > > > > > > >> > > > > > >> > > > > > >>> > this
> > > > > > > > >> > > > > > >> > > > > > >>> > > > is
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > really good
> writeup.
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > For
> > > > > > > /brokers/topics/[topic],
> > > > > > > > >> the
> > > > > > > > >> > > > schema
> > > > > > > > >> > > > > > >> version
> > > > > > > > >> > > > > > >> > > > > should
> > > > > > > > >> > > > > > >> > > > > > be
> > > > > > > > >> > > > > > >> > > > > > >>> > bumped
> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > version 3, rather
> > than
> > > 2.
> > > > > > > > KIP-455
> > > > > > > > >> > > bumped
> > > > > > > > >> > > > > the
> > > > > > > > >> > > > > > >> > version
> > > > > > > > >> > > > > > >> > > > of
> > > > > > > > >> > > > > > >> > > > > > this
> > > > > > > > >> > > > > > >> > > > > > >>> > znode
> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 2
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > already :)
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Given that
> we're
> > > > going
> > > > > to
> > > > > > > be
> > > > > > > > >> > seeing
> > > > > > > > >> > > > > these
> > > > > > > > >> > > > > > >> > things
> > > > > > > > >> > > > > > >> > > as
> > > > > > > > >> > > > > > >> > > > > > >>> strings
> > > > > > > > >> > > > > > >> > > > > > >>> > as
> > > > > > > > >> > > > > > >> > > > > > >>> > > > lot
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > (in
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > logs, in ZooKeeper,
> > on
> > > > the
> > > > > > > > >> > command-line,
> > > > > > > > >> > > > > > etc.),
> > > > > > > > >> > > > > > >> > does
> > > > > > > > >> > > > > > >> > > it
> > > > > > > > >> > > > > > >> > > > > > make
> > > > > > > > >> > > > > > >> > > > > > >>> > sense to
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > use
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > base64 when
> > converting
> > > > them
> > > > > > to
> > > > > > > > >> > strings?
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Here is an
> > example
> > > of
> > > > > the
> > > > > > > hex
> > > > > > > > >> > > > > > >> representation:
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > > > > > > >> > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > And here is an
> > > > example
> > > > > in
> > > > > > > > >> base64.
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > > b8tRS7h4TJ2Vt43Dp85v2A
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > The base64
> > version
> > > > > saves
> > > > > > 15
> > > > > > > > >> > letters
> > > > > > > > >> > > > (to
> > > > > > > > >> > > > > be
> > > > > > > > >> > > > > > >> > fair,
> > > > > > > > >> > > > > > >> > > 4
> > > > > > > > >> > > > > > >> > > > of
> > > > > > > > >> > > > > > >> > > > > > >>> those
> > > > > > > > >> > > > > > >> > > > > > >>> > were
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > dashes that we
> could
> > > have
> > > > > > > elided
> > > > > > > > in
> > > > > > > > >> > the
> > > > > > > > >> > > > hex
> > > > > > > > >> > > > > > >> > > > > > representation.)
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Another thing
> to
> > > > > consider
> > > > > > > is
> > > > > > > > >> that
> > > > > > > > >> > we
> > > > > > > > >> > > > > > should
> > > > > > > > >> > > > > > >> > > specify
> > > > > > > > >> > > > > > >> > > > > > that
> > > > > > > > >> > > > > > >> > > > > > >>> the
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > all-zeroes UUID is
> > not
> > > a
> > > > > > valid
> > > > > > > > >> topic
> > > > > > > > >> > > UUID.
> > > > > > > > >> > > > > >  We
> > > > > > > > >> > > > > > >> > can't
> > > > > > > > >> > > > > > >> > > > use
> > > > > > > > >> > > > > > >> > > > > > >>> null
> > > > > > > > >> > > > > > >> > > > > > >>> > for
> > > > > > > > >> > > > > > >> > > > > > >>> > > > this
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > because we can't
> > pass a
> > > > > null
> > > > > > > UUID
> > > > > > > > >> over
> > > > > > > > >> > > the
> > > > > > > > >> > > > > RPC
> > > > > > > > >> > > > > > >> > > protocol
> > > > > > > > >> > > > > > >> > > > > > >>> (there
> > > > > > > > >> > > > > > >> > > > > > >>> > is no
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > special pattern for
> > > null,
> > > > > nor
> > > > > > > do
> > > > > > > > we
> > > > > > > > >> > want
> > > > > > > > >> > > > to
> > > > > > > > >> > > > > > >> waste
> > > > > > > > >> > > > > > >> > > space
> > > > > > > > >> > > > > > >> > > > > > >>> reserving
> > > > > > > > >> > > > > > >> > > > > > >>> > > > such
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > pattern.)
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Maybe I missed
> > it,
> > > > but
> > > > > > did
> > > > > > > > you
> > > > > > > > >> > > > describe
> > > > > > > > >> > > > > > >> > > "migration
> > > > > > > > >> > > > > > >> > > > > > of...
> > > > > > > > >> > > > > > >> > > > > > >>> > existing
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > topic[s] without
> > topic
> > > > IDs"
> > > > > > in
> > > > > > > > >> detail
> > > > > > > > >> > in
> > > > > > > > >> > > > any
> > > > > > > > >> > > > > > >> > section?
> > > > > > > > >> > > > > > >> > > > It
> > > > > > > > >> > > > > > >> > > > > > >>> seems
> > > > > > > > >> > > > > > >> > > > > > >>> > like
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > when
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > the new controller
> > > > becomes
> > > > > > > > active,
> > > > > > > > >> it
> > > > > > > > >> > > > should
> > > > > > > > >> > > > > > >> just
> > > > > > > > >> > > > > > >> > > > > generate
> > > > > > > > >> > > > > > >> > > > > > >>> random
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > UUIDs for
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > these, and write
> the
> > > > random
> > > > > > > UUIDs
> > > > > > > > >> back
> > > > > > > > >> > > to
> > > > > > > > >> > > > > > >> > ZooKeeper.
> > > > > > > > >> > > > > > >> > > > It
> > > > > > > > >> > > > > > >> > > > > > >>> would be
> > > > > > > > >> > > > > > >> > > > > > >>> > > > good
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > to
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > spell that out.  We
> > > > should
> > > > > > make
> > > > > > > > it
> > > > > > > > >> > clear
> > > > > > > > >> > > > > that
> > > > > > > > >> > > > > > >> this
> > > > > > > > >> > > > > > >> > > > > happens
> > > > > > > > >> > > > > > >> > > > > > >>> > regardless
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > of
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > the inter-broker
> > > protocol
> > > > > > > version
> > > > > > > > >> > (it's
> > > > > > > > >> > > a
> > > > > > > > >> > > > > > >> > compatible
> > > > > > > > >> > > > > > >> > > > > > change).
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > "LeaderAndIsrRequests
> > > > > > > > >> including an
> > > > > > > > >> > > > > > >> > > > is_every_partition
> > > > > > > > >> > > > > > >> > > > > > >>> flag"
> > > > > > > > >> > > > > > >> > > > > > >>> > > > seems a
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > bit wordy.  Can we
> > just
> > > > > call
> > > > > > > > these
> > > > > > > > >> > "full
> > > > > > > > >> > > > > > >> > > > > > >>> LeaderAndIsrRequests"?
> > > > > > > > >> > > > > > >> > > > > > >>> > Then
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > the
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > RPC field could be
> > > named
> > > > > > > "full".
> > > > > > > > >> > Also,
> > > > > > > > >> > > it
> > > > > > > > >> > > > > > would
> > > > > > > > >> > > > > > >> > > > probably
> > > > > > > > >> > > > > > >> > > > > > be
> > > > > > > > >> > > > > > >> > > > > > >>> > better
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > for the
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > RPC field to be an
> > enum
> > > > of
> > > > > {
> > > > > > > > >> > > UNSPECIFIED,
> > > > > > > > >> > > > > > >> > > INCREMENTAL,
> > > > > > > > >> > > > > > >> > > > > FULL
> > > > > > > > >> > > > > > >> > > > > > >>> }, so
> > > > > > > > >> > > > > > >> > > > > > >>> > > > that
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > we
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > can cleanly handle
> > old
> > > > > > versions
> > > > > > > > (by
> > > > > > > > >> > > > treating
> > > > > > > > >> > > > > > >> them
> > > > > > > > >> > > > > > >> > as
> > > > > > > > >> > > > > > >> > > > > > >>> UNSPECIFIED)
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > In the
> > > > > > LeaderAndIsrRequest
> > > > > > > > >> > section,
> > > > > > > > >> > > > you
> > > > > > > > >> > > > > > >> write
> > > > > > > > >> > > > > > >> > "A
> > > > > > > > >> > > > > > >> > > > > final
> > > > > > > > >> > > > > > >> > > > > > >>> > deletion
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > event
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > will be secheduled
> > for
> > > X
> > > > ms
> > > > > > > after
> > > > > > > > >> the
> > > > > > > > >> > > > > > >> > > > LeaderAndIsrRequest
> > > > > > > > >> > > > > > >> > > > > > was
> > > > > > > > >> > > > > > >> > > > > > >>> > first
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > received..."  I
> guess
> > > > the X
> > > > > > > was a
> > > > > > > > >> > > > > placeholder
> > > > > > > > >> > > > > > >> that
> > > > > > > > >> > > > > > >> > > you
> > > > > > > > >> > > > > > >> > > > > > >>> intended
> > > > > > > > >> > > > > > >> > > > > > >>> > to
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > replace
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > before posting? :)
> > In
> > > > any
> > > > > > > case,
> > > > > > > > >> this
> > > > > > > > >> > > > seems
> > > > > > > > >> > > > > > like
> > > > > > > > >> > > > > > >> > the
> > > > > > > > >> > > > > > >> > > > kind
> > > > > > > > >> > > > > > >> > > > > > of
> > > > > > > > >> > > > > > >> > > > > > >>> > thing
> > > > > > > > >> > > > > > >> > > > > > >>> > > > we'd
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > want a
> configuration
> > > for.
> > > > > > > Let's
> > > > > > > > >> > > describe
> > > > > > > > >> > > > > that
> > > > > > > > >> > > > > > >> > > > > > configuration
> > > > > > > > >> > > > > > >> > > > > > >>> key
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > somewhere
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > in this KIP,
> > including
> > > > what
> > > > > > its
> > > > > > > > >> > default
> > > > > > > > >> > > > > value
> > > > > > > > >> > > > > > >> is.
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > We should
> > probably
> > > > also
> > > > > > > log a
> > > > > > > > >> > bunch
> > > > > > > > >> > > of
> > > > > > > > >> > > > > > >> messages
> > > > > > > > >> > > > > > >> > > at
> > > > > > > > >> > > > > > >> > > > > WARN
> > > > > > > > >> > > > > > >> > > > > > >>> level
> > > > > > > > >> > > > > > >> > > > > > >>> > > > when
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > something is
> > scheduled
> > > > for
> > > > > > > > >> deletion,
> > > > > > > > >> > as
> > > > > > > > >> > > > > well.
> > > > > > > > >> > > > > > >> > (Maybe
> > > > > > > > >> > > > > > >> > > > > this
> > > > > > > > >> > > > > > >> > > > > > >>> was
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > assumed, but
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > it would be good to
> > > > mention
> > > > > > > it).
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > I feel like
> there
> > > > are a
> > > > > > few
> > > > > > > > >> > sections
> > > > > > > > >> > > > > that
> > > > > > > > >> > > > > > >> > should
> > > > > > > > >> > > > > > >> > > be
> > > > > > > > >> > > > > > >> > > > > > >>> moved to
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > "rejected
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > alternatives."  For
> > > > > example,
> > > > > > in
> > > > > > > > the
> > > > > > > > >> > > > > > DeleteTopics
> > > > > > > > >> > > > > > >> > > > section,
> > > > > > > > >> > > > > > >> > > > > > >>> since
> > > > > > > > >> > > > > > >> > > > > > >>> > we're
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > not
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > going to do option
> 1
> > or
> > > > 2,
> > > > > > > these
> > > > > > > > >> > should
> > > > > > > > >> > > be
> > > > > > > > >> > > > > > moved
> > > > > > > > >> > > > > > >> > into
> > > > > > > > >> > > > > > >> > > > > > >>> "rejected
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > alternatives,"
> > rather
> > > > than
> > > > > > > > >> appearing
> > > > > > > > >> > > > > inline.
> > > > > > > > >> > > > > > >> > > Another
> > > > > > > > >> > > > > > >> > > > > case
> > > > > > > > >> > > > > > >> > > > > > >>> is
> > > > > > > > >> > > > > > >> > > > > > >>> > the
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > "Should
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > we remove topic
> name
> > > from
> > > > > the
> > > > > > > > >> protocol
> > > > > > > > >> > > > where
> > > > > > > > >> > > > > > >> > > possible"
> > > > > > > > >> > > > > > >> > > > > > >>> section.
> > > > > > > > >> > > > > > >> > > > > > >>> > This
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > is
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > clearly discussing
> a
> > > > design
> > > > > > > > >> > alternative
> > > > > > > > >> > > > that
> > > > > > > > >> > > > > > >> we're
> > > > > > > > >> > > > > > >> > > not
> > > > > > > > >> > > > > > >> > > > > > >>> proposing
> > > > > > > > >> > > > > > >> > > > > > >>> > to
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > implement: removing
> > the
> > > > > topic
> > > > > > > > name
> > > > > > > > >> > from
> > > > > > > > >> > > > > those
> > > > > > > > >> > > > > > >> > > > protocols.
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Is it really
> > > > necessary
> > > > > to
> > > > > > > > have
> > > > > > > > >> a
> > > > > > > > >> > new
> > > > > > > > >> > > > > > >> > > > > > >>> > /admin/delete_topics_by_id
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > path
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > in ZooKeeper?  It
> > seems
> > > > > like
> > > > > > we
> > > > > > > > >> don't
> > > > > > > > >> > > > really
> > > > > > > > >> > > > > > >> need
> > > > > > > > >> > > > > > >> > > this.
> > > > > > > > >> > > > > > >> > > > > > >>> Whenever
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > there is
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > a new controller,
> > we'll
> > > > > send
> > > > > > > out
> > > > > > > > >> full
> > > > > > > > >> > > > > > >> > > > > LeaderAndIsrRequests
> > > > > > > > >> > > > > > >> > > > > > >>> which
> > > > > > > > >> > > > > > >> > > > > > >>> > will
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > trigger the stale
> > > topics
> > > > to
> > > > > > be
> > > > > > > > >> cleaned
> > > > > > > > >> > > up.
> > > > > > > > >> > > > > >  The
> > > > > > > > >> > > > > > >> > > active
> > > > > > > > >> > > > > > >> > > > > > >>> > controller
> > > > > > > > >> > > > > > >> > > > > > >>> > > > will
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > also send the full
> > > > > > > > >> LeaderAndIsrRequest
> > > > > > > > >> > > to
> > > > > > > > >> > > > > > >> brokers
> > > > > > > > >> > > > > > >> > > that
> > > > > > > > >> > > > > > >> > > > > are
> > > > > > > > >> > > > > > >> > > > > > >>> just
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > starting
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > up.    So we don't
> > > really
> > > > > > need
> > > > > > > > this
> > > > > > > > >> > kind
> > > > > > > > >> > > > of
> > > > > > > > >> > > > > > >> > two-phase
> > > > > > > > >> > > > > > >> > > > > > commit
> > > > > > > > >> > > > > > >> > > > > > >>> > (send
> > > > > > > > >> > > > > > >> > > > > > >>> > > > out
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> StopReplicasRequest,
> > > get
> > > > > ACKs
> > > > > > > > from
> > > > > > > > >> all
> > > > > > > > >> > > > > nodes,
> > > > > > > > >> > > > > > >> > commit
> > > > > > > > >> > > > > > >> > > by
> > > > > > > > >> > > > > > >> > > > > > >>> removing
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> /admin/delete_topics
> > > > node)
> > > > > > any
> > > > > > > > >> more.
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > You mention
> that
> > > > > > > FetchRequest
> > > > > > > > >> will
> > > > > > > > >> > > now
> > > > > > > > >> > > > > > >> include
> > > > > > > > >> > > > > > >> > > UUID
> > > > > > > > >> > > > > > >> > > > > to
> > > > > > > > >> > > > > > >> > > > > > >>> avoid
> > > > > > > > >> > > > > > >> > > > > > >>> > > > issues
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > where requests are
> > made
> > > > to
> > > > > > > stale
> > > > > > > > >> > > > partitions.
> > > > > > > > >> > > > > > >> > > However,
> > > > > > > > >> > > > > > >> > > > > > >>> adding a
> > > > > > > > >> > > > > > >> > > > > > >>> > UUID
> > > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > MetadataRequest is
> > > listed
> > > > > as
> > > > > > > > future
> > > > > > > > >> > > work,
> > > > > > > > >> > > > > out
> > > > > > > > >> > > > > > of
> > > > > > > > >> > > > > > >> > > scope
> > > > > > > > >> > > > > > >> > > > > for
> > > > > > > > >> > > > > > >> > > > > > >>> this
> > > > > > > > >> > > > > > >> > > > > > >>> > KIP.
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > How
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > will the client
> learn
> > > > what
> > > > > > the
> > > > > > > > >> topic
> > > > > > > > >> > > UUID
> > > > > > > > >> > > > > is,
> > > > > > > > >> > > > > > if
> > > > > > > > >> > > > > > >> > the
> > > > > > > > >> > > > > > >> > > > > > metadata
> > > > > > > > >> > > > > > >> > > > > > >>> > > > response
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > doesn't include
> that
> > > > > > > information?
> > > > > > > > >> It
> > > > > > > > >> > > > seems
> > > > > > > > >> > > > > > like
> > > > > > > > >> > > > > > >> > > adding
> > > > > > > > >> > > > > > >> > > > > the
> > > > > > > > >> > > > > > >> > > > > > >>> UUID
> > > > > > > > >> > > > > > >> > > > > > >>> > to
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > MetadataResponse
> > would
> > > be
> > > > > an
> > > > > > > > >> > improvement
> > > > > > > > >> > > > > here
> > > > > > > > >> > > > > > >> that
> > > > > > > > >> > > > > > >> > > > might
> > > > > > > > >> > > > > > >> > > > > > not
> > > > > > > > >> > > > > > >> > > > > > >>> be
> > > > > > > > >> > > > > > >> > > > > > >>> > too
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > hard to
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > make.
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > best,
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Colin
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > On Mon, Sep 9,
> > > 2019,
> > > > at
> > > > > > > > 17:48,
> > > > > > > > >> > > Ryanne
> > > > > > > > >> > > > > > Dolan
> > > > > > > > >> > > > > > >> > > wrote:
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > Lucas, this
> > would
> > > > be
> > > > > > > great.
> > > > > > > > >> I've
> > > > > > > > >> > > run
> > > > > > > > >> > > > > > into
> > > > > > > > >> > > > > > >> > > issues
> > > > > > > > >> > > > > > >> > > > > with
> > > > > > > > >> > > > > > >> > > > > > >>> > topics
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > being
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > resurrected
> > > > > > accidentally,
> > > > > > > > >> since
> > > > > > > > >> > a
> > > > > > > > >> > > > > client
> > > > > > > > >> > > > > > >> > cannot
> > > > > > > > >> > > > > > >> > > > > > easily
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > distinguish
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > between
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > a deleted
> topic
> > > > and a
> > > > > > new
> > > > > > > > >> topic
> > > > > > > > >> > > with
> > > > > > > > >> > > > > the
> > > > > > > > >> > > > > > >> same
> > > > > > > > >> > > > > > >> > > > name.
> > > > > > > > >> > > > > > >> > > > > > I'd
> > > > > > > > >> > > > > > >> > > > > > >>> > need
> > > > > > > > >> > > > > > >> > > > > > >>> > > > the
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > ID
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > accessible
> from
> > > the
> > > > > > > client
> > > > > > > > to
> > > > > > > > >> > > solve
> > > > > > > > >> > > > > that
> > > > > > > > >> > > > > > >> > issue,
> > > > > > > > >> > > > > > >> > > > but
> > > > > > > > >> > > > > > >> > > > > > >>> this
> > > > > > > > >> > > > > > >> > > > > > >>> > is a
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > good
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > first
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > step.
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > Ryanne
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > On Wed, Sep
> 4,
> > > 2019
> > > > > at
> > > > > > > 1:41
> > > > > > > > >> PM
> > > > > > > > >> > > Lucas
> > > > > > > > >> > > > > > >> > > Bradstreet <
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > lucas@confluent.io>
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > wrote:
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Hi all,
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > I would
> like
> > to
> > > > > kick
> > > > > > > off
> > > > > > > > >> > > > discussion
> > > > > > > > >> > > > > of
> > > > > > > > >> > > > > > >> > > KIP-516,
> > > > > > > > >> > > > > > >> > > > > an
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > implementation
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > of topic
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > IDs for
> > Kafka.
> > > > > Topic
> > > > > > > IDs
> > > > > > > > >> aim
> > > > > > > > >> > to
> > > > > > > > >> > > > > solve
> > > > > > > > >> > > > > > >> topic
> > > > > > > > >> > > > > > >> > > > > > >>> uniqueness
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > problems in
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Kafka,
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > where
> > referring
> > > > to
> > > > > a
> > > > > > > > topic
> > > > > > > > >> by
> > > > > > > > >> > > name
> > > > > > > > >> > > > > > >> alone is
> > > > > > > > >> > > > > > >> > > > > > >>> insufficient.
> > > > > > > > >> > > > > > >> > > > > > >>> > > > Such
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > cases
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > include
> when
> > a
> > > > > topic
> > > > > > > has
> > > > > > > > >> been
> > > > > > > > >> > > > > deleted
> > > > > > > > >> > > > > > >> and
> > > > > > > > >> > > > > > >> > > > > recreated
> > > > > > > > >> > > > > > >> > > > > > >>> with
> > > > > > > > >> > > > > > >> > > > > > >>> > the
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > same
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > name.
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Unique
> > > > identifiers
> > > > > > will
> > > > > > > > >> help
> > > > > > > > >> > > > > simplify
> > > > > > > > >> > > > > > >> and
> > > > > > > > >> > > > > > >> > > > improve
> > > > > > > > >> > > > > > >> > > > > > >>> Kafka's
> > > > > > > > >> > > > > > >> > > > > > >>> > > > topic
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > deletion
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > process, as
> > > well
> > > > as
> > > > > > > > prevent
> > > > > > > > >> > > cases
> > > > > > > > >> > > > > > where
> > > > > > > > >> > > > > > >> > > brokers
> > > > > > > > >> > > > > > >> > > > > may
> > > > > > > > >> > > > > > >> > > > > > >>> > > > incorrectly
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > interact
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > with stale
> > > > versions
> > > > > > of
> > > > > > > > >> topics.
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > >> > > > > > >> > > >
> > > > > > > > >> > > > > > >> > >
> > > > > > > > >> > > > > > >> >
> > > > > > > > >> > > > > > >>
> > > > > > > > >> > > > > >
> > > > > > > > >> > > > >
> > > > > > > > >> > > >
> > > > > > > > >> > >
> > > > > > > > >> >
> > > > > > > > >>
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Looking
> > forward
> > > > to
> > > > > > your
> > > > > > > > >> > > thoughts.
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Lucas
> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > > >> > > > > > >> > > > > > >>
> > > > > > > > >> > > > > > >> > > > > >
> > > > > > > > >> > > > > > >> > > > >
> > > > > > > > >> > > > > > >> > > >
> > > > > > > > >> > > > > > >> > >
> > > > > > > > >> > > > > > >> >
> > > > > > > > >> > > > > > >>
> > > > > > > > >> > > > > > >
> > > > > > > > >> > > > > >
> > > > > > > > >> > > > >
> > > > > > > > >> > > >
> > > > > > > > >> > >
> > > > > > > > >> >
> > > > > > > > >>
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Lucas Bradstreet <lu...@confluent.io>]

Hi Ismael,

If you do not store it in a metadata file or in the directory structure
would you then
require the LeaderAndIsrRequest following startup to give you some notion
of
topic name in memory? We would still need this information for the older
protocols, but
perhaps this is what's meant by tech debt.

Once we're free of the old non-topicID requests then I think you wouldn't
need to retain the topic name.
I think the ability to easily look up topic names associated with partition
directories would still be missed
when diagnosing issues, though maybe it wouldn't be a deal breaker with the
right tooling.

Thanks,

Lucas

On Fri, Sep 25, 2020 at 7:55 AM Ismael Juma <is...@juma.me.uk> wrote:

> Hi Lucas,
>
> Why would you include the name and id? I think you'd want to remove the
> name from the directory name right? Jason's suggestion was that if you
> remove the name from the directory, then why would you need the id name
> mapping file?
>
> Ismael
>
> On Thu, Sep 24, 2020 at 4:24 PM Lucas Bradstreet <lu...@confluent.io>
> wrote:
>
> > > 2. Part of the usage of the file is to have persistent storage of the
> > topic
> > ID and use it to compare with the ID supplied in the LeaderAndIsr
> Request.
> > There is some discussion in the KIP about changes to the directory
> > structure, but I believe directory changes were considered to be out of
> > scope when the KIP was written.
> >
> >
> > Yeah, I was hoping to get a better understanding of why it was taken out
> of
> > scope. Perhaps Lucas Bradstreet might have more insight about the
> decision.
> > Basically my point is that we have to create additional infrastructure
> here
> > to support the name/id mapping, so I wanted to understand if we just
> > consider this a sort of tech debt. It would be useful to cover how we
> > handle the case when this file gets corrupted. Seems like we just have to
> > trust that it matches whatever the controller tells us and rewrite it?
> >
> >
> > Hi Jason, Justine,
> >
> > My thought process is that we will likely need the metadata file whether
> we
> > rename the directories or not.
> > Linux supports filenames of up to 255 bytes and that would not be enough
> to
> > support a directory name
> >  including both the name and topic ID. Given that fact, it seemed
> > reasonable to add the metadata file
> > and leave the directory rename until some time in the future (possibly
> > never).
> >
> > If the file does get corrupted I think you're right. We would either have
> > to trust it matches what the controller tells us
> >  or error out and let an administrator resolve it by checking across
> > replicas for consistency.
> >
> > Lucas
> >
> >
> > On Thu, Sep 24, 2020 at 3:41 PM Jason Gustafson <ja...@confluent.io>
> > wrote:
> >
> > > Thanks Justine. Responses below:
> > >
> > > > 1. Yes, the directory will still be based on the topic names.
> > > LeaderAndIsrRequest is one of the few requests that will still contain
> > the
> > > topic name. So I think we have this covered. Sorry for confusion.
> > >
> > > Ah, you're right. My eyes passed right over the field.
> > >
> > > > 2. Part of the usage of the file is to have persistent storage of the
> > > topic
> > > ID and use it to compare with the ID supplied in the LeaderAndIsr
> > Request.
> > > There is some discussion in the KIP about changes to the directory
> > > structure, but I believe directory changes were considered to be out of
> > > scope when the KIP was written.
> > >
> > > Yeah, I was hoping to get a better understanding of why it was taken
> out
> > of
> > > scope. Perhaps Lucas Bradstreet might have more insight about the
> > decision.
> > > Basically my point is that we have to create additional infrastructure
> > here
> > > to support the name/id mapping, so I wanted to understand if we just
> > > consider this a sort of tech debt. It would be useful to cover how we
> > > handle the case when this file gets corrupted. Seems like we just have
> to
> > > trust that it matches whatever the controller tells us and rewrite it?
> > >
> > > > 3. I think this is a good point, but I again I wonder about the scope
> > of
> > > the KIP. Most of the changes mentioned in the KIP are for supporting
> > topic
> > > deletion and I believe that is why the produce request was listed under
> > > future work.
> > >
> > > That's fair. I brought it up since `Fetch` is already included. If
> we've
> > > got `Metadata` and `Fetch`, seems we may as well do `Produce` and save
> an
> > > extra kip. No strong objection though if you want to leave it out.
> > >
> > >
> > > -Jason
> > >
> > >
> > > On Thu, Sep 24, 2020 at 3:26 PM Justine Olshan <jo...@confluent.io>
> > > wrote:
> > >
> > > > Hi Jason,
> > > >
> > > > Thanks for your comments.
> > > >
> > > > 1. Yes, the directory will still be based on the topic names.
> > > > LeaderAndIsrRequest is one of the few requests that will still
> contain
> > > the
> > > > topic name. So I think we have this covered. Sorry for confusion.
> > > >
> > > > 2. Part of the usage of the file is to have persistent storage of the
> > > topic
> > > > ID and use it to compare with the ID supplied in the LeaderAndIsr
> > > Request.
> > > > There is some discussion in the KIP about changes to the directory
> > > > structure, but I believe directory changes were considered to be out
> of
> > > > scope when the KIP was written.
> > > >
> > > > 3. I think this is a good point, but I again I wonder about the scope
> > of
> > > > the KIP. Most of the changes mentioned in the KIP are for supporting
> > > topic
> > > > deletion and I believe that is why the produce request was listed
> under
> > > > future work.
> > > >
> > > > 4. This sounds like it might be a good solution, but I will need to
> > > discuss
> > > > more with KIP-500 folks to get the details right.
> > > >
> > > > Thanks,
> > > > Justine
> > > >
> > > > On Thu, Sep 24, 2020 at 12:30 PM Jason Gustafson <jason@confluent.io
> >
> > > > wrote:
> > > >
> > > > > Hi Justine,
> > > > >
> > > > > Thanks for picking up this work. I have a few questions/comments:
> > > > >
> > > > > 1. It sounds like the directory structure is still going to be
> based
> > on
> > > > > topic names. Do I have that right? One complication is that the
> > > > > LeaderAndIsr request does not include the topic name any longer.
> This
> > > > means
> > > > > that a replica must wait for the UpdateMetadata request to arrive
> > with
> > > > the
> > > > > topic name mapping before it can create the directory. I wonder if
> it
> > > > would
> > > > > be simpler to avoid assumptions on the ordering of UpdateMetadata
> and
> > > let
> > > > > LeaderAndIsr include the topic name as well. Feels like we are not
> > > saving
> > > > > that much by excluding it.
> > > > >
> > > > > 2. On a related note, it seems that the reason we have the
> partition
> > > > > metadata file is because we are not changing the directory
> structure.
> > > We
> > > > > need it so that we remember which directories map to which topic
> id.
> > I
> > > > > think it would be useful to add some detail about why changing the
> > > > > directory layout was rejected. Basically trying to understand how
> > much
> > > > > effort we are saving by skipping this step if we have to implement
> > this
> > > > > additional bookkeeping. One concern I have for example is that the
> > > > > partition metadata file gets corrupt and then all bets are off as
> far
> > > as
> > > > > topic consistency.
> > > > >
> > > > > 3. Is it worth adding support now for topic ids in the `Produce`
> > > request
> > > > > now? Overhead is mentioned as one of the motivations and the best
> > APIs
> > > to
> > > > > get savings from are `Produce` and `Fetch`.
> > > > >
> > > > > 4. When it comes to bootstrapping the metadata topic with respect
> to
> > > > > KIP-500, one suggestion would be to reserve a sentinel topic ID
> which
> > > can
> > > > > be used initially by new brokers. When the first controller is
> > elected,
> > > > it
> > > > > can assign a topicId to the metadata topic. This is a bit similar
> to
> > > how
> > > > we
> > > > > were planning to generate the clusterId.
> > > > >
> > > > > Thanks,
> > > > > Jason
> > > > >
> > > > >
> > > > > On Thu, Sep 24, 2020 at 11:10 AM Jun Rao <ju...@confluent.io> wrote:
> > > > >
> > > > > > Hi, Justine,
> > > > > >
> > > > > > Thanks for the updated KIP. A few more comments below.
> > > > > >
> > > > > > 30.  {"name": "id", "type": "string", "doc": "version id"}}: The
> > doc
> > > > > should
> > > > > > say UUID.
> > > > > >
> > > > > > 31. LeaderAndIsrResponse v5 and StopReplicaResponse v4 : It seems
> > > there
> > > > > is
> > > > > > no need to add topic_id at partitions level.
> > > > > >
> > > > > > 32. Regarding partition metadata file. Perhaps the key can be a
> > > single
> > > > > > word, sth like the following.
> > > > > > version: 0
> > > > > > topic_id: 46bdb63f-9e8d-4a38-bf7b-ee4eb2a794e4
> > > > > >
> > > > > > 33. Another tricky thing that I realized is how to support the
> > > metadata
> > > > > > topic introduced in KIP-595. It's a bit tricky to assign a UUID
> to
> > > the
> > > > > > metadata topic since we have a chicken-and-egg problem. The
> > > controller
> > > > > > needs to persist the UUID in the metadata topic in order to
> assign
> > > one
> > > > > > successfully, but the metadata topic is needed to elect a
> > controller
> > > > > > (KIP-631). So, this probably needs a bit more thought.
> > > > > >
> > > > > > Jun
> > > > > >
> > > > > > On Thu, Sep 24, 2020 at 3:04 AM Ismael Juma <is...@juma.me.uk>
> > > wrote:
> > > > > >
> > > > > > > Also, can we provide more details on how the Partition Metadata
> > > file
> > > > > will
> > > > > > > be used?
> > > > > > >
> > > > > > > Ismael
> > > > > > >
> > > > > > > On Thu, Sep 24, 2020 at 3:01 AM Ismael Juma <ismael@juma.me.uk
> >
> > > > wrote:
> > > > > > >
> > > > > > > > Hi Justine,
> > > > > > > >
> > > > > > > > I think we need to update the "Rejected Alternatives" section
> > to
> > > > take
> > > > > > > into
> > > > > > > > account that the proposal now removes the topic name from the
> > > fetch
> > > > > > > > request. Also, if we are removing it from the Fetch request,
> > does
> > > > it
> > > > > > make
> > > > > > > > sense not to remove it from similar requests like
> > > > ListOffsetRequest?
> > > > > > > >
> > > > > > > > Ismael
> > > > > > > >
> > > > > > > > On Thu, Sep 24, 2020 at 2:46 AM David Jacot <
> > djacot@confluent.io
> > > >
> > > > > > wrote:
> > > > > > > >
> > > > > > > >> Hi Justine,
> > > > > > > >>
> > > > > > > >> Thanks for the KIP. I finally had time to read it :). It is
> a
> > > > great
> > > > > > > >> improvement.
> > > > > > > >>
> > > > > > > >> I have a few comments/questions:
> > > > > > > >>
> > > > > > > >> 1. It seems that the schema of the StopReplicaRequest is
> > > slightly
> > > > > > > >> outdated.
> > > > > > > >> We
> > > > > > > >> did some changes as part of KIP-570. V3 is already organized
> > by
> > > > > > topics.
> > > > > > > >>
> > > > > > > >> 2. I just want to make sure that I understand the
> > reconciliation
> > > > > > > >> logic correctly. When
> > > > > > > >> an "INCREMENTAL" LeaderAndIsr Request is received, the
> broker
> > > will
> > > > > > also
> > > > > > > >> reconcile
> > > > > > > >> when the local uuid does not match the uuid in the request,
> > > right?
> > > > > In
> > > > > > > this
> > > > > > > >> case, the
> > > > > > > >> local log is staged for deletion.
> > > > > > > >>
> > > > > > > >> 3. In the documentation of the `delete.stale.topic.delay.ms
> `
> > > > > config,
> > > > > > it
> > > > > > > >> says "When a
> > > > > > > >> FULL LeaderAndIsrRequest is received..." but I suppose it
> > > applies
> > > > to
> > > > > > > both
> > > > > > > >> types.
> > > > > > > >>
> > > > > > > >> Best,
> > > > > > > >> David
> > > > > > > >>
> > > > > > > >> On Thu, Sep 24, 2020 at 1:40 AM Justine Olshan <
> > > > > jolshan@confluent.io>
> > > > > > > >> wrote:
> > > > > > > >>
> > > > > > > >> > Hi Jun,
> > > > > > > >> >
> > > > > > > >> > Thanks for the comments. I apologize for some of the typos
> > and
> > > > > > > >> confusion.
> > > > > > > >> > I’ve updated the KIP to fix some of the issues you
> > mentioned.
> > > > > > > >> >
> > > > > > > >> > 20.2 I’ve changed the type to String.
> > > > > > > >> > 20.1/3 I’ve updated the TopicZNode to fix formatting and
> > > reflect
> > > > > the
> > > > > > > >> latest
> > > > > > > >> > version before this change.
> > > > > > > >> >
> > > > > > > >> > 21. You are correct and I’ve removed this line. I’ve also
> > > added
> > > > a
> > > > > > line
> > > > > > > >> > mentioning an IBP bump is necessary for migration
> > > > > > > >> >
> > > > > > > >> > 22. I think the wording was unclear but your summary is
> what
> > > was
> > > > > > > >> intended
> > > > > > > >> > by this line. I’ve updated to make this point more clear.
> > > > “Remove
> > > > > > > >> deleted
> > > > > > > >> > topics from replicas by sending StopReplicaRequest V3
> before
> > > the
> > > > > IBP
> > > > > > > >> bump
> > > > > > > >> > using the old logic, and using V4 and the new logic with
> > topic
> > > > IDs
> > > > > > > after
> > > > > > > >> > the IBP bump.”
> > > > > > > >> >
> > > > > > > >> > 23. I’ve removed the unspecified type from the KIP and
> > mention
> > > > > that
> > > > > > > IBP
> > > > > > > >> > will be used to handle this request. “IBP will be used to
> > > > > determine
> > > > > > > >> whether
> > > > > > > >> > this new form of the request will be used. For older
> > requests,
> > > > we
> > > > > > will
> > > > > > > >> > ignore this field and default to previous behavior.”
> > > > > > > >> >
> > > > > > > >> > 24. I’ve fixed this typo.
> > > > > > > >> >
> > > > > > > >> > 25. I've added a topics at a higher level for
> > > > LeaderAndIsrResponse
> > > > > > v5,
> > > > > > > >> > StopReplicaResponse v4. I've also changed
> StopReplicaRequest
> > > v4
> > > > to
> > > > > > > have
> > > > > > > >> > topics at a higher level.
> > > > > > > >> >
> > > > > > > >> > 26. I’ve updated forgotten_topics_data--added the topic ID
> > and
> > > > > > removed
> > > > > > > >> the
> > > > > > > >> > topic name
> > > > > > > >> >
> > > > > > > >> > 27. I’ve decided on plain text, and I’ve added an example.
> > > > > > > >> >
> > > > > > > >> > 28. I’ve added this idea to future work.
> > > > > > > >> >
> > > > > > > >> > Thanks again for taking a look,
> > > > > > > >> >
> > > > > > > >> > Justine
> > > > > > > >> >
> > > > > > > >> > On Wed, Sep 23, 2020 at 10:28 AM Jun Rao <
> jun@confluent.io>
> > > > > wrote:
> > > > > > > >> >
> > > > > > > >> > > Hi, Justine,
> > > > > > > >> > >
> > > > > > > >> > > Thanks for the response. Made another pass. A few more
> > > > comments
> > > > > > > below.
> > > > > > > >> > >
> > > > > > > >> > > 20. znode schema:
> > > > > > > >> > > 20.1 It seems that {"name": "version", "type": "int",
> > "id":
> > > > > > "UUID",
> > > > > > > >> > "doc":
> > > > > > > >> > > "version id"} should be {"name": "version", "type":
> > "int"},
> > > > > > {"name":
> > > > > > > >> > "id",
> > > > > > > >> > > "type": "UUID", "doc": "version id"}.
> > > > > > > >> > > 20.2 The znode format is JSON which doesn't have UUID
> > type.
> > > So
> > > > > the
> > > > > > > >> type
> > > > > > > >> > > probably should be string?
> > > > > > > >> > > 20.3 Also, the existing format used seems outdated. It
> > > should
> > > > > have
> > > > > > > the
> > > > > > > >> > > following format.
> > > > > > > >> > >     Json.encodeAsBytes(Map(
> > > > > > > >> > >       "version" -> 2,
> > > > > > > >> > >       "partitions" -> replicaAssignmentJson.asJava,
> > > > > > > >> > >       "adding_replicas" ->
> > > > addingReplicasAssignmentJson.asJava,
> > > > > > > >> > >       "removing_replicas" ->
> > > > > removingReplicasAssignmentJson.asJava
> > > > > > > >> > >     ).asJava)
> > > > > > > >> > >   }
> > > > > > > >> > >
> > > > > > > >> > > 21. Migration: The KIP says "The migration process can
> > take
> > > > > place
> > > > > > > >> without
> > > > > > > >> > > an inter-broker protocol bump, as the format stored in
> > > > > > > >> > > /brokers/topics/[topic] will be compatible with older
> > broker
> > > > > > > >> versions."
> > > > > > > >> > > However, since we are bumping up the version of
> > inter-broker
> > > > > > > >> requests, it
> > > > > > > >> > > seems that we need to use IBP for migration.
> > > > > > > >> > >
> > > > > > > >> > > 22. The KIP says "Remove deleted topics from replicas by
> > > > sending
> > > > > > > >> > > StopReplicaRequest V3 for any topics which do not
> contain
> > a
> > > > > topic
> > > > > > > ID,
> > > > > > > >> and
> > > > > > > >> > > V4 for any topics which do contain a topic ID.".
> However,
> > if
> > > > we
> > > > > > use
> > > > > > > >> IBP,
> > > > > > > >> > it
> > > > > > > >> > > seems that the controller will either send
> > > StopReplicaRequest
> > > > V3
> > > > > > > >> > > or StopReplicaRequest V4, but never mixed V3 and V4 for
> > > > > different
> > > > > > > >> topics.
> > > > > > > >> > > Basically, before the IBP bump, V3 will be used. After
> the
> > > IBP
> > > > > > bump,
> > > > > > > >> > > topicId will be created and V4 will be used.
> > > > > > > >> > >
> > > > > > > >> > > 23. Given that we depend on IBP, do we still need "0
> > > > > UNSPECIFIED"
> > > > > > > >> > > in LeaderAndIsr?
> > > > > > > >> > >
> > > > > > > >> > > 24. LeaderAndIsrResponse v5 : It still has the topic
> > field.
> > > > > > > >> > >
> > > > > > > >> > > 25. LeaderAndIsrResponse v5, StopReplicaResponse v4:
> Could
> > > we
> > > > > use
> > > > > > > this
> > > > > > > >> > > opportunity to organize the response in 2 levels, first
> by
> > > > > topic,
> > > > > > > >> then by
> > > > > > > >> > > partition, as most other requests/responses?
> > > > > > > >> > >
> > > > > > > >> > > 26. FetchRequest v13 : Should forgotten_topics_data use
> > > > topicId
> > > > > > too?
> > > > > > > >> > >
> > > > > > > >> > > 27. "This file can either be plain text (key/value
> pairs)
> > or
> > > > > > JSON."
> > > > > > > >> Have
> > > > > > > >> > we
> > > > > > > >> > > decided which one to use? Also, it would be helpful to
> > > provide
> > > > > an
> > > > > > > >> > example.
> > > > > > > >> > >
> > > > > > > >> > > 28. Future improvement: Another future improvement
> > > opportunity
> > > > > is
> > > > > > to
> > > > > > > >> use
> > > > > > > >> > > topicId in GroupMetadataManager.offsetCommitKey in the
> > > > > > offset_commit
> > > > > > > >> > topic.
> > > > > > > >> > > This may save some space.
> > > > > > > >> > >
> > > > > > > >> > > Thanks,
> > > > > > > >> > >
> > > > > > > >> > > Jun
> > > > > > > >> > >
> > > > > > > >> > > On Wed, Sep 23, 2020 at 8:50 AM Justine Olshan <
> > > > > > > jolshan@confluent.io>
> > > > > > > >> > > wrote:
> > > > > > > >> > >
> > > > > > > >> > > > Hi Tom,
> > > > > > > >> > > >
> > > > > > > >> > > > Thanks for the comment. I think this is a really good
> > idea
> > > > and
> > > > > > it
> > > > > > > >> has
> > > > > > > >> > > been
> > > > > > > >> > > > added to the KIP under the newly added tooling
> section.
> > > > > > > >> > > >
> > > > > > > >> > > > Thanks again,
> > > > > > > >> > > > Justine
> > > > > > > >> > > >
> > > > > > > >> > > > On Wed, Sep 23, 2020 at 3:17 AM Tom Bentley <
> > > > > > tbentley@redhat.com>
> > > > > > > >> > wrote:
> > > > > > > >> > > >
> > > > > > > >> > > > > Hi Justine,
> > > > > > > >> > > > >
> > > > > > > >> > > > > I know you started the vote thread, but on
> re-reading
> > > the
> > > > > KIP
> > > > > > I
> > > > > > > >> > noticed
> > > > > > > >> > > > > that although the topic id is included in the
> > > > > MetadataResponse
> > > > > > > >> it's
> > > > > > > >> > not
> > > > > > > >> > > > > surfaced in the output from `kafka-topics.sh
> > > --describe`.
> > > > > > Maybe
> > > > > > > >> that
> > > > > > > >> > > was
> > > > > > > >> > > > > intentional because ids are intentionally not really
> > > > > something
> > > > > > > the
> > > > > > > >> > user
> > > > > > > >> > > > > should care deeply about, but it would also make
> life
> > > > harder
> > > > > > for
> > > > > > > >> > anyone
> > > > > > > >> > > > > debugging Kafka and this would likely get worse the
> > more
> > > > > topic
> > > > > > > ids
> > > > > > > >> > got
> > > > > > > >> > > > > rolled out across the protocols, clients etc. It
> seems
> > > > > likely
> > > > > > > that
> > > > > > > >> > > > > `kafka-topics.sh` will eventually need the ability
> to
> > > show
> > > > > the
> > > > > > > id
> > > > > > > >> of
> > > > > > > >> > a
> > > > > > > >> > > > > topic and perhaps find a topic name given an id. Is
> > > there
> > > > > any
> > > > > > > >> reason
> > > > > > > >> > > not
> > > > > > > >> > > > to
> > > > > > > >> > > > > implement that in this KIP?
> > > > > > > >> > > > >
> > > > > > > >> > > > > Many thanks,
> > > > > > > >> > > > >
> > > > > > > >> > > > > Tom
> > > > > > > >> > > > >
> > > > > > > >> > > > > On Mon, Sep 21, 2020 at 9:54 PM Justine Olshan <
> > > > > > > >> jolshan@confluent.io
> > > > > > > >> > >
> > > > > > > >> > > > > wrote:
> > > > > > > >> > > > >
> > > > > > > >> > > > > > Hi all,
> > > > > > > >> > > > > >
> > > > > > > >> > > > > > After thinking about it, I've decided to remove
> the
> > > > topic
> > > > > > name
> > > > > > > >> from
> > > > > > > >> > > the
> > > > > > > >> > > > > > Fetch Request and Response after all. Since there
> > are
> > > so
> > > > > > many
> > > > > > > of
> > > > > > > >> > > these
> > > > > > > >> > > > > > requests per second, it is worth removing the
> extra
> > > > > > > information.
> > > > > > > >> > I've
> > > > > > > >> > > > > > updated the KIP to reflect this change.
> > > > > > > >> > > > > >
> > > > > > > >> > > > > > Please let me know if there is anything else we
> > should
> > > > > > discuss
> > > > > > > >> > before
> > > > > > > >> > > > > > voting.
> > > > > > > >> > > > > >
> > > > > > > >> > > > > > Thank you,
> > > > > > > >> > > > > > Justine
> > > > > > > >> > > > > >
> > > > > > > >> > > > > > On Fri, Sep 18, 2020 at 9:46 AM Justine Olshan <
> > > > > > > >> > jolshan@confluent.io
> > > > > > > >> > > >
> > > > > > > >> > > > > > wrote:
> > > > > > > >> > > > > >
> > > > > > > >> > > > > > > Hi Jun,
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > > I see what you are saying. For now we can remove
> > the
> > > > > extra
> > > > > > > >> > > > information.
> > > > > > > >> > > > > > > I'll leave the option to add more fields to the
> > file
> > > > in
> > > > > > the
> > > > > > > >> > future.
> > > > > > > >> > > > The
> > > > > > > >> > > > > > KIP
> > > > > > > >> > > > > > > has been updated to reflect this change.
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > > Thanks,
> > > > > > > >> > > > > > > Justine
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > > On Fri, Sep 18, 2020 at 8:46 AM Jun Rao <
> > > > > jun@confluent.io
> > > > > > >
> > > > > > > >> > wrote:
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > >> Hi, Justine,
> > > > > > > >> > > > > > >>
> > > > > > > >> > > > > > >> Thanks for the reply.
> > > > > > > >> > > > > > >>
> > > > > > > >> > > > > > >> 13. If the log directory is the source of
> truth,
> > it
> > > > > means
> > > > > > > >> that
> > > > > > > >> > the
> > > > > > > >> > > > > > >> redundant info in the metadata file will be
> > > ignored.
> > > > > Then
> > > > > > > the
> > > > > > > >> > > > question
> > > > > > > >> > > > > > is
> > > > > > > >> > > > > > >> why do we need to put the redundant info in the
> > > > > metadata
> > > > > > > file
> > > > > > > >> > now?
> > > > > > > >> > > > > > >>
> > > > > > > >> > > > > > >> Thanks,
> > > > > > > >> > > > > > >>
> > > > > > > >> > > > > > >> Jun
> > > > > > > >> > > > > > >>
> > > > > > > >> > > > > > >> On Thu, Sep 17, 2020 at 5:07 PM Justine Olshan
> <
> > > > > > > >> > > > jolshan@confluent.io>
> > > > > > > >> > > > > > >> wrote:
> > > > > > > >> > > > > > >>
> > > > > > > >> > > > > > >> > Hi Jun,
> > > > > > > >> > > > > > >> > Thanks for the quick response!
> > > > > > > >> > > > > > >> >
> > > > > > > >> > > > > > >> > 12. I've decided to bump up the versions on
> the
> > > > > > requests
> > > > > > > >> and
> > > > > > > >> > > > updated
> > > > > > > >> > > > > > the
> > > > > > > >> > > > > > >> > KIP. I think it's good we thoroughly
> discussed
> > > the
> > > > > > > options
> > > > > > > >> > here,
> > > > > > > >> > > > so
> > > > > > > >> > > > > we
> > > > > > > >> > > > > > >> know
> > > > > > > >> > > > > > >> > we made a good choice. :)
> > > > > > > >> > > > > > >> >
> > > > > > > >> > > > > > >> > 13. This is an interesting situation. I think
> > if
> > > > this
> > > > > > > does
> > > > > > > >> > occur
> > > > > > > >> > > > we
> > > > > > > >> > > > > > >> should
> > > > > > > >> > > > > > >> > give a warning. I agree that it's hard to
> know
> > > the
> > > > > > source
> > > > > > > >> of
> > > > > > > >> > > truth
> > > > > > > >> > > > > for
> > > > > > > >> > > > > > >> sure
> > > > > > > >> > > > > > >> > since the directory or the file could be
> > manually
> > > > > > > >> modified. I
> > > > > > > >> > > > guess
> > > > > > > >> > > > > > the
> > > > > > > >> > > > > > >> > directory could be used as the source of
> truth.
> > > To
> > > > be
> > > > > > > >> honest,
> > > > > > > >> > > I'm
> > > > > > > >> > > > > not
> > > > > > > >> > > > > > >> > really sure what happens in kafka when the
> log
> > > > > > directory
> > > > > > > is
> > > > > > > >> > > > renamed
> > > > > > > >> > > > > > >> > manually in such a way. I'm also wondering if
> > the
> > > > > > > >> situation is
> > > > > > > >> > > > > > >> recoverable
> > > > > > > >> > > > > > >> > in this scenario.
> > > > > > > >> > > > > > >> >
> > > > > > > >> > > > > > >> > Thanks,
> > > > > > > >> > > > > > >> > Justine
> > > > > > > >> > > > > > >> >
> > > > > > > >> > > > > > >> > On Thu, Sep 17, 2020 at 4:28 PM Jun Rao <
> > > > > > > jun@confluent.io>
> > > > > > > >> > > wrote:
> > > > > > > >> > > > > > >> >
> > > > > > > >> > > > > > >> > > Hi, Justine,
> > > > > > > >> > > > > > >> > >
> > > > > > > >> > > > > > >> > > Thanks for the reply.
> > > > > > > >> > > > > > >> > >
> > > > > > > >> > > > > > >> > > 12. I don't have a strong preference
> either.
> > > > > However,
> > > > > > > if
> > > > > > > >> we
> > > > > > > >> > > need
> > > > > > > >> > > > > IBP
> > > > > > > >> > > > > > >> > > anyway, maybe it's easier to just bump up
> the
> > > > > version
> > > > > > > for
> > > > > > > >> > all
> > > > > > > >> > > > > inter
> > > > > > > >> > > > > > >> > broker
> > > > > > > >> > > > > > >> > > requests and add the topic id field as a
> > > regular
> > > > > > > field. A
> > > > > > > >> > > > regular
> > > > > > > >> > > > > > >> field
> > > > > > > >> > > > > > >> > is
> > > > > > > >> > > > > > >> > > a bit more concise in wire transfer than a
> > > > flexible
> > > > > > > >> field.
> > > > > > > >> > > > > > >> > >
> > > > > > > >> > > > > > >> > > 13. The confusion that I was referring to
> is
> > > > > between
> > > > > > > the
> > > > > > > >> > topic
> > > > > > > >> > > > > name
> > > > > > > >> > > > > > >> and
> > > > > > > >> > > > > > >> > > partition number between the log dir and
> the
> > > > > metadata
> > > > > > > >> file.
> > > > > > > >> > > For
> > > > > > > >> > > > > > >> example,
> > > > > > > >> > > > > > >> > if
> > > > > > > >> > > > > > >> > > the log dir is topicA-1 and the metadata
> file
> > > in
> > > > it
> > > > > > has
> > > > > > > >> > topicB
> > > > > > > >> > > > and
> > > > > > > >> > > > > > >> > > partition 0 (say due to a bug or manual
> > > > > > modification),
> > > > > > > >> which
> > > > > > > >> > > one
> > > > > > > >> > > > > do
> > > > > > > >> > > > > > we
> > > > > > > >> > > > > > >> > use
> > > > > > > >> > > > > > >> > > as the source of truth?
> > > > > > > >> > > > > > >> > >
> > > > > > > >> > > > > > >> > > Jun
> > > > > > > >> > > > > > >> > >
> > > > > > > >> > > > > > >> > > On Thu, Sep 17, 2020 at 3:43 PM Justine
> > Olshan
> > > <
> > > > > > > >> > > > > > jolshan@confluent.io>
> > > > > > > >> > > > > > >> > > wrote:
> > > > > > > >> > > > > > >> > >
> > > > > > > >> > > > > > >> > > > Hi Jun,
> > > > > > > >> > > > > > >> > > > Thanks for the comments.
> > > > > > > >> > > > > > >> > > >
> > > > > > > >> > > > > > >> > > > 12. I bumped the LeaderAndIsrRequest
> > because
> > > I
> > > > > > > removed
> > > > > > > >> the
> > > > > > > >> > > > topic
> > > > > > > >> > > > > > >> name
> > > > > > > >> > > > > > >> > > field
> > > > > > > >> > > > > > >> > > > in the response. It may be possible to
> > avoid
> > > > > > bumping
> > > > > > > >> the
> > > > > > > >> > > > version
> > > > > > > >> > > > > > >> > without
> > > > > > > >> > > > > > >> > > > that change, but I may be missing
> > something.
> > > > > > > >> > > > > > >> > > > I believe StopReplica is actually on
> > version
> > > 3
> > > > > now,
> > > > > > > but
> > > > > > > >> > > > because
> > > > > > > >> > > > > > >> > version 2
> > > > > > > >> > > > > > >> > > > is flexible, I kept that listed as
> version
> > 2
> > > on
> > > > > the
> > > > > > > KIP
> > > > > > > >> > > page.
> > > > > > > >> > > > > > >> However,
> > > > > > > >> > > > > > >> > > you
> > > > > > > >> > > > > > >> > > > may be right in that we may need to bump
> > the
> > > > > > version
> > > > > > > on
> > > > > > > >> > > > > > StopReplica
> > > > > > > >> > > > > > >> to
> > > > > > > >> > > > > > >> > > deal
> > > > > > > >> > > > > > >> > > > with deletion differently as mentioned
> > > above. I
> > > > > > don't
> > > > > > > >> know
> > > > > > > >> > > if
> > > > > > > >> > > > I
> > > > > > > >> > > > > > >> have a
> > > > > > > >> > > > > > >> > > big
> > > > > > > >> > > > > > >> > > > preference over used tagged fields or
> not.
> > > > > > > >> > > > > > >> > > >
> > > > > > > >> > > > > > >> > > > 13. I was thinking that in the case where
> > the
> > > > > file
> > > > > > > and
> > > > > > > >> the
> > > > > > > >> > > > > request
> > > > > > > >> > > > > > >> > topic
> > > > > > > >> > > > > > >> > > > ids don't match, it means that the
> broker's
> > > > > > topic/the
> > > > > > > >> one
> > > > > > > >> > in
> > > > > > > >> > > > the
> > > > > > > >> > > > > > >> file
> > > > > > > >> > > > > > >> > has
> > > > > > > >> > > > > > >> > > > been deleted. In that case, we would need
> > to
> > > > > delete
> > > > > > > the
> > > > > > > >> > old
> > > > > > > >> > > > > topic
> > > > > > > >> > > > > > >> and
> > > > > > > >> > > > > > >> > > start
> > > > > > > >> > > > > > >> > > > receiving the new version. If the topic
> > name
> > > > were
> > > > > > to
> > > > > > > >> > change,
> > > > > > > >> > > > but
> > > > > > > >> > > > > > the
> > > > > > > >> > > > > > >> > ids
> > > > > > > >> > > > > > >> > > > still matched, the file would also need
> to
> > > > > update.
> > > > > > > Am I
> > > > > > > >> > > > missing
> > > > > > > >> > > > > a
> > > > > > > >> > > > > > >> case
> > > > > > > >> > > > > > >> > > > where the file would be correct and not
> the
> > > > > > request?
> > > > > > > >> > > > > > >> > > >
> > > > > > > >> > > > > > >> > > > Thanks,
> > > > > > > >> > > > > > >> > > > Justine
> > > > > > > >> > > > > > >> > > >
> > > > > > > >> > > > > > >> > > > On Thu, Sep 17, 2020 at 3:18 PM Jun Rao <
> > > > > > > >> jun@confluent.io
> > > > > > > >> > >
> > > > > > > >> > > > > wrote:
> > > > > > > >> > > > > > >> > > >
> > > > > > > >> > > > > > >> > > > > Hi, Justine,
> > > > > > > >> > > > > > >> > > > >
> > > > > > > >> > > > > > >> > > > > Thanks for the reply. A couple of more
> > > > comments
> > > > > > > >> below.
> > > > > > > >> > > > > > >> > > > >
> > > > > > > >> > > > > > >> > > > > 12. ListOffset and OffsetForLeader
> > > currently
> > > > > > don't
> > > > > > > >> > support
> > > > > > > >> > > > > > >> flexible
> > > > > > > >> > > > > > >> > > > fields.
> > > > > > > >> > > > > > >> > > > > So, we have to bump up the version
> number
> > > and
> > > > > use
> > > > > > > >> IBP at
> > > > > > > >> > > > least
> > > > > > > >> > > > > > for
> > > > > > > >> > > > > > >> > > these
> > > > > > > >> > > > > > >> > > > > two requests. Note that it seems 2.7.0
> > will
> > > > > > require
> > > > > > > >> IBP
> > > > > > > >> > > > anyway
> > > > > > > >> > > > > > >> > because
> > > > > > > >> > > > > > >> > > of
> > > > > > > >> > > > > > >> > > > > changes in KAFKA-10435. Also, it seems
> > that
> > > > the
> > > > > > > >> version
> > > > > > > >> > > for
> > > > > > > >> > > > > > >> > > > > LeaderAndIsrRequest and StopReplica are
> > > > bumped
> > > > > > even
> > > > > > > >> > though
> > > > > > > >> > > > we
> > > > > > > >> > > > > > only
> > > > > > > >> > > > > > >> > > added
> > > > > > > >> > > > > > >> > > > a
> > > > > > > >> > > > > > >> > > > > tagged field. But since IBP is needed
> > > anyway,
> > > > > we
> > > > > > > may
> > > > > > > >> > want
> > > > > > > >> > > to
> > > > > > > >> > > > > > >> revisit
> > > > > > > >> > > > > > >> > > the
> > > > > > > >> > > > > > >> > > > > overall tagged field choice.
> > > > > > > >> > > > > > >> > > > >
> > > > > > > >> > > > > > >> > > > > 13. The only downside is the potential
> > > > > confusion
> > > > > > on
> > > > > > > >> > which
> > > > > > > >> > > > one
> > > > > > > >> > > > > is
> > > > > > > >> > > > > > >> the
> > > > > > > >> > > > > > >> > > > source
> > > > > > > >> > > > > > >> > > > > of truth if they don't match. Another
> > > option
> > > > is
> > > > > > to
> > > > > > > >> > include
> > > > > > > >> > > > > those
> > > > > > > >> > > > > > >> > fields
> > > > > > > >> > > > > > >> > > > in
> > > > > > > >> > > > > > >> > > > > the metadata file when we actually
> change
> > > the
> > > > > > > >> directory
> > > > > > > >> > > > > > structure.
> > > > > > > >> > > > > > >> > > > >
> > > > > > > >> > > > > > >> > > > > Thanks,
> > > > > > > >> > > > > > >> > > > >
> > > > > > > >> > > > > > >> > > > > Jun
> > > > > > > >> > > > > > >> > > > >
> > > > > > > >> > > > > > >> > > > > On Thu, Sep 17, 2020 at 2:01 PM Justine
> > > > Olshan
> > > > > <
> > > > > > > >> > > > > > >> jolshan@confluent.io
> > > > > > > >> > > > > > >> > >
> > > > > > > >> > > > > > >> > > > > wrote:
> > > > > > > >> > > > > > >> > > > >
> > > > > > > >> > > > > > >> > > > > > Hello all,
> > > > > > > >> > > > > > >> > > > > >
> > > > > > > >> > > > > > >> > > > > > I've thought some more about removing
> > the
> > > > > topic
> > > > > > > >> name
> > > > > > > >> > > field
> > > > > > > >> > > > > > from
> > > > > > > >> > > > > > >> > some
> > > > > > > >> > > > > > >> > > of
> > > > > > > >> > > > > > >> > > > > the
> > > > > > > >> > > > > > >> > > > > > requests. On closer inspection of the
> > > > > > > >> > > requests/responses,
> > > > > > > >> > > > it
> > > > > > > >> > > > > > >> seems
> > > > > > > >> > > > > > >> > > that
> > > > > > > >> > > > > > >> > > > > the
> > > > > > > >> > > > > > >> > > > > > internal changes would be much larger
> > > than
> > > > I
> > > > > > > >> expected.
> > > > > > > >> > > > Some
> > > > > > > >> > > > > > >> > protocols
> > > > > > > >> > > > > > >> > > > > > involve clients, so they would
> require
> > > > > changes
> > > > > > > too.
> > > > > > > >> > I'm
> > > > > > > >> > > > > > thinking
> > > > > > > >> > > > > > >> > that
> > > > > > > >> > > > > > >> > > > for
> > > > > > > >> > > > > > >> > > > > > now, removing the topic name from
> these
> > > > > > requests
> > > > > > > >> and
> > > > > > > >> > > > > responses
> > > > > > > >> > > > > > >> are
> > > > > > > >> > > > > > >> > > out
> > > > > > > >> > > > > > >> > > > of
> > > > > > > >> > > > > > >> > > > > > scope.
> > > > > > > >> > > > > > >> > > > > >
> > > > > > > >> > > > > > >> > > > > > I have decided to just keep the
> change
> > > > > > > >> > > > LeaderAndIsrResponse
> > > > > > > >> > > > > to
> > > > > > > >> > > > > > >> > remove
> > > > > > > >> > > > > > >> > > > the
> > > > > > > >> > > > > > >> > > > > > topic name, and have updated the KIP
> to
> > > > > reflect
> > > > > > > >> this
> > > > > > > >> > > > > change. I
> > > > > > > >> > > > > > >> have
> > > > > > > >> > > > > > >> > > > also
> > > > > > > >> > > > > > >> > > > > > mentioned the other requests and
> > > responses
> > > > in
> > > > > > > >> future
> > > > > > > >> > > work.
> > > > > > > >> > > > > > >> > > > > >
> > > > > > > >> > > > > > >> > > > > > I'm hoping to start the voting
> process
> > > > soon,
> > > > > so
> > > > > > > >> let me
> > > > > > > >> > > > know
> > > > > > > >> > > > > if
> > > > > > > >> > > > > > >> > there
> > > > > > > >> > > > > > >> > > is
> > > > > > > >> > > > > > >> > > > > > anything else we should discuss.
> > > > > > > >> > > > > > >> > > > > >
> > > > > > > >> > > > > > >> > > > > > Thank you,
> > > > > > > >> > > > > > >> > > > > > Justine
> > > > > > > >> > > > > > >> > > > > >
> > > > > > > >> > > > > > >> > > > > > On Tue, Sep 15, 2020 at 3:57 PM
> Justine
> > > > > Olshan
> > > > > > <
> > > > > > > >> > > > > > >> > jolshan@confluent.io
> > > > > > > >> > > > > > >> > > >
> > > > > > > >> > > > > > >> > > > > > wrote:
> > > > > > > >> > > > > > >> > > > > >
> > > > > > > >> > > > > > >> > > > > > > Hello again,
> > > > > > > >> > > > > > >> > > > > > > To follow up on some of the other
> > > > comments:
> > > > > > > >> > > > > > >> > > > > > >
> > > > > > > >> > > > > > >> > > > > > > 10/11) We can remove the topic name
> > > from
> > > > > > these
> > > > > > > >> > > > > > >> > requests/responses,
> > > > > > > >> > > > > > >> > > > and
> > > > > > > >> > > > > > >> > > > > > > that means we will just have to
> make
> > a
> > > > few
> > > > > > > >> internal
> > > > > > > >> > > > > changes
> > > > > > > >> > > > > > to
> > > > > > > >> > > > > > >> > make
> > > > > > > >> > > > > > >> > > > > > > partitions accessible by topic id
> and
> > > > > > > partition.
> > > > > > > >> I
> > > > > > > >> > can
> > > > > > > >> > > > > > update
> > > > > > > >> > > > > > >> the
> > > > > > > >> > > > > > >> > > KIP
> > > > > > > >> > > > > > >> > > > > to
> > > > > > > >> > > > > > >> > > > > > > remove them unless anyone thinks
> they
> > > > > should
> > > > > > > >> stay.
> > > > > > > >> > > > > > >> > > > > > >
> > > > > > > >> > > > > > >> > > > > > > 12) Addressed in the previous
> email.
> > > I've
> > > > > > > updated
> > > > > > > >> > the
> > > > > > > >> > > > KIP
> > > > > > > >> > > > > to
> > > > > > > >> > > > > > >> > > include
> > > > > > > >> > > > > > >> > > > > > > tagged fields for the requests and
> > > > > responses.
> > > > > > > >> (More
> > > > > > > >> > on
> > > > > > > >> > > > > that
> > > > > > > >> > > > > > >> > below)
> > > > > > > >> > > > > > >> > > > > > >
> > > > > > > >> > > > > > >> > > > > > > 13) I think part of the idea for
> > > > including
> > > > > > this
> > > > > > > >> > > > > information
> > > > > > > >> > > > > > >> is to
> > > > > > > >> > > > > > >> > > > > prepare
> > > > > > > >> > > > > > >> > > > > > > for future changes. Perhaps the
> > > directory
> > > > > > > >> structure
> > > > > > > >> > > > might
> > > > > > > >> > > > > > >> change
> > > > > > > >> > > > > > >> > > from
> > > > > > > >> > > > > > >> > > > > > > topicName_partitionNumber to
> > something
> > > > like
> > > > > > > >> > > > > > >> > > topicID_partitionNumber.
> > > > > > > >> > > > > > >> > > > > Then
> > > > > > > >> > > > > > >> > > > > > > it would be useful to have the
> topic
> > > name
> > > > > in
> > > > > > > the
> > > > > > > >> > file
> > > > > > > >> > > > > since
> > > > > > > >> > > > > > it
> > > > > > > >> > > > > > >> > > would
> > > > > > > >> > > > > > >> > > > > not
> > > > > > > >> > > > > > >> > > > > > be
> > > > > > > >> > > > > > >> > > > > > > in the directory structure.
> > Supporting
> > > > > topic
> > > > > > > >> renames
> > > > > > > >> > > > might
> > > > > > > >> > > > > > be
> > > > > > > >> > > > > > >> > > easier
> > > > > > > >> > > > > > >> > > > if
> > > > > > > >> > > > > > >> > > > > > the
> > > > > > > >> > > > > > >> > > > > > > other fields are included. Would
> > there
> > > be
> > > > > any
> > > > > > > >> > > downsides
> > > > > > > >> > > > to
> > > > > > > >> > > > > > >> > > including
> > > > > > > >> > > > > > >> > > > > this
> > > > > > > >> > > > > > >> > > > > > > information?
> > > > > > > >> > > > > > >> > > > > > >
> > > > > > > >> > > > > > >> > > > > > > 14)  Yes, we would need to copy the
> > > > > partition
> > > > > > > >> > metadata
> > > > > > > >> > > > > file
> > > > > > > >> > > > > > in
> > > > > > > >> > > > > > >> > this
> > > > > > > >> > > > > > >> > > > > > > process. I've updated the KIP to
> > > include
> > > > > > this.
> > > > > > > >> > > > > > >> > > > > > >
> > > > > > > >> > > > > > >> > > > > > > 15) I believe Lucas meant v1 and v2
> > > here.
> > > > > He
> > > > > > > was
> > > > > > > >> > > > referring
> > > > > > > >> > > > > > to
> > > > > > > >> > > > > > >> how
> > > > > > > >> > > > > > >> > > the
> > > > > > > >> > > > > > >> > > > > > > requests would fall under different
> > IBP
> > > > and
> > > > > > > meant
> > > > > > > >> > that
> > > > > > > >> > > > > older
> > > > > > > >> > > > > > >> > > brokers
> > > > > > > >> > > > > > >> > > > > > would
> > > > > > > >> > > > > > >> > > > > > > have to use the older version of
> the
> > > > > request
> > > > > > > and
> > > > > > > >> the
> > > > > > > >> > > > > > existing
> > > > > > > >> > > > > > >> > topic
> > > > > > > >> > > > > > >> > > > > > > deletion process. At first, it
> seemed
> > > > like
> > > > > > > tagged
> > > > > > > >> > > fields
> > > > > > > >> > > > > > would
> > > > > > > >> > > > > > >> > > > resolve
> > > > > > > >> > > > > > >> > > > > > > the IBP issue. However, we may need
> > IBP
> > > > for
> > > > > > > this
> > > > > > > >> > > request
> > > > > > > >> > > > > > after
> > > > > > > >> > > > > > >> > all
> > > > > > > >> > > > > > >> > > > > since
> > > > > > > >> > > > > > >> > > > > > > the controller handles the topic
> > > deletion
> > > > > > > >> > differently
> > > > > > > >> > > > > > >> depending
> > > > > > > >> > > > > > >> > on
> > > > > > > >> > > > > > >> > > > the
> > > > > > > >> > > > > > >> > > > > > IBP
> > > > > > > >> > > > > > >> > > > > > > version. In an older version, we
> > can't
> > > > just
> > > > > > > send
> > > > > > > >> a
> > > > > > > >> > > > > > StopReplica
> > > > > > > >> > > > > > >> > > delete
> > > > > > > >> > > > > > >> > > > > the
> > > > > > > >> > > > > > >> > > > > > > topic immediately like we'd want to
> > for
> > > > > this
> > > > > > > KIP.
> > > > > > > >> > > > > > >> > > > > > >
> > > > > > > >> > > > > > >> > > > > > > This makes me wonder if we want
> > tagged
> > > > > fields
> > > > > > > on
> > > > > > > >> all
> > > > > > > >> > > the
> > > > > > > >> > > > > > >> requests
> > > > > > > >> > > > > > >> > > > after
> > > > > > > >> > > > > > >> > > > > > > all. Let me know your thoughts!
> > > > > > > >> > > > > > >> > > > > > >
> > > > > > > >> > > > > > >> > > > > > > Justine
> > > > > > > >> > > > > > >> > > > > > >
> > > > > > > >> > > > > > >> > > > > > > On Tue, Sep 15, 2020 at 1:03 PM
> > Justine
> > > > > > Olshan
> > > > > > > <
> > > > > > > >> > > > > > >> > > jolshan@confluent.io
> > > > > > > >> > > > > > >> > > > >
> > > > > > > >> > > > > > >> > > > > > > wrote:
> > > > > > > >> > > > > > >> > > > > > >
> > > > > > > >> > > > > > >> > > > > > >> Hi all,
> > > > > > > >> > > > > > >> > > > > > >> Jun brought up a good point in his
> > > last
> > > > > > email
> > > > > > > >> about
> > > > > > > >> > > > > tagged
> > > > > > > >> > > > > > >> > fields,
> > > > > > > >> > > > > > >> > > > and
> > > > > > > >> > > > > > >> > > > > > >> I've updated the KIP to reflect
> that
> > > the
> > > > > > > >> changes to
> > > > > > > >> > > > > > requests
> > > > > > > >> > > > > > >> and
> > > > > > > >> > > > > > >> > > > > > responses
> > > > > > > >> > > > > > >> > > > > > >> will be in the form of tagged
> fields
> > > to
> > > > > > avoid
> > > > > > > >> > > changing
> > > > > > > >> > > > > IBP.
> > > > > > > >> > > > > > >> > > > > > >>
> > > > > > > >> > > > > > >> > > > > > >> Jun: I plan on sending a followup
> > > email
> > > > to
> > > > > > > >> address
> > > > > > > >> > > some
> > > > > > > >> > > > > of
> > > > > > > >> > > > > > >> the
> > > > > > > >> > > > > > >> > > other
> > > > > > > >> > > > > > >> > > > > > >> points.
> > > > > > > >> > > > > > >> > > > > > >>
> > > > > > > >> > > > > > >> > > > > > >> Thanks,
> > > > > > > >> > > > > > >> > > > > > >> Justine
> > > > > > > >> > > > > > >> > > > > > >>
> > > > > > > >> > > > > > >> > > > > > >> On Mon, Sep 14, 2020 at 4:25 PM
> Jun
> > > Rao
> > > > <
> > > > > > > >> > > > > jun@confluent.io>
> > > > > > > >> > > > > > >> > wrote:
> > > > > > > >> > > > > > >> > > > > > >>
> > > > > > > >> > > > > > >> > > > > > >>> Hi, Justine,
> > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > >> > > > > > >> > > > > > >>> Thanks for the updated KIP. A few
> > > > > comments
> > > > > > > >> below.
> > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > >> > > > > > >> > > > > > >>> 10. LeaderAndIsr Response: Do we
> > need
> > > > the
> > > > > > > topic
> > > > > > > >> > > name?
> > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > >> > > > > > >> > > > > > >>> 11. For the changed
> > request/response,
> > > > > other
> > > > > > > >> than
> > > > > > > >> > > > > > >> LeaderAndIsr,
> > > > > > > >> > > > > > >> > > > > > >>> UpdateMetadata, Metadata, do we
> > need
> > > to
> > > > > > > include
> > > > > > > >> > the
> > > > > > > >> > > > > topic
> > > > > > > >> > > > > > >> name?
> > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > >> > > > > > >> > > > > > >>> 12. It seems that upgrades don't
> > > > require
> > > > > > IBP.
> > > > > > > >> Does
> > > > > > > >> > > > that
> > > > > > > >> > > > > > mean
> > > > > > > >> > > > > > >> > the
> > > > > > > >> > > > > > >> > > > new
> > > > > > > >> > > > > > >> > > > > > >>> fields
> > > > > > > >> > > > > > >> > > > > > >>> in all the request/response are
> > added
> > > > as
> > > > > > > tagged
> > > > > > > >> > > fields
> > > > > > > >> > > > > > >> without
> > > > > > > >> > > > > > >> > > > > bumping
> > > > > > > >> > > > > > >> > > > > > up
> > > > > > > >> > > > > > >> > > > > > >>> the request version? It would be
> > > useful
> > > > > to
> > > > > > > make
> > > > > > > >> > that
> > > > > > > >> > > > > > clear.
> > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > >> > > > > > >> > > > > > >>> 13. Partition Metadata file: Do
> we
> > > need
> > > > > to
> > > > > > > >> include
> > > > > > > >> > > the
> > > > > > > >> > > > > > topic
> > > > > > > >> > > > > > >> > name
> > > > > > > >> > > > > > >> > > > and
> > > > > > > >> > > > > > >> > > > > > the
> > > > > > > >> > > > > > >> > > > > > >>> partition id since they are
> implied
> > > in
> > > > > the
> > > > > > > >> > directory
> > > > > > > >> > > > > name?
> > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > >> > > > > > >> > > > > > >>> 14. In the JBOD mode, we support
> > > > moving a
> > > > > > > >> > > partition's
> > > > > > > >> > > > > data
> > > > > > > >> > > > > > >> from
> > > > > > > >> > > > > > >> > > one
> > > > > > > >> > > > > > >> > > > > > disk
> > > > > > > >> > > > > > >> > > > > > >>> to
> > > > > > > >> > > > > > >> > > > > > >>> another. Will the new partition
> > > > metadata
> > > > > > file
> > > > > > > >> be
> > > > > > > >> > > > copied
> > > > > > > >> > > > > > >> during
> > > > > > > >> > > > > > >> > > that
> > > > > > > >> > > > > > >> > > > > > >>> process?
> > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > >> > > > > > >> > > > > > >>> 15. The KIP says "Remove deleted
> > > topics
> > > > > > from
> > > > > > > >> > > replicas
> > > > > > > >> > > > by
> > > > > > > >> > > > > > >> > sending
> > > > > > > >> > > > > > >> > > > > > >>> StopReplicaRequest V2 for any
> > topics
> > > > > which
> > > > > > do
> > > > > > > >> not
> > > > > > > >> > > > > contain
> > > > > > > >> > > > > > a
> > > > > > > >> > > > > > >> > topic
> > > > > > > >> > > > > > >> > > > ID,
> > > > > > > >> > > > > > >> > > > > > and
> > > > > > > >> > > > > > >> > > > > > >>> V3 for any topics which do
> contain
> > a
> > > > > topic
> > > > > > > >> ID.".
> > > > > > > >> > > > > However,
> > > > > > > >> > > > > > it
> > > > > > > >> > > > > > >> > > seems
> > > > > > > >> > > > > > >> > > > > the
> > > > > > > >> > > > > > >> > > > > > >>> updated controller will create
> all
> > > > > missing
> > > > > > > >> topic
> > > > > > > >> > IDs
> > > > > > > >> > > > > first
> > > > > > > >> > > > > > >> > before
> > > > > > > >> > > > > > >> > > > > doing
> > > > > > > >> > > > > > >> > > > > > >>> other actions. So, is
> > > > StopReplicaRequest
> > > > > V2
> > > > > > > >> > needed?
> > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > >> > > > > > >> > > > > > >>> Jun
> > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > >> > > > > > >> > > > > > >>> On Fri, Sep 11, 2020 at 10:31 AM
> > John
> > > > > > > Roesler <
> > > > > > > >> > > > > > >> > > vvcephei@apache.org
> > > > > > > >> > > > > > >> > > > >
> > > > > > > >> > > > > > >> > > > > > >>> wrote:
> > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > >> > > > > > >> > > > > > >>> > Thanks, Justine!
> > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > >> > > > > > >> > > > > > >>> > Your response seems compelling
> to
> > > me.
> > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > >> > > > > > >> > > > > > >>> > -John
> > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > >> > > > > > >> > > > > > >>> > On Fri, 2020-09-11 at 10:17
> > -0700,
> > > > > > Justine
> > > > > > > >> > Olshan
> > > > > > > >> > > > > wrote:
> > > > > > > >> > > > > > >> > > > > > >>> > > Hello all,
> > > > > > > >> > > > > > >> > > > > > >>> > > Thanks for continuing the
> > > > > discussion! I
> > > > > > > >> have a
> > > > > > > >> > > few
> > > > > > > >> > > > > > >> > responses
> > > > > > > >> > > > > > >> > > to
> > > > > > > >> > > > > > >> > > > > > your
> > > > > > > >> > > > > > >> > > > > > >>> > points.
> > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > >> > > > > > >> > > > > > >>> > > Tom: You are correct in that
> > this
> > > > KIP
> > > > > > has
> > > > > > > >> not
> > > > > > > >> > > > > > mentioned
> > > > > > > >> > > > > > >> the
> > > > > > > >> > > > > > >> > > > > > >>> > > DeleteTopicsRequest. I think
> > that
> > > > > this
> > > > > > > >> would
> > > > > > > >> > be
> > > > > > > >> > > > out
> > > > > > > >> > > > > of
> > > > > > > >> > > > > > >> > scope
> > > > > > > >> > > > > > >> > > > for
> > > > > > > >> > > > > > >> > > > > > >>> now, but
> > > > > > > >> > > > > > >> > > > > > >>> > > may be something worth adding
> > in
> > > > the
> > > > > > > >> future.
> > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > >> > > > > > >> > > > > > >>> > > John: We did consider
> sequence
> > > ids,
> > > > > but
> > > > > > > >> there
> > > > > > > >> > > are
> > > > > > > >> > > > a
> > > > > > > >> > > > > > few
> > > > > > > >> > > > > > >> > > reasons
> > > > > > > >> > > > > > >> > > > > to
> > > > > > > >> > > > > > >> > > > > > >>> favor
> > > > > > > >> > > > > > >> > > > > > >>> > > UUIDs. There are several
> cases
> > > > where
> > > > > > > topics
> > > > > > > >> > from
> > > > > > > >> > > > > > >> different
> > > > > > > >> > > > > > >> > > > > clusters
> > > > > > > >> > > > > > >> > > > > > >>> may
> > > > > > > >> > > > > > >> > > > > > >>> > > interact now and in the
> future.
> > > For
> > > > > > > >> example,
> > > > > > > >> > > > Mirror
> > > > > > > >> > > > > > >> Maker 2
> > > > > > > >> > > > > > >> > > may
> > > > > > > >> > > > > > >> > > > > > >>> benefit
> > > > > > > >> > > > > > >> > > > > > >>> > > from being able to detect
> when
> > a
> > > > > > cluster
> > > > > > > >> being
> > > > > > > >> > > > > > mirrored
> > > > > > > >> > > > > > >> is
> > > > > > > >> > > > > > >> > > > > deleted
> > > > > > > >> > > > > > >> > > > > > >>> and
> > > > > > > >> > > > > > >> > > > > > >>> > > recreated and globally unique
> > > > > > identifiers
> > > > > > > >> > would
> > > > > > > >> > > > make
> > > > > > > >> > > > > > >> > > resolving
> > > > > > > >> > > > > > >> > > > > > issues
> > > > > > > >> > > > > > >> > > > > > >>> > > easier than sequence IDs
> which
> > > may
> > > > > > > collide
> > > > > > > >> > > between
> > > > > > > >> > > > > > >> > clusters.
> > > > > > > >> > > > > > >> > > > > > KIP-405
> > > > > > > >> > > > > > >> > > > > > >>> > > (tiered storage) will also
> > > benefit
> > > > > from
> > > > > > > >> > globally
> > > > > > > >> > > > > > unique
> > > > > > > >> > > > > > >> IDs
> > > > > > > >> > > > > > >> > > as
> > > > > > > >> > > > > > >> > > > > > shared
> > > > > > > >> > > > > > >> > > > > > >>> > > buckets may be used between
> > > > clusters.
> > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > >> > > > > > >> > > > > > >>> > > Globally unique IDs would
> also
> > > make
> > > > > > > >> > > functionality
> > > > > > > >> > > > > like
> > > > > > > >> > > > > > >> > moving
> > > > > > > >> > > > > > >> > > > > > topics
> > > > > > > >> > > > > > >> > > > > > >>> > > between disparate clusters
> > easier
> > > > in
> > > > > > the
> > > > > > > >> > future,
> > > > > > > >> > > > > > >> simplify
> > > > > > > >> > > > > > >> > any
> > > > > > > >> > > > > > >> > > > > > future
> > > > > > > >> > > > > > >> > > > > > >>> > > implementations of backups
> and
> > > > > > restores,
> > > > > > > >> and
> > > > > > > >> > > more.
> > > > > > > >> > > > > In
> > > > > > > >> > > > > > >> > > general,
> > > > > > > >> > > > > > >> > > > > > >>> unique IDs
> > > > > > > >> > > > > > >> > > > > > >>> > > would ensure that the source
> > > > cluster
> > > > > > > >> topics do
> > > > > > > >> > > not
> > > > > > > >> > > > > > >> conflict
> > > > > > > >> > > > > > >> > > > with
> > > > > > > >> > > > > > >> > > > > > the
> > > > > > > >> > > > > > >> > > > > > >>> > > destination cluster topics.
> > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > >> > > > > > >> > > > > > >>> > > If we were to use sequence
> ids,
> > > we
> > > > > > would
> > > > > > > >> need
> > > > > > > >> > > > > > >> sufficiently
> > > > > > > >> > > > > > >> > > > large
> > > > > > > >> > > > > > >> > > > > > >>> cluster
> > > > > > > >> > > > > > >> > > > > > >>> > > ids to be stored with the
> topic
> > > > > > > >> identifiers or
> > > > > > > >> > > we
> > > > > > > >> > > > > run
> > > > > > > >> > > > > > >> the
> > > > > > > >> > > > > > >> > > risk
> > > > > > > >> > > > > > >> > > > of
> > > > > > > >> > > > > > >> > > > > > >>> > > collisions. This will give up
> > any
> > > > > > > >> advantage in
> > > > > > > >> > > > > > >> compactness
> > > > > > > >> > > > > > >> > > that
> > > > > > > >> > > > > > >> > > > > > >>> sequence
> > > > > > > >> > > > > > >> > > > > > >>> > > numbers may bring. Given
> these
> > > > > > > advantages I
> > > > > > > >> > > think
> > > > > > > >> > > > it
> > > > > > > >> > > > > > >> makes
> > > > > > > >> > > > > > >> > > > sense
> > > > > > > >> > > > > > >> > > > > to
> > > > > > > >> > > > > > >> > > > > > >>> use
> > > > > > > >> > > > > > >> > > > > > >>> > > UUIDs.
> > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > >> > > > > > >> > > > > > >>> > > Gokul: This is an interesting
> > > idea,
> > > > > but
> > > > > > > >> this
> > > > > > > >> > is
> > > > > > > >> > > a
> > > > > > > >> > > > > > >> breaking
> > > > > > > >> > > > > > >> > > > > change.
> > > > > > > >> > > > > > >> > > > > > >>> Out of
> > > > > > > >> > > > > > >> > > > > > >>> > > scope for now, but maybe
> worth
> > > > > > discussing
> > > > > > > >> in
> > > > > > > >> > the
> > > > > > > >> > > > > > future.
> > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > >> > > > > > >> > > > > > >>> > > Hope this explains some of
> the
> > > > > > decisions,
> > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > >> > > > > > >> > > > > > >>> > > Justine
> > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > >> > > > > > >> > > > > > >>> > > On Fri, Sep 11, 2020 at 8:27
> AM
> > > > Gokul
> > > > > > > >> Ramanan
> > > > > > > >> > > > > > >> Subramanian <
> > > > > > > >> > > > > > >> > > > > > >>> > > gokul2411s@gmail.com> wrote:
> > > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > > >> > > > > > >> > > > > > >>> > > > Hi.
> > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > > > >> > > > > > >> > > > > > >>> > > > Thanks for the KIP.
> > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > > > >> > > > > > >> > > > > > >>> > > > Have you thought about
> > whether
> > > it
> > > > > > makes
> > > > > > > >> > sense
> > > > > > > >> > > to
> > > > > > > >> > > > > > >> support
> > > > > > > >> > > > > > >> > > > > > >>> authorizing a
> > > > > > > >> > > > > > >> > > > > > >>> > > > principal for a topic ID
> > rather
> > > > > than
> > > > > > a
> > > > > > > >> topic
> > > > > > > >> > > > name
> > > > > > > >> > > > > to
> > > > > > > >> > > > > > >> > > achieve
> > > > > > > >> > > > > > >> > > > > > >>> tighter
> > > > > > > >> > > > > > >> > > > > > >>> > > > security?
> > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > > > >> > > > > > >> > > > > > >>> > > > Or is the topic ID
> > > fundamentally
> > > > an
> > > > > > > >> internal
> > > > > > > >> > > > > detail
> > > > > > > >> > > > > > >> > similar
> > > > > > > >> > > > > > >> > > > to
> > > > > > > >> > > > > > >> > > > > > >>> epochs
> > > > > > > >> > > > > > >> > > > > > >>> > used
> > > > > > > >> > > > > > >> > > > > > >>> > > > in a bunch of other places
> in
> > > > > Kafka?
> > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > > > >> > > > > > >> > > > > > >>> > > > Thanks.
> > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > > > >> > > > > > >> > > > > > >>> > > > On Fri, Sep 11, 2020 at
> 4:06
> > PM
> > > > > John
> > > > > > > >> > Roesler <
> > > > > > > >> > > > > > >> > > > > > vvcephei@apache.org>
> > > > > > > >> > > > > > >> > > > > > >>> > wrote:
> > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > > > >> > > > > > >> > > > > > >>> > > > > Hello Justine,
> > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > > >> > > > > > >> > > > > > >>> > > > > Thanks for the KIP!
> > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > > >> > > > > > >> > > > > > >>> > > > > I happen to have been
> > > > confronted
> > > > > > > >> recently
> > > > > > > >> > > with
> > > > > > > >> > > > > the
> > > > > > > >> > > > > > >> need
> > > > > > > >> > > > > > >> > > to
> > > > > > > >> > > > > > >> > > > > keep
> > > > > > > >> > > > > > >> > > > > > >>> > track of
> > > > > > > >> > > > > > >> > > > > > >>> > > > a
> > > > > > > >> > > > > > >> > > > > > >>> > > > > large number of topics as
> > > > > compactly
> > > > > > > as
> > > > > > > >> > > > > possible. I
> > > > > > > >> > > > > > >> was
> > > > > > > >> > > > > > >> > > > going
> > > > > > > >> > > > > > >> > > > > to
> > > > > > > >> > > > > > >> > > > > > >>> come
> > > > > > > >> > > > > > >> > > > > > >>> > up
> > > > > > > >> > > > > > >> > > > > > >>> > > > > with some way to
> dictionary
> > > > > encode
> > > > > > > the
> > > > > > > >> > topic
> > > > > > > >> > > > > names
> > > > > > > >> > > > > > >> as
> > > > > > > >> > > > > > >> > > > > integers,
> > > > > > > >> > > > > > >> > > > > > >>> but
> > > > > > > >> > > > > > >> > > > > > >>> > this
> > > > > > > >> > > > > > >> > > > > > >>> > > > > seems much better!
> > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > > >> > > > > > >> > > > > > >>> > > > > Apologies if this has
> been
> > > > raised
> > > > > > > >> before,
> > > > > > > >> > > but
> > > > > > > >> > > > > I’m
> > > > > > > >> > > > > > >> > > wondering
> > > > > > > >> > > > > > >> > > > > > >>> about the
> > > > > > > >> > > > > > >> > > > > > >>> > > > > choice of UUID vs
> sequence
> > > > number
> > > > > > for
> > > > > > > >> the
> > > > > > > >> > > ids.
> > > > > > > >> > > > > > >> > Typically,
> > > > > > > >> > > > > > >> > > > > I’ve
> > > > > > > >> > > > > > >> > > > > > >>> seen
> > > > > > > >> > > > > > >> > > > > > >>> > UUIDs
> > > > > > > >> > > > > > >> > > > > > >>> > > > > in two situations:
> > > > > > > >> > > > > > >> > > > > > >>> > > > > 1. When processes need to
> > > > > generate
> > > > > > > >> > > > non-colliding
> > > > > > > >> > > > > > >> > > > identifiers
> > > > > > > >> > > > > > >> > > > > > >>> without
> > > > > > > >> > > > > > >> > > > > > >>> > > > > coordination.
> > > > > > > >> > > > > > >> > > > > > >>> > > > > 2. When the identifier
> > needs
> > > to
> > > > > be
> > > > > > > >> > > > “universally
> > > > > > > >> > > > > > >> > unique”;
> > > > > > > >> > > > > > >> > > > > I.e.,
> > > > > > > >> > > > > > >> > > > > > >>> the
> > > > > > > >> > > > > > >> > > > > > >>> > > > > identifier needs to
> > > distinguish
> > > > > the
> > > > > > > >> entity
> > > > > > > >> > > > from
> > > > > > > >> > > > > > all
> > > > > > > >> > > > > > >> > other
> > > > > > > >> > > > > > >> > > > > > >>> entities
> > > > > > > >> > > > > > >> > > > > > >>> > that
> > > > > > > >> > > > > > >> > > > > > >>> > > > > could ever exist. This is
> > > > useful
> > > > > in
> > > > > > > >> cases
> > > > > > > >> > > > where
> > > > > > > >> > > > > > >> > entities
> > > > > > > >> > > > > > >> > > > from
> > > > > > > >> > > > > > >> > > > > > all
> > > > > > > >> > > > > > >> > > > > > >>> > kinds
> > > > > > > >> > > > > > >> > > > > > >>> > > > of
> > > > > > > >> > > > > > >> > > > > > >>> > > > > systems get mixed
> together,
> > > > such
> > > > > as
> > > > > > > >> when
> > > > > > > >> > > > dumping
> > > > > > > >> > > > > > >> logs
> > > > > > > >> > > > > > >> > > from
> > > > > > > >> > > > > > >> > > > > all
> > > > > > > >> > > > > > >> > > > > > >>> > processes
> > > > > > > >> > > > > > >> > > > > > >>> > > > in
> > > > > > > >> > > > > > >> > > > > > >>> > > > > a company into a common
> > > system.
> > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > > >> > > > > > >> > > > > > >>> > > > > Maybe I’m being
> > > short-sighted,
> > > > > but
> > > > > > it
> > > > > > > >> > > doesn’t
> > > > > > > >> > > > > seem
> > > > > > > >> > > > > > >> like
> > > > > > > >> > > > > > >> > > > > either
> > > > > > > >> > > > > > >> > > > > > >>> really
> > > > > > > >> > > > > > >> > > > > > >>> > > > > applies here. It seems
> like
> > > the
> > > > > > > brokers
> > > > > > > >> > > could
> > > > > > > >> > > > > and
> > > > > > > >> > > > > > >> would
> > > > > > > >> > > > > > >> > > > > achieve
> > > > > > > >> > > > > > >> > > > > > >>> > consensus
> > > > > > > >> > > > > > >> > > > > > >>> > > > > when creating a topic
> > anyway,
> > > > > which
> > > > > > > is
> > > > > > > >> all
> > > > > > > >> > > > > that’s
> > > > > > > >> > > > > > >> > > required
> > > > > > > >> > > > > > >> > > > to
> > > > > > > >> > > > > > >> > > > > > >>> > generate
> > > > > > > >> > > > > > >> > > > > > >>> > > > > non-colliding sequence
> ids.
> > > For
> > > > > the
> > > > > > > >> > second,
> > > > > > > >> > > as
> > > > > > > >> > > > > you
> > > > > > > >> > > > > > >> > > mention,
> > > > > > > >> > > > > > >> > > > > we
> > > > > > > >> > > > > > >> > > > > > >>> could
> > > > > > > >> > > > > > >> > > > > > >>> > > > assign
> > > > > > > >> > > > > > >> > > > > > >>> > > > > a UUID to the cluster as
> a
> > > > whole,
> > > > > > > which
> > > > > > > >> > > would
> > > > > > > >> > > > > > render
> > > > > > > >> > > > > > >> > any
> > > > > > > >> > > > > > >> > > > > > resource
> > > > > > > >> > > > > > >> > > > > > >>> > scoped
> > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > > > > > > >> > > > > > >> > > > > > >>> > > > > the broker universally
> > unique
> > > > as
> > > > > > > well.
> > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > > >> > > > > > >> > > > > > >>> > > > > The reason I mention this
> > is
> > > > > that,
> > > > > > > >> > although
> > > > > > > >> > > a
> > > > > > > >> > > > > UUID
> > > > > > > >> > > > > > >> is
> > > > > > > >> > > > > > >> > way
> > > > > > > >> > > > > > >> > > > > more
> > > > > > > >> > > > > > >> > > > > > >>> > compact
> > > > > > > >> > > > > > >> > > > > > >>> > > > > than topic names, it’s
> > still
> > > 16
> > > > > > > bytes.
> > > > > > > >> In
> > > > > > > >> > > > > > contrast,
> > > > > > > >> > > > > > >> a
> > > > > > > >> > > > > > >> > > > 4-byte
> > > > > > > >> > > > > > >> > > > > > >>> integer
> > > > > > > >> > > > > > >> > > > > > >>> > > > > sequence id would give
> us 4
> > > > > billion
> > > > > > > >> unique
> > > > > > > >> > > > > topics
> > > > > > > >> > > > > > >> per
> > > > > > > >> > > > > > >> > > > > cluster,
> > > > > > > >> > > > > > >> > > > > > >>> which
> > > > > > > >> > > > > > >> > > > > > >>> > > > seems
> > > > > > > >> > > > > > >> > > > > > >>> > > > > like enough ;)
> > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > > >> > > > > > >> > > > > > >>> > > > > Considering the number of
> > > > > different
> > > > > > > >> times
> > > > > > > >> > > > these
> > > > > > > >> > > > > > >> topic
> > > > > > > >> > > > > > >> > > > > > >>> identifiers are
> > > > > > > >> > > > > > >> > > > > > >>> > > > sent
> > > > > > > >> > > > > > >> > > > > > >>> > > > > over the wire or stored
> in
> > > > > memory,
> > > > > > it
> > > > > > > >> > seems
> > > > > > > >> > > > like
> > > > > > > >> > > > > > it
> > > > > > > >> > > > > > >> > might
> > > > > > > >> > > > > > >> > > > be
> > > > > > > >> > > > > > >> > > > > > >>> worth
> > > > > > > >> > > > > > >> > > > > > >>> > the
> > > > > > > >> > > > > > >> > > > > > >>> > > > > additional 4x space
> > savings.
> > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > > >> > > > > > >> > > > > > >>> > > > > What do you think about
> > this?
> > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > > >> > > > > > >> > > > > > >>> > > > > Thanks,
> > > > > > > >> > > > > > >> > > > > > >>> > > > > John
> > > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > > >> > > > > > >> > > > > > >>> > > > > On Fri, Sep 11, 2020, at
> > > 03:20,
> > > > > Tom
> > > > > > > >> > Bentley
> > > > > > > >> > > > > wrote:
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > Hi Justine,
> > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > This looks like a very
> > > > welcome
> > > > > > > >> > > improvement.
> > > > > > > >> > > > > > >> Thanks!
> > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > Maybe I missed it, but
> > the
> > > > KIP
> > > > > > > >> doesn't
> > > > > > > >> > > seem
> > > > > > > >> > > > to
> > > > > > > >> > > > > > >> > mention
> > > > > > > >> > > > > > >> > > > > > changing
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > DeleteTopicsRequest to
> > > > identify
> > > > > > the
> > > > > > > >> > topic
> > > > > > > >> > > > > using
> > > > > > > >> > > > > > an
> > > > > > > >> > > > > > >> > id.
> > > > > > > >> > > > > > >> > > > > Maybe
> > > > > > > >> > > > > > >> > > > > > >>> > that's out
> > > > > > > >> > > > > > >> > > > > > >>> > > > > of
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > scope, but
> > > > DeleteTopicsRequest
> > > > > is
> > > > > > > not
> > > > > > > >> > > listed
> > > > > > > >> > > > > > among
> > > > > > > >> > > > > > >> > the
> > > > > > > >> > > > > > >> > > > > Future
> > > > > > > >> > > > > > >> > > > > > >>> Work
> > > > > > > >> > > > > > >> > > > > > >>> > APIs
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > either.
> > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > Kind regards,
> > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > Tom
> > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > On Thu, Sep 10, 2020 at
> > > 3:59
> > > > PM
> > > > > > > >> Satish
> > > > > > > >> > > > > Duggana <
> > > > > > > >> > > > > > >> > > > > > >>> > > > satish.duggana@gmail.com
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > wrote:
> > > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Thanks Lucas/Justine
> > for
> > > > the
> > > > > > nice
> > > > > > > >> KIP.
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > It has several
> benefits
> > > > which
> > > > > > > also
> > > > > > > >> > > include
> > > > > > > >> > > > > > >> > > simplifying
> > > > > > > >> > > > > > >> > > > > the
> > > > > > > >> > > > > > >> > > > > > >>> topic
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > deletion process by
> > > > > controller
> > > > > > > and
> > > > > > > >> > logs
> > > > > > > >> > > > > > cleanup
> > > > > > > >> > > > > > >> by
> > > > > > > >> > > > > > >> > > > > brokers
> > > > > > > >> > > > > > >> > > > > > in
> > > > > > > >> > > > > > >> > > > > > >>> > corner
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > cases.
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Best,
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Satish.
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > On Wed, Sep 9, 2020
> at
> > > > 10:07
> > > > > PM
> > > > > > > >> > Justine
> > > > > > > >> > > > > > Olshan <
> > > > > > > >> > > > > > >> > > > > > >>> > jolshan@confluent.io
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > wrote:
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Hello all, it's
> been
> > > > > almost a
> > > > > > > >> year!
> > > > > > > >> > > I've
> > > > > > > >> > > > > > made
> > > > > > > >> > > > > > >> > some
> > > > > > > >> > > > > > >> > > > > > changes
> > > > > > > >> > > > > > >> > > > > > >>> to
> > > > > > > >> > > > > > >> > > > > > >>> > this
> > > > > > > >> > > > > > >> > > > > > >>> > > > > KIP
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > and hope to continue
> > the
> > > > > > > >> discussion.
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > One of the main
> > changes
> > > > > I've
> > > > > > > >> added
> > > > > > > >> > is
> > > > > > > >> > > > now
> > > > > > > >> > > > > > the
> > > > > > > >> > > > > > >> > > > metadata
> > > > > > > >> > > > > > >> > > > > > >>> response
> > > > > > > >> > > > > > >> > > > > > >>> > > > will
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > include the topic ID
> > (as
> > > > > Colin
> > > > > > > >> > > suggested).
> > > > > > > >> > > > > > >> Clients
> > > > > > > >> > > > > > >> > > can
> > > > > > > >> > > > > > >> > > > > > >>> obtain the
> > > > > > > >> > > > > > >> > > > > > >>> > > > > topicID
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > of a given topic
> > through
> > > a
> > > > > > > >> > > > TopicDescription.
> > > > > > > >> > > > > > The
> > > > > > > >> > > > > > >> > > > topicId
> > > > > > > >> > > > > > >> > > > > > will
> > > > > > > >> > > > > > >> > > > > > >>> > also be
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > included with the
> > > > > > UpdateMetadata
> > > > > > > >> > > request.
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Let me know what
> you
> > > all
> > > > > > think.
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Thank you,
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Justine
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > On 2019/09/13
> > 16:38:26,
> > > > > > "Colin
> > > > > > > >> > > McCabe" <
> > > > > > > >> > > > > > >> > > > > > cmccabe@apache.org
> > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > >> > > > > > >> > > > > > >>> > wrote:
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Hi Lucas,
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Thanks for
> tackling
> > > > this.
> > > > > > > >> Topic
> > > > > > > >> > IDs
> > > > > > > >> > > > > are a
> > > > > > > >> > > > > > >> > great
> > > > > > > >> > > > > > >> > > > > idea,
> > > > > > > >> > > > > > >> > > > > > >>> and
> > > > > > > >> > > > > > >> > > > > > >>> > this
> > > > > > > >> > > > > > >> > > > > > >>> > > > is
> > > > > > > >> > > > > > >> > > > > > >>> > > > > a
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > really good writeup.
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > For
> > > > > > /brokers/topics/[topic],
> > > > > > > >> the
> > > > > > > >> > > > schema
> > > > > > > >> > > > > > >> version
> > > > > > > >> > > > > > >> > > > > should
> > > > > > > >> > > > > > >> > > > > > be
> > > > > > > >> > > > > > >> > > > > > >>> > bumped
> > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > version 3, rather
> than
> > 2.
> > > > > > > KIP-455
> > > > > > > >> > > bumped
> > > > > > > >> > > > > the
> > > > > > > >> > > > > > >> > version
> > > > > > > >> > > > > > >> > > > of
> > > > > > > >> > > > > > >> > > > > > this
> > > > > > > >> > > > > > >> > > > > > >>> > znode
> > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > > > > > > >> > > > > > >> > > > > > >>> > > > > 2
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > already :)
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Given that we're
> > > going
> > > > to
> > > > > > be
> > > > > > > >> > seeing
> > > > > > > >> > > > > these
> > > > > > > >> > > > > > >> > things
> > > > > > > >> > > > > > >> > > as
> > > > > > > >> > > > > > >> > > > > > >>> strings
> > > > > > > >> > > > > > >> > > > > > >>> > as
> > > > > > > >> > > > > > >> > > > > > >>> > > > lot
> > > > > > > >> > > > > > >> > > > > > >>> > > > > (in
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > logs, in ZooKeeper,
> on
> > > the
> > > > > > > >> > command-line,
> > > > > > > >> > > > > > etc.),
> > > > > > > >> > > > > > >> > does
> > > > > > > >> > > > > > >> > > it
> > > > > > > >> > > > > > >> > > > > > make
> > > > > > > >> > > > > > >> > > > > > >>> > sense to
> > > > > > > >> > > > > > >> > > > > > >>> > > > > use
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > base64 when
> converting
> > > them
> > > > > to
> > > > > > > >> > strings?
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Here is an
> example
> > of
> > > > the
> > > > > > hex
> > > > > > > >> > > > > > >> representation:
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > > > > > >> > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > And here is an
> > > example
> > > > in
> > > > > > > >> base64.
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > b8tRS7h4TJ2Vt43Dp85v2A
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > The base64
> version
> > > > saves
> > > > > 15
> > > > > > > >> > letters
> > > > > > > >> > > > (to
> > > > > > > >> > > > > be
> > > > > > > >> > > > > > >> > fair,
> > > > > > > >> > > > > > >> > > 4
> > > > > > > >> > > > > > >> > > > of
> > > > > > > >> > > > > > >> > > > > > >>> those
> > > > > > > >> > > > > > >> > > > > > >>> > were
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > dashes that we could
> > have
> > > > > > elided
> > > > > > > in
> > > > > > > >> > the
> > > > > > > >> > > > hex
> > > > > > > >> > > > > > >> > > > > > representation.)
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Another thing to
> > > > consider
> > > > > > is
> > > > > > > >> that
> > > > > > > >> > we
> > > > > > > >> > > > > > should
> > > > > > > >> > > > > > >> > > specify
> > > > > > > >> > > > > > >> > > > > > that
> > > > > > > >> > > > > > >> > > > > > >>> the
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > all-zeroes UUID is
> not
> > a
> > > > > valid
> > > > > > > >> topic
> > > > > > > >> > > UUID.
> > > > > > > >> > > > > >  We
> > > > > > > >> > > > > > >> > can't
> > > > > > > >> > > > > > >> > > > use
> > > > > > > >> > > > > > >> > > > > > >>> null
> > > > > > > >> > > > > > >> > > > > > >>> > for
> > > > > > > >> > > > > > >> > > > > > >>> > > > this
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > because we can't
> pass a
> > > > null
> > > > > > UUID
> > > > > > > >> over
> > > > > > > >> > > the
> > > > > > > >> > > > > RPC
> > > > > > > >> > > > > > >> > > protocol
> > > > > > > >> > > > > > >> > > > > > >>> (there
> > > > > > > >> > > > > > >> > > > > > >>> > is no
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > special pattern for
> > null,
> > > > nor
> > > > > > do
> > > > > > > we
> > > > > > > >> > want
> > > > > > > >> > > > to
> > > > > > > >> > > > > > >> waste
> > > > > > > >> > > > > > >> > > space
> > > > > > > >> > > > > > >> > > > > > >>> reserving
> > > > > > > >> > > > > > >> > > > > > >>> > > > such
> > > > > > > >> > > > > > >> > > > > > >>> > > > > a
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > pattern.)
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Maybe I missed
> it,
> > > but
> > > > > did
> > > > > > > you
> > > > > > > >> > > > describe
> > > > > > > >> > > > > > >> > > "migration
> > > > > > > >> > > > > > >> > > > > > of...
> > > > > > > >> > > > > > >> > > > > > >>> > existing
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > topic[s] without
> topic
> > > IDs"
> > > > > in
> > > > > > > >> detail
> > > > > > > >> > in
> > > > > > > >> > > > any
> > > > > > > >> > > > > > >> > section?
> > > > > > > >> > > > > > >> > > > It
> > > > > > > >> > > > > > >> > > > > > >>> seems
> > > > > > > >> > > > > > >> > > > > > >>> > like
> > > > > > > >> > > > > > >> > > > > > >>> > > > > when
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > the new controller
> > > becomes
> > > > > > > active,
> > > > > > > >> it
> > > > > > > >> > > > should
> > > > > > > >> > > > > > >> just
> > > > > > > >> > > > > > >> > > > > generate
> > > > > > > >> > > > > > >> > > > > > >>> random
> > > > > > > >> > > > > > >> > > > > > >>> > > > > UUIDs for
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > these, and write the
> > > random
> > > > > > UUIDs
> > > > > > > >> back
> > > > > > > >> > > to
> > > > > > > >> > > > > > >> > ZooKeeper.
> > > > > > > >> > > > > > >> > > > It
> > > > > > > >> > > > > > >> > > > > > >>> would be
> > > > > > > >> > > > > > >> > > > > > >>> > > > good
> > > > > > > >> > > > > > >> > > > > > >>> > > > > to
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > spell that out.  We
> > > should
> > > > > make
> > > > > > > it
> > > > > > > >> > clear
> > > > > > > >> > > > > that
> > > > > > > >> > > > > > >> this
> > > > > > > >> > > > > > >> > > > > happens
> > > > > > > >> > > > > > >> > > > > > >>> > regardless
> > > > > > > >> > > > > > >> > > > > > >>> > > > > of
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > the inter-broker
> > protocol
> > > > > > version
> > > > > > > >> > (it's
> > > > > > > >> > > a
> > > > > > > >> > > > > > >> > compatible
> > > > > > > >> > > > > > >> > > > > > change).
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > "LeaderAndIsrRequests
> > > > > > > >> including an
> > > > > > > >> > > > > > >> > > > is_every_partition
> > > > > > > >> > > > > > >> > > > > > >>> flag"
> > > > > > > >> > > > > > >> > > > > > >>> > > > seems a
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > bit wordy.  Can we
> just
> > > > call
> > > > > > > these
> > > > > > > >> > "full
> > > > > > > >> > > > > > >> > > > > > >>> LeaderAndIsrRequests"?
> > > > > > > >> > > > > > >> > > > > > >>> > Then
> > > > > > > >> > > > > > >> > > > > > >>> > > > > the
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > RPC field could be
> > named
> > > > > > "full".
> > > > > > > >> > Also,
> > > > > > > >> > > it
> > > > > > > >> > > > > > would
> > > > > > > >> > > > > > >> > > > probably
> > > > > > > >> > > > > > >> > > > > > be
> > > > > > > >> > > > > > >> > > > > > >>> > better
> > > > > > > >> > > > > > >> > > > > > >>> > > > > for the
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > RPC field to be an
> enum
> > > of
> > > > {
> > > > > > > >> > > UNSPECIFIED,
> > > > > > > >> > > > > > >> > > INCREMENTAL,
> > > > > > > >> > > > > > >> > > > > FULL
> > > > > > > >> > > > > > >> > > > > > >>> }, so
> > > > > > > >> > > > > > >> > > > > > >>> > > > that
> > > > > > > >> > > > > > >> > > > > > >>> > > > > we
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > can cleanly handle
> old
> > > > > versions
> > > > > > > (by
> > > > > > > >> > > > treating
> > > > > > > >> > > > > > >> them
> > > > > > > >> > > > > > >> > as
> > > > > > > >> > > > > > >> > > > > > >>> UNSPECIFIED)
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > In the
> > > > > LeaderAndIsrRequest
> > > > > > > >> > section,
> > > > > > > >> > > > you
> > > > > > > >> > > > > > >> write
> > > > > > > >> > > > > > >> > "A
> > > > > > > >> > > > > > >> > > > > final
> > > > > > > >> > > > > > >> > > > > > >>> > deletion
> > > > > > > >> > > > > > >> > > > > > >>> > > > > event
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > will be secheduled
> for
> > X
> > > ms
> > > > > > after
> > > > > > > >> the
> > > > > > > >> > > > > > >> > > > LeaderAndIsrRequest
> > > > > > > >> > > > > > >> > > > > > was
> > > > > > > >> > > > > > >> > > > > > >>> > first
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > received..."  I guess
> > > the X
> > > > > > was a
> > > > > > > >> > > > > placeholder
> > > > > > > >> > > > > > >> that
> > > > > > > >> > > > > > >> > > you
> > > > > > > >> > > > > > >> > > > > > >>> intended
> > > > > > > >> > > > > > >> > > > > > >>> > to
> > > > > > > >> > > > > > >> > > > > > >>> > > > > replace
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > before posting? :)
> In
> > > any
> > > > > > case,
> > > > > > > >> this
> > > > > > > >> > > > seems
> > > > > > > >> > > > > > like
> > > > > > > >> > > > > > >> > the
> > > > > > > >> > > > > > >> > > > kind
> > > > > > > >> > > > > > >> > > > > > of
> > > > > > > >> > > > > > >> > > > > > >>> > thing
> > > > > > > >> > > > > > >> > > > > > >>> > > > we'd
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > want a configuration
> > for.
> > > > > > Let's
> > > > > > > >> > > describe
> > > > > > > >> > > > > that
> > > > > > > >> > > > > > >> > > > > > configuration
> > > > > > > >> > > > > > >> > > > > > >>> key
> > > > > > > >> > > > > > >> > > > > > >>> > > > > somewhere
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > in this KIP,
> including
> > > what
> > > > > its
> > > > > > > >> > default
> > > > > > > >> > > > > value
> > > > > > > >> > > > > > >> is.
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > We should
> probably
> > > also
> > > > > > log a
> > > > > > > >> > bunch
> > > > > > > >> > > of
> > > > > > > >> > > > > > >> messages
> > > > > > > >> > > > > > >> > > at
> > > > > > > >> > > > > > >> > > > > WARN
> > > > > > > >> > > > > > >> > > > > > >>> level
> > > > > > > >> > > > > > >> > > > > > >>> > > > when
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > something is
> scheduled
> > > for
> > > > > > > >> deletion,
> > > > > > > >> > as
> > > > > > > >> > > > > well.
> > > > > > > >> > > > > > >> > (Maybe
> > > > > > > >> > > > > > >> > > > > this
> > > > > > > >> > > > > > >> > > > > > >>> was
> > > > > > > >> > > > > > >> > > > > > >>> > > > > assumed, but
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > it would be good to
> > > mention
> > > > > > it).
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > I feel like there
> > > are a
> > > > > few
> > > > > > > >> > sections
> > > > > > > >> > > > > that
> > > > > > > >> > > > > > >> > should
> > > > > > > >> > > > > > >> > > be
> > > > > > > >> > > > > > >> > > > > > >>> moved to
> > > > > > > >> > > > > > >> > > > > > >>> > > > > "rejected
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > alternatives."  For
> > > > example,
> > > > > in
> > > > > > > the
> > > > > > > >> > > > > > DeleteTopics
> > > > > > > >> > > > > > >> > > > section,
> > > > > > > >> > > > > > >> > > > > > >>> since
> > > > > > > >> > > > > > >> > > > > > >>> > we're
> > > > > > > >> > > > > > >> > > > > > >>> > > > > not
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > going to do option 1
> or
> > > 2,
> > > > > > these
> > > > > > > >> > should
> > > > > > > >> > > be
> > > > > > > >> > > > > > moved
> > > > > > > >> > > > > > >> > into
> > > > > > > >> > > > > > >> > > > > > >>> "rejected
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > alternatives,"
> rather
> > > than
> > > > > > > >> appearing
> > > > > > > >> > > > > inline.
> > > > > > > >> > > > > > >> > > Another
> > > > > > > >> > > > > > >> > > > > case
> > > > > > > >> > > > > > >> > > > > > >>> is
> > > > > > > >> > > > > > >> > > > > > >>> > the
> > > > > > > >> > > > > > >> > > > > > >>> > > > > "Should
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > we remove topic name
> > from
> > > > the
> > > > > > > >> protocol
> > > > > > > >> > > > where
> > > > > > > >> > > > > > >> > > possible"
> > > > > > > >> > > > > > >> > > > > > >>> section.
> > > > > > > >> > > > > > >> > > > > > >>> > This
> > > > > > > >> > > > > > >> > > > > > >>> > > > > is
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > clearly discussing a
> > > design
> > > > > > > >> > alternative
> > > > > > > >> > > > that
> > > > > > > >> > > > > > >> we're
> > > > > > > >> > > > > > >> > > not
> > > > > > > >> > > > > > >> > > > > > >>> proposing
> > > > > > > >> > > > > > >> > > > > > >>> > to
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > implement: removing
> the
> > > > topic
> > > > > > > name
> > > > > > > >> > from
> > > > > > > >> > > > > those
> > > > > > > >> > > > > > >> > > > protocols.
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Is it really
> > > necessary
> > > > to
> > > > > > > have
> > > > > > > >> a
> > > > > > > >> > new
> > > > > > > >> > > > > > >> > > > > > >>> > /admin/delete_topics_by_id
> > > > > > > >> > > > > > >> > > > > > >>> > > > > path
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > in ZooKeeper?  It
> seems
> > > > like
> > > > > we
> > > > > > > >> don't
> > > > > > > >> > > > really
> > > > > > > >> > > > > > >> need
> > > > > > > >> > > > > > >> > > this.
> > > > > > > >> > > > > > >> > > > > > >>> Whenever
> > > > > > > >> > > > > > >> > > > > > >>> > > > > there is
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > a new controller,
> we'll
> > > > send
> > > > > > out
> > > > > > > >> full
> > > > > > > >> > > > > > >> > > > > LeaderAndIsrRequests
> > > > > > > >> > > > > > >> > > > > > >>> which
> > > > > > > >> > > > > > >> > > > > > >>> > will
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > trigger the stale
> > topics
> > > to
> > > > > be
> > > > > > > >> cleaned
> > > > > > > >> > > up.
> > > > > > > >> > > > > >  The
> > > > > > > >> > > > > > >> > > active
> > > > > > > >> > > > > > >> > > > > > >>> > controller
> > > > > > > >> > > > > > >> > > > > > >>> > > > will
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > also send the full
> > > > > > > >> LeaderAndIsrRequest
> > > > > > > >> > > to
> > > > > > > >> > > > > > >> brokers
> > > > > > > >> > > > > > >> > > that
> > > > > > > >> > > > > > >> > > > > are
> > > > > > > >> > > > > > >> > > > > > >>> just
> > > > > > > >> > > > > > >> > > > > > >>> > > > > starting
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > up.    So we don't
> > really
> > > > > need
> > > > > > > this
> > > > > > > >> > kind
> > > > > > > >> > > > of
> > > > > > > >> > > > > > >> > two-phase
> > > > > > > >> > > > > > >> > > > > > commit
> > > > > > > >> > > > > > >> > > > > > >>> > (send
> > > > > > > >> > > > > > >> > > > > > >>> > > > out
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > StopReplicasRequest,
> > get
> > > > ACKs
> > > > > > > from
> > > > > > > >> all
> > > > > > > >> > > > > nodes,
> > > > > > > >> > > > > > >> > commit
> > > > > > > >> > > > > > >> > > by
> > > > > > > >> > > > > > >> > > > > > >>> removing
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > /admin/delete_topics
> > > node)
> > > > > any
> > > > > > > >> more.
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > You mention that
> > > > > > FetchRequest
> > > > > > > >> will
> > > > > > > >> > > now
> > > > > > > >> > > > > > >> include
> > > > > > > >> > > > > > >> > > UUID
> > > > > > > >> > > > > > >> > > > > to
> > > > > > > >> > > > > > >> > > > > > >>> avoid
> > > > > > > >> > > > > > >> > > > > > >>> > > > issues
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > where requests are
> made
> > > to
> > > > > > stale
> > > > > > > >> > > > partitions.
> > > > > > > >> > > > > > >> > > However,
> > > > > > > >> > > > > > >> > > > > > >>> adding a
> > > > > > > >> > > > > > >> > > > > > >>> > UUID
> > > > > > > >> > > > > > >> > > > > > >>> > > > to
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > MetadataRequest is
> > listed
> > > > as
> > > > > > > future
> > > > > > > >> > > work,
> > > > > > > >> > > > > out
> > > > > > > >> > > > > > of
> > > > > > > >> > > > > > >> > > scope
> > > > > > > >> > > > > > >> > > > > for
> > > > > > > >> > > > > > >> > > > > > >>> this
> > > > > > > >> > > > > > >> > > > > > >>> > KIP.
> > > > > > > >> > > > > > >> > > > > > >>> > > > > How
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > will the client learn
> > > what
> > > > > the
> > > > > > > >> topic
> > > > > > > >> > > UUID
> > > > > > > >> > > > > is,
> > > > > > > >> > > > > > if
> > > > > > > >> > > > > > >> > the
> > > > > > > >> > > > > > >> > > > > > metadata
> > > > > > > >> > > > > > >> > > > > > >>> > > > response
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > doesn't include that
> > > > > > information?
> > > > > > > >> It
> > > > > > > >> > > > seems
> > > > > > > >> > > > > > like
> > > > > > > >> > > > > > >> > > adding
> > > > > > > >> > > > > > >> > > > > the
> > > > > > > >> > > > > > >> > > > > > >>> UUID
> > > > > > > >> > > > > > >> > > > > > >>> > to
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > MetadataResponse
> would
> > be
> > > > an
> > > > > > > >> > improvement
> > > > > > > >> > > > > here
> > > > > > > >> > > > > > >> that
> > > > > > > >> > > > > > >> > > > might
> > > > > > > >> > > > > > >> > > > > > not
> > > > > > > >> > > > > > >> > > > > > >>> be
> > > > > > > >> > > > > > >> > > > > > >>> > too
> > > > > > > >> > > > > > >> > > > > > >>> > > > > hard to
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > make.
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > best,
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Colin
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > On Mon, Sep 9,
> > 2019,
> > > at
> > > > > > > 17:48,
> > > > > > > >> > > Ryanne
> > > > > > > >> > > > > > Dolan
> > > > > > > >> > > > > > >> > > wrote:
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > Lucas, this
> would
> > > be
> > > > > > great.
> > > > > > > >> I've
> > > > > > > >> > > run
> > > > > > > >> > > > > > into
> > > > > > > >> > > > > > >> > > issues
> > > > > > > >> > > > > > >> > > > > with
> > > > > > > >> > > > > > >> > > > > > >>> > topics
> > > > > > > >> > > > > > >> > > > > > >>> > > > > being
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > resurrected
> > > > > accidentally,
> > > > > > > >> since
> > > > > > > >> > a
> > > > > > > >> > > > > client
> > > > > > > >> > > > > > >> > cannot
> > > > > > > >> > > > > > >> > > > > > easily
> > > > > > > >> > > > > > >> > > > > > >>> > > > > distinguish
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > between
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > a deleted topic
> > > and a
> > > > > new
> > > > > > > >> topic
> > > > > > > >> > > with
> > > > > > > >> > > > > the
> > > > > > > >> > > > > > >> same
> > > > > > > >> > > > > > >> > > > name.
> > > > > > > >> > > > > > >> > > > > > I'd
> > > > > > > >> > > > > > >> > > > > > >>> > need
> > > > > > > >> > > > > > >> > > > > > >>> > > > the
> > > > > > > >> > > > > > >> > > > > > >>> > > > > ID
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > accessible from
> > the
> > > > > > client
> > > > > > > to
> > > > > > > >> > > solve
> > > > > > > >> > > > > that
> > > > > > > >> > > > > > >> > issue,
> > > > > > > >> > > > > > >> > > > but
> > > > > > > >> > > > > > >> > > > > > >>> this
> > > > > > > >> > > > > > >> > > > > > >>> > is a
> > > > > > > >> > > > > > >> > > > > > >>> > > > > good
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > first
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > step.
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > Ryanne
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > On Wed, Sep 4,
> > 2019
> > > > at
> > > > > > 1:41
> > > > > > > >> PM
> > > > > > > >> > > Lucas
> > > > > > > >> > > > > > >> > > Bradstreet <
> > > > > > > >> > > > > > >> > > > > > >>> > > > > lucas@confluent.io>
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > wrote:
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Hi all,
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > I would like
> to
> > > > kick
> > > > > > off
> > > > > > > >> > > > discussion
> > > > > > > >> > > > > of
> > > > > > > >> > > > > > >> > > KIP-516,
> > > > > > > >> > > > > > >> > > > > an
> > > > > > > >> > > > > > >> > > > > > >>> > > > > implementation
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > of topic
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > IDs for
> Kafka.
> > > > Topic
> > > > > > IDs
> > > > > > > >> aim
> > > > > > > >> > to
> > > > > > > >> > > > > solve
> > > > > > > >> > > > > > >> topic
> > > > > > > >> > > > > > >> > > > > > >>> uniqueness
> > > > > > > >> > > > > > >> > > > > > >>> > > > > problems in
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Kafka,
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > where
> referring
> > > to
> > > > a
> > > > > > > topic
> > > > > > > >> by
> > > > > > > >> > > name
> > > > > > > >> > > > > > >> alone is
> > > > > > > >> > > > > > >> > > > > > >>> insufficient.
> > > > > > > >> > > > > > >> > > > > > >>> > > > Such
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > cases
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > include when
> a
> > > > topic
> > > > > > has
> > > > > > > >> been
> > > > > > > >> > > > > deleted
> > > > > > > >> > > > > > >> and
> > > > > > > >> > > > > > >> > > > > recreated
> > > > > > > >> > > > > > >> > > > > > >>> with
> > > > > > > >> > > > > > >> > > > > > >>> > the
> > > > > > > >> > > > > > >> > > > > > >>> > > > > same
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > name.
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Unique
> > > identifiers
> > > > > will
> > > > > > > >> help
> > > > > > > >> > > > > simplify
> > > > > > > >> > > > > > >> and
> > > > > > > >> > > > > > >> > > > improve
> > > > > > > >> > > > > > >> > > > > > >>> Kafka's
> > > > > > > >> > > > > > >> > > > > > >>> > > > topic
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > deletion
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > process, as
> > well
> > > as
> > > > > > > prevent
> > > > > > > >> > > cases
> > > > > > > >> > > > > > where
> > > > > > > >> > > > > > >> > > brokers
> > > > > > > >> > > > > > >> > > > > may
> > > > > > > >> > > > > > >> > > > > > >>> > > > incorrectly
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > interact
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > with stale
> > > versions
> > > > > of
> > > > > > > >> topics.
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > >> > > > > > >> > > > > >
> > > > > > > >> > > > > > >> > > > >
> > > > > > > >> > > > > > >> > > >
> > > > > > > >> > > > > > >> > >
> > > > > > > >> > > > > > >> >
> > > > > > > >> > > > > > >>
> > > > > > > >> > > > > >
> > > > > > > >> > > > >
> > > > > > > >> > > >
> > > > > > > >> > >
> > > > > > > >> >
> > > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Looking
> forward
> > > to
> > > > > your
> > > > > > > >> > > thoughts.
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Lucas
> > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > > >> > > > > > >> > > > > > >>>
> > > > > > > >> > > > > > >> > > > > > >>
> > > > > > > >> > > > > > >> > > > > >
> > > > > > > >> > > > > > >> > > > >
> > > > > > > >> > > > > > >> > > >
> > > > > > > >> > > > > > >> > >
> > > > > > > >> > > > > > >> >
> > > > > > > >> > > > > > >>
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > >
> > > > > > > >> > > > >
> > > > > > > >> > > >
> > > > > > > >> > >
> > > > > > > >> >
> > > > > > > >>
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Ismael Juma <is...@juma.me.uk>]

Hi Lucas,

Why would you include the name and id? I think you'd want to remove the
name from the directory name right? Jason's suggestion was that if you
remove the name from the directory, then why would you need the id name
mapping file?

Ismael

On Thu, Sep 24, 2020 at 4:24 PM Lucas Bradstreet <lu...@confluent.io> wrote:

> > 2. Part of the usage of the file is to have persistent storage of the
> topic
> ID and use it to compare with the ID supplied in the LeaderAndIsr Request.
> There is some discussion in the KIP about changes to the directory
> structure, but I believe directory changes were considered to be out of
> scope when the KIP was written.
>
>
> Yeah, I was hoping to get a better understanding of why it was taken out of
> scope. Perhaps Lucas Bradstreet might have more insight about the decision.
> Basically my point is that we have to create additional infrastructure here
> to support the name/id mapping, so I wanted to understand if we just
> consider this a sort of tech debt. It would be useful to cover how we
> handle the case when this file gets corrupted. Seems like we just have to
> trust that it matches whatever the controller tells us and rewrite it?
>
>
> Hi Jason, Justine,
>
> My thought process is that we will likely need the metadata file whether we
> rename the directories or not.
> Linux supports filenames of up to 255 bytes and that would not be enough to
> support a directory name
>  including both the name and topic ID. Given that fact, it seemed
> reasonable to add the metadata file
> and leave the directory rename until some time in the future (possibly
> never).
>
> If the file does get corrupted I think you're right. We would either have
> to trust it matches what the controller tells us
>  or error out and let an administrator resolve it by checking across
> replicas for consistency.
>
> Lucas
>
>
> On Thu, Sep 24, 2020 at 3:41 PM Jason Gustafson <ja...@confluent.io>
> wrote:
>
> > Thanks Justine. Responses below:
> >
> > > 1. Yes, the directory will still be based on the topic names.
> > LeaderAndIsrRequest is one of the few requests that will still contain
> the
> > topic name. So I think we have this covered. Sorry for confusion.
> >
> > Ah, you're right. My eyes passed right over the field.
> >
> > > 2. Part of the usage of the file is to have persistent storage of the
> > topic
> > ID and use it to compare with the ID supplied in the LeaderAndIsr
> Request.
> > There is some discussion in the KIP about changes to the directory
> > structure, but I believe directory changes were considered to be out of
> > scope when the KIP was written.
> >
> > Yeah, I was hoping to get a better understanding of why it was taken out
> of
> > scope. Perhaps Lucas Bradstreet might have more insight about the
> decision.
> > Basically my point is that we have to create additional infrastructure
> here
> > to support the name/id mapping, so I wanted to understand if we just
> > consider this a sort of tech debt. It would be useful to cover how we
> > handle the case when this file gets corrupted. Seems like we just have to
> > trust that it matches whatever the controller tells us and rewrite it?
> >
> > > 3. I think this is a good point, but I again I wonder about the scope
> of
> > the KIP. Most of the changes mentioned in the KIP are for supporting
> topic
> > deletion and I believe that is why the produce request was listed under
> > future work.
> >
> > That's fair. I brought it up since `Fetch` is already included. If we've
> > got `Metadata` and `Fetch`, seems we may as well do `Produce` and save an
> > extra kip. No strong objection though if you want to leave it out.
> >
> >
> > -Jason
> >
> >
> > On Thu, Sep 24, 2020 at 3:26 PM Justine Olshan <jo...@confluent.io>
> > wrote:
> >
> > > Hi Jason,
> > >
> > > Thanks for your comments.
> > >
> > > 1. Yes, the directory will still be based on the topic names.
> > > LeaderAndIsrRequest is one of the few requests that will still contain
> > the
> > > topic name. So I think we have this covered. Sorry for confusion.
> > >
> > > 2. Part of the usage of the file is to have persistent storage of the
> > topic
> > > ID and use it to compare with the ID supplied in the LeaderAndIsr
> > Request.
> > > There is some discussion in the KIP about changes to the directory
> > > structure, but I believe directory changes were considered to be out of
> > > scope when the KIP was written.
> > >
> > > 3. I think this is a good point, but I again I wonder about the scope
> of
> > > the KIP. Most of the changes mentioned in the KIP are for supporting
> > topic
> > > deletion and I believe that is why the produce request was listed under
> > > future work.
> > >
> > > 4. This sounds like it might be a good solution, but I will need to
> > discuss
> > > more with KIP-500 folks to get the details right.
> > >
> > > Thanks,
> > > Justine
> > >
> > > On Thu, Sep 24, 2020 at 12:30 PM Jason Gustafson <ja...@confluent.io>
> > > wrote:
> > >
> > > > Hi Justine,
> > > >
> > > > Thanks for picking up this work. I have a few questions/comments:
> > > >
> > > > 1. It sounds like the directory structure is still going to be based
> on
> > > > topic names. Do I have that right? One complication is that the
> > > > LeaderAndIsr request does not include the topic name any longer. This
> > > means
> > > > that a replica must wait for the UpdateMetadata request to arrive
> with
> > > the
> > > > topic name mapping before it can create the directory. I wonder if it
> > > would
> > > > be simpler to avoid assumptions on the ordering of UpdateMetadata and
> > let
> > > > LeaderAndIsr include the topic name as well. Feels like we are not
> > saving
> > > > that much by excluding it.
> > > >
> > > > 2. On a related note, it seems that the reason we have the partition
> > > > metadata file is because we are not changing the directory structure.
> > We
> > > > need it so that we remember which directories map to which topic id.
> I
> > > > think it would be useful to add some detail about why changing the
> > > > directory layout was rejected. Basically trying to understand how
> much
> > > > effort we are saving by skipping this step if we have to implement
> this
> > > > additional bookkeeping. One concern I have for example is that the
> > > > partition metadata file gets corrupt and then all bets are off as far
> > as
> > > > topic consistency.
> > > >
> > > > 3. Is it worth adding support now for topic ids in the `Produce`
> > request
> > > > now? Overhead is mentioned as one of the motivations and the best
> APIs
> > to
> > > > get savings from are `Produce` and `Fetch`.
> > > >
> > > > 4. When it comes to bootstrapping the metadata topic with respect to
> > > > KIP-500, one suggestion would be to reserve a sentinel topic ID which
> > can
> > > > be used initially by new brokers. When the first controller is
> elected,
> > > it
> > > > can assign a topicId to the metadata topic. This is a bit similar to
> > how
> > > we
> > > > were planning to generate the clusterId.
> > > >
> > > > Thanks,
> > > > Jason
> > > >
> > > >
> > > > On Thu, Sep 24, 2020 at 11:10 AM Jun Rao <ju...@confluent.io> wrote:
> > > >
> > > > > Hi, Justine,
> > > > >
> > > > > Thanks for the updated KIP. A few more comments below.
> > > > >
> > > > > 30.  {"name": "id", "type": "string", "doc": "version id"}}: The
> doc
> > > > should
> > > > > say UUID.
> > > > >
> > > > > 31. LeaderAndIsrResponse v5 and StopReplicaResponse v4 : It seems
> > there
> > > > is
> > > > > no need to add topic_id at partitions level.
> > > > >
> > > > > 32. Regarding partition metadata file. Perhaps the key can be a
> > single
> > > > > word, sth like the following.
> > > > > version: 0
> > > > > topic_id: 46bdb63f-9e8d-4a38-bf7b-ee4eb2a794e4
> > > > >
> > > > > 33. Another tricky thing that I realized is how to support the
> > metadata
> > > > > topic introduced in KIP-595. It's a bit tricky to assign a UUID to
> > the
> > > > > metadata topic since we have a chicken-and-egg problem. The
> > controller
> > > > > needs to persist the UUID in the metadata topic in order to assign
> > one
> > > > > successfully, but the metadata topic is needed to elect a
> controller
> > > > > (KIP-631). So, this probably needs a bit more thought.
> > > > >
> > > > > Jun
> > > > >
> > > > > On Thu, Sep 24, 2020 at 3:04 AM Ismael Juma <is...@juma.me.uk>
> > wrote:
> > > > >
> > > > > > Also, can we provide more details on how the Partition Metadata
> > file
> > > > will
> > > > > > be used?
> > > > > >
> > > > > > Ismael
> > > > > >
> > > > > > On Thu, Sep 24, 2020 at 3:01 AM Ismael Juma <is...@juma.me.uk>
> > > wrote:
> > > > > >
> > > > > > > Hi Justine,
> > > > > > >
> > > > > > > I think we need to update the "Rejected Alternatives" section
> to
> > > take
> > > > > > into
> > > > > > > account that the proposal now removes the topic name from the
> > fetch
> > > > > > > request. Also, if we are removing it from the Fetch request,
> does
> > > it
> > > > > make
> > > > > > > sense not to remove it from similar requests like
> > > ListOffsetRequest?
> > > > > > >
> > > > > > > Ismael
> > > > > > >
> > > > > > > On Thu, Sep 24, 2020 at 2:46 AM David Jacot <
> djacot@confluent.io
> > >
> > > > > wrote:
> > > > > > >
> > > > > > >> Hi Justine,
> > > > > > >>
> > > > > > >> Thanks for the KIP. I finally had time to read it :). It is a
> > > great
> > > > > > >> improvement.
> > > > > > >>
> > > > > > >> I have a few comments/questions:
> > > > > > >>
> > > > > > >> 1. It seems that the schema of the StopReplicaRequest is
> > slightly
> > > > > > >> outdated.
> > > > > > >> We
> > > > > > >> did some changes as part of KIP-570. V3 is already organized
> by
> > > > > topics.
> > > > > > >>
> > > > > > >> 2. I just want to make sure that I understand the
> reconciliation
> > > > > > >> logic correctly. When
> > > > > > >> an "INCREMENTAL" LeaderAndIsr Request is received, the broker
> > will
> > > > > also
> > > > > > >> reconcile
> > > > > > >> when the local uuid does not match the uuid in the request,
> > right?
> > > > In
> > > > > > this
> > > > > > >> case, the
> > > > > > >> local log is staged for deletion.
> > > > > > >>
> > > > > > >> 3. In the documentation of the `delete.stale.topic.delay.ms`
> > > > config,
> > > > > it
> > > > > > >> says "When a
> > > > > > >> FULL LeaderAndIsrRequest is received..." but I suppose it
> > applies
> > > to
> > > > > > both
> > > > > > >> types.
> > > > > > >>
> > > > > > >> Best,
> > > > > > >> David
> > > > > > >>
> > > > > > >> On Thu, Sep 24, 2020 at 1:40 AM Justine Olshan <
> > > > jolshan@confluent.io>
> > > > > > >> wrote:
> > > > > > >>
> > > > > > >> > Hi Jun,
> > > > > > >> >
> > > > > > >> > Thanks for the comments. I apologize for some of the typos
> and
> > > > > > >> confusion.
> > > > > > >> > I’ve updated the KIP to fix some of the issues you
> mentioned.
> > > > > > >> >
> > > > > > >> > 20.2 I’ve changed the type to String.
> > > > > > >> > 20.1/3 I’ve updated the TopicZNode to fix formatting and
> > reflect
> > > > the
> > > > > > >> latest
> > > > > > >> > version before this change.
> > > > > > >> >
> > > > > > >> > 21. You are correct and I’ve removed this line. I’ve also
> > added
> > > a
> > > > > line
> > > > > > >> > mentioning an IBP bump is necessary for migration
> > > > > > >> >
> > > > > > >> > 22. I think the wording was unclear but your summary is what
> > was
> > > > > > >> intended
> > > > > > >> > by this line. I’ve updated to make this point more clear.
> > > “Remove
> > > > > > >> deleted
> > > > > > >> > topics from replicas by sending StopReplicaRequest V3 before
> > the
> > > > IBP
> > > > > > >> bump
> > > > > > >> > using the old logic, and using V4 and the new logic with
> topic
> > > IDs
> > > > > > after
> > > > > > >> > the IBP bump.”
> > > > > > >> >
> > > > > > >> > 23. I’ve removed the unspecified type from the KIP and
> mention
> > > > that
> > > > > > IBP
> > > > > > >> > will be used to handle this request. “IBP will be used to
> > > > determine
> > > > > > >> whether
> > > > > > >> > this new form of the request will be used. For older
> requests,
> > > we
> > > > > will
> > > > > > >> > ignore this field and default to previous behavior.”
> > > > > > >> >
> > > > > > >> > 24. I’ve fixed this typo.
> > > > > > >> >
> > > > > > >> > 25. I've added a topics at a higher level for
> > > LeaderAndIsrResponse
> > > > > v5,
> > > > > > >> > StopReplicaResponse v4. I've also changed StopReplicaRequest
> > v4
> > > to
> > > > > > have
> > > > > > >> > topics at a higher level.
> > > > > > >> >
> > > > > > >> > 26. I’ve updated forgotten_topics_data--added the topic ID
> and
> > > > > removed
> > > > > > >> the
> > > > > > >> > topic name
> > > > > > >> >
> > > > > > >> > 27. I’ve decided on plain text, and I’ve added an example.
> > > > > > >> >
> > > > > > >> > 28. I’ve added this idea to future work.
> > > > > > >> >
> > > > > > >> > Thanks again for taking a look,
> > > > > > >> >
> > > > > > >> > Justine
> > > > > > >> >
> > > > > > >> > On Wed, Sep 23, 2020 at 10:28 AM Jun Rao <ju...@confluent.io>
> > > > wrote:
> > > > > > >> >
> > > > > > >> > > Hi, Justine,
> > > > > > >> > >
> > > > > > >> > > Thanks for the response. Made another pass. A few more
> > > comments
> > > > > > below.
> > > > > > >> > >
> > > > > > >> > > 20. znode schema:
> > > > > > >> > > 20.1 It seems that {"name": "version", "type": "int",
> "id":
> > > > > "UUID",
> > > > > > >> > "doc":
> > > > > > >> > > "version id"} should be {"name": "version", "type":
> "int"},
> > > > > {"name":
> > > > > > >> > "id",
> > > > > > >> > > "type": "UUID", "doc": "version id"}.
> > > > > > >> > > 20.2 The znode format is JSON which doesn't have UUID
> type.
> > So
> > > > the
> > > > > > >> type
> > > > > > >> > > probably should be string?
> > > > > > >> > > 20.3 Also, the existing format used seems outdated. It
> > should
> > > > have
> > > > > > the
> > > > > > >> > > following format.
> > > > > > >> > >     Json.encodeAsBytes(Map(
> > > > > > >> > >       "version" -> 2,
> > > > > > >> > >       "partitions" -> replicaAssignmentJson.asJava,
> > > > > > >> > >       "adding_replicas" ->
> > > addingReplicasAssignmentJson.asJava,
> > > > > > >> > >       "removing_replicas" ->
> > > > removingReplicasAssignmentJson.asJava
> > > > > > >> > >     ).asJava)
> > > > > > >> > >   }
> > > > > > >> > >
> > > > > > >> > > 21. Migration: The KIP says "The migration process can
> take
> > > > place
> > > > > > >> without
> > > > > > >> > > an inter-broker protocol bump, as the format stored in
> > > > > > >> > > /brokers/topics/[topic] will be compatible with older
> broker
> > > > > > >> versions."
> > > > > > >> > > However, since we are bumping up the version of
> inter-broker
> > > > > > >> requests, it
> > > > > > >> > > seems that we need to use IBP for migration.
> > > > > > >> > >
> > > > > > >> > > 22. The KIP says "Remove deleted topics from replicas by
> > > sending
> > > > > > >> > > StopReplicaRequest V3 for any topics which do not contain
> a
> > > > topic
> > > > > > ID,
> > > > > > >> and
> > > > > > >> > > V4 for any topics which do contain a topic ID.". However,
> if
> > > we
> > > > > use
> > > > > > >> IBP,
> > > > > > >> > it
> > > > > > >> > > seems that the controller will either send
> > StopReplicaRequest
> > > V3
> > > > > > >> > > or StopReplicaRequest V4, but never mixed V3 and V4 for
> > > > different
> > > > > > >> topics.
> > > > > > >> > > Basically, before the IBP bump, V3 will be used. After the
> > IBP
> > > > > bump,
> > > > > > >> > > topicId will be created and V4 will be used.
> > > > > > >> > >
> > > > > > >> > > 23. Given that we depend on IBP, do we still need "0
> > > > UNSPECIFIED"
> > > > > > >> > > in LeaderAndIsr?
> > > > > > >> > >
> > > > > > >> > > 24. LeaderAndIsrResponse v5 : It still has the topic
> field.
> > > > > > >> > >
> > > > > > >> > > 25. LeaderAndIsrResponse v5, StopReplicaResponse v4: Could
> > we
> > > > use
> > > > > > this
> > > > > > >> > > opportunity to organize the response in 2 levels, first by
> > > > topic,
> > > > > > >> then by
> > > > > > >> > > partition, as most other requests/responses?
> > > > > > >> > >
> > > > > > >> > > 26. FetchRequest v13 : Should forgotten_topics_data use
> > > topicId
> > > > > too?
> > > > > > >> > >
> > > > > > >> > > 27. "This file can either be plain text (key/value pairs)
> or
> > > > > JSON."
> > > > > > >> Have
> > > > > > >> > we
> > > > > > >> > > decided which one to use? Also, it would be helpful to
> > provide
> > > > an
> > > > > > >> > example.
> > > > > > >> > >
> > > > > > >> > > 28. Future improvement: Another future improvement
> > opportunity
> > > > is
> > > > > to
> > > > > > >> use
> > > > > > >> > > topicId in GroupMetadataManager.offsetCommitKey in the
> > > > > offset_commit
> > > > > > >> > topic.
> > > > > > >> > > This may save some space.
> > > > > > >> > >
> > > > > > >> > > Thanks,
> > > > > > >> > >
> > > > > > >> > > Jun
> > > > > > >> > >
> > > > > > >> > > On Wed, Sep 23, 2020 at 8:50 AM Justine Olshan <
> > > > > > jolshan@confluent.io>
> > > > > > >> > > wrote:
> > > > > > >> > >
> > > > > > >> > > > Hi Tom,
> > > > > > >> > > >
> > > > > > >> > > > Thanks for the comment. I think this is a really good
> idea
> > > and
> > > > > it
> > > > > > >> has
> > > > > > >> > > been
> > > > > > >> > > > added to the KIP under the newly added tooling section.
> > > > > > >> > > >
> > > > > > >> > > > Thanks again,
> > > > > > >> > > > Justine
> > > > > > >> > > >
> > > > > > >> > > > On Wed, Sep 23, 2020 at 3:17 AM Tom Bentley <
> > > > > tbentley@redhat.com>
> > > > > > >> > wrote:
> > > > > > >> > > >
> > > > > > >> > > > > Hi Justine,
> > > > > > >> > > > >
> > > > > > >> > > > > I know you started the vote thread, but on re-reading
> > the
> > > > KIP
> > > > > I
> > > > > > >> > noticed
> > > > > > >> > > > > that although the topic id is included in the
> > > > MetadataResponse
> > > > > > >> it's
> > > > > > >> > not
> > > > > > >> > > > > surfaced in the output from `kafka-topics.sh
> > --describe`.
> > > > > Maybe
> > > > > > >> that
> > > > > > >> > > was
> > > > > > >> > > > > intentional because ids are intentionally not really
> > > > something
> > > > > > the
> > > > > > >> > user
> > > > > > >> > > > > should care deeply about, but it would also make life
> > > harder
> > > > > for
> > > > > > >> > anyone
> > > > > > >> > > > > debugging Kafka and this would likely get worse the
> more
> > > > topic
> > > > > > ids
> > > > > > >> > got
> > > > > > >> > > > > rolled out across the protocols, clients etc. It seems
> > > > likely
> > > > > > that
> > > > > > >> > > > > `kafka-topics.sh` will eventually need the ability to
> > show
> > > > the
> > > > > > id
> > > > > > >> of
> > > > > > >> > a
> > > > > > >> > > > > topic and perhaps find a topic name given an id. Is
> > there
> > > > any
> > > > > > >> reason
> > > > > > >> > > not
> > > > > > >> > > > to
> > > > > > >> > > > > implement that in this KIP?
> > > > > > >> > > > >
> > > > > > >> > > > > Many thanks,
> > > > > > >> > > > >
> > > > > > >> > > > > Tom
> > > > > > >> > > > >
> > > > > > >> > > > > On Mon, Sep 21, 2020 at 9:54 PM Justine Olshan <
> > > > > > >> jolshan@confluent.io
> > > > > > >> > >
> > > > > > >> > > > > wrote:
> > > > > > >> > > > >
> > > > > > >> > > > > > Hi all,
> > > > > > >> > > > > >
> > > > > > >> > > > > > After thinking about it, I've decided to remove the
> > > topic
> > > > > name
> > > > > > >> from
> > > > > > >> > > the
> > > > > > >> > > > > > Fetch Request and Response after all. Since there
> are
> > so
> > > > > many
> > > > > > of
> > > > > > >> > > these
> > > > > > >> > > > > > requests per second, it is worth removing the extra
> > > > > > information.
> > > > > > >> > I've
> > > > > > >> > > > > > updated the KIP to reflect this change.
> > > > > > >> > > > > >
> > > > > > >> > > > > > Please let me know if there is anything else we
> should
> > > > > discuss
> > > > > > >> > before
> > > > > > >> > > > > > voting.
> > > > > > >> > > > > >
> > > > > > >> > > > > > Thank you,
> > > > > > >> > > > > > Justine
> > > > > > >> > > > > >
> > > > > > >> > > > > > On Fri, Sep 18, 2020 at 9:46 AM Justine Olshan <
> > > > > > >> > jolshan@confluent.io
> > > > > > >> > > >
> > > > > > >> > > > > > wrote:
> > > > > > >> > > > > >
> > > > > > >> > > > > > > Hi Jun,
> > > > > > >> > > > > > >
> > > > > > >> > > > > > > I see what you are saying. For now we can remove
> the
> > > > extra
> > > > > > >> > > > information.
> > > > > > >> > > > > > > I'll leave the option to add more fields to the
> file
> > > in
> > > > > the
> > > > > > >> > future.
> > > > > > >> > > > The
> > > > > > >> > > > > > KIP
> > > > > > >> > > > > > > has been updated to reflect this change.
> > > > > > >> > > > > > >
> > > > > > >> > > > > > > Thanks,
> > > > > > >> > > > > > > Justine
> > > > > > >> > > > > > >
> > > > > > >> > > > > > > On Fri, Sep 18, 2020 at 8:46 AM Jun Rao <
> > > > jun@confluent.io
> > > > > >
> > > > > > >> > wrote:
> > > > > > >> > > > > > >
> > > > > > >> > > > > > >> Hi, Justine,
> > > > > > >> > > > > > >>
> > > > > > >> > > > > > >> Thanks for the reply.
> > > > > > >> > > > > > >>
> > > > > > >> > > > > > >> 13. If the log directory is the source of truth,
> it
> > > > means
> > > > > > >> that
> > > > > > >> > the
> > > > > > >> > > > > > >> redundant info in the metadata file will be
> > ignored.
> > > > Then
> > > > > > the
> > > > > > >> > > > question
> > > > > > >> > > > > > is
> > > > > > >> > > > > > >> why do we need to put the redundant info in the
> > > > metadata
> > > > > > file
> > > > > > >> > now?
> > > > > > >> > > > > > >>
> > > > > > >> > > > > > >> Thanks,
> > > > > > >> > > > > > >>
> > > > > > >> > > > > > >> Jun
> > > > > > >> > > > > > >>
> > > > > > >> > > > > > >> On Thu, Sep 17, 2020 at 5:07 PM Justine Olshan <
> > > > > > >> > > > jolshan@confluent.io>
> > > > > > >> > > > > > >> wrote:
> > > > > > >> > > > > > >>
> > > > > > >> > > > > > >> > Hi Jun,
> > > > > > >> > > > > > >> > Thanks for the quick response!
> > > > > > >> > > > > > >> >
> > > > > > >> > > > > > >> > 12. I've decided to bump up the versions on the
> > > > > requests
> > > > > > >> and
> > > > > > >> > > > updated
> > > > > > >> > > > > > the
> > > > > > >> > > > > > >> > KIP. I think it's good we thoroughly discussed
> > the
> > > > > > options
> > > > > > >> > here,
> > > > > > >> > > > so
> > > > > > >> > > > > we
> > > > > > >> > > > > > >> know
> > > > > > >> > > > > > >> > we made a good choice. :)
> > > > > > >> > > > > > >> >
> > > > > > >> > > > > > >> > 13. This is an interesting situation. I think
> if
> > > this
> > > > > > does
> > > > > > >> > occur
> > > > > > >> > > > we
> > > > > > >> > > > > > >> should
> > > > > > >> > > > > > >> > give a warning. I agree that it's hard to know
> > the
> > > > > source
> > > > > > >> of
> > > > > > >> > > truth
> > > > > > >> > > > > for
> > > > > > >> > > > > > >> sure
> > > > > > >> > > > > > >> > since the directory or the file could be
> manually
> > > > > > >> modified. I
> > > > > > >> > > > guess
> > > > > > >> > > > > > the
> > > > > > >> > > > > > >> > directory could be used as the source of truth.
> > To
> > > be
> > > > > > >> honest,
> > > > > > >> > > I'm
> > > > > > >> > > > > not
> > > > > > >> > > > > > >> > really sure what happens in kafka when the log
> > > > > directory
> > > > > > is
> > > > > > >> > > > renamed
> > > > > > >> > > > > > >> > manually in such a way. I'm also wondering if
> the
> > > > > > >> situation is
> > > > > > >> > > > > > >> recoverable
> > > > > > >> > > > > > >> > in this scenario.
> > > > > > >> > > > > > >> >
> > > > > > >> > > > > > >> > Thanks,
> > > > > > >> > > > > > >> > Justine
> > > > > > >> > > > > > >> >
> > > > > > >> > > > > > >> > On Thu, Sep 17, 2020 at 4:28 PM Jun Rao <
> > > > > > jun@confluent.io>
> > > > > > >> > > wrote:
> > > > > > >> > > > > > >> >
> > > > > > >> > > > > > >> > > Hi, Justine,
> > > > > > >> > > > > > >> > >
> > > > > > >> > > > > > >> > > Thanks for the reply.
> > > > > > >> > > > > > >> > >
> > > > > > >> > > > > > >> > > 12. I don't have a strong preference either.
> > > > However,
> > > > > > if
> > > > > > >> we
> > > > > > >> > > need
> > > > > > >> > > > > IBP
> > > > > > >> > > > > > >> > > anyway, maybe it's easier to just bump up the
> > > > version
> > > > > > for
> > > > > > >> > all
> > > > > > >> > > > > inter
> > > > > > >> > > > > > >> > broker
> > > > > > >> > > > > > >> > > requests and add the topic id field as a
> > regular
> > > > > > field. A
> > > > > > >> > > > regular
> > > > > > >> > > > > > >> field
> > > > > > >> > > > > > >> > is
> > > > > > >> > > > > > >> > > a bit more concise in wire transfer than a
> > > flexible
> > > > > > >> field.
> > > > > > >> > > > > > >> > >
> > > > > > >> > > > > > >> > > 13. The confusion that I was referring to is
> > > > between
> > > > > > the
> > > > > > >> > topic
> > > > > > >> > > > > name
> > > > > > >> > > > > > >> and
> > > > > > >> > > > > > >> > > partition number between the log dir and the
> > > > metadata
> > > > > > >> file.
> > > > > > >> > > For
> > > > > > >> > > > > > >> example,
> > > > > > >> > > > > > >> > if
> > > > > > >> > > > > > >> > > the log dir is topicA-1 and the metadata file
> > in
> > > it
> > > > > has
> > > > > > >> > topicB
> > > > > > >> > > > and
> > > > > > >> > > > > > >> > > partition 0 (say due to a bug or manual
> > > > > modification),
> > > > > > >> which
> > > > > > >> > > one
> > > > > > >> > > > > do
> > > > > > >> > > > > > we
> > > > > > >> > > > > > >> > use
> > > > > > >> > > > > > >> > > as the source of truth?
> > > > > > >> > > > > > >> > >
> > > > > > >> > > > > > >> > > Jun
> > > > > > >> > > > > > >> > >
> > > > > > >> > > > > > >> > > On Thu, Sep 17, 2020 at 3:43 PM Justine
> Olshan
> > <
> > > > > > >> > > > > > jolshan@confluent.io>
> > > > > > >> > > > > > >> > > wrote:
> > > > > > >> > > > > > >> > >
> > > > > > >> > > > > > >> > > > Hi Jun,
> > > > > > >> > > > > > >> > > > Thanks for the comments.
> > > > > > >> > > > > > >> > > >
> > > > > > >> > > > > > >> > > > 12. I bumped the LeaderAndIsrRequest
> because
> > I
> > > > > > removed
> > > > > > >> the
> > > > > > >> > > > topic
> > > > > > >> > > > > > >> name
> > > > > > >> > > > > > >> > > field
> > > > > > >> > > > > > >> > > > in the response. It may be possible to
> avoid
> > > > > bumping
> > > > > > >> the
> > > > > > >> > > > version
> > > > > > >> > > > > > >> > without
> > > > > > >> > > > > > >> > > > that change, but I may be missing
> something.
> > > > > > >> > > > > > >> > > > I believe StopReplica is actually on
> version
> > 3
> > > > now,
> > > > > > but
> > > > > > >> > > > because
> > > > > > >> > > > > > >> > version 2
> > > > > > >> > > > > > >> > > > is flexible, I kept that listed as version
> 2
> > on
> > > > the
> > > > > > KIP
> > > > > > >> > > page.
> > > > > > >> > > > > > >> However,
> > > > > > >> > > > > > >> > > you
> > > > > > >> > > > > > >> > > > may be right in that we may need to bump
> the
> > > > > version
> > > > > > on
> > > > > > >> > > > > > StopReplica
> > > > > > >> > > > > > >> to
> > > > > > >> > > > > > >> > > deal
> > > > > > >> > > > > > >> > > > with deletion differently as mentioned
> > above. I
> > > > > don't
> > > > > > >> know
> > > > > > >> > > if
> > > > > > >> > > > I
> > > > > > >> > > > > > >> have a
> > > > > > >> > > > > > >> > > big
> > > > > > >> > > > > > >> > > > preference over used tagged fields or not.
> > > > > > >> > > > > > >> > > >
> > > > > > >> > > > > > >> > > > 13. I was thinking that in the case where
> the
> > > > file
> > > > > > and
> > > > > > >> the
> > > > > > >> > > > > request
> > > > > > >> > > > > > >> > topic
> > > > > > >> > > > > > >> > > > ids don't match, it means that the broker's
> > > > > topic/the
> > > > > > >> one
> > > > > > >> > in
> > > > > > >> > > > the
> > > > > > >> > > > > > >> file
> > > > > > >> > > > > > >> > has
> > > > > > >> > > > > > >> > > > been deleted. In that case, we would need
> to
> > > > delete
> > > > > > the
> > > > > > >> > old
> > > > > > >> > > > > topic
> > > > > > >> > > > > > >> and
> > > > > > >> > > > > > >> > > start
> > > > > > >> > > > > > >> > > > receiving the new version. If the topic
> name
> > > were
> > > > > to
> > > > > > >> > change,
> > > > > > >> > > > but
> > > > > > >> > > > > > the
> > > > > > >> > > > > > >> > ids
> > > > > > >> > > > > > >> > > > still matched, the file would also need to
> > > > update.
> > > > > > Am I
> > > > > > >> > > > missing
> > > > > > >> > > > > a
> > > > > > >> > > > > > >> case
> > > > > > >> > > > > > >> > > > where the file would be correct and not the
> > > > > request?
> > > > > > >> > > > > > >> > > >
> > > > > > >> > > > > > >> > > > Thanks,
> > > > > > >> > > > > > >> > > > Justine
> > > > > > >> > > > > > >> > > >
> > > > > > >> > > > > > >> > > > On Thu, Sep 17, 2020 at 3:18 PM Jun Rao <
> > > > > > >> jun@confluent.io
> > > > > > >> > >
> > > > > > >> > > > > wrote:
> > > > > > >> > > > > > >> > > >
> > > > > > >> > > > > > >> > > > > Hi, Justine,
> > > > > > >> > > > > > >> > > > >
> > > > > > >> > > > > > >> > > > > Thanks for the reply. A couple of more
> > > comments
> > > > > > >> below.
> > > > > > >> > > > > > >> > > > >
> > > > > > >> > > > > > >> > > > > 12. ListOffset and OffsetForLeader
> > currently
> > > > > don't
> > > > > > >> > support
> > > > > > >> > > > > > >> flexible
> > > > > > >> > > > > > >> > > > fields.
> > > > > > >> > > > > > >> > > > > So, we have to bump up the version number
> > and
> > > > use
> > > > > > >> IBP at
> > > > > > >> > > > least
> > > > > > >> > > > > > for
> > > > > > >> > > > > > >> > > these
> > > > > > >> > > > > > >> > > > > two requests. Note that it seems 2.7.0
> will
> > > > > require
> > > > > > >> IBP
> > > > > > >> > > > anyway
> > > > > > >> > > > > > >> > because
> > > > > > >> > > > > > >> > > of
> > > > > > >> > > > > > >> > > > > changes in KAFKA-10435. Also, it seems
> that
> > > the
> > > > > > >> version
> > > > > > >> > > for
> > > > > > >> > > > > > >> > > > > LeaderAndIsrRequest and StopReplica are
> > > bumped
> > > > > even
> > > > > > >> > though
> > > > > > >> > > > we
> > > > > > >> > > > > > only
> > > > > > >> > > > > > >> > > added
> > > > > > >> > > > > > >> > > > a
> > > > > > >> > > > > > >> > > > > tagged field. But since IBP is needed
> > anyway,
> > > > we
> > > > > > may
> > > > > > >> > want
> > > > > > >> > > to
> > > > > > >> > > > > > >> revisit
> > > > > > >> > > > > > >> > > the
> > > > > > >> > > > > > >> > > > > overall tagged field choice.
> > > > > > >> > > > > > >> > > > >
> > > > > > >> > > > > > >> > > > > 13. The only downside is the potential
> > > > confusion
> > > > > on
> > > > > > >> > which
> > > > > > >> > > > one
> > > > > > >> > > > > is
> > > > > > >> > > > > > >> the
> > > > > > >> > > > > > >> > > > source
> > > > > > >> > > > > > >> > > > > of truth if they don't match. Another
> > option
> > > is
> > > > > to
> > > > > > >> > include
> > > > > > >> > > > > those
> > > > > > >> > > > > > >> > fields
> > > > > > >> > > > > > >> > > > in
> > > > > > >> > > > > > >> > > > > the metadata file when we actually change
> > the
> > > > > > >> directory
> > > > > > >> > > > > > structure.
> > > > > > >> > > > > > >> > > > >
> > > > > > >> > > > > > >> > > > > Thanks,
> > > > > > >> > > > > > >> > > > >
> > > > > > >> > > > > > >> > > > > Jun
> > > > > > >> > > > > > >> > > > >
> > > > > > >> > > > > > >> > > > > On Thu, Sep 17, 2020 at 2:01 PM Justine
> > > Olshan
> > > > <
> > > > > > >> > > > > > >> jolshan@confluent.io
> > > > > > >> > > > > > >> > >
> > > > > > >> > > > > > >> > > > > wrote:
> > > > > > >> > > > > > >> > > > >
> > > > > > >> > > > > > >> > > > > > Hello all,
> > > > > > >> > > > > > >> > > > > >
> > > > > > >> > > > > > >> > > > > > I've thought some more about removing
> the
> > > > topic
> > > > > > >> name
> > > > > > >> > > field
> > > > > > >> > > > > > from
> > > > > > >> > > > > > >> > some
> > > > > > >> > > > > > >> > > of
> > > > > > >> > > > > > >> > > > > the
> > > > > > >> > > > > > >> > > > > > requests. On closer inspection of the
> > > > > > >> > > requests/responses,
> > > > > > >> > > > it
> > > > > > >> > > > > > >> seems
> > > > > > >> > > > > > >> > > that
> > > > > > >> > > > > > >> > > > > the
> > > > > > >> > > > > > >> > > > > > internal changes would be much larger
> > than
> > > I
> > > > > > >> expected.
> > > > > > >> > > > Some
> > > > > > >> > > > > > >> > protocols
> > > > > > >> > > > > > >> > > > > > involve clients, so they would require
> > > > changes
> > > > > > too.
> > > > > > >> > I'm
> > > > > > >> > > > > > thinking
> > > > > > >> > > > > > >> > that
> > > > > > >> > > > > > >> > > > for
> > > > > > >> > > > > > >> > > > > > now, removing the topic name from these
> > > > > requests
> > > > > > >> and
> > > > > > >> > > > > responses
> > > > > > >> > > > > > >> are
> > > > > > >> > > > > > >> > > out
> > > > > > >> > > > > > >> > > > of
> > > > > > >> > > > > > >> > > > > > scope.
> > > > > > >> > > > > > >> > > > > >
> > > > > > >> > > > > > >> > > > > > I have decided to just keep the change
> > > > > > >> > > > LeaderAndIsrResponse
> > > > > > >> > > > > to
> > > > > > >> > > > > > >> > remove
> > > > > > >> > > > > > >> > > > the
> > > > > > >> > > > > > >> > > > > > topic name, and have updated the KIP to
> > > > reflect
> > > > > > >> this
> > > > > > >> > > > > change. I
> > > > > > >> > > > > > >> have
> > > > > > >> > > > > > >> > > > also
> > > > > > >> > > > > > >> > > > > > mentioned the other requests and
> > responses
> > > in
> > > > > > >> future
> > > > > > >> > > work.
> > > > > > >> > > > > > >> > > > > >
> > > > > > >> > > > > > >> > > > > > I'm hoping to start the voting process
> > > soon,
> > > > so
> > > > > > >> let me
> > > > > > >> > > > know
> > > > > > >> > > > > if
> > > > > > >> > > > > > >> > there
> > > > > > >> > > > > > >> > > is
> > > > > > >> > > > > > >> > > > > > anything else we should discuss.
> > > > > > >> > > > > > >> > > > > >
> > > > > > >> > > > > > >> > > > > > Thank you,
> > > > > > >> > > > > > >> > > > > > Justine
> > > > > > >> > > > > > >> > > > > >
> > > > > > >> > > > > > >> > > > > > On Tue, Sep 15, 2020 at 3:57 PM Justine
> > > > Olshan
> > > > > <
> > > > > > >> > > > > > >> > jolshan@confluent.io
> > > > > > >> > > > > > >> > > >
> > > > > > >> > > > > > >> > > > > > wrote:
> > > > > > >> > > > > > >> > > > > >
> > > > > > >> > > > > > >> > > > > > > Hello again,
> > > > > > >> > > > > > >> > > > > > > To follow up on some of the other
> > > comments:
> > > > > > >> > > > > > >> > > > > > >
> > > > > > >> > > > > > >> > > > > > > 10/11) We can remove the topic name
> > from
> > > > > these
> > > > > > >> > > > > > >> > requests/responses,
> > > > > > >> > > > > > >> > > > and
> > > > > > >> > > > > > >> > > > > > > that means we will just have to make
> a
> > > few
> > > > > > >> internal
> > > > > > >> > > > > changes
> > > > > > >> > > > > > to
> > > > > > >> > > > > > >> > make
> > > > > > >> > > > > > >> > > > > > > partitions accessible by topic id and
> > > > > > partition.
> > > > > > >> I
> > > > > > >> > can
> > > > > > >> > > > > > update
> > > > > > >> > > > > > >> the
> > > > > > >> > > > > > >> > > KIP
> > > > > > >> > > > > > >> > > > > to
> > > > > > >> > > > > > >> > > > > > > remove them unless anyone thinks they
> > > > should
> > > > > > >> stay.
> > > > > > >> > > > > > >> > > > > > >
> > > > > > >> > > > > > >> > > > > > > 12) Addressed in the previous email.
> > I've
> > > > > > updated
> > > > > > >> > the
> > > > > > >> > > > KIP
> > > > > > >> > > > > to
> > > > > > >> > > > > > >> > > include
> > > > > > >> > > > > > >> > > > > > > tagged fields for the requests and
> > > > responses.
> > > > > > >> (More
> > > > > > >> > on
> > > > > > >> > > > > that
> > > > > > >> > > > > > >> > below)
> > > > > > >> > > > > > >> > > > > > >
> > > > > > >> > > > > > >> > > > > > > 13) I think part of the idea for
> > > including
> > > > > this
> > > > > > >> > > > > information
> > > > > > >> > > > > > >> is to
> > > > > > >> > > > > > >> > > > > prepare
> > > > > > >> > > > > > >> > > > > > > for future changes. Perhaps the
> > directory
> > > > > > >> structure
> > > > > > >> > > > might
> > > > > > >> > > > > > >> change
> > > > > > >> > > > > > >> > > from
> > > > > > >> > > > > > >> > > > > > > topicName_partitionNumber to
> something
> > > like
> > > > > > >> > > > > > >> > > topicID_partitionNumber.
> > > > > > >> > > > > > >> > > > > Then
> > > > > > >> > > > > > >> > > > > > > it would be useful to have the topic
> > name
> > > > in
> > > > > > the
> > > > > > >> > file
> > > > > > >> > > > > since
> > > > > > >> > > > > > it
> > > > > > >> > > > > > >> > > would
> > > > > > >> > > > > > >> > > > > not
> > > > > > >> > > > > > >> > > > > > be
> > > > > > >> > > > > > >> > > > > > > in the directory structure.
> Supporting
> > > > topic
> > > > > > >> renames
> > > > > > >> > > > might
> > > > > > >> > > > > > be
> > > > > > >> > > > > > >> > > easier
> > > > > > >> > > > > > >> > > > if
> > > > > > >> > > > > > >> > > > > > the
> > > > > > >> > > > > > >> > > > > > > other fields are included. Would
> there
> > be
> > > > any
> > > > > > >> > > downsides
> > > > > > >> > > > to
> > > > > > >> > > > > > >> > > including
> > > > > > >> > > > > > >> > > > > this
> > > > > > >> > > > > > >> > > > > > > information?
> > > > > > >> > > > > > >> > > > > > >
> > > > > > >> > > > > > >> > > > > > > 14)  Yes, we would need to copy the
> > > > partition
> > > > > > >> > metadata
> > > > > > >> > > > > file
> > > > > > >> > > > > > in
> > > > > > >> > > > > > >> > this
> > > > > > >> > > > > > >> > > > > > > process. I've updated the KIP to
> > include
> > > > > this.
> > > > > > >> > > > > > >> > > > > > >
> > > > > > >> > > > > > >> > > > > > > 15) I believe Lucas meant v1 and v2
> > here.
> > > > He
> > > > > > was
> > > > > > >> > > > referring
> > > > > > >> > > > > > to
> > > > > > >> > > > > > >> how
> > > > > > >> > > > > > >> > > the
> > > > > > >> > > > > > >> > > > > > > requests would fall under different
> IBP
> > > and
> > > > > > meant
> > > > > > >> > that
> > > > > > >> > > > > older
> > > > > > >> > > > > > >> > > brokers
> > > > > > >> > > > > > >> > > > > > would
> > > > > > >> > > > > > >> > > > > > > have to use the older version of the
> > > > request
> > > > > > and
> > > > > > >> the
> > > > > > >> > > > > > existing
> > > > > > >> > > > > > >> > topic
> > > > > > >> > > > > > >> > > > > > > deletion process. At first, it seemed
> > > like
> > > > > > tagged
> > > > > > >> > > fields
> > > > > > >> > > > > > would
> > > > > > >> > > > > > >> > > > resolve
> > > > > > >> > > > > > >> > > > > > > the IBP issue. However, we may need
> IBP
> > > for
> > > > > > this
> > > > > > >> > > request
> > > > > > >> > > > > > after
> > > > > > >> > > > > > >> > all
> > > > > > >> > > > > > >> > > > > since
> > > > > > >> > > > > > >> > > > > > > the controller handles the topic
> > deletion
> > > > > > >> > differently
> > > > > > >> > > > > > >> depending
> > > > > > >> > > > > > >> > on
> > > > > > >> > > > > > >> > > > the
> > > > > > >> > > > > > >> > > > > > IBP
> > > > > > >> > > > > > >> > > > > > > version. In an older version, we
> can't
> > > just
> > > > > > send
> > > > > > >> a
> > > > > > >> > > > > > StopReplica
> > > > > > >> > > > > > >> > > delete
> > > > > > >> > > > > > >> > > > > the
> > > > > > >> > > > > > >> > > > > > > topic immediately like we'd want to
> for
> > > > this
> > > > > > KIP.
> > > > > > >> > > > > > >> > > > > > >
> > > > > > >> > > > > > >> > > > > > > This makes me wonder if we want
> tagged
> > > > fields
> > > > > > on
> > > > > > >> all
> > > > > > >> > > the
> > > > > > >> > > > > > >> requests
> > > > > > >> > > > > > >> > > > after
> > > > > > >> > > > > > >> > > > > > > all. Let me know your thoughts!
> > > > > > >> > > > > > >> > > > > > >
> > > > > > >> > > > > > >> > > > > > > Justine
> > > > > > >> > > > > > >> > > > > > >
> > > > > > >> > > > > > >> > > > > > > On Tue, Sep 15, 2020 at 1:03 PM
> Justine
> > > > > Olshan
> > > > > > <
> > > > > > >> > > > > > >> > > jolshan@confluent.io
> > > > > > >> > > > > > >> > > > >
> > > > > > >> > > > > > >> > > > > > > wrote:
> > > > > > >> > > > > > >> > > > > > >
> > > > > > >> > > > > > >> > > > > > >> Hi all,
> > > > > > >> > > > > > >> > > > > > >> Jun brought up a good point in his
> > last
> > > > > email
> > > > > > >> about
> > > > > > >> > > > > tagged
> > > > > > >> > > > > > >> > fields,
> > > > > > >> > > > > > >> > > > and
> > > > > > >> > > > > > >> > > > > > >> I've updated the KIP to reflect that
> > the
> > > > > > >> changes to
> > > > > > >> > > > > > requests
> > > > > > >> > > > > > >> and
> > > > > > >> > > > > > >> > > > > > responses
> > > > > > >> > > > > > >> > > > > > >> will be in the form of tagged fields
> > to
> > > > > avoid
> > > > > > >> > > changing
> > > > > > >> > > > > IBP.
> > > > > > >> > > > > > >> > > > > > >>
> > > > > > >> > > > > > >> > > > > > >> Jun: I plan on sending a followup
> > email
> > > to
> > > > > > >> address
> > > > > > >> > > some
> > > > > > >> > > > > of
> > > > > > >> > > > > > >> the
> > > > > > >> > > > > > >> > > other
> > > > > > >> > > > > > >> > > > > > >> points.
> > > > > > >> > > > > > >> > > > > > >>
> > > > > > >> > > > > > >> > > > > > >> Thanks,
> > > > > > >> > > > > > >> > > > > > >> Justine
> > > > > > >> > > > > > >> > > > > > >>
> > > > > > >> > > > > > >> > > > > > >> On Mon, Sep 14, 2020 at 4:25 PM Jun
> > Rao
> > > <
> > > > > > >> > > > > jun@confluent.io>
> > > > > > >> > > > > > >> > wrote:
> > > > > > >> > > > > > >> > > > > > >>
> > > > > > >> > > > > > >> > > > > > >>> Hi, Justine,
> > > > > > >> > > > > > >> > > > > > >>>
> > > > > > >> > > > > > >> > > > > > >>> Thanks for the updated KIP. A few
> > > > comments
> > > > > > >> below.
> > > > > > >> > > > > > >> > > > > > >>>
> > > > > > >> > > > > > >> > > > > > >>> 10. LeaderAndIsr Response: Do we
> need
> > > the
> > > > > > topic
> > > > > > >> > > name?
> > > > > > >> > > > > > >> > > > > > >>>
> > > > > > >> > > > > > >> > > > > > >>> 11. For the changed
> request/response,
> > > > other
> > > > > > >> than
> > > > > > >> > > > > > >> LeaderAndIsr,
> > > > > > >> > > > > > >> > > > > > >>> UpdateMetadata, Metadata, do we
> need
> > to
> > > > > > include
> > > > > > >> > the
> > > > > > >> > > > > topic
> > > > > > >> > > > > > >> name?
> > > > > > >> > > > > > >> > > > > > >>>
> > > > > > >> > > > > > >> > > > > > >>> 12. It seems that upgrades don't
> > > require
> > > > > IBP.
> > > > > > >> Does
> > > > > > >> > > > that
> > > > > > >> > > > > > mean
> > > > > > >> > > > > > >> > the
> > > > > > >> > > > > > >> > > > new
> > > > > > >> > > > > > >> > > > > > >>> fields
> > > > > > >> > > > > > >> > > > > > >>> in all the request/response are
> added
> > > as
> > > > > > tagged
> > > > > > >> > > fields
> > > > > > >> > > > > > >> without
> > > > > > >> > > > > > >> > > > > bumping
> > > > > > >> > > > > > >> > > > > > up
> > > > > > >> > > > > > >> > > > > > >>> the request version? It would be
> > useful
> > > > to
> > > > > > make
> > > > > > >> > that
> > > > > > >> > > > > > clear.
> > > > > > >> > > > > > >> > > > > > >>>
> > > > > > >> > > > > > >> > > > > > >>> 13. Partition Metadata file: Do we
> > need
> > > > to
> > > > > > >> include
> > > > > > >> > > the
> > > > > > >> > > > > > topic
> > > > > > >> > > > > > >> > name
> > > > > > >> > > > > > >> > > > and
> > > > > > >> > > > > > >> > > > > > the
> > > > > > >> > > > > > >> > > > > > >>> partition id since they are implied
> > in
> > > > the
> > > > > > >> > directory
> > > > > > >> > > > > name?
> > > > > > >> > > > > > >> > > > > > >>>
> > > > > > >> > > > > > >> > > > > > >>> 14. In the JBOD mode, we support
> > > moving a
> > > > > > >> > > partition's
> > > > > > >> > > > > data
> > > > > > >> > > > > > >> from
> > > > > > >> > > > > > >> > > one
> > > > > > >> > > > > > >> > > > > > disk
> > > > > > >> > > > > > >> > > > > > >>> to
> > > > > > >> > > > > > >> > > > > > >>> another. Will the new partition
> > > metadata
> > > > > file
> > > > > > >> be
> > > > > > >> > > > copied
> > > > > > >> > > > > > >> during
> > > > > > >> > > > > > >> > > that
> > > > > > >> > > > > > >> > > > > > >>> process?
> > > > > > >> > > > > > >> > > > > > >>>
> > > > > > >> > > > > > >> > > > > > >>> 15. The KIP says "Remove deleted
> > topics
> > > > > from
> > > > > > >> > > replicas
> > > > > > >> > > > by
> > > > > > >> > > > > > >> > sending
> > > > > > >> > > > > > >> > > > > > >>> StopReplicaRequest V2 for any
> topics
> > > > which
> > > > > do
> > > > > > >> not
> > > > > > >> > > > > contain
> > > > > > >> > > > > > a
> > > > > > >> > > > > > >> > topic
> > > > > > >> > > > > > >> > > > ID,
> > > > > > >> > > > > > >> > > > > > and
> > > > > > >> > > > > > >> > > > > > >>> V3 for any topics which do contain
> a
> > > > topic
> > > > > > >> ID.".
> > > > > > >> > > > > However,
> > > > > > >> > > > > > it
> > > > > > >> > > > > > >> > > seems
> > > > > > >> > > > > > >> > > > > the
> > > > > > >> > > > > > >> > > > > > >>> updated controller will create all
> > > > missing
> > > > > > >> topic
> > > > > > >> > IDs
> > > > > > >> > > > > first
> > > > > > >> > > > > > >> > before
> > > > > > >> > > > > > >> > > > > doing
> > > > > > >> > > > > > >> > > > > > >>> other actions. So, is
> > > StopReplicaRequest
> > > > V2
> > > > > > >> > needed?
> > > > > > >> > > > > > >> > > > > > >>>
> > > > > > >> > > > > > >> > > > > > >>> Jun
> > > > > > >> > > > > > >> > > > > > >>>
> > > > > > >> > > > > > >> > > > > > >>> On Fri, Sep 11, 2020 at 10:31 AM
> John
> > > > > > Roesler <
> > > > > > >> > > > > > >> > > vvcephei@apache.org
> > > > > > >> > > > > > >> > > > >
> > > > > > >> > > > > > >> > > > > > >>> wrote:
> > > > > > >> > > > > > >> > > > > > >>>
> > > > > > >> > > > > > >> > > > > > >>> > Thanks, Justine!
> > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > >> > > > > > >> > > > > > >>> > Your response seems compelling to
> > me.
> > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > >> > > > > > >> > > > > > >>> > -John
> > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > >> > > > > > >> > > > > > >>> > On Fri, 2020-09-11 at 10:17
> -0700,
> > > > > Justine
> > > > > > >> > Olshan
> > > > > > >> > > > > wrote:
> > > > > > >> > > > > > >> > > > > > >>> > > Hello all,
> > > > > > >> > > > > > >> > > > > > >>> > > Thanks for continuing the
> > > > discussion! I
> > > > > > >> have a
> > > > > > >> > > few
> > > > > > >> > > > > > >> > responses
> > > > > > >> > > > > > >> > > to
> > > > > > >> > > > > > >> > > > > > your
> > > > > > >> > > > > > >> > > > > > >>> > points.
> > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > >> > > > > > >> > > > > > >>> > > Tom: You are correct in that
> this
> > > KIP
> > > > > has
> > > > > > >> not
> > > > > > >> > > > > > mentioned
> > > > > > >> > > > > > >> the
> > > > > > >> > > > > > >> > > > > > >>> > > DeleteTopicsRequest. I think
> that
> > > > this
> > > > > > >> would
> > > > > > >> > be
> > > > > > >> > > > out
> > > > > > >> > > > > of
> > > > > > >> > > > > > >> > scope
> > > > > > >> > > > > > >> > > > for
> > > > > > >> > > > > > >> > > > > > >>> now, but
> > > > > > >> > > > > > >> > > > > > >>> > > may be something worth adding
> in
> > > the
> > > > > > >> future.
> > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > >> > > > > > >> > > > > > >>> > > John: We did consider sequence
> > ids,
> > > > but
> > > > > > >> there
> > > > > > >> > > are
> > > > > > >> > > > a
> > > > > > >> > > > > > few
> > > > > > >> > > > > > >> > > reasons
> > > > > > >> > > > > > >> > > > > to
> > > > > > >> > > > > > >> > > > > > >>> favor
> > > > > > >> > > > > > >> > > > > > >>> > > UUIDs. There are several cases
> > > where
> > > > > > topics
> > > > > > >> > from
> > > > > > >> > > > > > >> different
> > > > > > >> > > > > > >> > > > > clusters
> > > > > > >> > > > > > >> > > > > > >>> may
> > > > > > >> > > > > > >> > > > > > >>> > > interact now and in the future.
> > For
> > > > > > >> example,
> > > > > > >> > > > Mirror
> > > > > > >> > > > > > >> Maker 2
> > > > > > >> > > > > > >> > > may
> > > > > > >> > > > > > >> > > > > > >>> benefit
> > > > > > >> > > > > > >> > > > > > >>> > > from being able to detect when
> a
> > > > > cluster
> > > > > > >> being
> > > > > > >> > > > > > mirrored
> > > > > > >> > > > > > >> is
> > > > > > >> > > > > > >> > > > > deleted
> > > > > > >> > > > > > >> > > > > > >>> and
> > > > > > >> > > > > > >> > > > > > >>> > > recreated and globally unique
> > > > > identifiers
> > > > > > >> > would
> > > > > > >> > > > make
> > > > > > >> > > > > > >> > > resolving
> > > > > > >> > > > > > >> > > > > > issues
> > > > > > >> > > > > > >> > > > > > >>> > > easier than sequence IDs which
> > may
> > > > > > collide
> > > > > > >> > > between
> > > > > > >> > > > > > >> > clusters.
> > > > > > >> > > > > > >> > > > > > KIP-405
> > > > > > >> > > > > > >> > > > > > >>> > > (tiered storage) will also
> > benefit
> > > > from
> > > > > > >> > globally
> > > > > > >> > > > > > unique
> > > > > > >> > > > > > >> IDs
> > > > > > >> > > > > > >> > > as
> > > > > > >> > > > > > >> > > > > > shared
> > > > > > >> > > > > > >> > > > > > >>> > > buckets may be used between
> > > clusters.
> > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > >> > > > > > >> > > > > > >>> > > Globally unique IDs would also
> > make
> > > > > > >> > > functionality
> > > > > > >> > > > > like
> > > > > > >> > > > > > >> > moving
> > > > > > >> > > > > > >> > > > > > topics
> > > > > > >> > > > > > >> > > > > > >>> > > between disparate clusters
> easier
> > > in
> > > > > the
> > > > > > >> > future,
> > > > > > >> > > > > > >> simplify
> > > > > > >> > > > > > >> > any
> > > > > > >> > > > > > >> > > > > > future
> > > > > > >> > > > > > >> > > > > > >>> > > implementations of backups and
> > > > > restores,
> > > > > > >> and
> > > > > > >> > > more.
> > > > > > >> > > > > In
> > > > > > >> > > > > > >> > > general,
> > > > > > >> > > > > > >> > > > > > >>> unique IDs
> > > > > > >> > > > > > >> > > > > > >>> > > would ensure that the source
> > > cluster
> > > > > > >> topics do
> > > > > > >> > > not
> > > > > > >> > > > > > >> conflict
> > > > > > >> > > > > > >> > > > with
> > > > > > >> > > > > > >> > > > > > the
> > > > > > >> > > > > > >> > > > > > >>> > > destination cluster topics.
> > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > >> > > > > > >> > > > > > >>> > > If we were to use sequence ids,
> > we
> > > > > would
> > > > > > >> need
> > > > > > >> > > > > > >> sufficiently
> > > > > > >> > > > > > >> > > > large
> > > > > > >> > > > > > >> > > > > > >>> cluster
> > > > > > >> > > > > > >> > > > > > >>> > > ids to be stored with the topic
> > > > > > >> identifiers or
> > > > > > >> > > we
> > > > > > >> > > > > run
> > > > > > >> > > > > > >> the
> > > > > > >> > > > > > >> > > risk
> > > > > > >> > > > > > >> > > > of
> > > > > > >> > > > > > >> > > > > > >>> > > collisions. This will give up
> any
> > > > > > >> advantage in
> > > > > > >> > > > > > >> compactness
> > > > > > >> > > > > > >> > > that
> > > > > > >> > > > > > >> > > > > > >>> sequence
> > > > > > >> > > > > > >> > > > > > >>> > > numbers may bring. Given these
> > > > > > advantages I
> > > > > > >> > > think
> > > > > > >> > > > it
> > > > > > >> > > > > > >> makes
> > > > > > >> > > > > > >> > > > sense
> > > > > > >> > > > > > >> > > > > to
> > > > > > >> > > > > > >> > > > > > >>> use
> > > > > > >> > > > > > >> > > > > > >>> > > UUIDs.
> > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > >> > > > > > >> > > > > > >>> > > Gokul: This is an interesting
> > idea,
> > > > but
> > > > > > >> this
> > > > > > >> > is
> > > > > > >> > > a
> > > > > > >> > > > > > >> breaking
> > > > > > >> > > > > > >> > > > > change.
> > > > > > >> > > > > > >> > > > > > >>> Out of
> > > > > > >> > > > > > >> > > > > > >>> > > scope for now, but maybe worth
> > > > > discussing
> > > > > > >> in
> > > > > > >> > the
> > > > > > >> > > > > > future.
> > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > >> > > > > > >> > > > > > >>> > > Hope this explains some of the
> > > > > decisions,
> > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > >> > > > > > >> > > > > > >>> > > Justine
> > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > >> > > > > > >> > > > > > >>> > > On Fri, Sep 11, 2020 at 8:27 AM
> > > Gokul
> > > > > > >> Ramanan
> > > > > > >> > > > > > >> Subramanian <
> > > > > > >> > > > > > >> > > > > > >>> > > gokul2411s@gmail.com> wrote:
> > > > > > >> > > > > > >> > > > > > >>> > >
> > > > > > >> > > > > > >> > > > > > >>> > > > Hi.
> > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > > >> > > > > > >> > > > > > >>> > > > Thanks for the KIP.
> > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > > >> > > > > > >> > > > > > >>> > > > Have you thought about
> whether
> > it
> > > > > makes
> > > > > > >> > sense
> > > > > > >> > > to
> > > > > > >> > > > > > >> support
> > > > > > >> > > > > > >> > > > > > >>> authorizing a
> > > > > > >> > > > > > >> > > > > > >>> > > > principal for a topic ID
> rather
> > > > than
> > > > > a
> > > > > > >> topic
> > > > > > >> > > > name
> > > > > > >> > > > > to
> > > > > > >> > > > > > >> > > achieve
> > > > > > >> > > > > > >> > > > > > >>> tighter
> > > > > > >> > > > > > >> > > > > > >>> > > > security?
> > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > > >> > > > > > >> > > > > > >>> > > > Or is the topic ID
> > fundamentally
> > > an
> > > > > > >> internal
> > > > > > >> > > > > detail
> > > > > > >> > > > > > >> > similar
> > > > > > >> > > > > > >> > > > to
> > > > > > >> > > > > > >> > > > > > >>> epochs
> > > > > > >> > > > > > >> > > > > > >>> > used
> > > > > > >> > > > > > >> > > > > > >>> > > > in a bunch of other places in
> > > > Kafka?
> > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > > >> > > > > > >> > > > > > >>> > > > Thanks.
> > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > > >> > > > > > >> > > > > > >>> > > > On Fri, Sep 11, 2020 at 4:06
> PM
> > > > John
> > > > > > >> > Roesler <
> > > > > > >> > > > > > >> > > > > > vvcephei@apache.org>
> > > > > > >> > > > > > >> > > > > > >>> > wrote:
> > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > > >> > > > > > >> > > > > > >>> > > > > Hello Justine,
> > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > >> > > > > > >> > > > > > >>> > > > > Thanks for the KIP!
> > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > >> > > > > > >> > > > > > >>> > > > > I happen to have been
> > > confronted
> > > > > > >> recently
> > > > > > >> > > with
> > > > > > >> > > > > the
> > > > > > >> > > > > > >> need
> > > > > > >> > > > > > >> > > to
> > > > > > >> > > > > > >> > > > > keep
> > > > > > >> > > > > > >> > > > > > >>> > track of
> > > > > > >> > > > > > >> > > > > > >>> > > > a
> > > > > > >> > > > > > >> > > > > > >>> > > > > large number of topics as
> > > > compactly
> > > > > > as
> > > > > > >> > > > > possible. I
> > > > > > >> > > > > > >> was
> > > > > > >> > > > > > >> > > > going
> > > > > > >> > > > > > >> > > > > to
> > > > > > >> > > > > > >> > > > > > >>> come
> > > > > > >> > > > > > >> > > > > > >>> > up
> > > > > > >> > > > > > >> > > > > > >>> > > > > with some way to dictionary
> > > > encode
> > > > > > the
> > > > > > >> > topic
> > > > > > >> > > > > names
> > > > > > >> > > > > > >> as
> > > > > > >> > > > > > >> > > > > integers,
> > > > > > >> > > > > > >> > > > > > >>> but
> > > > > > >> > > > > > >> > > > > > >>> > this
> > > > > > >> > > > > > >> > > > > > >>> > > > > seems much better!
> > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > >> > > > > > >> > > > > > >>> > > > > Apologies if this has been
> > > raised
> > > > > > >> before,
> > > > > > >> > > but
> > > > > > >> > > > > I’m
> > > > > > >> > > > > > >> > > wondering
> > > > > > >> > > > > > >> > > > > > >>> about the
> > > > > > >> > > > > > >> > > > > > >>> > > > > choice of UUID vs sequence
> > > number
> > > > > for
> > > > > > >> the
> > > > > > >> > > ids.
> > > > > > >> > > > > > >> > Typically,
> > > > > > >> > > > > > >> > > > > I’ve
> > > > > > >> > > > > > >> > > > > > >>> seen
> > > > > > >> > > > > > >> > > > > > >>> > UUIDs
> > > > > > >> > > > > > >> > > > > > >>> > > > > in two situations:
> > > > > > >> > > > > > >> > > > > > >>> > > > > 1. When processes need to
> > > > generate
> > > > > > >> > > > non-colliding
> > > > > > >> > > > > > >> > > > identifiers
> > > > > > >> > > > > > >> > > > > > >>> without
> > > > > > >> > > > > > >> > > > > > >>> > > > > coordination.
> > > > > > >> > > > > > >> > > > > > >>> > > > > 2. When the identifier
> needs
> > to
> > > > be
> > > > > > >> > > > “universally
> > > > > > >> > > > > > >> > unique”;
> > > > > > >> > > > > > >> > > > > I.e.,
> > > > > > >> > > > > > >> > > > > > >>> the
> > > > > > >> > > > > > >> > > > > > >>> > > > > identifier needs to
> > distinguish
> > > > the
> > > > > > >> entity
> > > > > > >> > > > from
> > > > > > >> > > > > > all
> > > > > > >> > > > > > >> > other
> > > > > > >> > > > > > >> > > > > > >>> entities
> > > > > > >> > > > > > >> > > > > > >>> > that
> > > > > > >> > > > > > >> > > > > > >>> > > > > could ever exist. This is
> > > useful
> > > > in
> > > > > > >> cases
> > > > > > >> > > > where
> > > > > > >> > > > > > >> > entities
> > > > > > >> > > > > > >> > > > from
> > > > > > >> > > > > > >> > > > > > all
> > > > > > >> > > > > > >> > > > > > >>> > kinds
> > > > > > >> > > > > > >> > > > > > >>> > > > of
> > > > > > >> > > > > > >> > > > > > >>> > > > > systems get mixed together,
> > > such
> > > > as
> > > > > > >> when
> > > > > > >> > > > dumping
> > > > > > >> > > > > > >> logs
> > > > > > >> > > > > > >> > > from
> > > > > > >> > > > > > >> > > > > all
> > > > > > >> > > > > > >> > > > > > >>> > processes
> > > > > > >> > > > > > >> > > > > > >>> > > > in
> > > > > > >> > > > > > >> > > > > > >>> > > > > a company into a common
> > system.
> > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > >> > > > > > >> > > > > > >>> > > > > Maybe I’m being
> > short-sighted,
> > > > but
> > > > > it
> > > > > > >> > > doesn’t
> > > > > > >> > > > > seem
> > > > > > >> > > > > > >> like
> > > > > > >> > > > > > >> > > > > either
> > > > > > >> > > > > > >> > > > > > >>> really
> > > > > > >> > > > > > >> > > > > > >>> > > > > applies here. It seems like
> > the
> > > > > > brokers
> > > > > > >> > > could
> > > > > > >> > > > > and
> > > > > > >> > > > > > >> would
> > > > > > >> > > > > > >> > > > > achieve
> > > > > > >> > > > > > >> > > > > > >>> > consensus
> > > > > > >> > > > > > >> > > > > > >>> > > > > when creating a topic
> anyway,
> > > > which
> > > > > > is
> > > > > > >> all
> > > > > > >> > > > > that’s
> > > > > > >> > > > > > >> > > required
> > > > > > >> > > > > > >> > > > to
> > > > > > >> > > > > > >> > > > > > >>> > generate
> > > > > > >> > > > > > >> > > > > > >>> > > > > non-colliding sequence ids.
> > For
> > > > the
> > > > > > >> > second,
> > > > > > >> > > as
> > > > > > >> > > > > you
> > > > > > >> > > > > > >> > > mention,
> > > > > > >> > > > > > >> > > > > we
> > > > > > >> > > > > > >> > > > > > >>> could
> > > > > > >> > > > > > >> > > > > > >>> > > > assign
> > > > > > >> > > > > > >> > > > > > >>> > > > > a UUID to the cluster as a
> > > whole,
> > > > > > which
> > > > > > >> > > would
> > > > > > >> > > > > > render
> > > > > > >> > > > > > >> > any
> > > > > > >> > > > > > >> > > > > > resource
> > > > > > >> > > > > > >> > > > > > >>> > scoped
> > > > > > >> > > > > > >> > > > > > >>> > > > to
> > > > > > >> > > > > > >> > > > > > >>> > > > > the broker universally
> unique
> > > as
> > > > > > well.
> > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > >> > > > > > >> > > > > > >>> > > > > The reason I mention this
> is
> > > > that,
> > > > > > >> > although
> > > > > > >> > > a
> > > > > > >> > > > > UUID
> > > > > > >> > > > > > >> is
> > > > > > >> > > > > > >> > way
> > > > > > >> > > > > > >> > > > > more
> > > > > > >> > > > > > >> > > > > > >>> > compact
> > > > > > >> > > > > > >> > > > > > >>> > > > > than topic names, it’s
> still
> > 16
> > > > > > bytes.
> > > > > > >> In
> > > > > > >> > > > > > contrast,
> > > > > > >> > > > > > >> a
> > > > > > >> > > > > > >> > > > 4-byte
> > > > > > >> > > > > > >> > > > > > >>> integer
> > > > > > >> > > > > > >> > > > > > >>> > > > > sequence id would give us 4
> > > > billion
> > > > > > >> unique
> > > > > > >> > > > > topics
> > > > > > >> > > > > > >> per
> > > > > > >> > > > > > >> > > > > cluster,
> > > > > > >> > > > > > >> > > > > > >>> which
> > > > > > >> > > > > > >> > > > > > >>> > > > seems
> > > > > > >> > > > > > >> > > > > > >>> > > > > like enough ;)
> > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > >> > > > > > >> > > > > > >>> > > > > Considering the number of
> > > > different
> > > > > > >> times
> > > > > > >> > > > these
> > > > > > >> > > > > > >> topic
> > > > > > >> > > > > > >> > > > > > >>> identifiers are
> > > > > > >> > > > > > >> > > > > > >>> > > > sent
> > > > > > >> > > > > > >> > > > > > >>> > > > > over the wire or stored in
> > > > memory,
> > > > > it
> > > > > > >> > seems
> > > > > > >> > > > like
> > > > > > >> > > > > > it
> > > > > > >> > > > > > >> > might
> > > > > > >> > > > > > >> > > > be
> > > > > > >> > > > > > >> > > > > > >>> worth
> > > > > > >> > > > > > >> > > > > > >>> > the
> > > > > > >> > > > > > >> > > > > > >>> > > > > additional 4x space
> savings.
> > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > >> > > > > > >> > > > > > >>> > > > > What do you think about
> this?
> > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > >> > > > > > >> > > > > > >>> > > > > Thanks,
> > > > > > >> > > > > > >> > > > > > >>> > > > > John
> > > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > > >> > > > > > >> > > > > > >>> > > > > On Fri, Sep 11, 2020, at
> > 03:20,
> > > > Tom
> > > > > > >> > Bentley
> > > > > > >> > > > > wrote:
> > > > > > >> > > > > > >> > > > > > >>> > > > > > Hi Justine,
> > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > > > >> > > > > > >> > > > > > >>> > > > > > This looks like a very
> > > welcome
> > > > > > >> > > improvement.
> > > > > > >> > > > > > >> Thanks!
> > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > > > >> > > > > > >> > > > > > >>> > > > > > Maybe I missed it, but
> the
> > > KIP
> > > > > > >> doesn't
> > > > > > >> > > seem
> > > > > > >> > > > to
> > > > > > >> > > > > > >> > mention
> > > > > > >> > > > > > >> > > > > > changing
> > > > > > >> > > > > > >> > > > > > >>> > > > > > DeleteTopicsRequest to
> > > identify
> > > > > the
> > > > > > >> > topic
> > > > > > >> > > > > using
> > > > > > >> > > > > > an
> > > > > > >> > > > > > >> > id.
> > > > > > >> > > > > > >> > > > > Maybe
> > > > > > >> > > > > > >> > > > > > >>> > that's out
> > > > > > >> > > > > > >> > > > > > >>> > > > > of
> > > > > > >> > > > > > >> > > > > > >>> > > > > > scope, but
> > > DeleteTopicsRequest
> > > > is
> > > > > > not
> > > > > > >> > > listed
> > > > > > >> > > > > > among
> > > > > > >> > > > > > >> > the
> > > > > > >> > > > > > >> > > > > Future
> > > > > > >> > > > > > >> > > > > > >>> Work
> > > > > > >> > > > > > >> > > > > > >>> > APIs
> > > > > > >> > > > > > >> > > > > > >>> > > > > > either.
> > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > > > >> > > > > > >> > > > > > >>> > > > > > Kind regards,
> > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > > > >> > > > > > >> > > > > > >>> > > > > > Tom
> > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > > > >> > > > > > >> > > > > > >>> > > > > > On Thu, Sep 10, 2020 at
> > 3:59
> > > PM
> > > > > > >> Satish
> > > > > > >> > > > > Duggana <
> > > > > > >> > > > > > >> > > > > > >>> > > > satish.duggana@gmail.com
> > > > > > >> > > > > > >> > > > > > >>> > > > > > wrote:
> > > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > Thanks Lucas/Justine
> for
> > > the
> > > > > nice
> > > > > > >> KIP.
> > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > It has several benefits
> > > which
> > > > > > also
> > > > > > >> > > include
> > > > > > >> > > > > > >> > > simplifying
> > > > > > >> > > > > > >> > > > > the
> > > > > > >> > > > > > >> > > > > > >>> topic
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > deletion process by
> > > > controller
> > > > > > and
> > > > > > >> > logs
> > > > > > >> > > > > > cleanup
> > > > > > >> > > > > > >> by
> > > > > > >> > > > > > >> > > > > brokers
> > > > > > >> > > > > > >> > > > > > in
> > > > > > >> > > > > > >> > > > > > >>> > corner
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > cases.
> > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > Best,
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > Satish.
> > > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > On Wed, Sep 9, 2020 at
> > > 10:07
> > > > PM
> > > > > > >> > Justine
> > > > > > >> > > > > > Olshan <
> > > > > > >> > > > > > >> > > > > > >>> > jolshan@confluent.io
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > wrote:
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Hello all, it's been
> > > > almost a
> > > > > > >> year!
> > > > > > >> > > I've
> > > > > > >> > > > > > made
> > > > > > >> > > > > > >> > some
> > > > > > >> > > > > > >> > > > > > changes
> > > > > > >> > > > > > >> > > > > > >>> to
> > > > > > >> > > > > > >> > > > > > >>> > this
> > > > > > >> > > > > > >> > > > > > >>> > > > > KIP
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > and hope to continue
> the
> > > > > > >> discussion.
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > One of the main
> changes
> > > > I've
> > > > > > >> added
> > > > > > >> > is
> > > > > > >> > > > now
> > > > > > >> > > > > > the
> > > > > > >> > > > > > >> > > > metadata
> > > > > > >> > > > > > >> > > > > > >>> response
> > > > > > >> > > > > > >> > > > > > >>> > > > will
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > include the topic ID
> (as
> > > > Colin
> > > > > > >> > > suggested).
> > > > > > >> > > > > > >> Clients
> > > > > > >> > > > > > >> > > can
> > > > > > >> > > > > > >> > > > > > >>> obtain the
> > > > > > >> > > > > > >> > > > > > >>> > > > > topicID
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > of a given topic
> through
> > a
> > > > > > >> > > > TopicDescription.
> > > > > > >> > > > > > The
> > > > > > >> > > > > > >> > > > topicId
> > > > > > >> > > > > > >> > > > > > will
> > > > > > >> > > > > > >> > > > > > >>> > also be
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > included with the
> > > > > UpdateMetadata
> > > > > > >> > > request.
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Let me know what you
> > all
> > > > > think.
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Thank you,
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Justine
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > On 2019/09/13
> 16:38:26,
> > > > > "Colin
> > > > > > >> > > McCabe" <
> > > > > > >> > > > > > >> > > > > > cmccabe@apache.org
> > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > >> > > > > > >> > > > > > >>> > wrote:
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Hi Lucas,
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Thanks for tackling
> > > this.
> > > > > > >> Topic
> > > > > > >> > IDs
> > > > > > >> > > > > are a
> > > > > > >> > > > > > >> > great
> > > > > > >> > > > > > >> > > > > idea,
> > > > > > >> > > > > > >> > > > > > >>> and
> > > > > > >> > > > > > >> > > > > > >>> > this
> > > > > > >> > > > > > >> > > > > > >>> > > > is
> > > > > > >> > > > > > >> > > > > > >>> > > > > a
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > really good writeup.
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > For
> > > > > /brokers/topics/[topic],
> > > > > > >> the
> > > > > > >> > > > schema
> > > > > > >> > > > > > >> version
> > > > > > >> > > > > > >> > > > > should
> > > > > > >> > > > > > >> > > > > > be
> > > > > > >> > > > > > >> > > > > > >>> > bumped
> > > > > > >> > > > > > >> > > > > > >>> > > > to
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > version 3, rather than
> 2.
> > > > > > KIP-455
> > > > > > >> > > bumped
> > > > > > >> > > > > the
> > > > > > >> > > > > > >> > version
> > > > > > >> > > > > > >> > > > of
> > > > > > >> > > > > > >> > > > > > this
> > > > > > >> > > > > > >> > > > > > >>> > znode
> > > > > > >> > > > > > >> > > > > > >>> > > > to
> > > > > > >> > > > > > >> > > > > > >>> > > > > 2
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > already :)
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Given that we're
> > going
> > > to
> > > > > be
> > > > > > >> > seeing
> > > > > > >> > > > > these
> > > > > > >> > > > > > >> > things
> > > > > > >> > > > > > >> > > as
> > > > > > >> > > > > > >> > > > > > >>> strings
> > > > > > >> > > > > > >> > > > > > >>> > as
> > > > > > >> > > > > > >> > > > > > >>> > > > lot
> > > > > > >> > > > > > >> > > > > > >>> > > > > (in
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > logs, in ZooKeeper, on
> > the
> > > > > > >> > command-line,
> > > > > > >> > > > > > etc.),
> > > > > > >> > > > > > >> > does
> > > > > > >> > > > > > >> > > it
> > > > > > >> > > > > > >> > > > > > make
> > > > > > >> > > > > > >> > > > > > >>> > sense to
> > > > > > >> > > > > > >> > > > > > >>> > > > > use
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > base64 when converting
> > them
> > > > to
> > > > > > >> > strings?
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Here is an example
> of
> > > the
> > > > > hex
> > > > > > >> > > > > > >> representation:
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > > > > >> > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > And here is an
> > example
> > > in
> > > > > > >> base64.
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > b8tRS7h4TJ2Vt43Dp85v2A
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > The base64 version
> > > saves
> > > > 15
> > > > > > >> > letters
> > > > > > >> > > > (to
> > > > > > >> > > > > be
> > > > > > >> > > > > > >> > fair,
> > > > > > >> > > > > > >> > > 4
> > > > > > >> > > > > > >> > > > of
> > > > > > >> > > > > > >> > > > > > >>> those
> > > > > > >> > > > > > >> > > > > > >>> > were
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > dashes that we could
> have
> > > > > elided
> > > > > > in
> > > > > > >> > the
> > > > > > >> > > > hex
> > > > > > >> > > > > > >> > > > > > representation.)
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Another thing to
> > > consider
> > > > > is
> > > > > > >> that
> > > > > > >> > we
> > > > > > >> > > > > > should
> > > > > > >> > > > > > >> > > specify
> > > > > > >> > > > > > >> > > > > > that
> > > > > > >> > > > > > >> > > > > > >>> the
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > all-zeroes UUID is not
> a
> > > > valid
> > > > > > >> topic
> > > > > > >> > > UUID.
> > > > > > >> > > > > >  We
> > > > > > >> > > > > > >> > can't
> > > > > > >> > > > > > >> > > > use
> > > > > > >> > > > > > >> > > > > > >>> null
> > > > > > >> > > > > > >> > > > > > >>> > for
> > > > > > >> > > > > > >> > > > > > >>> > > > this
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > because we can't pass a
> > > null
> > > > > UUID
> > > > > > >> over
> > > > > > >> > > the
> > > > > > >> > > > > RPC
> > > > > > >> > > > > > >> > > protocol
> > > > > > >> > > > > > >> > > > > > >>> (there
> > > > > > >> > > > > > >> > > > > > >>> > is no
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > special pattern for
> null,
> > > nor
> > > > > do
> > > > > > we
> > > > > > >> > want
> > > > > > >> > > > to
> > > > > > >> > > > > > >> waste
> > > > > > >> > > > > > >> > > space
> > > > > > >> > > > > > >> > > > > > >>> reserving
> > > > > > >> > > > > > >> > > > > > >>> > > > such
> > > > > > >> > > > > > >> > > > > > >>> > > > > a
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > pattern.)
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Maybe I missed it,
> > but
> > > > did
> > > > > > you
> > > > > > >> > > > describe
> > > > > > >> > > > > > >> > > "migration
> > > > > > >> > > > > > >> > > > > > of...
> > > > > > >> > > > > > >> > > > > > >>> > existing
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > topic[s] without topic
> > IDs"
> > > > in
> > > > > > >> detail
> > > > > > >> > in
> > > > > > >> > > > any
> > > > > > >> > > > > > >> > section?
> > > > > > >> > > > > > >> > > > It
> > > > > > >> > > > > > >> > > > > > >>> seems
> > > > > > >> > > > > > >> > > > > > >>> > like
> > > > > > >> > > > > > >> > > > > > >>> > > > > when
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > the new controller
> > becomes
> > > > > > active,
> > > > > > >> it
> > > > > > >> > > > should
> > > > > > >> > > > > > >> just
> > > > > > >> > > > > > >> > > > > generate
> > > > > > >> > > > > > >> > > > > > >>> random
> > > > > > >> > > > > > >> > > > > > >>> > > > > UUIDs for
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > these, and write the
> > random
> > > > > UUIDs
> > > > > > >> back
> > > > > > >> > > to
> > > > > > >> > > > > > >> > ZooKeeper.
> > > > > > >> > > > > > >> > > > It
> > > > > > >> > > > > > >> > > > > > >>> would be
> > > > > > >> > > > > > >> > > > > > >>> > > > good
> > > > > > >> > > > > > >> > > > > > >>> > > > > to
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > spell that out.  We
> > should
> > > > make
> > > > > > it
> > > > > > >> > clear
> > > > > > >> > > > > that
> > > > > > >> > > > > > >> this
> > > > > > >> > > > > > >> > > > > happens
> > > > > > >> > > > > > >> > > > > > >>> > regardless
> > > > > > >> > > > > > >> > > > > > >>> > > > > of
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > the inter-broker
> protocol
> > > > > version
> > > > > > >> > (it's
> > > > > > >> > > a
> > > > > > >> > > > > > >> > compatible
> > > > > > >> > > > > > >> > > > > > change).
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> "LeaderAndIsrRequests
> > > > > > >> including an
> > > > > > >> > > > > > >> > > > is_every_partition
> > > > > > >> > > > > > >> > > > > > >>> flag"
> > > > > > >> > > > > > >> > > > > > >>> > > > seems a
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > bit wordy.  Can we just
> > > call
> > > > > > these
> > > > > > >> > "full
> > > > > > >> > > > > > >> > > > > > >>> LeaderAndIsrRequests"?
> > > > > > >> > > > > > >> > > > > > >>> > Then
> > > > > > >> > > > > > >> > > > > > >>> > > > > the
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > RPC field could be
> named
> > > > > "full".
> > > > > > >> > Also,
> > > > > > >> > > it
> > > > > > >> > > > > > would
> > > > > > >> > > > > > >> > > > probably
> > > > > > >> > > > > > >> > > > > > be
> > > > > > >> > > > > > >> > > > > > >>> > better
> > > > > > >> > > > > > >> > > > > > >>> > > > > for the
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > RPC field to be an enum
> > of
> > > {
> > > > > > >> > > UNSPECIFIED,
> > > > > > >> > > > > > >> > > INCREMENTAL,
> > > > > > >> > > > > > >> > > > > FULL
> > > > > > >> > > > > > >> > > > > > >>> }, so
> > > > > > >> > > > > > >> > > > > > >>> > > > that
> > > > > > >> > > > > > >> > > > > > >>> > > > > we
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > can cleanly handle old
> > > > versions
> > > > > > (by
> > > > > > >> > > > treating
> > > > > > >> > > > > > >> them
> > > > > > >> > > > > > >> > as
> > > > > > >> > > > > > >> > > > > > >>> UNSPECIFIED)
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > In the
> > > > LeaderAndIsrRequest
> > > > > > >> > section,
> > > > > > >> > > > you
> > > > > > >> > > > > > >> write
> > > > > > >> > > > > > >> > "A
> > > > > > >> > > > > > >> > > > > final
> > > > > > >> > > > > > >> > > > > > >>> > deletion
> > > > > > >> > > > > > >> > > > > > >>> > > > > event
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > will be secheduled for
> X
> > ms
> > > > > after
> > > > > > >> the
> > > > > > >> > > > > > >> > > > LeaderAndIsrRequest
> > > > > > >> > > > > > >> > > > > > was
> > > > > > >> > > > > > >> > > > > > >>> > first
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > received..."  I guess
> > the X
> > > > > was a
> > > > > > >> > > > > placeholder
> > > > > > >> > > > > > >> that
> > > > > > >> > > > > > >> > > you
> > > > > > >> > > > > > >> > > > > > >>> intended
> > > > > > >> > > > > > >> > > > > > >>> > to
> > > > > > >> > > > > > >> > > > > > >>> > > > > replace
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > before posting? :)  In
> > any
> > > > > case,
> > > > > > >> this
> > > > > > >> > > > seems
> > > > > > >> > > > > > like
> > > > > > >> > > > > > >> > the
> > > > > > >> > > > > > >> > > > kind
> > > > > > >> > > > > > >> > > > > > of
> > > > > > >> > > > > > >> > > > > > >>> > thing
> > > > > > >> > > > > > >> > > > > > >>> > > > we'd
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > want a configuration
> for.
> > > > > Let's
> > > > > > >> > > describe
> > > > > > >> > > > > that
> > > > > > >> > > > > > >> > > > > > configuration
> > > > > > >> > > > > > >> > > > > > >>> key
> > > > > > >> > > > > > >> > > > > > >>> > > > > somewhere
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > in this KIP, including
> > what
> > > > its
> > > > > > >> > default
> > > > > > >> > > > > value
> > > > > > >> > > > > > >> is.
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > We should probably
> > also
> > > > > log a
> > > > > > >> > bunch
> > > > > > >> > > of
> > > > > > >> > > > > > >> messages
> > > > > > >> > > > > > >> > > at
> > > > > > >> > > > > > >> > > > > WARN
> > > > > > >> > > > > > >> > > > > > >>> level
> > > > > > >> > > > > > >> > > > > > >>> > > > when
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > something is scheduled
> > for
> > > > > > >> deletion,
> > > > > > >> > as
> > > > > > >> > > > > well.
> > > > > > >> > > > > > >> > (Maybe
> > > > > > >> > > > > > >> > > > > this
> > > > > > >> > > > > > >> > > > > > >>> was
> > > > > > >> > > > > > >> > > > > > >>> > > > > assumed, but
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > it would be good to
> > mention
> > > > > it).
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > I feel like there
> > are a
> > > > few
> > > > > > >> > sections
> > > > > > >> > > > > that
> > > > > > >> > > > > > >> > should
> > > > > > >> > > > > > >> > > be
> > > > > > >> > > > > > >> > > > > > >>> moved to
> > > > > > >> > > > > > >> > > > > > >>> > > > > "rejected
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > alternatives."  For
> > > example,
> > > > in
> > > > > > the
> > > > > > >> > > > > > DeleteTopics
> > > > > > >> > > > > > >> > > > section,
> > > > > > >> > > > > > >> > > > > > >>> since
> > > > > > >> > > > > > >> > > > > > >>> > we're
> > > > > > >> > > > > > >> > > > > > >>> > > > > not
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > going to do option 1 or
> > 2,
> > > > > these
> > > > > > >> > should
> > > > > > >> > > be
> > > > > > >> > > > > > moved
> > > > > > >> > > > > > >> > into
> > > > > > >> > > > > > >> > > > > > >>> "rejected
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > alternatives,"  rather
> > than
> > > > > > >> appearing
> > > > > > >> > > > > inline.
> > > > > > >> > > > > > >> > > Another
> > > > > > >> > > > > > >> > > > > case
> > > > > > >> > > > > > >> > > > > > >>> is
> > > > > > >> > > > > > >> > > > > > >>> > the
> > > > > > >> > > > > > >> > > > > > >>> > > > > "Should
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > we remove topic name
> from
> > > the
> > > > > > >> protocol
> > > > > > >> > > > where
> > > > > > >> > > > > > >> > > possible"
> > > > > > >> > > > > > >> > > > > > >>> section.
> > > > > > >> > > > > > >> > > > > > >>> > This
> > > > > > >> > > > > > >> > > > > > >>> > > > > is
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > clearly discussing a
> > design
> > > > > > >> > alternative
> > > > > > >> > > > that
> > > > > > >> > > > > > >> we're
> > > > > > >> > > > > > >> > > not
> > > > > > >> > > > > > >> > > > > > >>> proposing
> > > > > > >> > > > > > >> > > > > > >>> > to
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > implement: removing the
> > > topic
> > > > > > name
> > > > > > >> > from
> > > > > > >> > > > > those
> > > > > > >> > > > > > >> > > > protocols.
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Is it really
> > necessary
> > > to
> > > > > > have
> > > > > > >> a
> > > > > > >> > new
> > > > > > >> > > > > > >> > > > > > >>> > /admin/delete_topics_by_id
> > > > > > >> > > > > > >> > > > > > >>> > > > > path
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > in ZooKeeper?  It seems
> > > like
> > > > we
> > > > > > >> don't
> > > > > > >> > > > really
> > > > > > >> > > > > > >> need
> > > > > > >> > > > > > >> > > this.
> > > > > > >> > > > > > >> > > > > > >>> Whenever
> > > > > > >> > > > > > >> > > > > > >>> > > > > there is
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > a new controller, we'll
> > > send
> > > > > out
> > > > > > >> full
> > > > > > >> > > > > > >> > > > > LeaderAndIsrRequests
> > > > > > >> > > > > > >> > > > > > >>> which
> > > > > > >> > > > > > >> > > > > > >>> > will
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > trigger the stale
> topics
> > to
> > > > be
> > > > > > >> cleaned
> > > > > > >> > > up.
> > > > > > >> > > > > >  The
> > > > > > >> > > > > > >> > > active
> > > > > > >> > > > > > >> > > > > > >>> > controller
> > > > > > >> > > > > > >> > > > > > >>> > > > will
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > also send the full
> > > > > > >> LeaderAndIsrRequest
> > > > > > >> > > to
> > > > > > >> > > > > > >> brokers
> > > > > > >> > > > > > >> > > that
> > > > > > >> > > > > > >> > > > > are
> > > > > > >> > > > > > >> > > > > > >>> just
> > > > > > >> > > > > > >> > > > > > >>> > > > > starting
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > up.    So we don't
> really
> > > > need
> > > > > > this
> > > > > > >> > kind
> > > > > > >> > > > of
> > > > > > >> > > > > > >> > two-phase
> > > > > > >> > > > > > >> > > > > > commit
> > > > > > >> > > > > > >> > > > > > >>> > (send
> > > > > > >> > > > > > >> > > > > > >>> > > > out
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > StopReplicasRequest,
> get
> > > ACKs
> > > > > > from
> > > > > > >> all
> > > > > > >> > > > > nodes,
> > > > > > >> > > > > > >> > commit
> > > > > > >> > > > > > >> > > by
> > > > > > >> > > > > > >> > > > > > >>> removing
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > /admin/delete_topics
> > node)
> > > > any
> > > > > > >> more.
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > You mention that
> > > > > FetchRequest
> > > > > > >> will
> > > > > > >> > > now
> > > > > > >> > > > > > >> include
> > > > > > >> > > > > > >> > > UUID
> > > > > > >> > > > > > >> > > > > to
> > > > > > >> > > > > > >> > > > > > >>> avoid
> > > > > > >> > > > > > >> > > > > > >>> > > > issues
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > where requests are made
> > to
> > > > > stale
> > > > > > >> > > > partitions.
> > > > > > >> > > > > > >> > > However,
> > > > > > >> > > > > > >> > > > > > >>> adding a
> > > > > > >> > > > > > >> > > > > > >>> > UUID
> > > > > > >> > > > > > >> > > > > > >>> > > > to
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > MetadataRequest is
> listed
> > > as
> > > > > > future
> > > > > > >> > > work,
> > > > > > >> > > > > out
> > > > > > >> > > > > > of
> > > > > > >> > > > > > >> > > scope
> > > > > > >> > > > > > >> > > > > for
> > > > > > >> > > > > > >> > > > > > >>> this
> > > > > > >> > > > > > >> > > > > > >>> > KIP.
> > > > > > >> > > > > > >> > > > > > >>> > > > > How
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > will the client learn
> > what
> > > > the
> > > > > > >> topic
> > > > > > >> > > UUID
> > > > > > >> > > > > is,
> > > > > > >> > > > > > if
> > > > > > >> > > > > > >> > the
> > > > > > >> > > > > > >> > > > > > metadata
> > > > > > >> > > > > > >> > > > > > >>> > > > response
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > doesn't include that
> > > > > information?
> > > > > > >> It
> > > > > > >> > > > seems
> > > > > > >> > > > > > like
> > > > > > >> > > > > > >> > > adding
> > > > > > >> > > > > > >> > > > > the
> > > > > > >> > > > > > >> > > > > > >>> UUID
> > > > > > >> > > > > > >> > > > > > >>> > to
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > MetadataResponse would
> be
> > > an
> > > > > > >> > improvement
> > > > > > >> > > > > here
> > > > > > >> > > > > > >> that
> > > > > > >> > > > > > >> > > > might
> > > > > > >> > > > > > >> > > > > > not
> > > > > > >> > > > > > >> > > > > > >>> be
> > > > > > >> > > > > > >> > > > > > >>> > too
> > > > > > >> > > > > > >> > > > > > >>> > > > > hard to
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > make.
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > best,
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Colin
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > On Mon, Sep 9,
> 2019,
> > at
> > > > > > 17:48,
> > > > > > >> > > Ryanne
> > > > > > >> > > > > > Dolan
> > > > > > >> > > > > > >> > > wrote:
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > Lucas, this would
> > be
> > > > > great.
> > > > > > >> I've
> > > > > > >> > > run
> > > > > > >> > > > > > into
> > > > > > >> > > > > > >> > > issues
> > > > > > >> > > > > > >> > > > > with
> > > > > > >> > > > > > >> > > > > > >>> > topics
> > > > > > >> > > > > > >> > > > > > >>> > > > > being
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > resurrected
> > > > accidentally,
> > > > > > >> since
> > > > > > >> > a
> > > > > > >> > > > > client
> > > > > > >> > > > > > >> > cannot
> > > > > > >> > > > > > >> > > > > > easily
> > > > > > >> > > > > > >> > > > > > >>> > > > > distinguish
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > between
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > a deleted topic
> > and a
> > > > new
> > > > > > >> topic
> > > > > > >> > > with
> > > > > > >> > > > > the
> > > > > > >> > > > > > >> same
> > > > > > >> > > > > > >> > > > name.
> > > > > > >> > > > > > >> > > > > > I'd
> > > > > > >> > > > > > >> > > > > > >>> > need
> > > > > > >> > > > > > >> > > > > > >>> > > > the
> > > > > > >> > > > > > >> > > > > > >>> > > > > ID
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > accessible from
> the
> > > > > client
> > > > > > to
> > > > > > >> > > solve
> > > > > > >> > > > > that
> > > > > > >> > > > > > >> > issue,
> > > > > > >> > > > > > >> > > > but
> > > > > > >> > > > > > >> > > > > > >>> this
> > > > > > >> > > > > > >> > > > > > >>> > is a
> > > > > > >> > > > > > >> > > > > > >>> > > > > good
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > first
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > step.
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > Ryanne
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > On Wed, Sep 4,
> 2019
> > > at
> > > > > 1:41
> > > > > > >> PM
> > > > > > >> > > Lucas
> > > > > > >> > > > > > >> > > Bradstreet <
> > > > > > >> > > > > > >> > > > > > >>> > > > > lucas@confluent.io>
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > wrote:
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Hi all,
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > I would like to
> > > kick
> > > > > off
> > > > > > >> > > > discussion
> > > > > > >> > > > > of
> > > > > > >> > > > > > >> > > KIP-516,
> > > > > > >> > > > > > >> > > > > an
> > > > > > >> > > > > > >> > > > > > >>> > > > > implementation
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > of topic
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > IDs for Kafka.
> > > Topic
> > > > > IDs
> > > > > > >> aim
> > > > > > >> > to
> > > > > > >> > > > > solve
> > > > > > >> > > > > > >> topic
> > > > > > >> > > > > > >> > > > > > >>> uniqueness
> > > > > > >> > > > > > >> > > > > > >>> > > > > problems in
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > Kafka,
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > where referring
> > to
> > > a
> > > > > > topic
> > > > > > >> by
> > > > > > >> > > name
> > > > > > >> > > > > > >> alone is
> > > > > > >> > > > > > >> > > > > > >>> insufficient.
> > > > > > >> > > > > > >> > > > > > >>> > > > Such
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > cases
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > include when a
> > > topic
> > > > > has
> > > > > > >> been
> > > > > > >> > > > > deleted
> > > > > > >> > > > > > >> and
> > > > > > >> > > > > > >> > > > > recreated
> > > > > > >> > > > > > >> > > > > > >>> with
> > > > > > >> > > > > > >> > > > > > >>> > the
> > > > > > >> > > > > > >> > > > > > >>> > > > > same
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > name.
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Unique
> > identifiers
> > > > will
> > > > > > >> help
> > > > > > >> > > > > simplify
> > > > > > >> > > > > > >> and
> > > > > > >> > > > > > >> > > > improve
> > > > > > >> > > > > > >> > > > > > >>> Kafka's
> > > > > > >> > > > > > >> > > > > > >>> > > > topic
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > deletion
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > process, as
> well
> > as
> > > > > > prevent
> > > > > > >> > > cases
> > > > > > >> > > > > > where
> > > > > > >> > > > > > >> > > brokers
> > > > > > >> > > > > > >> > > > > may
> > > > > > >> > > > > > >> > > > > > >>> > > > incorrectly
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > interact
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > with stale
> > versions
> > > > of
> > > > > > >> topics.
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > >> > > > > > >> > > > > > >>>
> > > > > > >> > > > > > >> > > > > >
> > > > > > >> > > > > > >> > > > >
> > > > > > >> > > > > > >> > > >
> > > > > > >> > > > > > >> > >
> > > > > > >> > > > > > >> >
> > > > > > >> > > > > > >>
> > > > > > >> > > > > >
> > > > > > >> > > > >
> > > > > > >> > > >
> > > > > > >> > >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Looking forward
> > to
> > > > your
> > > > > > >> > > thoughts.
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Lucas
> > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > >> > > > > > >> > > > > > >>> >
> > > > > > >> > > > > > >> > > > > > >>>
> > > > > > >> > > > > > >> > > > > > >>
> > > > > > >> > > > > > >> > > > > >
> > > > > > >> > > > > > >> > > > >
> > > > > > >> > > > > > >> > > >
> > > > > > >> > > > > > >> > >
> > > > > > >> > > > > > >> >
> > > > > > >> > > > > > >>
> > > > > > >> > > > > > >
> > > > > > >> > > > > >
> > > > > > >> > > > >
> > > > > > >> > > >
> > > > > > >> > >
> > > > > > >> >
> > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Lucas Bradstreet <lu...@confluent.io>]

> 2. Part of the usage of the file is to have persistent storage of the
topic
ID and use it to compare with the ID supplied in the LeaderAndIsr Request.
There is some discussion in the KIP about changes to the directory
structure, but I believe directory changes were considered to be out of
scope when the KIP was written.


Yeah, I was hoping to get a better understanding of why it was taken out of
scope. Perhaps Lucas Bradstreet might have more insight about the decision.
Basically my point is that we have to create additional infrastructure here
to support the name/id mapping, so I wanted to understand if we just
consider this a sort of tech debt. It would be useful to cover how we
handle the case when this file gets corrupted. Seems like we just have to
trust that it matches whatever the controller tells us and rewrite it?


Hi Jason, Justine,

My thought process is that we will likely need the metadata file whether we
rename the directories or not.
Linux supports filenames of up to 255 bytes and that would not be enough to
support a directory name
 including both the name and topic ID. Given that fact, it seemed
reasonable to add the metadata file
and leave the directory rename until some time in the future (possibly
never).

If the file does get corrupted I think you're right. We would either have
to trust it matches what the controller tells us
 or error out and let an administrator resolve it by checking across
replicas for consistency.

Lucas


On Thu, Sep 24, 2020 at 3:41 PM Jason Gustafson <ja...@confluent.io> wrote:

> Thanks Justine. Responses below:
>
> > 1. Yes, the directory will still be based on the topic names.
> LeaderAndIsrRequest is one of the few requests that will still contain the
> topic name. So I think we have this covered. Sorry for confusion.
>
> Ah, you're right. My eyes passed right over the field.
>
> > 2. Part of the usage of the file is to have persistent storage of the
> topic
> ID and use it to compare with the ID supplied in the LeaderAndIsr Request.
> There is some discussion in the KIP about changes to the directory
> structure, but I believe directory changes were considered to be out of
> scope when the KIP was written.
>
> Yeah, I was hoping to get a better understanding of why it was taken out of
> scope. Perhaps Lucas Bradstreet might have more insight about the decision.
> Basically my point is that we have to create additional infrastructure here
> to support the name/id mapping, so I wanted to understand if we just
> consider this a sort of tech debt. It would be useful to cover how we
> handle the case when this file gets corrupted. Seems like we just have to
> trust that it matches whatever the controller tells us and rewrite it?
>
> > 3. I think this is a good point, but I again I wonder about the scope of
> the KIP. Most of the changes mentioned in the KIP are for supporting topic
> deletion and I believe that is why the produce request was listed under
> future work.
>
> That's fair. I brought it up since `Fetch` is already included. If we've
> got `Metadata` and `Fetch`, seems we may as well do `Produce` and save an
> extra kip. No strong objection though if you want to leave it out.
>
>
> -Jason
>
>
> On Thu, Sep 24, 2020 at 3:26 PM Justine Olshan <jo...@confluent.io>
> wrote:
>
> > Hi Jason,
> >
> > Thanks for your comments.
> >
> > 1. Yes, the directory will still be based on the topic names.
> > LeaderAndIsrRequest is one of the few requests that will still contain
> the
> > topic name. So I think we have this covered. Sorry for confusion.
> >
> > 2. Part of the usage of the file is to have persistent storage of the
> topic
> > ID and use it to compare with the ID supplied in the LeaderAndIsr
> Request.
> > There is some discussion in the KIP about changes to the directory
> > structure, but I believe directory changes were considered to be out of
> > scope when the KIP was written.
> >
> > 3. I think this is a good point, but I again I wonder about the scope of
> > the KIP. Most of the changes mentioned in the KIP are for supporting
> topic
> > deletion and I believe that is why the produce request was listed under
> > future work.
> >
> > 4. This sounds like it might be a good solution, but I will need to
> discuss
> > more with KIP-500 folks to get the details right.
> >
> > Thanks,
> > Justine
> >
> > On Thu, Sep 24, 2020 at 12:30 PM Jason Gustafson <ja...@confluent.io>
> > wrote:
> >
> > > Hi Justine,
> > >
> > > Thanks for picking up this work. I have a few questions/comments:
> > >
> > > 1. It sounds like the directory structure is still going to be based on
> > > topic names. Do I have that right? One complication is that the
> > > LeaderAndIsr request does not include the topic name any longer. This
> > means
> > > that a replica must wait for the UpdateMetadata request to arrive with
> > the
> > > topic name mapping before it can create the directory. I wonder if it
> > would
> > > be simpler to avoid assumptions on the ordering of UpdateMetadata and
> let
> > > LeaderAndIsr include the topic name as well. Feels like we are not
> saving
> > > that much by excluding it.
> > >
> > > 2. On a related note, it seems that the reason we have the partition
> > > metadata file is because we are not changing the directory structure.
> We
> > > need it so that we remember which directories map to which topic id. I
> > > think it would be useful to add some detail about why changing the
> > > directory layout was rejected. Basically trying to understand how much
> > > effort we are saving by skipping this step if we have to implement this
> > > additional bookkeeping. One concern I have for example is that the
> > > partition metadata file gets corrupt and then all bets are off as far
> as
> > > topic consistency.
> > >
> > > 3. Is it worth adding support now for topic ids in the `Produce`
> request
> > > now? Overhead is mentioned as one of the motivations and the best APIs
> to
> > > get savings from are `Produce` and `Fetch`.
> > >
> > > 4. When it comes to bootstrapping the metadata topic with respect to
> > > KIP-500, one suggestion would be to reserve a sentinel topic ID which
> can
> > > be used initially by new brokers. When the first controller is elected,
> > it
> > > can assign a topicId to the metadata topic. This is a bit similar to
> how
> > we
> > > were planning to generate the clusterId.
> > >
> > > Thanks,
> > > Jason
> > >
> > >
> > > On Thu, Sep 24, 2020 at 11:10 AM Jun Rao <ju...@confluent.io> wrote:
> > >
> > > > Hi, Justine,
> > > >
> > > > Thanks for the updated KIP. A few more comments below.
> > > >
> > > > 30.  {"name": "id", "type": "string", "doc": "version id"}}: The doc
> > > should
> > > > say UUID.
> > > >
> > > > 31. LeaderAndIsrResponse v5 and StopReplicaResponse v4 : It seems
> there
> > > is
> > > > no need to add topic_id at partitions level.
> > > >
> > > > 32. Regarding partition metadata file. Perhaps the key can be a
> single
> > > > word, sth like the following.
> > > > version: 0
> > > > topic_id: 46bdb63f-9e8d-4a38-bf7b-ee4eb2a794e4
> > > >
> > > > 33. Another tricky thing that I realized is how to support the
> metadata
> > > > topic introduced in KIP-595. It's a bit tricky to assign a UUID to
> the
> > > > metadata topic since we have a chicken-and-egg problem. The
> controller
> > > > needs to persist the UUID in the metadata topic in order to assign
> one
> > > > successfully, but the metadata topic is needed to elect a controller
> > > > (KIP-631). So, this probably needs a bit more thought.
> > > >
> > > > Jun
> > > >
> > > > On Thu, Sep 24, 2020 at 3:04 AM Ismael Juma <is...@juma.me.uk>
> wrote:
> > > >
> > > > > Also, can we provide more details on how the Partition Metadata
> file
> > > will
> > > > > be used?
> > > > >
> > > > > Ismael
> > > > >
> > > > > On Thu, Sep 24, 2020 at 3:01 AM Ismael Juma <is...@juma.me.uk>
> > wrote:
> > > > >
> > > > > > Hi Justine,
> > > > > >
> > > > > > I think we need to update the "Rejected Alternatives" section to
> > take
> > > > > into
> > > > > > account that the proposal now removes the topic name from the
> fetch
> > > > > > request. Also, if we are removing it from the Fetch request, does
> > it
> > > > make
> > > > > > sense not to remove it from similar requests like
> > ListOffsetRequest?
> > > > > >
> > > > > > Ismael
> > > > > >
> > > > > > On Thu, Sep 24, 2020 at 2:46 AM David Jacot <djacot@confluent.io
> >
> > > > wrote:
> > > > > >
> > > > > >> Hi Justine,
> > > > > >>
> > > > > >> Thanks for the KIP. I finally had time to read it :). It is a
> > great
> > > > > >> improvement.
> > > > > >>
> > > > > >> I have a few comments/questions:
> > > > > >>
> > > > > >> 1. It seems that the schema of the StopReplicaRequest is
> slightly
> > > > > >> outdated.
> > > > > >> We
> > > > > >> did some changes as part of KIP-570. V3 is already organized by
> > > > topics.
> > > > > >>
> > > > > >> 2. I just want to make sure that I understand the reconciliation
> > > > > >> logic correctly. When
> > > > > >> an "INCREMENTAL" LeaderAndIsr Request is received, the broker
> will
> > > > also
> > > > > >> reconcile
> > > > > >> when the local uuid does not match the uuid in the request,
> right?
> > > In
> > > > > this
> > > > > >> case, the
> > > > > >> local log is staged for deletion.
> > > > > >>
> > > > > >> 3. In the documentation of the `delete.stale.topic.delay.ms`
> > > config,
> > > > it
> > > > > >> says "When a
> > > > > >> FULL LeaderAndIsrRequest is received..." but I suppose it
> applies
> > to
> > > > > both
> > > > > >> types.
> > > > > >>
> > > > > >> Best,
> > > > > >> David
> > > > > >>
> > > > > >> On Thu, Sep 24, 2020 at 1:40 AM Justine Olshan <
> > > jolshan@confluent.io>
> > > > > >> wrote:
> > > > > >>
> > > > > >> > Hi Jun,
> > > > > >> >
> > > > > >> > Thanks for the comments. I apologize for some of the typos and
> > > > > >> confusion.
> > > > > >> > I’ve updated the KIP to fix some of the issues you mentioned.
> > > > > >> >
> > > > > >> > 20.2 I’ve changed the type to String.
> > > > > >> > 20.1/3 I’ve updated the TopicZNode to fix formatting and
> reflect
> > > the
> > > > > >> latest
> > > > > >> > version before this change.
> > > > > >> >
> > > > > >> > 21. You are correct and I’ve removed this line. I’ve also
> added
> > a
> > > > line
> > > > > >> > mentioning an IBP bump is necessary for migration
> > > > > >> >
> > > > > >> > 22. I think the wording was unclear but your summary is what
> was
> > > > > >> intended
> > > > > >> > by this line. I’ve updated to make this point more clear.
> > “Remove
> > > > > >> deleted
> > > > > >> > topics from replicas by sending StopReplicaRequest V3 before
> the
> > > IBP
> > > > > >> bump
> > > > > >> > using the old logic, and using V4 and the new logic with topic
> > IDs
> > > > > after
> > > > > >> > the IBP bump.”
> > > > > >> >
> > > > > >> > 23. I’ve removed the unspecified type from the KIP and mention
> > > that
> > > > > IBP
> > > > > >> > will be used to handle this request. “IBP will be used to
> > > determine
> > > > > >> whether
> > > > > >> > this new form of the request will be used. For older requests,
> > we
> > > > will
> > > > > >> > ignore this field and default to previous behavior.”
> > > > > >> >
> > > > > >> > 24. I’ve fixed this typo.
> > > > > >> >
> > > > > >> > 25. I've added a topics at a higher level for
> > LeaderAndIsrResponse
> > > > v5,
> > > > > >> > StopReplicaResponse v4. I've also changed StopReplicaRequest
> v4
> > to
> > > > > have
> > > > > >> > topics at a higher level.
> > > > > >> >
> > > > > >> > 26. I’ve updated forgotten_topics_data--added the topic ID and
> > > > removed
> > > > > >> the
> > > > > >> > topic name
> > > > > >> >
> > > > > >> > 27. I’ve decided on plain text, and I’ve added an example.
> > > > > >> >
> > > > > >> > 28. I’ve added this idea to future work.
> > > > > >> >
> > > > > >> > Thanks again for taking a look,
> > > > > >> >
> > > > > >> > Justine
> > > > > >> >
> > > > > >> > On Wed, Sep 23, 2020 at 10:28 AM Jun Rao <ju...@confluent.io>
> > > wrote:
> > > > > >> >
> > > > > >> > > Hi, Justine,
> > > > > >> > >
> > > > > >> > > Thanks for the response. Made another pass. A few more
> > comments
> > > > > below.
> > > > > >> > >
> > > > > >> > > 20. znode schema:
> > > > > >> > > 20.1 It seems that {"name": "version", "type": "int", "id":
> > > > "UUID",
> > > > > >> > "doc":
> > > > > >> > > "version id"} should be {"name": "version", "type": "int"},
> > > > {"name":
> > > > > >> > "id",
> > > > > >> > > "type": "UUID", "doc": "version id"}.
> > > > > >> > > 20.2 The znode format is JSON which doesn't have UUID type.
> So
> > > the
> > > > > >> type
> > > > > >> > > probably should be string?
> > > > > >> > > 20.3 Also, the existing format used seems outdated. It
> should
> > > have
> > > > > the
> > > > > >> > > following format.
> > > > > >> > >     Json.encodeAsBytes(Map(
> > > > > >> > >       "version" -> 2,
> > > > > >> > >       "partitions" -> replicaAssignmentJson.asJava,
> > > > > >> > >       "adding_replicas" ->
> > addingReplicasAssignmentJson.asJava,
> > > > > >> > >       "removing_replicas" ->
> > > removingReplicasAssignmentJson.asJava
> > > > > >> > >     ).asJava)
> > > > > >> > >   }
> > > > > >> > >
> > > > > >> > > 21. Migration: The KIP says "The migration process can take
> > > place
> > > > > >> without
> > > > > >> > > an inter-broker protocol bump, as the format stored in
> > > > > >> > > /brokers/topics/[topic] will be compatible with older broker
> > > > > >> versions."
> > > > > >> > > However, since we are bumping up the version of inter-broker
> > > > > >> requests, it
> > > > > >> > > seems that we need to use IBP for migration.
> > > > > >> > >
> > > > > >> > > 22. The KIP says "Remove deleted topics from replicas by
> > sending
> > > > > >> > > StopReplicaRequest V3 for any topics which do not contain a
> > > topic
> > > > > ID,
> > > > > >> and
> > > > > >> > > V4 for any topics which do contain a topic ID.". However, if
> > we
> > > > use
> > > > > >> IBP,
> > > > > >> > it
> > > > > >> > > seems that the controller will either send
> StopReplicaRequest
> > V3
> > > > > >> > > or StopReplicaRequest V4, but never mixed V3 and V4 for
> > > different
> > > > > >> topics.
> > > > > >> > > Basically, before the IBP bump, V3 will be used. After the
> IBP
> > > > bump,
> > > > > >> > > topicId will be created and V4 will be used.
> > > > > >> > >
> > > > > >> > > 23. Given that we depend on IBP, do we still need "0
> > > UNSPECIFIED"
> > > > > >> > > in LeaderAndIsr?
> > > > > >> > >
> > > > > >> > > 24. LeaderAndIsrResponse v5 : It still has the topic field.
> > > > > >> > >
> > > > > >> > > 25. LeaderAndIsrResponse v5, StopReplicaResponse v4: Could
> we
> > > use
> > > > > this
> > > > > >> > > opportunity to organize the response in 2 levels, first by
> > > topic,
> > > > > >> then by
> > > > > >> > > partition, as most other requests/responses?
> > > > > >> > >
> > > > > >> > > 26. FetchRequest v13 : Should forgotten_topics_data use
> > topicId
> > > > too?
> > > > > >> > >
> > > > > >> > > 27. "This file can either be plain text (key/value pairs) or
> > > > JSON."
> > > > > >> Have
> > > > > >> > we
> > > > > >> > > decided which one to use? Also, it would be helpful to
> provide
> > > an
> > > > > >> > example.
> > > > > >> > >
> > > > > >> > > 28. Future improvement: Another future improvement
> opportunity
> > > is
> > > > to
> > > > > >> use
> > > > > >> > > topicId in GroupMetadataManager.offsetCommitKey in the
> > > > offset_commit
> > > > > >> > topic.
> > > > > >> > > This may save some space.
> > > > > >> > >
> > > > > >> > > Thanks,
> > > > > >> > >
> > > > > >> > > Jun
> > > > > >> > >
> > > > > >> > > On Wed, Sep 23, 2020 at 8:50 AM Justine Olshan <
> > > > > jolshan@confluent.io>
> > > > > >> > > wrote:
> > > > > >> > >
> > > > > >> > > > Hi Tom,
> > > > > >> > > >
> > > > > >> > > > Thanks for the comment. I think this is a really good idea
> > and
> > > > it
> > > > > >> has
> > > > > >> > > been
> > > > > >> > > > added to the KIP under the newly added tooling section.
> > > > > >> > > >
> > > > > >> > > > Thanks again,
> > > > > >> > > > Justine
> > > > > >> > > >
> > > > > >> > > > On Wed, Sep 23, 2020 at 3:17 AM Tom Bentley <
> > > > tbentley@redhat.com>
> > > > > >> > wrote:
> > > > > >> > > >
> > > > > >> > > > > Hi Justine,
> > > > > >> > > > >
> > > > > >> > > > > I know you started the vote thread, but on re-reading
> the
> > > KIP
> > > > I
> > > > > >> > noticed
> > > > > >> > > > > that although the topic id is included in the
> > > MetadataResponse
> > > > > >> it's
> > > > > >> > not
> > > > > >> > > > > surfaced in the output from `kafka-topics.sh
> --describe`.
> > > > Maybe
> > > > > >> that
> > > > > >> > > was
> > > > > >> > > > > intentional because ids are intentionally not really
> > > something
> > > > > the
> > > > > >> > user
> > > > > >> > > > > should care deeply about, but it would also make life
> > harder
> > > > for
> > > > > >> > anyone
> > > > > >> > > > > debugging Kafka and this would likely get worse the more
> > > topic
> > > > > ids
> > > > > >> > got
> > > > > >> > > > > rolled out across the protocols, clients etc. It seems
> > > likely
> > > > > that
> > > > > >> > > > > `kafka-topics.sh` will eventually need the ability to
> show
> > > the
> > > > > id
> > > > > >> of
> > > > > >> > a
> > > > > >> > > > > topic and perhaps find a topic name given an id. Is
> there
> > > any
> > > > > >> reason
> > > > > >> > > not
> > > > > >> > > > to
> > > > > >> > > > > implement that in this KIP?
> > > > > >> > > > >
> > > > > >> > > > > Many thanks,
> > > > > >> > > > >
> > > > > >> > > > > Tom
> > > > > >> > > > >
> > > > > >> > > > > On Mon, Sep 21, 2020 at 9:54 PM Justine Olshan <
> > > > > >> jolshan@confluent.io
> > > > > >> > >
> > > > > >> > > > > wrote:
> > > > > >> > > > >
> > > > > >> > > > > > Hi all,
> > > > > >> > > > > >
> > > > > >> > > > > > After thinking about it, I've decided to remove the
> > topic
> > > > name
> > > > > >> from
> > > > > >> > > the
> > > > > >> > > > > > Fetch Request and Response after all. Since there are
> so
> > > > many
> > > > > of
> > > > > >> > > these
> > > > > >> > > > > > requests per second, it is worth removing the extra
> > > > > information.
> > > > > >> > I've
> > > > > >> > > > > > updated the KIP to reflect this change.
> > > > > >> > > > > >
> > > > > >> > > > > > Please let me know if there is anything else we should
> > > > discuss
> > > > > >> > before
> > > > > >> > > > > > voting.
> > > > > >> > > > > >
> > > > > >> > > > > > Thank you,
> > > > > >> > > > > > Justine
> > > > > >> > > > > >
> > > > > >> > > > > > On Fri, Sep 18, 2020 at 9:46 AM Justine Olshan <
> > > > > >> > jolshan@confluent.io
> > > > > >> > > >
> > > > > >> > > > > > wrote:
> > > > > >> > > > > >
> > > > > >> > > > > > > Hi Jun,
> > > > > >> > > > > > >
> > > > > >> > > > > > > I see what you are saying. For now we can remove the
> > > extra
> > > > > >> > > > information.
> > > > > >> > > > > > > I'll leave the option to add more fields to the file
> > in
> > > > the
> > > > > >> > future.
> > > > > >> > > > The
> > > > > >> > > > > > KIP
> > > > > >> > > > > > > has been updated to reflect this change.
> > > > > >> > > > > > >
> > > > > >> > > > > > > Thanks,
> > > > > >> > > > > > > Justine
> > > > > >> > > > > > >
> > > > > >> > > > > > > On Fri, Sep 18, 2020 at 8:46 AM Jun Rao <
> > > jun@confluent.io
> > > > >
> > > > > >> > wrote:
> > > > > >> > > > > > >
> > > > > >> > > > > > >> Hi, Justine,
> > > > > >> > > > > > >>
> > > > > >> > > > > > >> Thanks for the reply.
> > > > > >> > > > > > >>
> > > > > >> > > > > > >> 13. If the log directory is the source of truth, it
> > > means
> > > > > >> that
> > > > > >> > the
> > > > > >> > > > > > >> redundant info in the metadata file will be
> ignored.
> > > Then
> > > > > the
> > > > > >> > > > question
> > > > > >> > > > > > is
> > > > > >> > > > > > >> why do we need to put the redundant info in the
> > > metadata
> > > > > file
> > > > > >> > now?
> > > > > >> > > > > > >>
> > > > > >> > > > > > >> Thanks,
> > > > > >> > > > > > >>
> > > > > >> > > > > > >> Jun
> > > > > >> > > > > > >>
> > > > > >> > > > > > >> On Thu, Sep 17, 2020 at 5:07 PM Justine Olshan <
> > > > > >> > > > jolshan@confluent.io>
> > > > > >> > > > > > >> wrote:
> > > > > >> > > > > > >>
> > > > > >> > > > > > >> > Hi Jun,
> > > > > >> > > > > > >> > Thanks for the quick response!
> > > > > >> > > > > > >> >
> > > > > >> > > > > > >> > 12. I've decided to bump up the versions on the
> > > > requests
> > > > > >> and
> > > > > >> > > > updated
> > > > > >> > > > > > the
> > > > > >> > > > > > >> > KIP. I think it's good we thoroughly discussed
> the
> > > > > options
> > > > > >> > here,
> > > > > >> > > > so
> > > > > >> > > > > we
> > > > > >> > > > > > >> know
> > > > > >> > > > > > >> > we made a good choice. :)
> > > > > >> > > > > > >> >
> > > > > >> > > > > > >> > 13. This is an interesting situation. I think if
> > this
> > > > > does
> > > > > >> > occur
> > > > > >> > > > we
> > > > > >> > > > > > >> should
> > > > > >> > > > > > >> > give a warning. I agree that it's hard to know
> the
> > > > source
> > > > > >> of
> > > > > >> > > truth
> > > > > >> > > > > for
> > > > > >> > > > > > >> sure
> > > > > >> > > > > > >> > since the directory or the file could be manually
> > > > > >> modified. I
> > > > > >> > > > guess
> > > > > >> > > > > > the
> > > > > >> > > > > > >> > directory could be used as the source of truth.
> To
> > be
> > > > > >> honest,
> > > > > >> > > I'm
> > > > > >> > > > > not
> > > > > >> > > > > > >> > really sure what happens in kafka when the log
> > > > directory
> > > > > is
> > > > > >> > > > renamed
> > > > > >> > > > > > >> > manually in such a way. I'm also wondering if the
> > > > > >> situation is
> > > > > >> > > > > > >> recoverable
> > > > > >> > > > > > >> > in this scenario.
> > > > > >> > > > > > >> >
> > > > > >> > > > > > >> > Thanks,
> > > > > >> > > > > > >> > Justine
> > > > > >> > > > > > >> >
> > > > > >> > > > > > >> > On Thu, Sep 17, 2020 at 4:28 PM Jun Rao <
> > > > > jun@confluent.io>
> > > > > >> > > wrote:
> > > > > >> > > > > > >> >
> > > > > >> > > > > > >> > > Hi, Justine,
> > > > > >> > > > > > >> > >
> > > > > >> > > > > > >> > > Thanks for the reply.
> > > > > >> > > > > > >> > >
> > > > > >> > > > > > >> > > 12. I don't have a strong preference either.
> > > However,
> > > > > if
> > > > > >> we
> > > > > >> > > need
> > > > > >> > > > > IBP
> > > > > >> > > > > > >> > > anyway, maybe it's easier to just bump up the
> > > version
> > > > > for
> > > > > >> > all
> > > > > >> > > > > inter
> > > > > >> > > > > > >> > broker
> > > > > >> > > > > > >> > > requests and add the topic id field as a
> regular
> > > > > field. A
> > > > > >> > > > regular
> > > > > >> > > > > > >> field
> > > > > >> > > > > > >> > is
> > > > > >> > > > > > >> > > a bit more concise in wire transfer than a
> > flexible
> > > > > >> field.
> > > > > >> > > > > > >> > >
> > > > > >> > > > > > >> > > 13. The confusion that I was referring to is
> > > between
> > > > > the
> > > > > >> > topic
> > > > > >> > > > > name
> > > > > >> > > > > > >> and
> > > > > >> > > > > > >> > > partition number between the log dir and the
> > > metadata
> > > > > >> file.
> > > > > >> > > For
> > > > > >> > > > > > >> example,
> > > > > >> > > > > > >> > if
> > > > > >> > > > > > >> > > the log dir is topicA-1 and the metadata file
> in
> > it
> > > > has
> > > > > >> > topicB
> > > > > >> > > > and
> > > > > >> > > > > > >> > > partition 0 (say due to a bug or manual
> > > > modification),
> > > > > >> which
> > > > > >> > > one
> > > > > >> > > > > do
> > > > > >> > > > > > we
> > > > > >> > > > > > >> > use
> > > > > >> > > > > > >> > > as the source of truth?
> > > > > >> > > > > > >> > >
> > > > > >> > > > > > >> > > Jun
> > > > > >> > > > > > >> > >
> > > > > >> > > > > > >> > > On Thu, Sep 17, 2020 at 3:43 PM Justine Olshan
> <
> > > > > >> > > > > > jolshan@confluent.io>
> > > > > >> > > > > > >> > > wrote:
> > > > > >> > > > > > >> > >
> > > > > >> > > > > > >> > > > Hi Jun,
> > > > > >> > > > > > >> > > > Thanks for the comments.
> > > > > >> > > > > > >> > > >
> > > > > >> > > > > > >> > > > 12. I bumped the LeaderAndIsrRequest because
> I
> > > > > removed
> > > > > >> the
> > > > > >> > > > topic
> > > > > >> > > > > > >> name
> > > > > >> > > > > > >> > > field
> > > > > >> > > > > > >> > > > in the response. It may be possible to avoid
> > > > bumping
> > > > > >> the
> > > > > >> > > > version
> > > > > >> > > > > > >> > without
> > > > > >> > > > > > >> > > > that change, but I may be missing something.
> > > > > >> > > > > > >> > > > I believe StopReplica is actually on version
> 3
> > > now,
> > > > > but
> > > > > >> > > > because
> > > > > >> > > > > > >> > version 2
> > > > > >> > > > > > >> > > > is flexible, I kept that listed as version 2
> on
> > > the
> > > > > KIP
> > > > > >> > > page.
> > > > > >> > > > > > >> However,
> > > > > >> > > > > > >> > > you
> > > > > >> > > > > > >> > > > may be right in that we may need to bump the
> > > > version
> > > > > on
> > > > > >> > > > > > StopReplica
> > > > > >> > > > > > >> to
> > > > > >> > > > > > >> > > deal
> > > > > >> > > > > > >> > > > with deletion differently as mentioned
> above. I
> > > > don't
> > > > > >> know
> > > > > >> > > if
> > > > > >> > > > I
> > > > > >> > > > > > >> have a
> > > > > >> > > > > > >> > > big
> > > > > >> > > > > > >> > > > preference over used tagged fields or not.
> > > > > >> > > > > > >> > > >
> > > > > >> > > > > > >> > > > 13. I was thinking that in the case where the
> > > file
> > > > > and
> > > > > >> the
> > > > > >> > > > > request
> > > > > >> > > > > > >> > topic
> > > > > >> > > > > > >> > > > ids don't match, it means that the broker's
> > > > topic/the
> > > > > >> one
> > > > > >> > in
> > > > > >> > > > the
> > > > > >> > > > > > >> file
> > > > > >> > > > > > >> > has
> > > > > >> > > > > > >> > > > been deleted. In that case, we would need to
> > > delete
> > > > > the
> > > > > >> > old
> > > > > >> > > > > topic
> > > > > >> > > > > > >> and
> > > > > >> > > > > > >> > > start
> > > > > >> > > > > > >> > > > receiving the new version. If the topic name
> > were
> > > > to
> > > > > >> > change,
> > > > > >> > > > but
> > > > > >> > > > > > the
> > > > > >> > > > > > >> > ids
> > > > > >> > > > > > >> > > > still matched, the file would also need to
> > > update.
> > > > > Am I
> > > > > >> > > > missing
> > > > > >> > > > > a
> > > > > >> > > > > > >> case
> > > > > >> > > > > > >> > > > where the file would be correct and not the
> > > > request?
> > > > > >> > > > > > >> > > >
> > > > > >> > > > > > >> > > > Thanks,
> > > > > >> > > > > > >> > > > Justine
> > > > > >> > > > > > >> > > >
> > > > > >> > > > > > >> > > > On Thu, Sep 17, 2020 at 3:18 PM Jun Rao <
> > > > > >> jun@confluent.io
> > > > > >> > >
> > > > > >> > > > > wrote:
> > > > > >> > > > > > >> > > >
> > > > > >> > > > > > >> > > > > Hi, Justine,
> > > > > >> > > > > > >> > > > >
> > > > > >> > > > > > >> > > > > Thanks for the reply. A couple of more
> > comments
> > > > > >> below.
> > > > > >> > > > > > >> > > > >
> > > > > >> > > > > > >> > > > > 12. ListOffset and OffsetForLeader
> currently
> > > > don't
> > > > > >> > support
> > > > > >> > > > > > >> flexible
> > > > > >> > > > > > >> > > > fields.
> > > > > >> > > > > > >> > > > > So, we have to bump up the version number
> and
> > > use
> > > > > >> IBP at
> > > > > >> > > > least
> > > > > >> > > > > > for
> > > > > >> > > > > > >> > > these
> > > > > >> > > > > > >> > > > > two requests. Note that it seems 2.7.0 will
> > > > require
> > > > > >> IBP
> > > > > >> > > > anyway
> > > > > >> > > > > > >> > because
> > > > > >> > > > > > >> > > of
> > > > > >> > > > > > >> > > > > changes in KAFKA-10435. Also, it seems that
> > the
> > > > > >> version
> > > > > >> > > for
> > > > > >> > > > > > >> > > > > LeaderAndIsrRequest and StopReplica are
> > bumped
> > > > even
> > > > > >> > though
> > > > > >> > > > we
> > > > > >> > > > > > only
> > > > > >> > > > > > >> > > added
> > > > > >> > > > > > >> > > > a
> > > > > >> > > > > > >> > > > > tagged field. But since IBP is needed
> anyway,
> > > we
> > > > > may
> > > > > >> > want
> > > > > >> > > to
> > > > > >> > > > > > >> revisit
> > > > > >> > > > > > >> > > the
> > > > > >> > > > > > >> > > > > overall tagged field choice.
> > > > > >> > > > > > >> > > > >
> > > > > >> > > > > > >> > > > > 13. The only downside is the potential
> > > confusion
> > > > on
> > > > > >> > which
> > > > > >> > > > one
> > > > > >> > > > > is
> > > > > >> > > > > > >> the
> > > > > >> > > > > > >> > > > source
> > > > > >> > > > > > >> > > > > of truth if they don't match. Another
> option
> > is
> > > > to
> > > > > >> > include
> > > > > >> > > > > those
> > > > > >> > > > > > >> > fields
> > > > > >> > > > > > >> > > > in
> > > > > >> > > > > > >> > > > > the metadata file when we actually change
> the
> > > > > >> directory
> > > > > >> > > > > > structure.
> > > > > >> > > > > > >> > > > >
> > > > > >> > > > > > >> > > > > Thanks,
> > > > > >> > > > > > >> > > > >
> > > > > >> > > > > > >> > > > > Jun
> > > > > >> > > > > > >> > > > >
> > > > > >> > > > > > >> > > > > On Thu, Sep 17, 2020 at 2:01 PM Justine
> > Olshan
> > > <
> > > > > >> > > > > > >> jolshan@confluent.io
> > > > > >> > > > > > >> > >
> > > > > >> > > > > > >> > > > > wrote:
> > > > > >> > > > > > >> > > > >
> > > > > >> > > > > > >> > > > > > Hello all,
> > > > > >> > > > > > >> > > > > >
> > > > > >> > > > > > >> > > > > > I've thought some more about removing the
> > > topic
> > > > > >> name
> > > > > >> > > field
> > > > > >> > > > > > from
> > > > > >> > > > > > >> > some
> > > > > >> > > > > > >> > > of
> > > > > >> > > > > > >> > > > > the
> > > > > >> > > > > > >> > > > > > requests. On closer inspection of the
> > > > > >> > > requests/responses,
> > > > > >> > > > it
> > > > > >> > > > > > >> seems
> > > > > >> > > > > > >> > > that
> > > > > >> > > > > > >> > > > > the
> > > > > >> > > > > > >> > > > > > internal changes would be much larger
> than
> > I
> > > > > >> expected.
> > > > > >> > > > Some
> > > > > >> > > > > > >> > protocols
> > > > > >> > > > > > >> > > > > > involve clients, so they would require
> > > changes
> > > > > too.
> > > > > >> > I'm
> > > > > >> > > > > > thinking
> > > > > >> > > > > > >> > that
> > > > > >> > > > > > >> > > > for
> > > > > >> > > > > > >> > > > > > now, removing the topic name from these
> > > > requests
> > > > > >> and
> > > > > >> > > > > responses
> > > > > >> > > > > > >> are
> > > > > >> > > > > > >> > > out
> > > > > >> > > > > > >> > > > of
> > > > > >> > > > > > >> > > > > > scope.
> > > > > >> > > > > > >> > > > > >
> > > > > >> > > > > > >> > > > > > I have decided to just keep the change
> > > > > >> > > > LeaderAndIsrResponse
> > > > > >> > > > > to
> > > > > >> > > > > > >> > remove
> > > > > >> > > > > > >> > > > the
> > > > > >> > > > > > >> > > > > > topic name, and have updated the KIP to
> > > reflect
> > > > > >> this
> > > > > >> > > > > change. I
> > > > > >> > > > > > >> have
> > > > > >> > > > > > >> > > > also
> > > > > >> > > > > > >> > > > > > mentioned the other requests and
> responses
> > in
> > > > > >> future
> > > > > >> > > work.
> > > > > >> > > > > > >> > > > > >
> > > > > >> > > > > > >> > > > > > I'm hoping to start the voting process
> > soon,
> > > so
> > > > > >> let me
> > > > > >> > > > know
> > > > > >> > > > > if
> > > > > >> > > > > > >> > there
> > > > > >> > > > > > >> > > is
> > > > > >> > > > > > >> > > > > > anything else we should discuss.
> > > > > >> > > > > > >> > > > > >
> > > > > >> > > > > > >> > > > > > Thank you,
> > > > > >> > > > > > >> > > > > > Justine
> > > > > >> > > > > > >> > > > > >
> > > > > >> > > > > > >> > > > > > On Tue, Sep 15, 2020 at 3:57 PM Justine
> > > Olshan
> > > > <
> > > > > >> > > > > > >> > jolshan@confluent.io
> > > > > >> > > > > > >> > > >
> > > > > >> > > > > > >> > > > > > wrote:
> > > > > >> > > > > > >> > > > > >
> > > > > >> > > > > > >> > > > > > > Hello again,
> > > > > >> > > > > > >> > > > > > > To follow up on some of the other
> > comments:
> > > > > >> > > > > > >> > > > > > >
> > > > > >> > > > > > >> > > > > > > 10/11) We can remove the topic name
> from
> > > > these
> > > > > >> > > > > > >> > requests/responses,
> > > > > >> > > > > > >> > > > and
> > > > > >> > > > > > >> > > > > > > that means we will just have to make a
> > few
> > > > > >> internal
> > > > > >> > > > > changes
> > > > > >> > > > > > to
> > > > > >> > > > > > >> > make
> > > > > >> > > > > > >> > > > > > > partitions accessible by topic id and
> > > > > partition.
> > > > > >> I
> > > > > >> > can
> > > > > >> > > > > > update
> > > > > >> > > > > > >> the
> > > > > >> > > > > > >> > > KIP
> > > > > >> > > > > > >> > > > > to
> > > > > >> > > > > > >> > > > > > > remove them unless anyone thinks they
> > > should
> > > > > >> stay.
> > > > > >> > > > > > >> > > > > > >
> > > > > >> > > > > > >> > > > > > > 12) Addressed in the previous email.
> I've
> > > > > updated
> > > > > >> > the
> > > > > >> > > > KIP
> > > > > >> > > > > to
> > > > > >> > > > > > >> > > include
> > > > > >> > > > > > >> > > > > > > tagged fields for the requests and
> > > responses.
> > > > > >> (More
> > > > > >> > on
> > > > > >> > > > > that
> > > > > >> > > > > > >> > below)
> > > > > >> > > > > > >> > > > > > >
> > > > > >> > > > > > >> > > > > > > 13) I think part of the idea for
> > including
> > > > this
> > > > > >> > > > > information
> > > > > >> > > > > > >> is to
> > > > > >> > > > > > >> > > > > prepare
> > > > > >> > > > > > >> > > > > > > for future changes. Perhaps the
> directory
> > > > > >> structure
> > > > > >> > > > might
> > > > > >> > > > > > >> change
> > > > > >> > > > > > >> > > from
> > > > > >> > > > > > >> > > > > > > topicName_partitionNumber to something
> > like
> > > > > >> > > > > > >> > > topicID_partitionNumber.
> > > > > >> > > > > > >> > > > > Then
> > > > > >> > > > > > >> > > > > > > it would be useful to have the topic
> name
> > > in
> > > > > the
> > > > > >> > file
> > > > > >> > > > > since
> > > > > >> > > > > > it
> > > > > >> > > > > > >> > > would
> > > > > >> > > > > > >> > > > > not
> > > > > >> > > > > > >> > > > > > be
> > > > > >> > > > > > >> > > > > > > in the directory structure. Supporting
> > > topic
> > > > > >> renames
> > > > > >> > > > might
> > > > > >> > > > > > be
> > > > > >> > > > > > >> > > easier
> > > > > >> > > > > > >> > > > if
> > > > > >> > > > > > >> > > > > > the
> > > > > >> > > > > > >> > > > > > > other fields are included. Would there
> be
> > > any
> > > > > >> > > downsides
> > > > > >> > > > to
> > > > > >> > > > > > >> > > including
> > > > > >> > > > > > >> > > > > this
> > > > > >> > > > > > >> > > > > > > information?
> > > > > >> > > > > > >> > > > > > >
> > > > > >> > > > > > >> > > > > > > 14)  Yes, we would need to copy the
> > > partition
> > > > > >> > metadata
> > > > > >> > > > > file
> > > > > >> > > > > > in
> > > > > >> > > > > > >> > this
> > > > > >> > > > > > >> > > > > > > process. I've updated the KIP to
> include
> > > > this.
> > > > > >> > > > > > >> > > > > > >
> > > > > >> > > > > > >> > > > > > > 15) I believe Lucas meant v1 and v2
> here.
> > > He
> > > > > was
> > > > > >> > > > referring
> > > > > >> > > > > > to
> > > > > >> > > > > > >> how
> > > > > >> > > > > > >> > > the
> > > > > >> > > > > > >> > > > > > > requests would fall under different IBP
> > and
> > > > > meant
> > > > > >> > that
> > > > > >> > > > > older
> > > > > >> > > > > > >> > > brokers
> > > > > >> > > > > > >> > > > > > would
> > > > > >> > > > > > >> > > > > > > have to use the older version of the
> > > request
> > > > > and
> > > > > >> the
> > > > > >> > > > > > existing
> > > > > >> > > > > > >> > topic
> > > > > >> > > > > > >> > > > > > > deletion process. At first, it seemed
> > like
> > > > > tagged
> > > > > >> > > fields
> > > > > >> > > > > > would
> > > > > >> > > > > > >> > > > resolve
> > > > > >> > > > > > >> > > > > > > the IBP issue. However, we may need IBP
> > for
> > > > > this
> > > > > >> > > request
> > > > > >> > > > > > after
> > > > > >> > > > > > >> > all
> > > > > >> > > > > > >> > > > > since
> > > > > >> > > > > > >> > > > > > > the controller handles the topic
> deletion
> > > > > >> > differently
> > > > > >> > > > > > >> depending
> > > > > >> > > > > > >> > on
> > > > > >> > > > > > >> > > > the
> > > > > >> > > > > > >> > > > > > IBP
> > > > > >> > > > > > >> > > > > > > version. In an older version, we can't
> > just
> > > > > send
> > > > > >> a
> > > > > >> > > > > > StopReplica
> > > > > >> > > > > > >> > > delete
> > > > > >> > > > > > >> > > > > the
> > > > > >> > > > > > >> > > > > > > topic immediately like we'd want to for
> > > this
> > > > > KIP.
> > > > > >> > > > > > >> > > > > > >
> > > > > >> > > > > > >> > > > > > > This makes me wonder if we want tagged
> > > fields
> > > > > on
> > > > > >> all
> > > > > >> > > the
> > > > > >> > > > > > >> requests
> > > > > >> > > > > > >> > > > after
> > > > > >> > > > > > >> > > > > > > all. Let me know your thoughts!
> > > > > >> > > > > > >> > > > > > >
> > > > > >> > > > > > >> > > > > > > Justine
> > > > > >> > > > > > >> > > > > > >
> > > > > >> > > > > > >> > > > > > > On Tue, Sep 15, 2020 at 1:03 PM Justine
> > > > Olshan
> > > > > <
> > > > > >> > > > > > >> > > jolshan@confluent.io
> > > > > >> > > > > > >> > > > >
> > > > > >> > > > > > >> > > > > > > wrote:
> > > > > >> > > > > > >> > > > > > >
> > > > > >> > > > > > >> > > > > > >> Hi all,
> > > > > >> > > > > > >> > > > > > >> Jun brought up a good point in his
> last
> > > > email
> > > > > >> about
> > > > > >> > > > > tagged
> > > > > >> > > > > > >> > fields,
> > > > > >> > > > > > >> > > > and
> > > > > >> > > > > > >> > > > > > >> I've updated the KIP to reflect that
> the
> > > > > >> changes to
> > > > > >> > > > > > requests
> > > > > >> > > > > > >> and
> > > > > >> > > > > > >> > > > > > responses
> > > > > >> > > > > > >> > > > > > >> will be in the form of tagged fields
> to
> > > > avoid
> > > > > >> > > changing
> > > > > >> > > > > IBP.
> > > > > >> > > > > > >> > > > > > >>
> > > > > >> > > > > > >> > > > > > >> Jun: I plan on sending a followup
> email
> > to
> > > > > >> address
> > > > > >> > > some
> > > > > >> > > > > of
> > > > > >> > > > > > >> the
> > > > > >> > > > > > >> > > other
> > > > > >> > > > > > >> > > > > > >> points.
> > > > > >> > > > > > >> > > > > > >>
> > > > > >> > > > > > >> > > > > > >> Thanks,
> > > > > >> > > > > > >> > > > > > >> Justine
> > > > > >> > > > > > >> > > > > > >>
> > > > > >> > > > > > >> > > > > > >> On Mon, Sep 14, 2020 at 4:25 PM Jun
> Rao
> > <
> > > > > >> > > > > jun@confluent.io>
> > > > > >> > > > > > >> > wrote:
> > > > > >> > > > > > >> > > > > > >>
> > > > > >> > > > > > >> > > > > > >>> Hi, Justine,
> > > > > >> > > > > > >> > > > > > >>>
> > > > > >> > > > > > >> > > > > > >>> Thanks for the updated KIP. A few
> > > comments
> > > > > >> below.
> > > > > >> > > > > > >> > > > > > >>>
> > > > > >> > > > > > >> > > > > > >>> 10. LeaderAndIsr Response: Do we need
> > the
> > > > > topic
> > > > > >> > > name?
> > > > > >> > > > > > >> > > > > > >>>
> > > > > >> > > > > > >> > > > > > >>> 11. For the changed request/response,
> > > other
> > > > > >> than
> > > > > >> > > > > > >> LeaderAndIsr,
> > > > > >> > > > > > >> > > > > > >>> UpdateMetadata, Metadata, do we need
> to
> > > > > include
> > > > > >> > the
> > > > > >> > > > > topic
> > > > > >> > > > > > >> name?
> > > > > >> > > > > > >> > > > > > >>>
> > > > > >> > > > > > >> > > > > > >>> 12. It seems that upgrades don't
> > require
> > > > IBP.
> > > > > >> Does
> > > > > >> > > > that
> > > > > >> > > > > > mean
> > > > > >> > > > > > >> > the
> > > > > >> > > > > > >> > > > new
> > > > > >> > > > > > >> > > > > > >>> fields
> > > > > >> > > > > > >> > > > > > >>> in all the request/response are added
> > as
> > > > > tagged
> > > > > >> > > fields
> > > > > >> > > > > > >> without
> > > > > >> > > > > > >> > > > > bumping
> > > > > >> > > > > > >> > > > > > up
> > > > > >> > > > > > >> > > > > > >>> the request version? It would be
> useful
> > > to
> > > > > make
> > > > > >> > that
> > > > > >> > > > > > clear.
> > > > > >> > > > > > >> > > > > > >>>
> > > > > >> > > > > > >> > > > > > >>> 13. Partition Metadata file: Do we
> need
> > > to
> > > > > >> include
> > > > > >> > > the
> > > > > >> > > > > > topic
> > > > > >> > > > > > >> > name
> > > > > >> > > > > > >> > > > and
> > > > > >> > > > > > >> > > > > > the
> > > > > >> > > > > > >> > > > > > >>> partition id since they are implied
> in
> > > the
> > > > > >> > directory
> > > > > >> > > > > name?
> > > > > >> > > > > > >> > > > > > >>>
> > > > > >> > > > > > >> > > > > > >>> 14. In the JBOD mode, we support
> > moving a
> > > > > >> > > partition's
> > > > > >> > > > > data
> > > > > >> > > > > > >> from
> > > > > >> > > > > > >> > > one
> > > > > >> > > > > > >> > > > > > disk
> > > > > >> > > > > > >> > > > > > >>> to
> > > > > >> > > > > > >> > > > > > >>> another. Will the new partition
> > metadata
> > > > file
> > > > > >> be
> > > > > >> > > > copied
> > > > > >> > > > > > >> during
> > > > > >> > > > > > >> > > that
> > > > > >> > > > > > >> > > > > > >>> process?
> > > > > >> > > > > > >> > > > > > >>>
> > > > > >> > > > > > >> > > > > > >>> 15. The KIP says "Remove deleted
> topics
> > > > from
> > > > > >> > > replicas
> > > > > >> > > > by
> > > > > >> > > > > > >> > sending
> > > > > >> > > > > > >> > > > > > >>> StopReplicaRequest V2 for any topics
> > > which
> > > > do
> > > > > >> not
> > > > > >> > > > > contain
> > > > > >> > > > > > a
> > > > > >> > > > > > >> > topic
> > > > > >> > > > > > >> > > > ID,
> > > > > >> > > > > > >> > > > > > and
> > > > > >> > > > > > >> > > > > > >>> V3 for any topics which do contain a
> > > topic
> > > > > >> ID.".
> > > > > >> > > > > However,
> > > > > >> > > > > > it
> > > > > >> > > > > > >> > > seems
> > > > > >> > > > > > >> > > > > the
> > > > > >> > > > > > >> > > > > > >>> updated controller will create all
> > > missing
> > > > > >> topic
> > > > > >> > IDs
> > > > > >> > > > > first
> > > > > >> > > > > > >> > before
> > > > > >> > > > > > >> > > > > doing
> > > > > >> > > > > > >> > > > > > >>> other actions. So, is
> > StopReplicaRequest
> > > V2
> > > > > >> > needed?
> > > > > >> > > > > > >> > > > > > >>>
> > > > > >> > > > > > >> > > > > > >>> Jun
> > > > > >> > > > > > >> > > > > > >>>
> > > > > >> > > > > > >> > > > > > >>> On Fri, Sep 11, 2020 at 10:31 AM John
> > > > > Roesler <
> > > > > >> > > > > > >> > > vvcephei@apache.org
> > > > > >> > > > > > >> > > > >
> > > > > >> > > > > > >> > > > > > >>> wrote:
> > > > > >> > > > > > >> > > > > > >>>
> > > > > >> > > > > > >> > > > > > >>> > Thanks, Justine!
> > > > > >> > > > > > >> > > > > > >>> >
> > > > > >> > > > > > >> > > > > > >>> > Your response seems compelling to
> me.
> > > > > >> > > > > > >> > > > > > >>> >
> > > > > >> > > > > > >> > > > > > >>> > -John
> > > > > >> > > > > > >> > > > > > >>> >
> > > > > >> > > > > > >> > > > > > >>> > On Fri, 2020-09-11 at 10:17 -0700,
> > > > Justine
> > > > > >> > Olshan
> > > > > >> > > > > wrote:
> > > > > >> > > > > > >> > > > > > >>> > > Hello all,
> > > > > >> > > > > > >> > > > > > >>> > > Thanks for continuing the
> > > discussion! I
> > > > > >> have a
> > > > > >> > > few
> > > > > >> > > > > > >> > responses
> > > > > >> > > > > > >> > > to
> > > > > >> > > > > > >> > > > > > your
> > > > > >> > > > > > >> > > > > > >>> > points.
> > > > > >> > > > > > >> > > > > > >>> > >
> > > > > >> > > > > > >> > > > > > >>> > > Tom: You are correct in that this
> > KIP
> > > > has
> > > > > >> not
> > > > > >> > > > > > mentioned
> > > > > >> > > > > > >> the
> > > > > >> > > > > > >> > > > > > >>> > > DeleteTopicsRequest. I think that
> > > this
> > > > > >> would
> > > > > >> > be
> > > > > >> > > > out
> > > > > >> > > > > of
> > > > > >> > > > > > >> > scope
> > > > > >> > > > > > >> > > > for
> > > > > >> > > > > > >> > > > > > >>> now, but
> > > > > >> > > > > > >> > > > > > >>> > > may be something worth adding in
> > the
> > > > > >> future.
> > > > > >> > > > > > >> > > > > > >>> > >
> > > > > >> > > > > > >> > > > > > >>> > > John: We did consider sequence
> ids,
> > > but
> > > > > >> there
> > > > > >> > > are
> > > > > >> > > > a
> > > > > >> > > > > > few
> > > > > >> > > > > > >> > > reasons
> > > > > >> > > > > > >> > > > > to
> > > > > >> > > > > > >> > > > > > >>> favor
> > > > > >> > > > > > >> > > > > > >>> > > UUIDs. There are several cases
> > where
> > > > > topics
> > > > > >> > from
> > > > > >> > > > > > >> different
> > > > > >> > > > > > >> > > > > clusters
> > > > > >> > > > > > >> > > > > > >>> may
> > > > > >> > > > > > >> > > > > > >>> > > interact now and in the future.
> For
> > > > > >> example,
> > > > > >> > > > Mirror
> > > > > >> > > > > > >> Maker 2
> > > > > >> > > > > > >> > > may
> > > > > >> > > > > > >> > > > > > >>> benefit
> > > > > >> > > > > > >> > > > > > >>> > > from being able to detect when a
> > > > cluster
> > > > > >> being
> > > > > >> > > > > > mirrored
> > > > > >> > > > > > >> is
> > > > > >> > > > > > >> > > > > deleted
> > > > > >> > > > > > >> > > > > > >>> and
> > > > > >> > > > > > >> > > > > > >>> > > recreated and globally unique
> > > > identifiers
> > > > > >> > would
> > > > > >> > > > make
> > > > > >> > > > > > >> > > resolving
> > > > > >> > > > > > >> > > > > > issues
> > > > > >> > > > > > >> > > > > > >>> > > easier than sequence IDs which
> may
> > > > > collide
> > > > > >> > > between
> > > > > >> > > > > > >> > clusters.
> > > > > >> > > > > > >> > > > > > KIP-405
> > > > > >> > > > > > >> > > > > > >>> > > (tiered storage) will also
> benefit
> > > from
> > > > > >> > globally
> > > > > >> > > > > > unique
> > > > > >> > > > > > >> IDs
> > > > > >> > > > > > >> > > as
> > > > > >> > > > > > >> > > > > > shared
> > > > > >> > > > > > >> > > > > > >>> > > buckets may be used between
> > clusters.
> > > > > >> > > > > > >> > > > > > >>> > >
> > > > > >> > > > > > >> > > > > > >>> > > Globally unique IDs would also
> make
> > > > > >> > > functionality
> > > > > >> > > > > like
> > > > > >> > > > > > >> > moving
> > > > > >> > > > > > >> > > > > > topics
> > > > > >> > > > > > >> > > > > > >>> > > between disparate clusters easier
> > in
> > > > the
> > > > > >> > future,
> > > > > >> > > > > > >> simplify
> > > > > >> > > > > > >> > any
> > > > > >> > > > > > >> > > > > > future
> > > > > >> > > > > > >> > > > > > >>> > > implementations of backups and
> > > > restores,
> > > > > >> and
> > > > > >> > > more.
> > > > > >> > > > > In
> > > > > >> > > > > > >> > > general,
> > > > > >> > > > > > >> > > > > > >>> unique IDs
> > > > > >> > > > > > >> > > > > > >>> > > would ensure that the source
> > cluster
> > > > > >> topics do
> > > > > >> > > not
> > > > > >> > > > > > >> conflict
> > > > > >> > > > > > >> > > > with
> > > > > >> > > > > > >> > > > > > the
> > > > > >> > > > > > >> > > > > > >>> > > destination cluster topics.
> > > > > >> > > > > > >> > > > > > >>> > >
> > > > > >> > > > > > >> > > > > > >>> > > If we were to use sequence ids,
> we
> > > > would
> > > > > >> need
> > > > > >> > > > > > >> sufficiently
> > > > > >> > > > > > >> > > > large
> > > > > >> > > > > > >> > > > > > >>> cluster
> > > > > >> > > > > > >> > > > > > >>> > > ids to be stored with the topic
> > > > > >> identifiers or
> > > > > >> > > we
> > > > > >> > > > > run
> > > > > >> > > > > > >> the
> > > > > >> > > > > > >> > > risk
> > > > > >> > > > > > >> > > > of
> > > > > >> > > > > > >> > > > > > >>> > > collisions. This will give up any
> > > > > >> advantage in
> > > > > >> > > > > > >> compactness
> > > > > >> > > > > > >> > > that
> > > > > >> > > > > > >> > > > > > >>> sequence
> > > > > >> > > > > > >> > > > > > >>> > > numbers may bring. Given these
> > > > > advantages I
> > > > > >> > > think
> > > > > >> > > > it
> > > > > >> > > > > > >> makes
> > > > > >> > > > > > >> > > > sense
> > > > > >> > > > > > >> > > > > to
> > > > > >> > > > > > >> > > > > > >>> use
> > > > > >> > > > > > >> > > > > > >>> > > UUIDs.
> > > > > >> > > > > > >> > > > > > >>> > >
> > > > > >> > > > > > >> > > > > > >>> > > Gokul: This is an interesting
> idea,
> > > but
> > > > > >> this
> > > > > >> > is
> > > > > >> > > a
> > > > > >> > > > > > >> breaking
> > > > > >> > > > > > >> > > > > change.
> > > > > >> > > > > > >> > > > > > >>> Out of
> > > > > >> > > > > > >> > > > > > >>> > > scope for now, but maybe worth
> > > > discussing
> > > > > >> in
> > > > > >> > the
> > > > > >> > > > > > future.
> > > > > >> > > > > > >> > > > > > >>> > >
> > > > > >> > > > > > >> > > > > > >>> > > Hope this explains some of the
> > > > decisions,
> > > > > >> > > > > > >> > > > > > >>> > >
> > > > > >> > > > > > >> > > > > > >>> > > Justine
> > > > > >> > > > > > >> > > > > > >>> > >
> > > > > >> > > > > > >> > > > > > >>> > >
> > > > > >> > > > > > >> > > > > > >>> > >
> > > > > >> > > > > > >> > > > > > >>> > > On Fri, Sep 11, 2020 at 8:27 AM
> > Gokul
> > > > > >> Ramanan
> > > > > >> > > > > > >> Subramanian <
> > > > > >> > > > > > >> > > > > > >>> > > gokul2411s@gmail.com> wrote:
> > > > > >> > > > > > >> > > > > > >>> > >
> > > > > >> > > > > > >> > > > > > >>> > > > Hi.
> > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > >> > > > > > >> > > > > > >>> > > > Thanks for the KIP.
> > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > >> > > > > > >> > > > > > >>> > > > Have you thought about whether
> it
> > > > makes
> > > > > >> > sense
> > > > > >> > > to
> > > > > >> > > > > > >> support
> > > > > >> > > > > > >> > > > > > >>> authorizing a
> > > > > >> > > > > > >> > > > > > >>> > > > principal for a topic ID rather
> > > than
> > > > a
> > > > > >> topic
> > > > > >> > > > name
> > > > > >> > > > > to
> > > > > >> > > > > > >> > > achieve
> > > > > >> > > > > > >> > > > > > >>> tighter
> > > > > >> > > > > > >> > > > > > >>> > > > security?
> > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > >> > > > > > >> > > > > > >>> > > > Or is the topic ID
> fundamentally
> > an
> > > > > >> internal
> > > > > >> > > > > detail
> > > > > >> > > > > > >> > similar
> > > > > >> > > > > > >> > > > to
> > > > > >> > > > > > >> > > > > > >>> epochs
> > > > > >> > > > > > >> > > > > > >>> > used
> > > > > >> > > > > > >> > > > > > >>> > > > in a bunch of other places in
> > > Kafka?
> > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > >> > > > > > >> > > > > > >>> > > > Thanks.
> > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > >> > > > > > >> > > > > > >>> > > > On Fri, Sep 11, 2020 at 4:06 PM
> > > John
> > > > > >> > Roesler <
> > > > > >> > > > > > >> > > > > > vvcephei@apache.org>
> > > > > >> > > > > > >> > > > > > >>> > wrote:
> > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > >> > > > > > >> > > > > > >>> > > > > Hello Justine,
> > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > >> > > > > > >> > > > > > >>> > > > > Thanks for the KIP!
> > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > >> > > > > > >> > > > > > >>> > > > > I happen to have been
> > confronted
> > > > > >> recently
> > > > > >> > > with
> > > > > >> > > > > the
> > > > > >> > > > > > >> need
> > > > > >> > > > > > >> > > to
> > > > > >> > > > > > >> > > > > keep
> > > > > >> > > > > > >> > > > > > >>> > track of
> > > > > >> > > > > > >> > > > > > >>> > > > a
> > > > > >> > > > > > >> > > > > > >>> > > > > large number of topics as
> > > compactly
> > > > > as
> > > > > >> > > > > possible. I
> > > > > >> > > > > > >> was
> > > > > >> > > > > > >> > > > going
> > > > > >> > > > > > >> > > > > to
> > > > > >> > > > > > >> > > > > > >>> come
> > > > > >> > > > > > >> > > > > > >>> > up
> > > > > >> > > > > > >> > > > > > >>> > > > > with some way to dictionary
> > > encode
> > > > > the
> > > > > >> > topic
> > > > > >> > > > > names
> > > > > >> > > > > > >> as
> > > > > >> > > > > > >> > > > > integers,
> > > > > >> > > > > > >> > > > > > >>> but
> > > > > >> > > > > > >> > > > > > >>> > this
> > > > > >> > > > > > >> > > > > > >>> > > > > seems much better!
> > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > >> > > > > > >> > > > > > >>> > > > > Apologies if this has been
> > raised
> > > > > >> before,
> > > > > >> > > but
> > > > > >> > > > > I’m
> > > > > >> > > > > > >> > > wondering
> > > > > >> > > > > > >> > > > > > >>> about the
> > > > > >> > > > > > >> > > > > > >>> > > > > choice of UUID vs sequence
> > number
> > > > for
> > > > > >> the
> > > > > >> > > ids.
> > > > > >> > > > > > >> > Typically,
> > > > > >> > > > > > >> > > > > I’ve
> > > > > >> > > > > > >> > > > > > >>> seen
> > > > > >> > > > > > >> > > > > > >>> > UUIDs
> > > > > >> > > > > > >> > > > > > >>> > > > > in two situations:
> > > > > >> > > > > > >> > > > > > >>> > > > > 1. When processes need to
> > > generate
> > > > > >> > > > non-colliding
> > > > > >> > > > > > >> > > > identifiers
> > > > > >> > > > > > >> > > > > > >>> without
> > > > > >> > > > > > >> > > > > > >>> > > > > coordination.
> > > > > >> > > > > > >> > > > > > >>> > > > > 2. When the identifier needs
> to
> > > be
> > > > > >> > > > “universally
> > > > > >> > > > > > >> > unique”;
> > > > > >> > > > > > >> > > > > I.e.,
> > > > > >> > > > > > >> > > > > > >>> the
> > > > > >> > > > > > >> > > > > > >>> > > > > identifier needs to
> distinguish
> > > the
> > > > > >> entity
> > > > > >> > > > from
> > > > > >> > > > > > all
> > > > > >> > > > > > >> > other
> > > > > >> > > > > > >> > > > > > >>> entities
> > > > > >> > > > > > >> > > > > > >>> > that
> > > > > >> > > > > > >> > > > > > >>> > > > > could ever exist. This is
> > useful
> > > in
> > > > > >> cases
> > > > > >> > > > where
> > > > > >> > > > > > >> > entities
> > > > > >> > > > > > >> > > > from
> > > > > >> > > > > > >> > > > > > all
> > > > > >> > > > > > >> > > > > > >>> > kinds
> > > > > >> > > > > > >> > > > > > >>> > > > of
> > > > > >> > > > > > >> > > > > > >>> > > > > systems get mixed together,
> > such
> > > as
> > > > > >> when
> > > > > >> > > > dumping
> > > > > >> > > > > > >> logs
> > > > > >> > > > > > >> > > from
> > > > > >> > > > > > >> > > > > all
> > > > > >> > > > > > >> > > > > > >>> > processes
> > > > > >> > > > > > >> > > > > > >>> > > > in
> > > > > >> > > > > > >> > > > > > >>> > > > > a company into a common
> system.
> > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > >> > > > > > >> > > > > > >>> > > > > Maybe I’m being
> short-sighted,
> > > but
> > > > it
> > > > > >> > > doesn’t
> > > > > >> > > > > seem
> > > > > >> > > > > > >> like
> > > > > >> > > > > > >> > > > > either
> > > > > >> > > > > > >> > > > > > >>> really
> > > > > >> > > > > > >> > > > > > >>> > > > > applies here. It seems like
> the
> > > > > brokers
> > > > > >> > > could
> > > > > >> > > > > and
> > > > > >> > > > > > >> would
> > > > > >> > > > > > >> > > > > achieve
> > > > > >> > > > > > >> > > > > > >>> > consensus
> > > > > >> > > > > > >> > > > > > >>> > > > > when creating a topic anyway,
> > > which
> > > > > is
> > > > > >> all
> > > > > >> > > > > that’s
> > > > > >> > > > > > >> > > required
> > > > > >> > > > > > >> > > > to
> > > > > >> > > > > > >> > > > > > >>> > generate
> > > > > >> > > > > > >> > > > > > >>> > > > > non-colliding sequence ids.
> For
> > > the
> > > > > >> > second,
> > > > > >> > > as
> > > > > >> > > > > you
> > > > > >> > > > > > >> > > mention,
> > > > > >> > > > > > >> > > > > we
> > > > > >> > > > > > >> > > > > > >>> could
> > > > > >> > > > > > >> > > > > > >>> > > > assign
> > > > > >> > > > > > >> > > > > > >>> > > > > a UUID to the cluster as a
> > whole,
> > > > > which
> > > > > >> > > would
> > > > > >> > > > > > render
> > > > > >> > > > > > >> > any
> > > > > >> > > > > > >> > > > > > resource
> > > > > >> > > > > > >> > > > > > >>> > scoped
> > > > > >> > > > > > >> > > > > > >>> > > > to
> > > > > >> > > > > > >> > > > > > >>> > > > > the broker universally unique
> > as
> > > > > well.
> > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > >> > > > > > >> > > > > > >>> > > > > The reason I mention this is
> > > that,
> > > > > >> > although
> > > > > >> > > a
> > > > > >> > > > > UUID
> > > > > >> > > > > > >> is
> > > > > >> > > > > > >> > way
> > > > > >> > > > > > >> > > > > more
> > > > > >> > > > > > >> > > > > > >>> > compact
> > > > > >> > > > > > >> > > > > > >>> > > > > than topic names, it’s still
> 16
> > > > > bytes.
> > > > > >> In
> > > > > >> > > > > > contrast,
> > > > > >> > > > > > >> a
> > > > > >> > > > > > >> > > > 4-byte
> > > > > >> > > > > > >> > > > > > >>> integer
> > > > > >> > > > > > >> > > > > > >>> > > > > sequence id would give us 4
> > > billion
> > > > > >> unique
> > > > > >> > > > > topics
> > > > > >> > > > > > >> per
> > > > > >> > > > > > >> > > > > cluster,
> > > > > >> > > > > > >> > > > > > >>> which
> > > > > >> > > > > > >> > > > > > >>> > > > seems
> > > > > >> > > > > > >> > > > > > >>> > > > > like enough ;)
> > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > >> > > > > > >> > > > > > >>> > > > > Considering the number of
> > > different
> > > > > >> times
> > > > > >> > > > these
> > > > > >> > > > > > >> topic
> > > > > >> > > > > > >> > > > > > >>> identifiers are
> > > > > >> > > > > > >> > > > > > >>> > > > sent
> > > > > >> > > > > > >> > > > > > >>> > > > > over the wire or stored in
> > > memory,
> > > > it
> > > > > >> > seems
> > > > > >> > > > like
> > > > > >> > > > > > it
> > > > > >> > > > > > >> > might
> > > > > >> > > > > > >> > > > be
> > > > > >> > > > > > >> > > > > > >>> worth
> > > > > >> > > > > > >> > > > > > >>> > the
> > > > > >> > > > > > >> > > > > > >>> > > > > additional 4x space savings.
> > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > >> > > > > > >> > > > > > >>> > > > > What do you think about this?
> > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > >> > > > > > >> > > > > > >>> > > > > Thanks,
> > > > > >> > > > > > >> > > > > > >>> > > > > John
> > > > > >> > > > > > >> > > > > > >>> > > > >
> > > > > >> > > > > > >> > > > > > >>> > > > > On Fri, Sep 11, 2020, at
> 03:20,
> > > Tom
> > > > > >> > Bentley
> > > > > >> > > > > wrote:
> > > > > >> > > > > > >> > > > > > >>> > > > > > Hi Justine,
> > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > > >> > > > > > >> > > > > > >>> > > > > > This looks like a very
> > welcome
> > > > > >> > > improvement.
> > > > > >> > > > > > >> Thanks!
> > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > > >> > > > > > >> > > > > > >>> > > > > > Maybe I missed it, but the
> > KIP
> > > > > >> doesn't
> > > > > >> > > seem
> > > > > >> > > > to
> > > > > >> > > > > > >> > mention
> > > > > >> > > > > > >> > > > > > changing
> > > > > >> > > > > > >> > > > > > >>> > > > > > DeleteTopicsRequest to
> > identify
> > > > the
> > > > > >> > topic
> > > > > >> > > > > using
> > > > > >> > > > > > an
> > > > > >> > > > > > >> > id.
> > > > > >> > > > > > >> > > > > Maybe
> > > > > >> > > > > > >> > > > > > >>> > that's out
> > > > > >> > > > > > >> > > > > > >>> > > > > of
> > > > > >> > > > > > >> > > > > > >>> > > > > > scope, but
> > DeleteTopicsRequest
> > > is
> > > > > not
> > > > > >> > > listed
> > > > > >> > > > > > among
> > > > > >> > > > > > >> > the
> > > > > >> > > > > > >> > > > > Future
> > > > > >> > > > > > >> > > > > > >>> Work
> > > > > >> > > > > > >> > > > > > >>> > APIs
> > > > > >> > > > > > >> > > > > > >>> > > > > > either.
> > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > > >> > > > > > >> > > > > > >>> > > > > > Kind regards,
> > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > > >> > > > > > >> > > > > > >>> > > > > > Tom
> > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > > >> > > > > > >> > > > > > >>> > > > > > On Thu, Sep 10, 2020 at
> 3:59
> > PM
> > > > > >> Satish
> > > > > >> > > > > Duggana <
> > > > > >> > > > > > >> > > > > > >>> > > > satish.duggana@gmail.com
> > > > > >> > > > > > >> > > > > > >>> > > > > > wrote:
> > > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > > >> > > > > > >> > > > > > >>> > > > > > > Thanks Lucas/Justine for
> > the
> > > > nice
> > > > > >> KIP.
> > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > > >> > > > > > >> > > > > > >>> > > > > > > It has several benefits
> > which
> > > > > also
> > > > > >> > > include
> > > > > >> > > > > > >> > > simplifying
> > > > > >> > > > > > >> > > > > the
> > > > > >> > > > > > >> > > > > > >>> topic
> > > > > >> > > > > > >> > > > > > >>> > > > > > > deletion process by
> > > controller
> > > > > and
> > > > > >> > logs
> > > > > >> > > > > > cleanup
> > > > > >> > > > > > >> by
> > > > > >> > > > > > >> > > > > brokers
> > > > > >> > > > > > >> > > > > > in
> > > > > >> > > > > > >> > > > > > >>> > corner
> > > > > >> > > > > > >> > > > > > >>> > > > > > > cases.
> > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > > >> > > > > > >> > > > > > >>> > > > > > > Best,
> > > > > >> > > > > > >> > > > > > >>> > > > > > > Satish.
> > > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > > >> > > > > > >> > > > > > >>> > > > > > > On Wed, Sep 9, 2020 at
> > 10:07
> > > PM
> > > > > >> > Justine
> > > > > >> > > > > > Olshan <
> > > > > >> > > > > > >> > > > > > >>> > jolshan@confluent.io
> > > > > >> > > > > > >> > > > > > >>> > > > > > > wrote:
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > Hello all, it's been
> > > almost a
> > > > > >> year!
> > > > > >> > > I've
> > > > > >> > > > > > made
> > > > > >> > > > > > >> > some
> > > > > >> > > > > > >> > > > > > changes
> > > > > >> > > > > > >> > > > > > >>> to
> > > > > >> > > > > > >> > > > > > >>> > this
> > > > > >> > > > > > >> > > > > > >>> > > > > KIP
> > > > > >> > > > > > >> > > > > > >>> > > > > > > and hope to continue the
> > > > > >> discussion.
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > One of the main changes
> > > I've
> > > > > >> added
> > > > > >> > is
> > > > > >> > > > now
> > > > > >> > > > > > the
> > > > > >> > > > > > >> > > > metadata
> > > > > >> > > > > > >> > > > > > >>> response
> > > > > >> > > > > > >> > > > > > >>> > > > will
> > > > > >> > > > > > >> > > > > > >>> > > > > > > include the topic ID (as
> > > Colin
> > > > > >> > > suggested).
> > > > > >> > > > > > >> Clients
> > > > > >> > > > > > >> > > can
> > > > > >> > > > > > >> > > > > > >>> obtain the
> > > > > >> > > > > > >> > > > > > >>> > > > > topicID
> > > > > >> > > > > > >> > > > > > >>> > > > > > > of a given topic through
> a
> > > > > >> > > > TopicDescription.
> > > > > >> > > > > > The
> > > > > >> > > > > > >> > > > topicId
> > > > > >> > > > > > >> > > > > > will
> > > > > >> > > > > > >> > > > > > >>> > also be
> > > > > >> > > > > > >> > > > > > >>> > > > > > > included with the
> > > > UpdateMetadata
> > > > > >> > > request.
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > Let me know what you
> all
> > > > think.
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > Thank you,
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > Justine
> > > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > On 2019/09/13 16:38:26,
> > > > "Colin
> > > > > >> > > McCabe" <
> > > > > >> > > > > > >> > > > > > cmccabe@apache.org
> > > > > >> > > > > > >> > > > > > >>> >
> > > > > >> > > > > > >> > > > > > >>> > wrote:
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Hi Lucas,
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Thanks for tackling
> > this.
> > > > > >> Topic
> > > > > >> > IDs
> > > > > >> > > > > are a
> > > > > >> > > > > > >> > great
> > > > > >> > > > > > >> > > > > idea,
> > > > > >> > > > > > >> > > > > > >>> and
> > > > > >> > > > > > >> > > > > > >>> > this
> > > > > >> > > > > > >> > > > > > >>> > > > is
> > > > > >> > > > > > >> > > > > > >>> > > > > a
> > > > > >> > > > > > >> > > > > > >>> > > > > > > really good writeup.
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > For
> > > > /brokers/topics/[topic],
> > > > > >> the
> > > > > >> > > > schema
> > > > > >> > > > > > >> version
> > > > > >> > > > > > >> > > > > should
> > > > > >> > > > > > >> > > > > > be
> > > > > >> > > > > > >> > > > > > >>> > bumped
> > > > > >> > > > > > >> > > > > > >>> > > > to
> > > > > >> > > > > > >> > > > > > >>> > > > > > > version 3, rather than 2.
> > > > > KIP-455
> > > > > >> > > bumped
> > > > > >> > > > > the
> > > > > >> > > > > > >> > version
> > > > > >> > > > > > >> > > > of
> > > > > >> > > > > > >> > > > > > this
> > > > > >> > > > > > >> > > > > > >>> > znode
> > > > > >> > > > > > >> > > > > > >>> > > > to
> > > > > >> > > > > > >> > > > > > >>> > > > > 2
> > > > > >> > > > > > >> > > > > > >>> > > > > > > already :)
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Given that we're
> going
> > to
> > > > be
> > > > > >> > seeing
> > > > > >> > > > > these
> > > > > >> > > > > > >> > things
> > > > > >> > > > > > >> > > as
> > > > > >> > > > > > >> > > > > > >>> strings
> > > > > >> > > > > > >> > > > > > >>> > as
> > > > > >> > > > > > >> > > > > > >>> > > > lot
> > > > > >> > > > > > >> > > > > > >>> > > > > (in
> > > > > >> > > > > > >> > > > > > >>> > > > > > > logs, in ZooKeeper, on
> the
> > > > > >> > command-line,
> > > > > >> > > > > > etc.),
> > > > > >> > > > > > >> > does
> > > > > >> > > > > > >> > > it
> > > > > >> > > > > > >> > > > > > make
> > > > > >> > > > > > >> > > > > > >>> > sense to
> > > > > >> > > > > > >> > > > > > >>> > > > > use
> > > > > >> > > > > > >> > > > > > >>> > > > > > > base64 when converting
> them
> > > to
> > > > > >> > strings?
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Here is an example of
> > the
> > > > hex
> > > > > >> > > > > > >> representation:
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > > > >> > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > And here is an
> example
> > in
> > > > > >> base64.
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> b8tRS7h4TJ2Vt43Dp85v2A
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > The base64 version
> > saves
> > > 15
> > > > > >> > letters
> > > > > >> > > > (to
> > > > > >> > > > > be
> > > > > >> > > > > > >> > fair,
> > > > > >> > > > > > >> > > 4
> > > > > >> > > > > > >> > > > of
> > > > > >> > > > > > >> > > > > > >>> those
> > > > > >> > > > > > >> > > > > > >>> > were
> > > > > >> > > > > > >> > > > > > >>> > > > > > > dashes that we could have
> > > > elided
> > > > > in
> > > > > >> > the
> > > > > >> > > > hex
> > > > > >> > > > > > >> > > > > > representation.)
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Another thing to
> > consider
> > > > is
> > > > > >> that
> > > > > >> > we
> > > > > >> > > > > > should
> > > > > >> > > > > > >> > > specify
> > > > > >> > > > > > >> > > > > > that
> > > > > >> > > > > > >> > > > > > >>> the
> > > > > >> > > > > > >> > > > > > >>> > > > > > > all-zeroes UUID is not a
> > > valid
> > > > > >> topic
> > > > > >> > > UUID.
> > > > > >> > > > > >  We
> > > > > >> > > > > > >> > can't
> > > > > >> > > > > > >> > > > use
> > > > > >> > > > > > >> > > > > > >>> null
> > > > > >> > > > > > >> > > > > > >>> > for
> > > > > >> > > > > > >> > > > > > >>> > > > this
> > > > > >> > > > > > >> > > > > > >>> > > > > > > because we can't pass a
> > null
> > > > UUID
> > > > > >> over
> > > > > >> > > the
> > > > > >> > > > > RPC
> > > > > >> > > > > > >> > > protocol
> > > > > >> > > > > > >> > > > > > >>> (there
> > > > > >> > > > > > >> > > > > > >>> > is no
> > > > > >> > > > > > >> > > > > > >>> > > > > > > special pattern for null,
> > nor
> > > > do
> > > > > we
> > > > > >> > want
> > > > > >> > > > to
> > > > > >> > > > > > >> waste
> > > > > >> > > > > > >> > > space
> > > > > >> > > > > > >> > > > > > >>> reserving
> > > > > >> > > > > > >> > > > > > >>> > > > such
> > > > > >> > > > > > >> > > > > > >>> > > > > a
> > > > > >> > > > > > >> > > > > > >>> > > > > > > pattern.)
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Maybe I missed it,
> but
> > > did
> > > > > you
> > > > > >> > > > describe
> > > > > >> > > > > > >> > > "migration
> > > > > >> > > > > > >> > > > > > of...
> > > > > >> > > > > > >> > > > > > >>> > existing
> > > > > >> > > > > > >> > > > > > >>> > > > > > > topic[s] without topic
> IDs"
> > > in
> > > > > >> detail
> > > > > >> > in
> > > > > >> > > > any
> > > > > >> > > > > > >> > section?
> > > > > >> > > > > > >> > > > It
> > > > > >> > > > > > >> > > > > > >>> seems
> > > > > >> > > > > > >> > > > > > >>> > like
> > > > > >> > > > > > >> > > > > > >>> > > > > when
> > > > > >> > > > > > >> > > > > > >>> > > > > > > the new controller
> becomes
> > > > > active,
> > > > > >> it
> > > > > >> > > > should
> > > > > >> > > > > > >> just
> > > > > >> > > > > > >> > > > > generate
> > > > > >> > > > > > >> > > > > > >>> random
> > > > > >> > > > > > >> > > > > > >>> > > > > UUIDs for
> > > > > >> > > > > > >> > > > > > >>> > > > > > > these, and write the
> random
> > > > UUIDs
> > > > > >> back
> > > > > >> > > to
> > > > > >> > > > > > >> > ZooKeeper.
> > > > > >> > > > > > >> > > > It
> > > > > >> > > > > > >> > > > > > >>> would be
> > > > > >> > > > > > >> > > > > > >>> > > > good
> > > > > >> > > > > > >> > > > > > >>> > > > > to
> > > > > >> > > > > > >> > > > > > >>> > > > > > > spell that out.  We
> should
> > > make
> > > > > it
> > > > > >> > clear
> > > > > >> > > > > that
> > > > > >> > > > > > >> this
> > > > > >> > > > > > >> > > > > happens
> > > > > >> > > > > > >> > > > > > >>> > regardless
> > > > > >> > > > > > >> > > > > > >>> > > > > of
> > > > > >> > > > > > >> > > > > > >>> > > > > > > the inter-broker protocol
> > > > version
> > > > > >> > (it's
> > > > > >> > > a
> > > > > >> > > > > > >> > compatible
> > > > > >> > > > > > >> > > > > > change).
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > "LeaderAndIsrRequests
> > > > > >> including an
> > > > > >> > > > > > >> > > > is_every_partition
> > > > > >> > > > > > >> > > > > > >>> flag"
> > > > > >> > > > > > >> > > > > > >>> > > > seems a
> > > > > >> > > > > > >> > > > > > >>> > > > > > > bit wordy.  Can we just
> > call
> > > > > these
> > > > > >> > "full
> > > > > >> > > > > > >> > > > > > >>> LeaderAndIsrRequests"?
> > > > > >> > > > > > >> > > > > > >>> > Then
> > > > > >> > > > > > >> > > > > > >>> > > > > the
> > > > > >> > > > > > >> > > > > > >>> > > > > > > RPC field could be named
> > > > "full".
> > > > > >> > Also,
> > > > > >> > > it
> > > > > >> > > > > > would
> > > > > >> > > > > > >> > > > probably
> > > > > >> > > > > > >> > > > > > be
> > > > > >> > > > > > >> > > > > > >>> > better
> > > > > >> > > > > > >> > > > > > >>> > > > > for the
> > > > > >> > > > > > >> > > > > > >>> > > > > > > RPC field to be an enum
> of
> > {
> > > > > >> > > UNSPECIFIED,
> > > > > >> > > > > > >> > > INCREMENTAL,
> > > > > >> > > > > > >> > > > > FULL
> > > > > >> > > > > > >> > > > > > >>> }, so
> > > > > >> > > > > > >> > > > > > >>> > > > that
> > > > > >> > > > > > >> > > > > > >>> > > > > we
> > > > > >> > > > > > >> > > > > > >>> > > > > > > can cleanly handle old
> > > versions
> > > > > (by
> > > > > >> > > > treating
> > > > > >> > > > > > >> them
> > > > > >> > > > > > >> > as
> > > > > >> > > > > > >> > > > > > >>> UNSPECIFIED)
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > In the
> > > LeaderAndIsrRequest
> > > > > >> > section,
> > > > > >> > > > you
> > > > > >> > > > > > >> write
> > > > > >> > > > > > >> > "A
> > > > > >> > > > > > >> > > > > final
> > > > > >> > > > > > >> > > > > > >>> > deletion
> > > > > >> > > > > > >> > > > > > >>> > > > > event
> > > > > >> > > > > > >> > > > > > >>> > > > > > > will be secheduled for X
> ms
> > > > after
> > > > > >> the
> > > > > >> > > > > > >> > > > LeaderAndIsrRequest
> > > > > >> > > > > > >> > > > > > was
> > > > > >> > > > > > >> > > > > > >>> > first
> > > > > >> > > > > > >> > > > > > >>> > > > > > > received..."  I guess
> the X
> > > > was a
> > > > > >> > > > > placeholder
> > > > > >> > > > > > >> that
> > > > > >> > > > > > >> > > you
> > > > > >> > > > > > >> > > > > > >>> intended
> > > > > >> > > > > > >> > > > > > >>> > to
> > > > > >> > > > > > >> > > > > > >>> > > > > replace
> > > > > >> > > > > > >> > > > > > >>> > > > > > > before posting? :)  In
> any
> > > > case,
> > > > > >> this
> > > > > >> > > > seems
> > > > > >> > > > > > like
> > > > > >> > > > > > >> > the
> > > > > >> > > > > > >> > > > kind
> > > > > >> > > > > > >> > > > > > of
> > > > > >> > > > > > >> > > > > > >>> > thing
> > > > > >> > > > > > >> > > > > > >>> > > > we'd
> > > > > >> > > > > > >> > > > > > >>> > > > > > > want a configuration for.
> > > > Let's
> > > > > >> > > describe
> > > > > >> > > > > that
> > > > > >> > > > > > >> > > > > > configuration
> > > > > >> > > > > > >> > > > > > >>> key
> > > > > >> > > > > > >> > > > > > >>> > > > > somewhere
> > > > > >> > > > > > >> > > > > > >>> > > > > > > in this KIP, including
> what
> > > its
> > > > > >> > default
> > > > > >> > > > > value
> > > > > >> > > > > > >> is.
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > We should probably
> also
> > > > log a
> > > > > >> > bunch
> > > > > >> > > of
> > > > > >> > > > > > >> messages
> > > > > >> > > > > > >> > > at
> > > > > >> > > > > > >> > > > > WARN
> > > > > >> > > > > > >> > > > > > >>> level
> > > > > >> > > > > > >> > > > > > >>> > > > when
> > > > > >> > > > > > >> > > > > > >>> > > > > > > something is scheduled
> for
> > > > > >> deletion,
> > > > > >> > as
> > > > > >> > > > > well.
> > > > > >> > > > > > >> > (Maybe
> > > > > >> > > > > > >> > > > > this
> > > > > >> > > > > > >> > > > > > >>> was
> > > > > >> > > > > > >> > > > > > >>> > > > > assumed, but
> > > > > >> > > > > > >> > > > > > >>> > > > > > > it would be good to
> mention
> > > > it).
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > I feel like there
> are a
> > > few
> > > > > >> > sections
> > > > > >> > > > > that
> > > > > >> > > > > > >> > should
> > > > > >> > > > > > >> > > be
> > > > > >> > > > > > >> > > > > > >>> moved to
> > > > > >> > > > > > >> > > > > > >>> > > > > "rejected
> > > > > >> > > > > > >> > > > > > >>> > > > > > > alternatives."  For
> > example,
> > > in
> > > > > the
> > > > > >> > > > > > DeleteTopics
> > > > > >> > > > > > >> > > > section,
> > > > > >> > > > > > >> > > > > > >>> since
> > > > > >> > > > > > >> > > > > > >>> > we're
> > > > > >> > > > > > >> > > > > > >>> > > > > not
> > > > > >> > > > > > >> > > > > > >>> > > > > > > going to do option 1 or
> 2,
> > > > these
> > > > > >> > should
> > > > > >> > > be
> > > > > >> > > > > > moved
> > > > > >> > > > > > >> > into
> > > > > >> > > > > > >> > > > > > >>> "rejected
> > > > > >> > > > > > >> > > > > > >>> > > > > > > alternatives,"  rather
> than
> > > > > >> appearing
> > > > > >> > > > > inline.
> > > > > >> > > > > > >> > > Another
> > > > > >> > > > > > >> > > > > case
> > > > > >> > > > > > >> > > > > > >>> is
> > > > > >> > > > > > >> > > > > > >>> > the
> > > > > >> > > > > > >> > > > > > >>> > > > > "Should
> > > > > >> > > > > > >> > > > > > >>> > > > > > > we remove topic name from
> > the
> > > > > >> protocol
> > > > > >> > > > where
> > > > > >> > > > > > >> > > possible"
> > > > > >> > > > > > >> > > > > > >>> section.
> > > > > >> > > > > > >> > > > > > >>> > This
> > > > > >> > > > > > >> > > > > > >>> > > > > is
> > > > > >> > > > > > >> > > > > > >>> > > > > > > clearly discussing a
> design
> > > > > >> > alternative
> > > > > >> > > > that
> > > > > >> > > > > > >> we're
> > > > > >> > > > > > >> > > not
> > > > > >> > > > > > >> > > > > > >>> proposing
> > > > > >> > > > > > >> > > > > > >>> > to
> > > > > >> > > > > > >> > > > > > >>> > > > > > > implement: removing the
> > topic
> > > > > name
> > > > > >> > from
> > > > > >> > > > > those
> > > > > >> > > > > > >> > > > protocols.
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Is it really
> necessary
> > to
> > > > > have
> > > > > >> a
> > > > > >> > new
> > > > > >> > > > > > >> > > > > > >>> > /admin/delete_topics_by_id
> > > > > >> > > > > > >> > > > > > >>> > > > > path
> > > > > >> > > > > > >> > > > > > >>> > > > > > > in ZooKeeper?  It seems
> > like
> > > we
> > > > > >> don't
> > > > > >> > > > really
> > > > > >> > > > > > >> need
> > > > > >> > > > > > >> > > this.
> > > > > >> > > > > > >> > > > > > >>> Whenever
> > > > > >> > > > > > >> > > > > > >>> > > > > there is
> > > > > >> > > > > > >> > > > > > >>> > > > > > > a new controller, we'll
> > send
> > > > out
> > > > > >> full
> > > > > >> > > > > > >> > > > > LeaderAndIsrRequests
> > > > > >> > > > > > >> > > > > > >>> which
> > > > > >> > > > > > >> > > > > > >>> > will
> > > > > >> > > > > > >> > > > > > >>> > > > > > > trigger the stale topics
> to
> > > be
> > > > > >> cleaned
> > > > > >> > > up.
> > > > > >> > > > > >  The
> > > > > >> > > > > > >> > > active
> > > > > >> > > > > > >> > > > > > >>> > controller
> > > > > >> > > > > > >> > > > > > >>> > > > will
> > > > > >> > > > > > >> > > > > > >>> > > > > > > also send the full
> > > > > >> LeaderAndIsrRequest
> > > > > >> > > to
> > > > > >> > > > > > >> brokers
> > > > > >> > > > > > >> > > that
> > > > > >> > > > > > >> > > > > are
> > > > > >> > > > > > >> > > > > > >>> just
> > > > > >> > > > > > >> > > > > > >>> > > > > starting
> > > > > >> > > > > > >> > > > > > >>> > > > > > > up.    So we don't really
> > > need
> > > > > this
> > > > > >> > kind
> > > > > >> > > > of
> > > > > >> > > > > > >> > two-phase
> > > > > >> > > > > > >> > > > > > commit
> > > > > >> > > > > > >> > > > > > >>> > (send
> > > > > >> > > > > > >> > > > > > >>> > > > out
> > > > > >> > > > > > >> > > > > > >>> > > > > > > StopReplicasRequest, get
> > ACKs
> > > > > from
> > > > > >> all
> > > > > >> > > > > nodes,
> > > > > >> > > > > > >> > commit
> > > > > >> > > > > > >> > > by
> > > > > >> > > > > > >> > > > > > >>> removing
> > > > > >> > > > > > >> > > > > > >>> > > > > > > /admin/delete_topics
> node)
> > > any
> > > > > >> more.
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > You mention that
> > > > FetchRequest
> > > > > >> will
> > > > > >> > > now
> > > > > >> > > > > > >> include
> > > > > >> > > > > > >> > > UUID
> > > > > >> > > > > > >> > > > > to
> > > > > >> > > > > > >> > > > > > >>> avoid
> > > > > >> > > > > > >> > > > > > >>> > > > issues
> > > > > >> > > > > > >> > > > > > >>> > > > > > > where requests are made
> to
> > > > stale
> > > > > >> > > > partitions.
> > > > > >> > > > > > >> > > However,
> > > > > >> > > > > > >> > > > > > >>> adding a
> > > > > >> > > > > > >> > > > > > >>> > UUID
> > > > > >> > > > > > >> > > > > > >>> > > > to
> > > > > >> > > > > > >> > > > > > >>> > > > > > > MetadataRequest is listed
> > as
> > > > > future
> > > > > >> > > work,
> > > > > >> > > > > out
> > > > > >> > > > > > of
> > > > > >> > > > > > >> > > scope
> > > > > >> > > > > > >> > > > > for
> > > > > >> > > > > > >> > > > > > >>> this
> > > > > >> > > > > > >> > > > > > >>> > KIP.
> > > > > >> > > > > > >> > > > > > >>> > > > > How
> > > > > >> > > > > > >> > > > > > >>> > > > > > > will the client learn
> what
> > > the
> > > > > >> topic
> > > > > >> > > UUID
> > > > > >> > > > > is,
> > > > > >> > > > > > if
> > > > > >> > > > > > >> > the
> > > > > >> > > > > > >> > > > > > metadata
> > > > > >> > > > > > >> > > > > > >>> > > > response
> > > > > >> > > > > > >> > > > > > >>> > > > > > > doesn't include that
> > > > information?
> > > > > >> It
> > > > > >> > > > seems
> > > > > >> > > > > > like
> > > > > >> > > > > > >> > > adding
> > > > > >> > > > > > >> > > > > the
> > > > > >> > > > > > >> > > > > > >>> UUID
> > > > > >> > > > > > >> > > > > > >>> > to
> > > > > >> > > > > > >> > > > > > >>> > > > > > > MetadataResponse would be
> > an
> > > > > >> > improvement
> > > > > >> > > > > here
> > > > > >> > > > > > >> that
> > > > > >> > > > > > >> > > > might
> > > > > >> > > > > > >> > > > > > not
> > > > > >> > > > > > >> > > > > > >>> be
> > > > > >> > > > > > >> > > > > > >>> > too
> > > > > >> > > > > > >> > > > > > >>> > > > > hard to
> > > > > >> > > > > > >> > > > > > >>> > > > > > > make.
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > best,
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Colin
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > On Mon, Sep 9, 2019,
> at
> > > > > 17:48,
> > > > > >> > > Ryanne
> > > > > >> > > > > > Dolan
> > > > > >> > > > > > >> > > wrote:
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > Lucas, this would
> be
> > > > great.
> > > > > >> I've
> > > > > >> > > run
> > > > > >> > > > > > into
> > > > > >> > > > > > >> > > issues
> > > > > >> > > > > > >> > > > > with
> > > > > >> > > > > > >> > > > > > >>> > topics
> > > > > >> > > > > > >> > > > > > >>> > > > > being
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > resurrected
> > > accidentally,
> > > > > >> since
> > > > > >> > a
> > > > > >> > > > > client
> > > > > >> > > > > > >> > cannot
> > > > > >> > > > > > >> > > > > > easily
> > > > > >> > > > > > >> > > > > > >>> > > > > distinguish
> > > > > >> > > > > > >> > > > > > >>> > > > > > > between
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > a deleted topic
> and a
> > > new
> > > > > >> topic
> > > > > >> > > with
> > > > > >> > > > > the
> > > > > >> > > > > > >> same
> > > > > >> > > > > > >> > > > name.
> > > > > >> > > > > > >> > > > > > I'd
> > > > > >> > > > > > >> > > > > > >>> > need
> > > > > >> > > > > > >> > > > > > >>> > > > the
> > > > > >> > > > > > >> > > > > > >>> > > > > ID
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > accessible from the
> > > > client
> > > > > to
> > > > > >> > > solve
> > > > > >> > > > > that
> > > > > >> > > > > > >> > issue,
> > > > > >> > > > > > >> > > > but
> > > > > >> > > > > > >> > > > > > >>> this
> > > > > >> > > > > > >> > > > > > >>> > is a
> > > > > >> > > > > > >> > > > > > >>> > > > > good
> > > > > >> > > > > > >> > > > > > >>> > > > > > > first
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > step.
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > Ryanne
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > On Wed, Sep 4, 2019
> > at
> > > > 1:41
> > > > > >> PM
> > > > > >> > > Lucas
> > > > > >> > > > > > >> > > Bradstreet <
> > > > > >> > > > > > >> > > > > > >>> > > > > lucas@confluent.io>
> > > > > >> > > > > > >> > > > > > >>> > > > > > > wrote:
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Hi all,
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > I would like to
> > kick
> > > > off
> > > > > >> > > > discussion
> > > > > >> > > > > of
> > > > > >> > > > > > >> > > KIP-516,
> > > > > >> > > > > > >> > > > > an
> > > > > >> > > > > > >> > > > > > >>> > > > > implementation
> > > > > >> > > > > > >> > > > > > >>> > > > > > > of topic
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > IDs for Kafka.
> > Topic
> > > > IDs
> > > > > >> aim
> > > > > >> > to
> > > > > >> > > > > solve
> > > > > >> > > > > > >> topic
> > > > > >> > > > > > >> > > > > > >>> uniqueness
> > > > > >> > > > > > >> > > > > > >>> > > > > problems in
> > > > > >> > > > > > >> > > > > > >>> > > > > > > Kafka,
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > where referring
> to
> > a
> > > > > topic
> > > > > >> by
> > > > > >> > > name
> > > > > >> > > > > > >> alone is
> > > > > >> > > > > > >> > > > > > >>> insufficient.
> > > > > >> > > > > > >> > > > > > >>> > > > Such
> > > > > >> > > > > > >> > > > > > >>> > > > > > > cases
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > include when a
> > topic
> > > > has
> > > > > >> been
> > > > > >> > > > > deleted
> > > > > >> > > > > > >> and
> > > > > >> > > > > > >> > > > > recreated
> > > > > >> > > > > > >> > > > > > >>> with
> > > > > >> > > > > > >> > > > > > >>> > the
> > > > > >> > > > > > >> > > > > > >>> > > > > same
> > > > > >> > > > > > >> > > > > > >>> > > > > > > name.
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Unique
> identifiers
> > > will
> > > > > >> help
> > > > > >> > > > > simplify
> > > > > >> > > > > > >> and
> > > > > >> > > > > > >> > > > improve
> > > > > >> > > > > > >> > > > > > >>> Kafka's
> > > > > >> > > > > > >> > > > > > >>> > > > topic
> > > > > >> > > > > > >> > > > > > >>> > > > > > > deletion
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > process, as well
> as
> > > > > prevent
> > > > > >> > > cases
> > > > > >> > > > > > where
> > > > > >> > > > > > >> > > brokers
> > > > > >> > > > > > >> > > > > may
> > > > > >> > > > > > >> > > > > > >>> > > > incorrectly
> > > > > >> > > > > > >> > > > > > >>> > > > > > > interact
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > with stale
> versions
> > > of
> > > > > >> topics.
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > > >> > > > > > >> > > > > > >>> > > >
> > > > > >> > > > > > >> > > > > > >>> >
> > > > > >> > > > > > >> > > > > > >>>
> > > > > >> > > > > > >> > > > > >
> > > > > >> > > > > > >> > > > >
> > > > > >> > > > > > >> > > >
> > > > > >> > > > > > >> > >
> > > > > >> > > > > > >> >
> > > > > >> > > > > > >>
> > > > > >> > > > > >
> > > > > >> > > > >
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Looking forward
> to
> > > your
> > > > > >> > > thoughts.
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Lucas
> > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > > >> > > > > > >> > > > > > >>> >
> > > > > >> > > > > > >> > > > > > >>> >
> > > > > >> > > > > > >> > > > > > >>>
> > > > > >> > > > > > >> > > > > > >>
> > > > > >> > > > > > >> > > > > >
> > > > > >> > > > > > >> > > > >
> > > > > >> > > > > > >> > > >
> > > > > >> > > > > > >> > >
> > > > > >> > > > > > >> >
> > > > > >> > > > > > >>
> > > > > >> > > > > > >
> > > > > >> > > > > >
> > > > > >> > > > >
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Jason Gustafson <ja...@confluent.io>]

Thanks Justine. Responses below:

> 1. Yes, the directory will still be based on the topic names.
LeaderAndIsrRequest is one of the few requests that will still contain the
topic name. So I think we have this covered. Sorry for confusion.

Ah, you're right. My eyes passed right over the field.

> 2. Part of the usage of the file is to have persistent storage of the
topic
ID and use it to compare with the ID supplied in the LeaderAndIsr Request.
There is some discussion in the KIP about changes to the directory
structure, but I believe directory changes were considered to be out of
scope when the KIP was written.

Yeah, I was hoping to get a better understanding of why it was taken out of
scope. Perhaps Lucas Bradstreet might have more insight about the decision.
Basically my point is that we have to create additional infrastructure here
to support the name/id mapping, so I wanted to understand if we just
consider this a sort of tech debt. It would be useful to cover how we
handle the case when this file gets corrupted. Seems like we just have to
trust that it matches whatever the controller tells us and rewrite it?

> 3. I think this is a good point, but I again I wonder about the scope of
the KIP. Most of the changes mentioned in the KIP are for supporting topic
deletion and I believe that is why the produce request was listed under
future work.

That's fair. I brought it up since `Fetch` is already included. If we've
got `Metadata` and `Fetch`, seems we may as well do `Produce` and save an
extra kip. No strong objection though if you want to leave it out.


-Jason


On Thu, Sep 24, 2020 at 3:26 PM Justine Olshan <jo...@confluent.io> wrote:

> Hi Jason,
>
> Thanks for your comments.
>
> 1. Yes, the directory will still be based on the topic names.
> LeaderAndIsrRequest is one of the few requests that will still contain the
> topic name. So I think we have this covered. Sorry for confusion.
>
> 2. Part of the usage of the file is to have persistent storage of the topic
> ID and use it to compare with the ID supplied in the LeaderAndIsr Request.
> There is some discussion in the KIP about changes to the directory
> structure, but I believe directory changes were considered to be out of
> scope when the KIP was written.
>
> 3. I think this is a good point, but I again I wonder about the scope of
> the KIP. Most of the changes mentioned in the KIP are for supporting topic
> deletion and I believe that is why the produce request was listed under
> future work.
>
> 4. This sounds like it might be a good solution, but I will need to discuss
> more with KIP-500 folks to get the details right.
>
> Thanks,
> Justine
>
> On Thu, Sep 24, 2020 at 12:30 PM Jason Gustafson <ja...@confluent.io>
> wrote:
>
> > Hi Justine,
> >
> > Thanks for picking up this work. I have a few questions/comments:
> >
> > 1. It sounds like the directory structure is still going to be based on
> > topic names. Do I have that right? One complication is that the
> > LeaderAndIsr request does not include the topic name any longer. This
> means
> > that a replica must wait for the UpdateMetadata request to arrive with
> the
> > topic name mapping before it can create the directory. I wonder if it
> would
> > be simpler to avoid assumptions on the ordering of UpdateMetadata and let
> > LeaderAndIsr include the topic name as well. Feels like we are not saving
> > that much by excluding it.
> >
> > 2. On a related note, it seems that the reason we have the partition
> > metadata file is because we are not changing the directory structure. We
> > need it so that we remember which directories map to which topic id. I
> > think it would be useful to add some detail about why changing the
> > directory layout was rejected. Basically trying to understand how much
> > effort we are saving by skipping this step if we have to implement this
> > additional bookkeeping. One concern I have for example is that the
> > partition metadata file gets corrupt and then all bets are off as far as
> > topic consistency.
> >
> > 3. Is it worth adding support now for topic ids in the `Produce` request
> > now? Overhead is mentioned as one of the motivations and the best APIs to
> > get savings from are `Produce` and `Fetch`.
> >
> > 4. When it comes to bootstrapping the metadata topic with respect to
> > KIP-500, one suggestion would be to reserve a sentinel topic ID which can
> > be used initially by new brokers. When the first controller is elected,
> it
> > can assign a topicId to the metadata topic. This is a bit similar to how
> we
> > were planning to generate the clusterId.
> >
> > Thanks,
> > Jason
> >
> >
> > On Thu, Sep 24, 2020 at 11:10 AM Jun Rao <ju...@confluent.io> wrote:
> >
> > > Hi, Justine,
> > >
> > > Thanks for the updated KIP. A few more comments below.
> > >
> > > 30.  {"name": "id", "type": "string", "doc": "version id"}}: The doc
> > should
> > > say UUID.
> > >
> > > 31. LeaderAndIsrResponse v5 and StopReplicaResponse v4 : It seems there
> > is
> > > no need to add topic_id at partitions level.
> > >
> > > 32. Regarding partition metadata file. Perhaps the key can be a single
> > > word, sth like the following.
> > > version: 0
> > > topic_id: 46bdb63f-9e8d-4a38-bf7b-ee4eb2a794e4
> > >
> > > 33. Another tricky thing that I realized is how to support the metadata
> > > topic introduced in KIP-595. It's a bit tricky to assign a UUID to the
> > > metadata topic since we have a chicken-and-egg problem. The controller
> > > needs to persist the UUID in the metadata topic in order to assign one
> > > successfully, but the metadata topic is needed to elect a controller
> > > (KIP-631). So, this probably needs a bit more thought.
> > >
> > > Jun
> > >
> > > On Thu, Sep 24, 2020 at 3:04 AM Ismael Juma <is...@juma.me.uk> wrote:
> > >
> > > > Also, can we provide more details on how the Partition Metadata file
> > will
> > > > be used?
> > > >
> > > > Ismael
> > > >
> > > > On Thu, Sep 24, 2020 at 3:01 AM Ismael Juma <is...@juma.me.uk>
> wrote:
> > > >
> > > > > Hi Justine,
> > > > >
> > > > > I think we need to update the "Rejected Alternatives" section to
> take
> > > > into
> > > > > account that the proposal now removes the topic name from the fetch
> > > > > request. Also, if we are removing it from the Fetch request, does
> it
> > > make
> > > > > sense not to remove it from similar requests like
> ListOffsetRequest?
> > > > >
> > > > > Ismael
> > > > >
> > > > > On Thu, Sep 24, 2020 at 2:46 AM David Jacot <dj...@confluent.io>
> > > wrote:
> > > > >
> > > > >> Hi Justine,
> > > > >>
> > > > >> Thanks for the KIP. I finally had time to read it :). It is a
> great
> > > > >> improvement.
> > > > >>
> > > > >> I have a few comments/questions:
> > > > >>
> > > > >> 1. It seems that the schema of the StopReplicaRequest is slightly
> > > > >> outdated.
> > > > >> We
> > > > >> did some changes as part of KIP-570. V3 is already organized by
> > > topics.
> > > > >>
> > > > >> 2. I just want to make sure that I understand the reconciliation
> > > > >> logic correctly. When
> > > > >> an "INCREMENTAL" LeaderAndIsr Request is received, the broker will
> > > also
> > > > >> reconcile
> > > > >> when the local uuid does not match the uuid in the request, right?
> > In
> > > > this
> > > > >> case, the
> > > > >> local log is staged for deletion.
> > > > >>
> > > > >> 3. In the documentation of the `delete.stale.topic.delay.ms`
> > config,
> > > it
> > > > >> says "When a
> > > > >> FULL LeaderAndIsrRequest is received..." but I suppose it applies
> to
> > > > both
> > > > >> types.
> > > > >>
> > > > >> Best,
> > > > >> David
> > > > >>
> > > > >> On Thu, Sep 24, 2020 at 1:40 AM Justine Olshan <
> > jolshan@confluent.io>
> > > > >> wrote:
> > > > >>
> > > > >> > Hi Jun,
> > > > >> >
> > > > >> > Thanks for the comments. I apologize for some of the typos and
> > > > >> confusion.
> > > > >> > I’ve updated the KIP to fix some of the issues you mentioned.
> > > > >> >
> > > > >> > 20.2 I’ve changed the type to String.
> > > > >> > 20.1/3 I’ve updated the TopicZNode to fix formatting and reflect
> > the
> > > > >> latest
> > > > >> > version before this change.
> > > > >> >
> > > > >> > 21. You are correct and I’ve removed this line. I’ve also added
> a
> > > line
> > > > >> > mentioning an IBP bump is necessary for migration
> > > > >> >
> > > > >> > 22. I think the wording was unclear but your summary is what was
> > > > >> intended
> > > > >> > by this line. I’ve updated to make this point more clear.
> “Remove
> > > > >> deleted
> > > > >> > topics from replicas by sending StopReplicaRequest V3 before the
> > IBP
> > > > >> bump
> > > > >> > using the old logic, and using V4 and the new logic with topic
> IDs
> > > > after
> > > > >> > the IBP bump.”
> > > > >> >
> > > > >> > 23. I’ve removed the unspecified type from the KIP and mention
> > that
> > > > IBP
> > > > >> > will be used to handle this request. “IBP will be used to
> > determine
> > > > >> whether
> > > > >> > this new form of the request will be used. For older requests,
> we
> > > will
> > > > >> > ignore this field and default to previous behavior.”
> > > > >> >
> > > > >> > 24. I’ve fixed this typo.
> > > > >> >
> > > > >> > 25. I've added a topics at a higher level for
> LeaderAndIsrResponse
> > > v5,
> > > > >> > StopReplicaResponse v4. I've also changed StopReplicaRequest v4
> to
> > > > have
> > > > >> > topics at a higher level.
> > > > >> >
> > > > >> > 26. I’ve updated forgotten_topics_data--added the topic ID and
> > > removed
> > > > >> the
> > > > >> > topic name
> > > > >> >
> > > > >> > 27. I’ve decided on plain text, and I’ve added an example.
> > > > >> >
> > > > >> > 28. I’ve added this idea to future work.
> > > > >> >
> > > > >> > Thanks again for taking a look,
> > > > >> >
> > > > >> > Justine
> > > > >> >
> > > > >> > On Wed, Sep 23, 2020 at 10:28 AM Jun Rao <ju...@confluent.io>
> > wrote:
> > > > >> >
> > > > >> > > Hi, Justine,
> > > > >> > >
> > > > >> > > Thanks for the response. Made another pass. A few more
> comments
> > > > below.
> > > > >> > >
> > > > >> > > 20. znode schema:
> > > > >> > > 20.1 It seems that {"name": "version", "type": "int", "id":
> > > "UUID",
> > > > >> > "doc":
> > > > >> > > "version id"} should be {"name": "version", "type": "int"},
> > > {"name":
> > > > >> > "id",
> > > > >> > > "type": "UUID", "doc": "version id"}.
> > > > >> > > 20.2 The znode format is JSON which doesn't have UUID type. So
> > the
> > > > >> type
> > > > >> > > probably should be string?
> > > > >> > > 20.3 Also, the existing format used seems outdated. It should
> > have
> > > > the
> > > > >> > > following format.
> > > > >> > >     Json.encodeAsBytes(Map(
> > > > >> > >       "version" -> 2,
> > > > >> > >       "partitions" -> replicaAssignmentJson.asJava,
> > > > >> > >       "adding_replicas" ->
> addingReplicasAssignmentJson.asJava,
> > > > >> > >       "removing_replicas" ->
> > removingReplicasAssignmentJson.asJava
> > > > >> > >     ).asJava)
> > > > >> > >   }
> > > > >> > >
> > > > >> > > 21. Migration: The KIP says "The migration process can take
> > place
> > > > >> without
> > > > >> > > an inter-broker protocol bump, as the format stored in
> > > > >> > > /brokers/topics/[topic] will be compatible with older broker
> > > > >> versions."
> > > > >> > > However, since we are bumping up the version of inter-broker
> > > > >> requests, it
> > > > >> > > seems that we need to use IBP for migration.
> > > > >> > >
> > > > >> > > 22. The KIP says "Remove deleted topics from replicas by
> sending
> > > > >> > > StopReplicaRequest V3 for any topics which do not contain a
> > topic
> > > > ID,
> > > > >> and
> > > > >> > > V4 for any topics which do contain a topic ID.". However, if
> we
> > > use
> > > > >> IBP,
> > > > >> > it
> > > > >> > > seems that the controller will either send StopReplicaRequest
> V3
> > > > >> > > or StopReplicaRequest V4, but never mixed V3 and V4 for
> > different
> > > > >> topics.
> > > > >> > > Basically, before the IBP bump, V3 will be used. After the IBP
> > > bump,
> > > > >> > > topicId will be created and V4 will be used.
> > > > >> > >
> > > > >> > > 23. Given that we depend on IBP, do we still need "0
> > UNSPECIFIED"
> > > > >> > > in LeaderAndIsr?
> > > > >> > >
> > > > >> > > 24. LeaderAndIsrResponse v5 : It still has the topic field.
> > > > >> > >
> > > > >> > > 25. LeaderAndIsrResponse v5, StopReplicaResponse v4: Could we
> > use
> > > > this
> > > > >> > > opportunity to organize the response in 2 levels, first by
> > topic,
> > > > >> then by
> > > > >> > > partition, as most other requests/responses?
> > > > >> > >
> > > > >> > > 26. FetchRequest v13 : Should forgotten_topics_data use
> topicId
> > > too?
> > > > >> > >
> > > > >> > > 27. "This file can either be plain text (key/value pairs) or
> > > JSON."
> > > > >> Have
> > > > >> > we
> > > > >> > > decided which one to use? Also, it would be helpful to provide
> > an
> > > > >> > example.
> > > > >> > >
> > > > >> > > 28. Future improvement: Another future improvement opportunity
> > is
> > > to
> > > > >> use
> > > > >> > > topicId in GroupMetadataManager.offsetCommitKey in the
> > > offset_commit
> > > > >> > topic.
> > > > >> > > This may save some space.
> > > > >> > >
> > > > >> > > Thanks,
> > > > >> > >
> > > > >> > > Jun
> > > > >> > >
> > > > >> > > On Wed, Sep 23, 2020 at 8:50 AM Justine Olshan <
> > > > jolshan@confluent.io>
> > > > >> > > wrote:
> > > > >> > >
> > > > >> > > > Hi Tom,
> > > > >> > > >
> > > > >> > > > Thanks for the comment. I think this is a really good idea
> and
> > > it
> > > > >> has
> > > > >> > > been
> > > > >> > > > added to the KIP under the newly added tooling section.
> > > > >> > > >
> > > > >> > > > Thanks again,
> > > > >> > > > Justine
> > > > >> > > >
> > > > >> > > > On Wed, Sep 23, 2020 at 3:17 AM Tom Bentley <
> > > tbentley@redhat.com>
> > > > >> > wrote:
> > > > >> > > >
> > > > >> > > > > Hi Justine,
> > > > >> > > > >
> > > > >> > > > > I know you started the vote thread, but on re-reading the
> > KIP
> > > I
> > > > >> > noticed
> > > > >> > > > > that although the topic id is included in the
> > MetadataResponse
> > > > >> it's
> > > > >> > not
> > > > >> > > > > surfaced in the output from `kafka-topics.sh --describe`.
> > > Maybe
> > > > >> that
> > > > >> > > was
> > > > >> > > > > intentional because ids are intentionally not really
> > something
> > > > the
> > > > >> > user
> > > > >> > > > > should care deeply about, but it would also make life
> harder
> > > for
> > > > >> > anyone
> > > > >> > > > > debugging Kafka and this would likely get worse the more
> > topic
> > > > ids
> > > > >> > got
> > > > >> > > > > rolled out across the protocols, clients etc. It seems
> > likely
> > > > that
> > > > >> > > > > `kafka-topics.sh` will eventually need the ability to show
> > the
> > > > id
> > > > >> of
> > > > >> > a
> > > > >> > > > > topic and perhaps find a topic name given an id. Is there
> > any
> > > > >> reason
> > > > >> > > not
> > > > >> > > > to
> > > > >> > > > > implement that in this KIP?
> > > > >> > > > >
> > > > >> > > > > Many thanks,
> > > > >> > > > >
> > > > >> > > > > Tom
> > > > >> > > > >
> > > > >> > > > > On Mon, Sep 21, 2020 at 9:54 PM Justine Olshan <
> > > > >> jolshan@confluent.io
> > > > >> > >
> > > > >> > > > > wrote:
> > > > >> > > > >
> > > > >> > > > > > Hi all,
> > > > >> > > > > >
> > > > >> > > > > > After thinking about it, I've decided to remove the
> topic
> > > name
> > > > >> from
> > > > >> > > the
> > > > >> > > > > > Fetch Request and Response after all. Since there are so
> > > many
> > > > of
> > > > >> > > these
> > > > >> > > > > > requests per second, it is worth removing the extra
> > > > information.
> > > > >> > I've
> > > > >> > > > > > updated the KIP to reflect this change.
> > > > >> > > > > >
> > > > >> > > > > > Please let me know if there is anything else we should
> > > discuss
> > > > >> > before
> > > > >> > > > > > voting.
> > > > >> > > > > >
> > > > >> > > > > > Thank you,
> > > > >> > > > > > Justine
> > > > >> > > > > >
> > > > >> > > > > > On Fri, Sep 18, 2020 at 9:46 AM Justine Olshan <
> > > > >> > jolshan@confluent.io
> > > > >> > > >
> > > > >> > > > > > wrote:
> > > > >> > > > > >
> > > > >> > > > > > > Hi Jun,
> > > > >> > > > > > >
> > > > >> > > > > > > I see what you are saying. For now we can remove the
> > extra
> > > > >> > > > information.
> > > > >> > > > > > > I'll leave the option to add more fields to the file
> in
> > > the
> > > > >> > future.
> > > > >> > > > The
> > > > >> > > > > > KIP
> > > > >> > > > > > > has been updated to reflect this change.
> > > > >> > > > > > >
> > > > >> > > > > > > Thanks,
> > > > >> > > > > > > Justine
> > > > >> > > > > > >
> > > > >> > > > > > > On Fri, Sep 18, 2020 at 8:46 AM Jun Rao <
> > jun@confluent.io
> > > >
> > > > >> > wrote:
> > > > >> > > > > > >
> > > > >> > > > > > >> Hi, Justine,
> > > > >> > > > > > >>
> > > > >> > > > > > >> Thanks for the reply.
> > > > >> > > > > > >>
> > > > >> > > > > > >> 13. If the log directory is the source of truth, it
> > means
> > > > >> that
> > > > >> > the
> > > > >> > > > > > >> redundant info in the metadata file will be ignored.
> > Then
> > > > the
> > > > >> > > > question
> > > > >> > > > > > is
> > > > >> > > > > > >> why do we need to put the redundant info in the
> > metadata
> > > > file
> > > > >> > now?
> > > > >> > > > > > >>
> > > > >> > > > > > >> Thanks,
> > > > >> > > > > > >>
> > > > >> > > > > > >> Jun
> > > > >> > > > > > >>
> > > > >> > > > > > >> On Thu, Sep 17, 2020 at 5:07 PM Justine Olshan <
> > > > >> > > > jolshan@confluent.io>
> > > > >> > > > > > >> wrote:
> > > > >> > > > > > >>
> > > > >> > > > > > >> > Hi Jun,
> > > > >> > > > > > >> > Thanks for the quick response!
> > > > >> > > > > > >> >
> > > > >> > > > > > >> > 12. I've decided to bump up the versions on the
> > > requests
> > > > >> and
> > > > >> > > > updated
> > > > >> > > > > > the
> > > > >> > > > > > >> > KIP. I think it's good we thoroughly discussed the
> > > > options
> > > > >> > here,
> > > > >> > > > so
> > > > >> > > > > we
> > > > >> > > > > > >> know
> > > > >> > > > > > >> > we made a good choice. :)
> > > > >> > > > > > >> >
> > > > >> > > > > > >> > 13. This is an interesting situation. I think if
> this
> > > > does
> > > > >> > occur
> > > > >> > > > we
> > > > >> > > > > > >> should
> > > > >> > > > > > >> > give a warning. I agree that it's hard to know the
> > > source
> > > > >> of
> > > > >> > > truth
> > > > >> > > > > for
> > > > >> > > > > > >> sure
> > > > >> > > > > > >> > since the directory or the file could be manually
> > > > >> modified. I
> > > > >> > > > guess
> > > > >> > > > > > the
> > > > >> > > > > > >> > directory could be used as the source of truth. To
> be
> > > > >> honest,
> > > > >> > > I'm
> > > > >> > > > > not
> > > > >> > > > > > >> > really sure what happens in kafka when the log
> > > directory
> > > > is
> > > > >> > > > renamed
> > > > >> > > > > > >> > manually in such a way. I'm also wondering if the
> > > > >> situation is
> > > > >> > > > > > >> recoverable
> > > > >> > > > > > >> > in this scenario.
> > > > >> > > > > > >> >
> > > > >> > > > > > >> > Thanks,
> > > > >> > > > > > >> > Justine
> > > > >> > > > > > >> >
> > > > >> > > > > > >> > On Thu, Sep 17, 2020 at 4:28 PM Jun Rao <
> > > > jun@confluent.io>
> > > > >> > > wrote:
> > > > >> > > > > > >> >
> > > > >> > > > > > >> > > Hi, Justine,
> > > > >> > > > > > >> > >
> > > > >> > > > > > >> > > Thanks for the reply.
> > > > >> > > > > > >> > >
> > > > >> > > > > > >> > > 12. I don't have a strong preference either.
> > However,
> > > > if
> > > > >> we
> > > > >> > > need
> > > > >> > > > > IBP
> > > > >> > > > > > >> > > anyway, maybe it's easier to just bump up the
> > version
> > > > for
> > > > >> > all
> > > > >> > > > > inter
> > > > >> > > > > > >> > broker
> > > > >> > > > > > >> > > requests and add the topic id field as a regular
> > > > field. A
> > > > >> > > > regular
> > > > >> > > > > > >> field
> > > > >> > > > > > >> > is
> > > > >> > > > > > >> > > a bit more concise in wire transfer than a
> flexible
> > > > >> field.
> > > > >> > > > > > >> > >
> > > > >> > > > > > >> > > 13. The confusion that I was referring to is
> > between
> > > > the
> > > > >> > topic
> > > > >> > > > > name
> > > > >> > > > > > >> and
> > > > >> > > > > > >> > > partition number between the log dir and the
> > metadata
> > > > >> file.
> > > > >> > > For
> > > > >> > > > > > >> example,
> > > > >> > > > > > >> > if
> > > > >> > > > > > >> > > the log dir is topicA-1 and the metadata file in
> it
> > > has
> > > > >> > topicB
> > > > >> > > > and
> > > > >> > > > > > >> > > partition 0 (say due to a bug or manual
> > > modification),
> > > > >> which
> > > > >> > > one
> > > > >> > > > > do
> > > > >> > > > > > we
> > > > >> > > > > > >> > use
> > > > >> > > > > > >> > > as the source of truth?
> > > > >> > > > > > >> > >
> > > > >> > > > > > >> > > Jun
> > > > >> > > > > > >> > >
> > > > >> > > > > > >> > > On Thu, Sep 17, 2020 at 3:43 PM Justine Olshan <
> > > > >> > > > > > jolshan@confluent.io>
> > > > >> > > > > > >> > > wrote:
> > > > >> > > > > > >> > >
> > > > >> > > > > > >> > > > Hi Jun,
> > > > >> > > > > > >> > > > Thanks for the comments.
> > > > >> > > > > > >> > > >
> > > > >> > > > > > >> > > > 12. I bumped the LeaderAndIsrRequest because I
> > > > removed
> > > > >> the
> > > > >> > > > topic
> > > > >> > > > > > >> name
> > > > >> > > > > > >> > > field
> > > > >> > > > > > >> > > > in the response. It may be possible to avoid
> > > bumping
> > > > >> the
> > > > >> > > > version
> > > > >> > > > > > >> > without
> > > > >> > > > > > >> > > > that change, but I may be missing something.
> > > > >> > > > > > >> > > > I believe StopReplica is actually on version 3
> > now,
> > > > but
> > > > >> > > > because
> > > > >> > > > > > >> > version 2
> > > > >> > > > > > >> > > > is flexible, I kept that listed as version 2 on
> > the
> > > > KIP
> > > > >> > > page.
> > > > >> > > > > > >> However,
> > > > >> > > > > > >> > > you
> > > > >> > > > > > >> > > > may be right in that we may need to bump the
> > > version
> > > > on
> > > > >> > > > > > StopReplica
> > > > >> > > > > > >> to
> > > > >> > > > > > >> > > deal
> > > > >> > > > > > >> > > > with deletion differently as mentioned above. I
> > > don't
> > > > >> know
> > > > >> > > if
> > > > >> > > > I
> > > > >> > > > > > >> have a
> > > > >> > > > > > >> > > big
> > > > >> > > > > > >> > > > preference over used tagged fields or not.
> > > > >> > > > > > >> > > >
> > > > >> > > > > > >> > > > 13. I was thinking that in the case where the
> > file
> > > > and
> > > > >> the
> > > > >> > > > > request
> > > > >> > > > > > >> > topic
> > > > >> > > > > > >> > > > ids don't match, it means that the broker's
> > > topic/the
> > > > >> one
> > > > >> > in
> > > > >> > > > the
> > > > >> > > > > > >> file
> > > > >> > > > > > >> > has
> > > > >> > > > > > >> > > > been deleted. In that case, we would need to
> > delete
> > > > the
> > > > >> > old
> > > > >> > > > > topic
> > > > >> > > > > > >> and
> > > > >> > > > > > >> > > start
> > > > >> > > > > > >> > > > receiving the new version. If the topic name
> were
> > > to
> > > > >> > change,
> > > > >> > > > but
> > > > >> > > > > > the
> > > > >> > > > > > >> > ids
> > > > >> > > > > > >> > > > still matched, the file would also need to
> > update.
> > > > Am I
> > > > >> > > > missing
> > > > >> > > > > a
> > > > >> > > > > > >> case
> > > > >> > > > > > >> > > > where the file would be correct and not the
> > > request?
> > > > >> > > > > > >> > > >
> > > > >> > > > > > >> > > > Thanks,
> > > > >> > > > > > >> > > > Justine
> > > > >> > > > > > >> > > >
> > > > >> > > > > > >> > > > On Thu, Sep 17, 2020 at 3:18 PM Jun Rao <
> > > > >> jun@confluent.io
> > > > >> > >
> > > > >> > > > > wrote:
> > > > >> > > > > > >> > > >
> > > > >> > > > > > >> > > > > Hi, Justine,
> > > > >> > > > > > >> > > > >
> > > > >> > > > > > >> > > > > Thanks for the reply. A couple of more
> comments
> > > > >> below.
> > > > >> > > > > > >> > > > >
> > > > >> > > > > > >> > > > > 12. ListOffset and OffsetForLeader currently
> > > don't
> > > > >> > support
> > > > >> > > > > > >> flexible
> > > > >> > > > > > >> > > > fields.
> > > > >> > > > > > >> > > > > So, we have to bump up the version number and
> > use
> > > > >> IBP at
> > > > >> > > > least
> > > > >> > > > > > for
> > > > >> > > > > > >> > > these
> > > > >> > > > > > >> > > > > two requests. Note that it seems 2.7.0 will
> > > require
> > > > >> IBP
> > > > >> > > > anyway
> > > > >> > > > > > >> > because
> > > > >> > > > > > >> > > of
> > > > >> > > > > > >> > > > > changes in KAFKA-10435. Also, it seems that
> the
> > > > >> version
> > > > >> > > for
> > > > >> > > > > > >> > > > > LeaderAndIsrRequest and StopReplica are
> bumped
> > > even
> > > > >> > though
> > > > >> > > > we
> > > > >> > > > > > only
> > > > >> > > > > > >> > > added
> > > > >> > > > > > >> > > > a
> > > > >> > > > > > >> > > > > tagged field. But since IBP is needed anyway,
> > we
> > > > may
> > > > >> > want
> > > > >> > > to
> > > > >> > > > > > >> revisit
> > > > >> > > > > > >> > > the
> > > > >> > > > > > >> > > > > overall tagged field choice.
> > > > >> > > > > > >> > > > >
> > > > >> > > > > > >> > > > > 13. The only downside is the potential
> > confusion
> > > on
> > > > >> > which
> > > > >> > > > one
> > > > >> > > > > is
> > > > >> > > > > > >> the
> > > > >> > > > > > >> > > > source
> > > > >> > > > > > >> > > > > of truth if they don't match. Another option
> is
> > > to
> > > > >> > include
> > > > >> > > > > those
> > > > >> > > > > > >> > fields
> > > > >> > > > > > >> > > > in
> > > > >> > > > > > >> > > > > the metadata file when we actually change the
> > > > >> directory
> > > > >> > > > > > structure.
> > > > >> > > > > > >> > > > >
> > > > >> > > > > > >> > > > > Thanks,
> > > > >> > > > > > >> > > > >
> > > > >> > > > > > >> > > > > Jun
> > > > >> > > > > > >> > > > >
> > > > >> > > > > > >> > > > > On Thu, Sep 17, 2020 at 2:01 PM Justine
> Olshan
> > <
> > > > >> > > > > > >> jolshan@confluent.io
> > > > >> > > > > > >> > >
> > > > >> > > > > > >> > > > > wrote:
> > > > >> > > > > > >> > > > >
> > > > >> > > > > > >> > > > > > Hello all,
> > > > >> > > > > > >> > > > > >
> > > > >> > > > > > >> > > > > > I've thought some more about removing the
> > topic
> > > > >> name
> > > > >> > > field
> > > > >> > > > > > from
> > > > >> > > > > > >> > some
> > > > >> > > > > > >> > > of
> > > > >> > > > > > >> > > > > the
> > > > >> > > > > > >> > > > > > requests. On closer inspection of the
> > > > >> > > requests/responses,
> > > > >> > > > it
> > > > >> > > > > > >> seems
> > > > >> > > > > > >> > > that
> > > > >> > > > > > >> > > > > the
> > > > >> > > > > > >> > > > > > internal changes would be much larger than
> I
> > > > >> expected.
> > > > >> > > > Some
> > > > >> > > > > > >> > protocols
> > > > >> > > > > > >> > > > > > involve clients, so they would require
> > changes
> > > > too.
> > > > >> > I'm
> > > > >> > > > > > thinking
> > > > >> > > > > > >> > that
> > > > >> > > > > > >> > > > for
> > > > >> > > > > > >> > > > > > now, removing the topic name from these
> > > requests
> > > > >> and
> > > > >> > > > > responses
> > > > >> > > > > > >> are
> > > > >> > > > > > >> > > out
> > > > >> > > > > > >> > > > of
> > > > >> > > > > > >> > > > > > scope.
> > > > >> > > > > > >> > > > > >
> > > > >> > > > > > >> > > > > > I have decided to just keep the change
> > > > >> > > > LeaderAndIsrResponse
> > > > >> > > > > to
> > > > >> > > > > > >> > remove
> > > > >> > > > > > >> > > > the
> > > > >> > > > > > >> > > > > > topic name, and have updated the KIP to
> > reflect
> > > > >> this
> > > > >> > > > > change. I
> > > > >> > > > > > >> have
> > > > >> > > > > > >> > > > also
> > > > >> > > > > > >> > > > > > mentioned the other requests and responses
> in
> > > > >> future
> > > > >> > > work.
> > > > >> > > > > > >> > > > > >
> > > > >> > > > > > >> > > > > > I'm hoping to start the voting process
> soon,
> > so
> > > > >> let me
> > > > >> > > > know
> > > > >> > > > > if
> > > > >> > > > > > >> > there
> > > > >> > > > > > >> > > is
> > > > >> > > > > > >> > > > > > anything else we should discuss.
> > > > >> > > > > > >> > > > > >
> > > > >> > > > > > >> > > > > > Thank you,
> > > > >> > > > > > >> > > > > > Justine
> > > > >> > > > > > >> > > > > >
> > > > >> > > > > > >> > > > > > On Tue, Sep 15, 2020 at 3:57 PM Justine
> > Olshan
> > > <
> > > > >> > > > > > >> > jolshan@confluent.io
> > > > >> > > > > > >> > > >
> > > > >> > > > > > >> > > > > > wrote:
> > > > >> > > > > > >> > > > > >
> > > > >> > > > > > >> > > > > > > Hello again,
> > > > >> > > > > > >> > > > > > > To follow up on some of the other
> comments:
> > > > >> > > > > > >> > > > > > >
> > > > >> > > > > > >> > > > > > > 10/11) We can remove the topic name from
> > > these
> > > > >> > > > > > >> > requests/responses,
> > > > >> > > > > > >> > > > and
> > > > >> > > > > > >> > > > > > > that means we will just have to make a
> few
> > > > >> internal
> > > > >> > > > > changes
> > > > >> > > > > > to
> > > > >> > > > > > >> > make
> > > > >> > > > > > >> > > > > > > partitions accessible by topic id and
> > > > partition.
> > > > >> I
> > > > >> > can
> > > > >> > > > > > update
> > > > >> > > > > > >> the
> > > > >> > > > > > >> > > KIP
> > > > >> > > > > > >> > > > > to
> > > > >> > > > > > >> > > > > > > remove them unless anyone thinks they
> > should
> > > > >> stay.
> > > > >> > > > > > >> > > > > > >
> > > > >> > > > > > >> > > > > > > 12) Addressed in the previous email. I've
> > > > updated
> > > > >> > the
> > > > >> > > > KIP
> > > > >> > > > > to
> > > > >> > > > > > >> > > include
> > > > >> > > > > > >> > > > > > > tagged fields for the requests and
> > responses.
> > > > >> (More
> > > > >> > on
> > > > >> > > > > that
> > > > >> > > > > > >> > below)
> > > > >> > > > > > >> > > > > > >
> > > > >> > > > > > >> > > > > > > 13) I think part of the idea for
> including
> > > this
> > > > >> > > > > information
> > > > >> > > > > > >> is to
> > > > >> > > > > > >> > > > > prepare
> > > > >> > > > > > >> > > > > > > for future changes. Perhaps the directory
> > > > >> structure
> > > > >> > > > might
> > > > >> > > > > > >> change
> > > > >> > > > > > >> > > from
> > > > >> > > > > > >> > > > > > > topicName_partitionNumber to something
> like
> > > > >> > > > > > >> > > topicID_partitionNumber.
> > > > >> > > > > > >> > > > > Then
> > > > >> > > > > > >> > > > > > > it would be useful to have the topic name
> > in
> > > > the
> > > > >> > file
> > > > >> > > > > since
> > > > >> > > > > > it
> > > > >> > > > > > >> > > would
> > > > >> > > > > > >> > > > > not
> > > > >> > > > > > >> > > > > > be
> > > > >> > > > > > >> > > > > > > in the directory structure. Supporting
> > topic
> > > > >> renames
> > > > >> > > > might
> > > > >> > > > > > be
> > > > >> > > > > > >> > > easier
> > > > >> > > > > > >> > > > if
> > > > >> > > > > > >> > > > > > the
> > > > >> > > > > > >> > > > > > > other fields are included. Would there be
> > any
> > > > >> > > downsides
> > > > >> > > > to
> > > > >> > > > > > >> > > including
> > > > >> > > > > > >> > > > > this
> > > > >> > > > > > >> > > > > > > information?
> > > > >> > > > > > >> > > > > > >
> > > > >> > > > > > >> > > > > > > 14)  Yes, we would need to copy the
> > partition
> > > > >> > metadata
> > > > >> > > > > file
> > > > >> > > > > > in
> > > > >> > > > > > >> > this
> > > > >> > > > > > >> > > > > > > process. I've updated the KIP to include
> > > this.
> > > > >> > > > > > >> > > > > > >
> > > > >> > > > > > >> > > > > > > 15) I believe Lucas meant v1 and v2 here.
> > He
> > > > was
> > > > >> > > > referring
> > > > >> > > > > > to
> > > > >> > > > > > >> how
> > > > >> > > > > > >> > > the
> > > > >> > > > > > >> > > > > > > requests would fall under different IBP
> and
> > > > meant
> > > > >> > that
> > > > >> > > > > older
> > > > >> > > > > > >> > > brokers
> > > > >> > > > > > >> > > > > > would
> > > > >> > > > > > >> > > > > > > have to use the older version of the
> > request
> > > > and
> > > > >> the
> > > > >> > > > > > existing
> > > > >> > > > > > >> > topic
> > > > >> > > > > > >> > > > > > > deletion process. At first, it seemed
> like
> > > > tagged
> > > > >> > > fields
> > > > >> > > > > > would
> > > > >> > > > > > >> > > > resolve
> > > > >> > > > > > >> > > > > > > the IBP issue. However, we may need IBP
> for
> > > > this
> > > > >> > > request
> > > > >> > > > > > after
> > > > >> > > > > > >> > all
> > > > >> > > > > > >> > > > > since
> > > > >> > > > > > >> > > > > > > the controller handles the topic deletion
> > > > >> > differently
> > > > >> > > > > > >> depending
> > > > >> > > > > > >> > on
> > > > >> > > > > > >> > > > the
> > > > >> > > > > > >> > > > > > IBP
> > > > >> > > > > > >> > > > > > > version. In an older version, we can't
> just
> > > > send
> > > > >> a
> > > > >> > > > > > StopReplica
> > > > >> > > > > > >> > > delete
> > > > >> > > > > > >> > > > > the
> > > > >> > > > > > >> > > > > > > topic immediately like we'd want to for
> > this
> > > > KIP.
> > > > >> > > > > > >> > > > > > >
> > > > >> > > > > > >> > > > > > > This makes me wonder if we want tagged
> > fields
> > > > on
> > > > >> all
> > > > >> > > the
> > > > >> > > > > > >> requests
> > > > >> > > > > > >> > > > after
> > > > >> > > > > > >> > > > > > > all. Let me know your thoughts!
> > > > >> > > > > > >> > > > > > >
> > > > >> > > > > > >> > > > > > > Justine
> > > > >> > > > > > >> > > > > > >
> > > > >> > > > > > >> > > > > > > On Tue, Sep 15, 2020 at 1:03 PM Justine
> > > Olshan
> > > > <
> > > > >> > > > > > >> > > jolshan@confluent.io
> > > > >> > > > > > >> > > > >
> > > > >> > > > > > >> > > > > > > wrote:
> > > > >> > > > > > >> > > > > > >
> > > > >> > > > > > >> > > > > > >> Hi all,
> > > > >> > > > > > >> > > > > > >> Jun brought up a good point in his last
> > > email
> > > > >> about
> > > > >> > > > > tagged
> > > > >> > > > > > >> > fields,
> > > > >> > > > > > >> > > > and
> > > > >> > > > > > >> > > > > > >> I've updated the KIP to reflect that the
> > > > >> changes to
> > > > >> > > > > > requests
> > > > >> > > > > > >> and
> > > > >> > > > > > >> > > > > > responses
> > > > >> > > > > > >> > > > > > >> will be in the form of tagged fields to
> > > avoid
> > > > >> > > changing
> > > > >> > > > > IBP.
> > > > >> > > > > > >> > > > > > >>
> > > > >> > > > > > >> > > > > > >> Jun: I plan on sending a followup email
> to
> > > > >> address
> > > > >> > > some
> > > > >> > > > > of
> > > > >> > > > > > >> the
> > > > >> > > > > > >> > > other
> > > > >> > > > > > >> > > > > > >> points.
> > > > >> > > > > > >> > > > > > >>
> > > > >> > > > > > >> > > > > > >> Thanks,
> > > > >> > > > > > >> > > > > > >> Justine
> > > > >> > > > > > >> > > > > > >>
> > > > >> > > > > > >> > > > > > >> On Mon, Sep 14, 2020 at 4:25 PM Jun Rao
> <
> > > > >> > > > > jun@confluent.io>
> > > > >> > > > > > >> > wrote:
> > > > >> > > > > > >> > > > > > >>
> > > > >> > > > > > >> > > > > > >>> Hi, Justine,
> > > > >> > > > > > >> > > > > > >>>
> > > > >> > > > > > >> > > > > > >>> Thanks for the updated KIP. A few
> > comments
> > > > >> below.
> > > > >> > > > > > >> > > > > > >>>
> > > > >> > > > > > >> > > > > > >>> 10. LeaderAndIsr Response: Do we need
> the
> > > > topic
> > > > >> > > name?
> > > > >> > > > > > >> > > > > > >>>
> > > > >> > > > > > >> > > > > > >>> 11. For the changed request/response,
> > other
> > > > >> than
> > > > >> > > > > > >> LeaderAndIsr,
> > > > >> > > > > > >> > > > > > >>> UpdateMetadata, Metadata, do we need to
> > > > include
> > > > >> > the
> > > > >> > > > > topic
> > > > >> > > > > > >> name?
> > > > >> > > > > > >> > > > > > >>>
> > > > >> > > > > > >> > > > > > >>> 12. It seems that upgrades don't
> require
> > > IBP.
> > > > >> Does
> > > > >> > > > that
> > > > >> > > > > > mean
> > > > >> > > > > > >> > the
> > > > >> > > > > > >> > > > new
> > > > >> > > > > > >> > > > > > >>> fields
> > > > >> > > > > > >> > > > > > >>> in all the request/response are added
> as
> > > > tagged
> > > > >> > > fields
> > > > >> > > > > > >> without
> > > > >> > > > > > >> > > > > bumping
> > > > >> > > > > > >> > > > > > up
> > > > >> > > > > > >> > > > > > >>> the request version? It would be useful
> > to
> > > > make
> > > > >> > that
> > > > >> > > > > > clear.
> > > > >> > > > > > >> > > > > > >>>
> > > > >> > > > > > >> > > > > > >>> 13. Partition Metadata file: Do we need
> > to
> > > > >> include
> > > > >> > > the
> > > > >> > > > > > topic
> > > > >> > > > > > >> > name
> > > > >> > > > > > >> > > > and
> > > > >> > > > > > >> > > > > > the
> > > > >> > > > > > >> > > > > > >>> partition id since they are implied in
> > the
> > > > >> > directory
> > > > >> > > > > name?
> > > > >> > > > > > >> > > > > > >>>
> > > > >> > > > > > >> > > > > > >>> 14. In the JBOD mode, we support
> moving a
> > > > >> > > partition's
> > > > >> > > > > data
> > > > >> > > > > > >> from
> > > > >> > > > > > >> > > one
> > > > >> > > > > > >> > > > > > disk
> > > > >> > > > > > >> > > > > > >>> to
> > > > >> > > > > > >> > > > > > >>> another. Will the new partition
> metadata
> > > file
> > > > >> be
> > > > >> > > > copied
> > > > >> > > > > > >> during
> > > > >> > > > > > >> > > that
> > > > >> > > > > > >> > > > > > >>> process?
> > > > >> > > > > > >> > > > > > >>>
> > > > >> > > > > > >> > > > > > >>> 15. The KIP says "Remove deleted topics
> > > from
> > > > >> > > replicas
> > > > >> > > > by
> > > > >> > > > > > >> > sending
> > > > >> > > > > > >> > > > > > >>> StopReplicaRequest V2 for any topics
> > which
> > > do
> > > > >> not
> > > > >> > > > > contain
> > > > >> > > > > > a
> > > > >> > > > > > >> > topic
> > > > >> > > > > > >> > > > ID,
> > > > >> > > > > > >> > > > > > and
> > > > >> > > > > > >> > > > > > >>> V3 for any topics which do contain a
> > topic
> > > > >> ID.".
> > > > >> > > > > However,
> > > > >> > > > > > it
> > > > >> > > > > > >> > > seems
> > > > >> > > > > > >> > > > > the
> > > > >> > > > > > >> > > > > > >>> updated controller will create all
> > missing
> > > > >> topic
> > > > >> > IDs
> > > > >> > > > > first
> > > > >> > > > > > >> > before
> > > > >> > > > > > >> > > > > doing
> > > > >> > > > > > >> > > > > > >>> other actions. So, is
> StopReplicaRequest
> > V2
> > > > >> > needed?
> > > > >> > > > > > >> > > > > > >>>
> > > > >> > > > > > >> > > > > > >>> Jun
> > > > >> > > > > > >> > > > > > >>>
> > > > >> > > > > > >> > > > > > >>> On Fri, Sep 11, 2020 at 10:31 AM John
> > > > Roesler <
> > > > >> > > > > > >> > > vvcephei@apache.org
> > > > >> > > > > > >> > > > >
> > > > >> > > > > > >> > > > > > >>> wrote:
> > > > >> > > > > > >> > > > > > >>>
> > > > >> > > > > > >> > > > > > >>> > Thanks, Justine!
> > > > >> > > > > > >> > > > > > >>> >
> > > > >> > > > > > >> > > > > > >>> > Your response seems compelling to me.
> > > > >> > > > > > >> > > > > > >>> >
> > > > >> > > > > > >> > > > > > >>> > -John
> > > > >> > > > > > >> > > > > > >>> >
> > > > >> > > > > > >> > > > > > >>> > On Fri, 2020-09-11 at 10:17 -0700,
> > > Justine
> > > > >> > Olshan
> > > > >> > > > > wrote:
> > > > >> > > > > > >> > > > > > >>> > > Hello all,
> > > > >> > > > > > >> > > > > > >>> > > Thanks for continuing the
> > discussion! I
> > > > >> have a
> > > > >> > > few
> > > > >> > > > > > >> > responses
> > > > >> > > > > > >> > > to
> > > > >> > > > > > >> > > > > > your
> > > > >> > > > > > >> > > > > > >>> > points.
> > > > >> > > > > > >> > > > > > >>> > >
> > > > >> > > > > > >> > > > > > >>> > > Tom: You are correct in that this
> KIP
> > > has
> > > > >> not
> > > > >> > > > > > mentioned
> > > > >> > > > > > >> the
> > > > >> > > > > > >> > > > > > >>> > > DeleteTopicsRequest. I think that
> > this
> > > > >> would
> > > > >> > be
> > > > >> > > > out
> > > > >> > > > > of
> > > > >> > > > > > >> > scope
> > > > >> > > > > > >> > > > for
> > > > >> > > > > > >> > > > > > >>> now, but
> > > > >> > > > > > >> > > > > > >>> > > may be something worth adding in
> the
> > > > >> future.
> > > > >> > > > > > >> > > > > > >>> > >
> > > > >> > > > > > >> > > > > > >>> > > John: We did consider sequence ids,
> > but
> > > > >> there
> > > > >> > > are
> > > > >> > > > a
> > > > >> > > > > > few
> > > > >> > > > > > >> > > reasons
> > > > >> > > > > > >> > > > > to
> > > > >> > > > > > >> > > > > > >>> favor
> > > > >> > > > > > >> > > > > > >>> > > UUIDs. There are several cases
> where
> > > > topics
> > > > >> > from
> > > > >> > > > > > >> different
> > > > >> > > > > > >> > > > > clusters
> > > > >> > > > > > >> > > > > > >>> may
> > > > >> > > > > > >> > > > > > >>> > > interact now and in the future. For
> > > > >> example,
> > > > >> > > > Mirror
> > > > >> > > > > > >> Maker 2
> > > > >> > > > > > >> > > may
> > > > >> > > > > > >> > > > > > >>> benefit
> > > > >> > > > > > >> > > > > > >>> > > from being able to detect when a
> > > cluster
> > > > >> being
> > > > >> > > > > > mirrored
> > > > >> > > > > > >> is
> > > > >> > > > > > >> > > > > deleted
> > > > >> > > > > > >> > > > > > >>> and
> > > > >> > > > > > >> > > > > > >>> > > recreated and globally unique
> > > identifiers
> > > > >> > would
> > > > >> > > > make
> > > > >> > > > > > >> > > resolving
> > > > >> > > > > > >> > > > > > issues
> > > > >> > > > > > >> > > > > > >>> > > easier than sequence IDs which may
> > > > collide
> > > > >> > > between
> > > > >> > > > > > >> > clusters.
> > > > >> > > > > > >> > > > > > KIP-405
> > > > >> > > > > > >> > > > > > >>> > > (tiered storage) will also benefit
> > from
> > > > >> > globally
> > > > >> > > > > > unique
> > > > >> > > > > > >> IDs
> > > > >> > > > > > >> > > as
> > > > >> > > > > > >> > > > > > shared
> > > > >> > > > > > >> > > > > > >>> > > buckets may be used between
> clusters.
> > > > >> > > > > > >> > > > > > >>> > >
> > > > >> > > > > > >> > > > > > >>> > > Globally unique IDs would also make
> > > > >> > > functionality
> > > > >> > > > > like
> > > > >> > > > > > >> > moving
> > > > >> > > > > > >> > > > > > topics
> > > > >> > > > > > >> > > > > > >>> > > between disparate clusters easier
> in
> > > the
> > > > >> > future,
> > > > >> > > > > > >> simplify
> > > > >> > > > > > >> > any
> > > > >> > > > > > >> > > > > > future
> > > > >> > > > > > >> > > > > > >>> > > implementations of backups and
> > > restores,
> > > > >> and
> > > > >> > > more.
> > > > >> > > > > In
> > > > >> > > > > > >> > > general,
> > > > >> > > > > > >> > > > > > >>> unique IDs
> > > > >> > > > > > >> > > > > > >>> > > would ensure that the source
> cluster
> > > > >> topics do
> > > > >> > > not
> > > > >> > > > > > >> conflict
> > > > >> > > > > > >> > > > with
> > > > >> > > > > > >> > > > > > the
> > > > >> > > > > > >> > > > > > >>> > > destination cluster topics.
> > > > >> > > > > > >> > > > > > >>> > >
> > > > >> > > > > > >> > > > > > >>> > > If we were to use sequence ids, we
> > > would
> > > > >> need
> > > > >> > > > > > >> sufficiently
> > > > >> > > > > > >> > > > large
> > > > >> > > > > > >> > > > > > >>> cluster
> > > > >> > > > > > >> > > > > > >>> > > ids to be stored with the topic
> > > > >> identifiers or
> > > > >> > > we
> > > > >> > > > > run
> > > > >> > > > > > >> the
> > > > >> > > > > > >> > > risk
> > > > >> > > > > > >> > > > of
> > > > >> > > > > > >> > > > > > >>> > > collisions. This will give up any
> > > > >> advantage in
> > > > >> > > > > > >> compactness
> > > > >> > > > > > >> > > that
> > > > >> > > > > > >> > > > > > >>> sequence
> > > > >> > > > > > >> > > > > > >>> > > numbers may bring. Given these
> > > > advantages I
> > > > >> > > think
> > > > >> > > > it
> > > > >> > > > > > >> makes
> > > > >> > > > > > >> > > > sense
> > > > >> > > > > > >> > > > > to
> > > > >> > > > > > >> > > > > > >>> use
> > > > >> > > > > > >> > > > > > >>> > > UUIDs.
> > > > >> > > > > > >> > > > > > >>> > >
> > > > >> > > > > > >> > > > > > >>> > > Gokul: This is an interesting idea,
> > but
> > > > >> this
> > > > >> > is
> > > > >> > > a
> > > > >> > > > > > >> breaking
> > > > >> > > > > > >> > > > > change.
> > > > >> > > > > > >> > > > > > >>> Out of
> > > > >> > > > > > >> > > > > > >>> > > scope for now, but maybe worth
> > > discussing
> > > > >> in
> > > > >> > the
> > > > >> > > > > > future.
> > > > >> > > > > > >> > > > > > >>> > >
> > > > >> > > > > > >> > > > > > >>> > > Hope this explains some of the
> > > decisions,
> > > > >> > > > > > >> > > > > > >>> > >
> > > > >> > > > > > >> > > > > > >>> > > Justine
> > > > >> > > > > > >> > > > > > >>> > >
> > > > >> > > > > > >> > > > > > >>> > >
> > > > >> > > > > > >> > > > > > >>> > >
> > > > >> > > > > > >> > > > > > >>> > > On Fri, Sep 11, 2020 at 8:27 AM
> Gokul
> > > > >> Ramanan
> > > > >> > > > > > >> Subramanian <
> > > > >> > > > > > >> > > > > > >>> > > gokul2411s@gmail.com> wrote:
> > > > >> > > > > > >> > > > > > >>> > >
> > > > >> > > > > > >> > > > > > >>> > > > Hi.
> > > > >> > > > > > >> > > > > > >>> > > >
> > > > >> > > > > > >> > > > > > >>> > > > Thanks for the KIP.
> > > > >> > > > > > >> > > > > > >>> > > >
> > > > >> > > > > > >> > > > > > >>> > > > Have you thought about whether it
> > > makes
> > > > >> > sense
> > > > >> > > to
> > > > >> > > > > > >> support
> > > > >> > > > > > >> > > > > > >>> authorizing a
> > > > >> > > > > > >> > > > > > >>> > > > principal for a topic ID rather
> > than
> > > a
> > > > >> topic
> > > > >> > > > name
> > > > >> > > > > to
> > > > >> > > > > > >> > > achieve
> > > > >> > > > > > >> > > > > > >>> tighter
> > > > >> > > > > > >> > > > > > >>> > > > security?
> > > > >> > > > > > >> > > > > > >>> > > >
> > > > >> > > > > > >> > > > > > >>> > > > Or is the topic ID fundamentally
> an
> > > > >> internal
> > > > >> > > > > detail
> > > > >> > > > > > >> > similar
> > > > >> > > > > > >> > > > to
> > > > >> > > > > > >> > > > > > >>> epochs
> > > > >> > > > > > >> > > > > > >>> > used
> > > > >> > > > > > >> > > > > > >>> > > > in a bunch of other places in
> > Kafka?
> > > > >> > > > > > >> > > > > > >>> > > >
> > > > >> > > > > > >> > > > > > >>> > > > Thanks.
> > > > >> > > > > > >> > > > > > >>> > > >
> > > > >> > > > > > >> > > > > > >>> > > > On Fri, Sep 11, 2020 at 4:06 PM
> > John
> > > > >> > Roesler <
> > > > >> > > > > > >> > > > > > vvcephei@apache.org>
> > > > >> > > > > > >> > > > > > >>> > wrote:
> > > > >> > > > > > >> > > > > > >>> > > >
> > > > >> > > > > > >> > > > > > >>> > > > > Hello Justine,
> > > > >> > > > > > >> > > > > > >>> > > > >
> > > > >> > > > > > >> > > > > > >>> > > > > Thanks for the KIP!
> > > > >> > > > > > >> > > > > > >>> > > > >
> > > > >> > > > > > >> > > > > > >>> > > > > I happen to have been
> confronted
> > > > >> recently
> > > > >> > > with
> > > > >> > > > > the
> > > > >> > > > > > >> need
> > > > >> > > > > > >> > > to
> > > > >> > > > > > >> > > > > keep
> > > > >> > > > > > >> > > > > > >>> > track of
> > > > >> > > > > > >> > > > > > >>> > > > a
> > > > >> > > > > > >> > > > > > >>> > > > > large number of topics as
> > compactly
> > > > as
> > > > >> > > > > possible. I
> > > > >> > > > > > >> was
> > > > >> > > > > > >> > > > going
> > > > >> > > > > > >> > > > > to
> > > > >> > > > > > >> > > > > > >>> come
> > > > >> > > > > > >> > > > > > >>> > up
> > > > >> > > > > > >> > > > > > >>> > > > > with some way to dictionary
> > encode
> > > > the
> > > > >> > topic
> > > > >> > > > > names
> > > > >> > > > > > >> as
> > > > >> > > > > > >> > > > > integers,
> > > > >> > > > > > >> > > > > > >>> but
> > > > >> > > > > > >> > > > > > >>> > this
> > > > >> > > > > > >> > > > > > >>> > > > > seems much better!
> > > > >> > > > > > >> > > > > > >>> > > > >
> > > > >> > > > > > >> > > > > > >>> > > > > Apologies if this has been
> raised
> > > > >> before,
> > > > >> > > but
> > > > >> > > > > I’m
> > > > >> > > > > > >> > > wondering
> > > > >> > > > > > >> > > > > > >>> about the
> > > > >> > > > > > >> > > > > > >>> > > > > choice of UUID vs sequence
> number
> > > for
> > > > >> the
> > > > >> > > ids.
> > > > >> > > > > > >> > Typically,
> > > > >> > > > > > >> > > > > I’ve
> > > > >> > > > > > >> > > > > > >>> seen
> > > > >> > > > > > >> > > > > > >>> > UUIDs
> > > > >> > > > > > >> > > > > > >>> > > > > in two situations:
> > > > >> > > > > > >> > > > > > >>> > > > > 1. When processes need to
> > generate
> > > > >> > > > non-colliding
> > > > >> > > > > > >> > > > identifiers
> > > > >> > > > > > >> > > > > > >>> without
> > > > >> > > > > > >> > > > > > >>> > > > > coordination.
> > > > >> > > > > > >> > > > > > >>> > > > > 2. When the identifier needs to
> > be
> > > > >> > > > “universally
> > > > >> > > > > > >> > unique”;
> > > > >> > > > > > >> > > > > I.e.,
> > > > >> > > > > > >> > > > > > >>> the
> > > > >> > > > > > >> > > > > > >>> > > > > identifier needs to distinguish
> > the
> > > > >> entity
> > > > >> > > > from
> > > > >> > > > > > all
> > > > >> > > > > > >> > other
> > > > >> > > > > > >> > > > > > >>> entities
> > > > >> > > > > > >> > > > > > >>> > that
> > > > >> > > > > > >> > > > > > >>> > > > > could ever exist. This is
> useful
> > in
> > > > >> cases
> > > > >> > > > where
> > > > >> > > > > > >> > entities
> > > > >> > > > > > >> > > > from
> > > > >> > > > > > >> > > > > > all
> > > > >> > > > > > >> > > > > > >>> > kinds
> > > > >> > > > > > >> > > > > > >>> > > > of
> > > > >> > > > > > >> > > > > > >>> > > > > systems get mixed together,
> such
> > as
> > > > >> when
> > > > >> > > > dumping
> > > > >> > > > > > >> logs
> > > > >> > > > > > >> > > from
> > > > >> > > > > > >> > > > > all
> > > > >> > > > > > >> > > > > > >>> > processes
> > > > >> > > > > > >> > > > > > >>> > > > in
> > > > >> > > > > > >> > > > > > >>> > > > > a company into a common system.
> > > > >> > > > > > >> > > > > > >>> > > > >
> > > > >> > > > > > >> > > > > > >>> > > > > Maybe I’m being short-sighted,
> > but
> > > it
> > > > >> > > doesn’t
> > > > >> > > > > seem
> > > > >> > > > > > >> like
> > > > >> > > > > > >> > > > > either
> > > > >> > > > > > >> > > > > > >>> really
> > > > >> > > > > > >> > > > > > >>> > > > > applies here. It seems like the
> > > > brokers
> > > > >> > > could
> > > > >> > > > > and
> > > > >> > > > > > >> would
> > > > >> > > > > > >> > > > > achieve
> > > > >> > > > > > >> > > > > > >>> > consensus
> > > > >> > > > > > >> > > > > > >>> > > > > when creating a topic anyway,
> > which
> > > > is
> > > > >> all
> > > > >> > > > > that’s
> > > > >> > > > > > >> > > required
> > > > >> > > > > > >> > > > to
> > > > >> > > > > > >> > > > > > >>> > generate
> > > > >> > > > > > >> > > > > > >>> > > > > non-colliding sequence ids. For
> > the
> > > > >> > second,
> > > > >> > > as
> > > > >> > > > > you
> > > > >> > > > > > >> > > mention,
> > > > >> > > > > > >> > > > > we
> > > > >> > > > > > >> > > > > > >>> could
> > > > >> > > > > > >> > > > > > >>> > > > assign
> > > > >> > > > > > >> > > > > > >>> > > > > a UUID to the cluster as a
> whole,
> > > > which
> > > > >> > > would
> > > > >> > > > > > render
> > > > >> > > > > > >> > any
> > > > >> > > > > > >> > > > > > resource
> > > > >> > > > > > >> > > > > > >>> > scoped
> > > > >> > > > > > >> > > > > > >>> > > > to
> > > > >> > > > > > >> > > > > > >>> > > > > the broker universally unique
> as
> > > > well.
> > > > >> > > > > > >> > > > > > >>> > > > >
> > > > >> > > > > > >> > > > > > >>> > > > > The reason I mention this is
> > that,
> > > > >> > although
> > > > >> > > a
> > > > >> > > > > UUID
> > > > >> > > > > > >> is
> > > > >> > > > > > >> > way
> > > > >> > > > > > >> > > > > more
> > > > >> > > > > > >> > > > > > >>> > compact
> > > > >> > > > > > >> > > > > > >>> > > > > than topic names, it’s still 16
> > > > bytes.
> > > > >> In
> > > > >> > > > > > contrast,
> > > > >> > > > > > >> a
> > > > >> > > > > > >> > > > 4-byte
> > > > >> > > > > > >> > > > > > >>> integer
> > > > >> > > > > > >> > > > > > >>> > > > > sequence id would give us 4
> > billion
> > > > >> unique
> > > > >> > > > > topics
> > > > >> > > > > > >> per
> > > > >> > > > > > >> > > > > cluster,
> > > > >> > > > > > >> > > > > > >>> which
> > > > >> > > > > > >> > > > > > >>> > > > seems
> > > > >> > > > > > >> > > > > > >>> > > > > like enough ;)
> > > > >> > > > > > >> > > > > > >>> > > > >
> > > > >> > > > > > >> > > > > > >>> > > > > Considering the number of
> > different
> > > > >> times
> > > > >> > > > these
> > > > >> > > > > > >> topic
> > > > >> > > > > > >> > > > > > >>> identifiers are
> > > > >> > > > > > >> > > > > > >>> > > > sent
> > > > >> > > > > > >> > > > > > >>> > > > > over the wire or stored in
> > memory,
> > > it
> > > > >> > seems
> > > > >> > > > like
> > > > >> > > > > > it
> > > > >> > > > > > >> > might
> > > > >> > > > > > >> > > > be
> > > > >> > > > > > >> > > > > > >>> worth
> > > > >> > > > > > >> > > > > > >>> > the
> > > > >> > > > > > >> > > > > > >>> > > > > additional 4x space savings.
> > > > >> > > > > > >> > > > > > >>> > > > >
> > > > >> > > > > > >> > > > > > >>> > > > > What do you think about this?
> > > > >> > > > > > >> > > > > > >>> > > > >
> > > > >> > > > > > >> > > > > > >>> > > > > Thanks,
> > > > >> > > > > > >> > > > > > >>> > > > > John
> > > > >> > > > > > >> > > > > > >>> > > > >
> > > > >> > > > > > >> > > > > > >>> > > > > On Fri, Sep 11, 2020, at 03:20,
> > Tom
> > > > >> > Bentley
> > > > >> > > > > wrote:
> > > > >> > > > > > >> > > > > > >>> > > > > > Hi Justine,
> > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > >> > > > > > >> > > > > > >>> > > > > > This looks like a very
> welcome
> > > > >> > > improvement.
> > > > >> > > > > > >> Thanks!
> > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > >> > > > > > >> > > > > > >>> > > > > > Maybe I missed it, but the
> KIP
> > > > >> doesn't
> > > > >> > > seem
> > > > >> > > > to
> > > > >> > > > > > >> > mention
> > > > >> > > > > > >> > > > > > changing
> > > > >> > > > > > >> > > > > > >>> > > > > > DeleteTopicsRequest to
> identify
> > > the
> > > > >> > topic
> > > > >> > > > > using
> > > > >> > > > > > an
> > > > >> > > > > > >> > id.
> > > > >> > > > > > >> > > > > Maybe
> > > > >> > > > > > >> > > > > > >>> > that's out
> > > > >> > > > > > >> > > > > > >>> > > > > of
> > > > >> > > > > > >> > > > > > >>> > > > > > scope, but
> DeleteTopicsRequest
> > is
> > > > not
> > > > >> > > listed
> > > > >> > > > > > among
> > > > >> > > > > > >> > the
> > > > >> > > > > > >> > > > > Future
> > > > >> > > > > > >> > > > > > >>> Work
> > > > >> > > > > > >> > > > > > >>> > APIs
> > > > >> > > > > > >> > > > > > >>> > > > > > either.
> > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > >> > > > > > >> > > > > > >>> > > > > > Kind regards,
> > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > >> > > > > > >> > > > > > >>> > > > > > Tom
> > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > >> > > > > > >> > > > > > >>> > > > > > On Thu, Sep 10, 2020 at 3:59
> PM
> > > > >> Satish
> > > > >> > > > > Duggana <
> > > > >> > > > > > >> > > > > > >>> > > > satish.duggana@gmail.com
> > > > >> > > > > > >> > > > > > >>> > > > > > wrote:
> > > > >> > > > > > >> > > > > > >>> > > > > >
> > > > >> > > > > > >> > > > > > >>> > > > > > > Thanks Lucas/Justine for
> the
> > > nice
> > > > >> KIP.
> > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > >> > > > > > >> > > > > > >>> > > > > > > It has several benefits
> which
> > > > also
> > > > >> > > include
> > > > >> > > > > > >> > > simplifying
> > > > >> > > > > > >> > > > > the
> > > > >> > > > > > >> > > > > > >>> topic
> > > > >> > > > > > >> > > > > > >>> > > > > > > deletion process by
> > controller
> > > > and
> > > > >> > logs
> > > > >> > > > > > cleanup
> > > > >> > > > > > >> by
> > > > >> > > > > > >> > > > > brokers
> > > > >> > > > > > >> > > > > > in
> > > > >> > > > > > >> > > > > > >>> > corner
> > > > >> > > > > > >> > > > > > >>> > > > > > > cases.
> > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > >> > > > > > >> > > > > > >>> > > > > > > Best,
> > > > >> > > > > > >> > > > > > >>> > > > > > > Satish.
> > > > >> > > > > > >> > > > > > >>> > > > > > >
> > > > >> > > > > > >> > > > > > >>> > > > > > > On Wed, Sep 9, 2020 at
> 10:07
> > PM
> > > > >> > Justine
> > > > >> > > > > > Olshan <
> > > > >> > > > > > >> > > > > > >>> > jolshan@confluent.io
> > > > >> > > > > > >> > > > > > >>> > > > > > > wrote:
> > > > >> > > > > > >> > > > > > >>> > > > > > > > Hello all, it's been
> > almost a
> > > > >> year!
> > > > >> > > I've
> > > > >> > > > > > made
> > > > >> > > > > > >> > some
> > > > >> > > > > > >> > > > > > changes
> > > > >> > > > > > >> > > > > > >>> to
> > > > >> > > > > > >> > > > > > >>> > this
> > > > >> > > > > > >> > > > > > >>> > > > > KIP
> > > > >> > > > > > >> > > > > > >>> > > > > > > and hope to continue the
> > > > >> discussion.
> > > > >> > > > > > >> > > > > > >>> > > > > > > > One of the main changes
> > I've
> > > > >> added
> > > > >> > is
> > > > >> > > > now
> > > > >> > > > > > the
> > > > >> > > > > > >> > > > metadata
> > > > >> > > > > > >> > > > > > >>> response
> > > > >> > > > > > >> > > > > > >>> > > > will
> > > > >> > > > > > >> > > > > > >>> > > > > > > include the topic ID (as
> > Colin
> > > > >> > > suggested).
> > > > >> > > > > > >> Clients
> > > > >> > > > > > >> > > can
> > > > >> > > > > > >> > > > > > >>> obtain the
> > > > >> > > > > > >> > > > > > >>> > > > > topicID
> > > > >> > > > > > >> > > > > > >>> > > > > > > of a given topic through a
> > > > >> > > > TopicDescription.
> > > > >> > > > > > The
> > > > >> > > > > > >> > > > topicId
> > > > >> > > > > > >> > > > > > will
> > > > >> > > > > > >> > > > > > >>> > also be
> > > > >> > > > > > >> > > > > > >>> > > > > > > included with the
> > > UpdateMetadata
> > > > >> > > request.
> > > > >> > > > > > >> > > > > > >>> > > > > > > > Let me know what you all
> > > think.
> > > > >> > > > > > >> > > > > > >>> > > > > > > > Thank you,
> > > > >> > > > > > >> > > > > > >>> > > > > > > > Justine
> > > > >> > > > > > >> > > > > > >>> > > > > > > >
> > > > >> > > > > > >> > > > > > >>> > > > > > > > On 2019/09/13 16:38:26,
> > > "Colin
> > > > >> > > McCabe" <
> > > > >> > > > > > >> > > > > > cmccabe@apache.org
> > > > >> > > > > > >> > > > > > >>> >
> > > > >> > > > > > >> > > > > > >>> > wrote:
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > Hi Lucas,
> > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > Thanks for tackling
> this.
> > > > >> Topic
> > > > >> > IDs
> > > > >> > > > > are a
> > > > >> > > > > > >> > great
> > > > >> > > > > > >> > > > > idea,
> > > > >> > > > > > >> > > > > > >>> and
> > > > >> > > > > > >> > > > > > >>> > this
> > > > >> > > > > > >> > > > > > >>> > > > is
> > > > >> > > > > > >> > > > > > >>> > > > > a
> > > > >> > > > > > >> > > > > > >>> > > > > > > really good writeup.
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > For
> > > /brokers/topics/[topic],
> > > > >> the
> > > > >> > > > schema
> > > > >> > > > > > >> version
> > > > >> > > > > > >> > > > > should
> > > > >> > > > > > >> > > > > > be
> > > > >> > > > > > >> > > > > > >>> > bumped
> > > > >> > > > > > >> > > > > > >>> > > > to
> > > > >> > > > > > >> > > > > > >>> > > > > > > version 3, rather than 2.
> > > > KIP-455
> > > > >> > > bumped
> > > > >> > > > > the
> > > > >> > > > > > >> > version
> > > > >> > > > > > >> > > > of
> > > > >> > > > > > >> > > > > > this
> > > > >> > > > > > >> > > > > > >>> > znode
> > > > >> > > > > > >> > > > > > >>> > > > to
> > > > >> > > > > > >> > > > > > >>> > > > > 2
> > > > >> > > > > > >> > > > > > >>> > > > > > > already :)
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > Given that we're going
> to
> > > be
> > > > >> > seeing
> > > > >> > > > > these
> > > > >> > > > > > >> > things
> > > > >> > > > > > >> > > as
> > > > >> > > > > > >> > > > > > >>> strings
> > > > >> > > > > > >> > > > > > >>> > as
> > > > >> > > > > > >> > > > > > >>> > > > lot
> > > > >> > > > > > >> > > > > > >>> > > > > (in
> > > > >> > > > > > >> > > > > > >>> > > > > > > logs, in ZooKeeper, on the
> > > > >> > command-line,
> > > > >> > > > > > etc.),
> > > > >> > > > > > >> > does
> > > > >> > > > > > >> > > it
> > > > >> > > > > > >> > > > > > make
> > > > >> > > > > > >> > > > > > >>> > sense to
> > > > >> > > > > > >> > > > > > >>> > > > > use
> > > > >> > > > > > >> > > > > > >>> > > > > > > base64 when converting them
> > to
> > > > >> > strings?
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > Here is an example of
> the
> > > hex
> > > > >> > > > > > >> representation:
> > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > > >> > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > And here is an example
> in
> > > > >> base64.
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > b8tRS7h4TJ2Vt43Dp85v2A
> > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > The base64 version
> saves
> > 15
> > > > >> > letters
> > > > >> > > > (to
> > > > >> > > > > be
> > > > >> > > > > > >> > fair,
> > > > >> > > > > > >> > > 4
> > > > >> > > > > > >> > > > of
> > > > >> > > > > > >> > > > > > >>> those
> > > > >> > > > > > >> > > > > > >>> > were
> > > > >> > > > > > >> > > > > > >>> > > > > > > dashes that we could have
> > > elided
> > > > in
> > > > >> > the
> > > > >> > > > hex
> > > > >> > > > > > >> > > > > > representation.)
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > Another thing to
> consider
> > > is
> > > > >> that
> > > > >> > we
> > > > >> > > > > > should
> > > > >> > > > > > >> > > specify
> > > > >> > > > > > >> > > > > > that
> > > > >> > > > > > >> > > > > > >>> the
> > > > >> > > > > > >> > > > > > >>> > > > > > > all-zeroes UUID is not a
> > valid
> > > > >> topic
> > > > >> > > UUID.
> > > > >> > > > > >  We
> > > > >> > > > > > >> > can't
> > > > >> > > > > > >> > > > use
> > > > >> > > > > > >> > > > > > >>> null
> > > > >> > > > > > >> > > > > > >>> > for
> > > > >> > > > > > >> > > > > > >>> > > > this
> > > > >> > > > > > >> > > > > > >>> > > > > > > because we can't pass a
> null
> > > UUID
> > > > >> over
> > > > >> > > the
> > > > >> > > > > RPC
> > > > >> > > > > > >> > > protocol
> > > > >> > > > > > >> > > > > > >>> (there
> > > > >> > > > > > >> > > > > > >>> > is no
> > > > >> > > > > > >> > > > > > >>> > > > > > > special pattern for null,
> nor
> > > do
> > > > we
> > > > >> > want
> > > > >> > > > to
> > > > >> > > > > > >> waste
> > > > >> > > > > > >> > > space
> > > > >> > > > > > >> > > > > > >>> reserving
> > > > >> > > > > > >> > > > > > >>> > > > such
> > > > >> > > > > > >> > > > > > >>> > > > > a
> > > > >> > > > > > >> > > > > > >>> > > > > > > pattern.)
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > Maybe I missed it, but
> > did
> > > > you
> > > > >> > > > describe
> > > > >> > > > > > >> > > "migration
> > > > >> > > > > > >> > > > > > of...
> > > > >> > > > > > >> > > > > > >>> > existing
> > > > >> > > > > > >> > > > > > >>> > > > > > > topic[s] without topic IDs"
> > in
> > > > >> detail
> > > > >> > in
> > > > >> > > > any
> > > > >> > > > > > >> > section?
> > > > >> > > > > > >> > > > It
> > > > >> > > > > > >> > > > > > >>> seems
> > > > >> > > > > > >> > > > > > >>> > like
> > > > >> > > > > > >> > > > > > >>> > > > > when
> > > > >> > > > > > >> > > > > > >>> > > > > > > the new controller becomes
> > > > active,
> > > > >> it
> > > > >> > > > should
> > > > >> > > > > > >> just
> > > > >> > > > > > >> > > > > generate
> > > > >> > > > > > >> > > > > > >>> random
> > > > >> > > > > > >> > > > > > >>> > > > > UUIDs for
> > > > >> > > > > > >> > > > > > >>> > > > > > > these, and write the random
> > > UUIDs
> > > > >> back
> > > > >> > > to
> > > > >> > > > > > >> > ZooKeeper.
> > > > >> > > > > > >> > > > It
> > > > >> > > > > > >> > > > > > >>> would be
> > > > >> > > > > > >> > > > > > >>> > > > good
> > > > >> > > > > > >> > > > > > >>> > > > > to
> > > > >> > > > > > >> > > > > > >>> > > > > > > spell that out.  We should
> > make
> > > > it
> > > > >> > clear
> > > > >> > > > > that
> > > > >> > > > > > >> this
> > > > >> > > > > > >> > > > > happens
> > > > >> > > > > > >> > > > > > >>> > regardless
> > > > >> > > > > > >> > > > > > >>> > > > > of
> > > > >> > > > > > >> > > > > > >>> > > > > > > the inter-broker protocol
> > > version
> > > > >> > (it's
> > > > >> > > a
> > > > >> > > > > > >> > compatible
> > > > >> > > > > > >> > > > > > change).
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > "LeaderAndIsrRequests
> > > > >> including an
> > > > >> > > > > > >> > > > is_every_partition
> > > > >> > > > > > >> > > > > > >>> flag"
> > > > >> > > > > > >> > > > > > >>> > > > seems a
> > > > >> > > > > > >> > > > > > >>> > > > > > > bit wordy.  Can we just
> call
> > > > these
> > > > >> > "full
> > > > >> > > > > > >> > > > > > >>> LeaderAndIsrRequests"?
> > > > >> > > > > > >> > > > > > >>> > Then
> > > > >> > > > > > >> > > > > > >>> > > > > the
> > > > >> > > > > > >> > > > > > >>> > > > > > > RPC field could be named
> > > "full".
> > > > >> > Also,
> > > > >> > > it
> > > > >> > > > > > would
> > > > >> > > > > > >> > > > probably
> > > > >> > > > > > >> > > > > > be
> > > > >> > > > > > >> > > > > > >>> > better
> > > > >> > > > > > >> > > > > > >>> > > > > for the
> > > > >> > > > > > >> > > > > > >>> > > > > > > RPC field to be an enum of
> {
> > > > >> > > UNSPECIFIED,
> > > > >> > > > > > >> > > INCREMENTAL,
> > > > >> > > > > > >> > > > > FULL
> > > > >> > > > > > >> > > > > > >>> }, so
> > > > >> > > > > > >> > > > > > >>> > > > that
> > > > >> > > > > > >> > > > > > >>> > > > > we
> > > > >> > > > > > >> > > > > > >>> > > > > > > can cleanly handle old
> > versions
> > > > (by
> > > > >> > > > treating
> > > > >> > > > > > >> them
> > > > >> > > > > > >> > as
> > > > >> > > > > > >> > > > > > >>> UNSPECIFIED)
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > In the
> > LeaderAndIsrRequest
> > > > >> > section,
> > > > >> > > > you
> > > > >> > > > > > >> write
> > > > >> > > > > > >> > "A
> > > > >> > > > > > >> > > > > final
> > > > >> > > > > > >> > > > > > >>> > deletion
> > > > >> > > > > > >> > > > > > >>> > > > > event
> > > > >> > > > > > >> > > > > > >>> > > > > > > will be secheduled for X ms
> > > after
> > > > >> the
> > > > >> > > > > > >> > > > LeaderAndIsrRequest
> > > > >> > > > > > >> > > > > > was
> > > > >> > > > > > >> > > > > > >>> > first
> > > > >> > > > > > >> > > > > > >>> > > > > > > received..."  I guess the X
> > > was a
> > > > >> > > > > placeholder
> > > > >> > > > > > >> that
> > > > >> > > > > > >> > > you
> > > > >> > > > > > >> > > > > > >>> intended
> > > > >> > > > > > >> > > > > > >>> > to
> > > > >> > > > > > >> > > > > > >>> > > > > replace
> > > > >> > > > > > >> > > > > > >>> > > > > > > before posting? :)  In any
> > > case,
> > > > >> this
> > > > >> > > > seems
> > > > >> > > > > > like
> > > > >> > > > > > >> > the
> > > > >> > > > > > >> > > > kind
> > > > >> > > > > > >> > > > > > of
> > > > >> > > > > > >> > > > > > >>> > thing
> > > > >> > > > > > >> > > > > > >>> > > > we'd
> > > > >> > > > > > >> > > > > > >>> > > > > > > want a configuration for.
> > > Let's
> > > > >> > > describe
> > > > >> > > > > that
> > > > >> > > > > > >> > > > > > configuration
> > > > >> > > > > > >> > > > > > >>> key
> > > > >> > > > > > >> > > > > > >>> > > > > somewhere
> > > > >> > > > > > >> > > > > > >>> > > > > > > in this KIP, including what
> > its
> > > > >> > default
> > > > >> > > > > value
> > > > >> > > > > > >> is.
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > We should probably also
> > > log a
> > > > >> > bunch
> > > > >> > > of
> > > > >> > > > > > >> messages
> > > > >> > > > > > >> > > at
> > > > >> > > > > > >> > > > > WARN
> > > > >> > > > > > >> > > > > > >>> level
> > > > >> > > > > > >> > > > > > >>> > > > when
> > > > >> > > > > > >> > > > > > >>> > > > > > > something is scheduled for
> > > > >> deletion,
> > > > >> > as
> > > > >> > > > > well.
> > > > >> > > > > > >> > (Maybe
> > > > >> > > > > > >> > > > > this
> > > > >> > > > > > >> > > > > > >>> was
> > > > >> > > > > > >> > > > > > >>> > > > > assumed, but
> > > > >> > > > > > >> > > > > > >>> > > > > > > it would be good to mention
> > > it).
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > I feel like there are a
> > few
> > > > >> > sections
> > > > >> > > > > that
> > > > >> > > > > > >> > should
> > > > >> > > > > > >> > > be
> > > > >> > > > > > >> > > > > > >>> moved to
> > > > >> > > > > > >> > > > > > >>> > > > > "rejected
> > > > >> > > > > > >> > > > > > >>> > > > > > > alternatives."  For
> example,
> > in
> > > > the
> > > > >> > > > > > DeleteTopics
> > > > >> > > > > > >> > > > section,
> > > > >> > > > > > >> > > > > > >>> since
> > > > >> > > > > > >> > > > > > >>> > we're
> > > > >> > > > > > >> > > > > > >>> > > > > not
> > > > >> > > > > > >> > > > > > >>> > > > > > > going to do option 1 or 2,
> > > these
> > > > >> > should
> > > > >> > > be
> > > > >> > > > > > moved
> > > > >> > > > > > >> > into
> > > > >> > > > > > >> > > > > > >>> "rejected
> > > > >> > > > > > >> > > > > > >>> > > > > > > alternatives,"  rather than
> > > > >> appearing
> > > > >> > > > > inline.
> > > > >> > > > > > >> > > Another
> > > > >> > > > > > >> > > > > case
> > > > >> > > > > > >> > > > > > >>> is
> > > > >> > > > > > >> > > > > > >>> > the
> > > > >> > > > > > >> > > > > > >>> > > > > "Should
> > > > >> > > > > > >> > > > > > >>> > > > > > > we remove topic name from
> the
> > > > >> protocol
> > > > >> > > > where
> > > > >> > > > > > >> > > possible"
> > > > >> > > > > > >> > > > > > >>> section.
> > > > >> > > > > > >> > > > > > >>> > This
> > > > >> > > > > > >> > > > > > >>> > > > > is
> > > > >> > > > > > >> > > > > > >>> > > > > > > clearly discussing a design
> > > > >> > alternative
> > > > >> > > > that
> > > > >> > > > > > >> we're
> > > > >> > > > > > >> > > not
> > > > >> > > > > > >> > > > > > >>> proposing
> > > > >> > > > > > >> > > > > > >>> > to
> > > > >> > > > > > >> > > > > > >>> > > > > > > implement: removing the
> topic
> > > > name
> > > > >> > from
> > > > >> > > > > those
> > > > >> > > > > > >> > > > protocols.
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > Is it really necessary
> to
> > > > have
> > > > >> a
> > > > >> > new
> > > > >> > > > > > >> > > > > > >>> > /admin/delete_topics_by_id
> > > > >> > > > > > >> > > > > > >>> > > > > path
> > > > >> > > > > > >> > > > > > >>> > > > > > > in ZooKeeper?  It seems
> like
> > we
> > > > >> don't
> > > > >> > > > really
> > > > >> > > > > > >> need
> > > > >> > > > > > >> > > this.
> > > > >> > > > > > >> > > > > > >>> Whenever
> > > > >> > > > > > >> > > > > > >>> > > > > there is
> > > > >> > > > > > >> > > > > > >>> > > > > > > a new controller, we'll
> send
> > > out
> > > > >> full
> > > > >> > > > > > >> > > > > LeaderAndIsrRequests
> > > > >> > > > > > >> > > > > > >>> which
> > > > >> > > > > > >> > > > > > >>> > will
> > > > >> > > > > > >> > > > > > >>> > > > > > > trigger the stale topics to
> > be
> > > > >> cleaned
> > > > >> > > up.
> > > > >> > > > > >  The
> > > > >> > > > > > >> > > active
> > > > >> > > > > > >> > > > > > >>> > controller
> > > > >> > > > > > >> > > > > > >>> > > > will
> > > > >> > > > > > >> > > > > > >>> > > > > > > also send the full
> > > > >> LeaderAndIsrRequest
> > > > >> > > to
> > > > >> > > > > > >> brokers
> > > > >> > > > > > >> > > that
> > > > >> > > > > > >> > > > > are
> > > > >> > > > > > >> > > > > > >>> just
> > > > >> > > > > > >> > > > > > >>> > > > > starting
> > > > >> > > > > > >> > > > > > >>> > > > > > > up.    So we don't really
> > need
> > > > this
> > > > >> > kind
> > > > >> > > > of
> > > > >> > > > > > >> > two-phase
> > > > >> > > > > > >> > > > > > commit
> > > > >> > > > > > >> > > > > > >>> > (send
> > > > >> > > > > > >> > > > > > >>> > > > out
> > > > >> > > > > > >> > > > > > >>> > > > > > > StopReplicasRequest, get
> ACKs
> > > > from
> > > > >> all
> > > > >> > > > > nodes,
> > > > >> > > > > > >> > commit
> > > > >> > > > > > >> > > by
> > > > >> > > > > > >> > > > > > >>> removing
> > > > >> > > > > > >> > > > > > >>> > > > > > > /admin/delete_topics node)
> > any
> > > > >> more.
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > You mention that
> > > FetchRequest
> > > > >> will
> > > > >> > > now
> > > > >> > > > > > >> include
> > > > >> > > > > > >> > > UUID
> > > > >> > > > > > >> > > > > to
> > > > >> > > > > > >> > > > > > >>> avoid
> > > > >> > > > > > >> > > > > > >>> > > > issues
> > > > >> > > > > > >> > > > > > >>> > > > > > > where requests are made to
> > > stale
> > > > >> > > > partitions.
> > > > >> > > > > > >> > > However,
> > > > >> > > > > > >> > > > > > >>> adding a
> > > > >> > > > > > >> > > > > > >>> > UUID
> > > > >> > > > > > >> > > > > > >>> > > > to
> > > > >> > > > > > >> > > > > > >>> > > > > > > MetadataRequest is listed
> as
> > > > future
> > > > >> > > work,
> > > > >> > > > > out
> > > > >> > > > > > of
> > > > >> > > > > > >> > > scope
> > > > >> > > > > > >> > > > > for
> > > > >> > > > > > >> > > > > > >>> this
> > > > >> > > > > > >> > > > > > >>> > KIP.
> > > > >> > > > > > >> > > > > > >>> > > > > How
> > > > >> > > > > > >> > > > > > >>> > > > > > > will the client learn what
> > the
> > > > >> topic
> > > > >> > > UUID
> > > > >> > > > > is,
> > > > >> > > > > > if
> > > > >> > > > > > >> > the
> > > > >> > > > > > >> > > > > > metadata
> > > > >> > > > > > >> > > > > > >>> > > > response
> > > > >> > > > > > >> > > > > > >>> > > > > > > doesn't include that
> > > information?
> > > > >> It
> > > > >> > > > seems
> > > > >> > > > > > like
> > > > >> > > > > > >> > > adding
> > > > >> > > > > > >> > > > > the
> > > > >> > > > > > >> > > > > > >>> UUID
> > > > >> > > > > > >> > > > > > >>> > to
> > > > >> > > > > > >> > > > > > >>> > > > > > > MetadataResponse would be
> an
> > > > >> > improvement
> > > > >> > > > > here
> > > > >> > > > > > >> that
> > > > >> > > > > > >> > > > might
> > > > >> > > > > > >> > > > > > not
> > > > >> > > > > > >> > > > > > >>> be
> > > > >> > > > > > >> > > > > > >>> > too
> > > > >> > > > > > >> > > > > > >>> > > > > hard to
> > > > >> > > > > > >> > > > > > >>> > > > > > > make.
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > best,
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > Colin
> > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > On Mon, Sep 9, 2019, at
> > > > 17:48,
> > > > >> > > Ryanne
> > > > >> > > > > > Dolan
> > > > >> > > > > > >> > > wrote:
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > > Lucas, this would be
> > > great.
> > > > >> I've
> > > > >> > > run
> > > > >> > > > > > into
> > > > >> > > > > > >> > > issues
> > > > >> > > > > > >> > > > > with
> > > > >> > > > > > >> > > > > > >>> > topics
> > > > >> > > > > > >> > > > > > >>> > > > > being
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > > resurrected
> > accidentally,
> > > > >> since
> > > > >> > a
> > > > >> > > > > client
> > > > >> > > > > > >> > cannot
> > > > >> > > > > > >> > > > > > easily
> > > > >> > > > > > >> > > > > > >>> > > > > distinguish
> > > > >> > > > > > >> > > > > > >>> > > > > > > between
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > > a deleted topic and a
> > new
> > > > >> topic
> > > > >> > > with
> > > > >> > > > > the
> > > > >> > > > > > >> same
> > > > >> > > > > > >> > > > name.
> > > > >> > > > > > >> > > > > > I'd
> > > > >> > > > > > >> > > > > > >>> > need
> > > > >> > > > > > >> > > > > > >>> > > > the
> > > > >> > > > > > >> > > > > > >>> > > > > ID
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > > accessible from the
> > > client
> > > > to
> > > > >> > > solve
> > > > >> > > > > that
> > > > >> > > > > > >> > issue,
> > > > >> > > > > > >> > > > but
> > > > >> > > > > > >> > > > > > >>> this
> > > > >> > > > > > >> > > > > > >>> > is a
> > > > >> > > > > > >> > > > > > >>> > > > > good
> > > > >> > > > > > >> > > > > > >>> > > > > > > first
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > > step.
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > > Ryanne
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > > On Wed, Sep 4, 2019
> at
> > > 1:41
> > > > >> PM
> > > > >> > > Lucas
> > > > >> > > > > > >> > > Bradstreet <
> > > > >> > > > > > >> > > > > > >>> > > > > lucas@confluent.io>
> > > > >> > > > > > >> > > > > > >>> > > > > > > wrote:
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Hi all,
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > I would like to
> kick
> > > off
> > > > >> > > > discussion
> > > > >> > > > > of
> > > > >> > > > > > >> > > KIP-516,
> > > > >> > > > > > >> > > > > an
> > > > >> > > > > > >> > > > > > >>> > > > > implementation
> > > > >> > > > > > >> > > > > > >>> > > > > > > of topic
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > IDs for Kafka.
> Topic
> > > IDs
> > > > >> aim
> > > > >> > to
> > > > >> > > > > solve
> > > > >> > > > > > >> topic
> > > > >> > > > > > >> > > > > > >>> uniqueness
> > > > >> > > > > > >> > > > > > >>> > > > > problems in
> > > > >> > > > > > >> > > > > > >>> > > > > > > Kafka,
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > where referring to
> a
> > > > topic
> > > > >> by
> > > > >> > > name
> > > > >> > > > > > >> alone is
> > > > >> > > > > > >> > > > > > >>> insufficient.
> > > > >> > > > > > >> > > > > > >>> > > > Such
> > > > >> > > > > > >> > > > > > >>> > > > > > > cases
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > include when a
> topic
> > > has
> > > > >> been
> > > > >> > > > > deleted
> > > > >> > > > > > >> and
> > > > >> > > > > > >> > > > > recreated
> > > > >> > > > > > >> > > > > > >>> with
> > > > >> > > > > > >> > > > > > >>> > the
> > > > >> > > > > > >> > > > > > >>> > > > > same
> > > > >> > > > > > >> > > > > > >>> > > > > > > name.
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Unique identifiers
> > will
> > > > >> help
> > > > >> > > > > simplify
> > > > >> > > > > > >> and
> > > > >> > > > > > >> > > > improve
> > > > >> > > > > > >> > > > > > >>> Kafka's
> > > > >> > > > > > >> > > > > > >>> > > > topic
> > > > >> > > > > > >> > > > > > >>> > > > > > > deletion
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > process, as well as
> > > > prevent
> > > > >> > > cases
> > > > >> > > > > > where
> > > > >> > > > > > >> > > brokers
> > > > >> > > > > > >> > > > > may
> > > > >> > > > > > >> > > > > > >>> > > > incorrectly
> > > > >> > > > > > >> > > > > > >>> > > > > > > interact
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > with stale versions
> > of
> > > > >> topics.
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > >> > > > > > >> > > > > > >>> > > >
> > > > >> > > > > > >> > > > > > >>> >
> > > > >> > > > > > >> > > > > > >>>
> > > > >> > > > > > >> > > > > >
> > > > >> > > > > > >> > > > >
> > > > >> > > > > > >> > > >
> > > > >> > > > > > >> > >
> > > > >> > > > > > >> >
> > > > >> > > > > > >>
> > > > >> > > > > >
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Looking forward to
> > your
> > > > >> > > thoughts.
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Lucas
> > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > >> > > > > > >> > > > > > >>> >
> > > > >> > > > > > >> > > > > > >>> >
> > > > >> > > > > > >> > > > > > >>>
> > > > >> > > > > > >> > > > > > >>
> > > > >> > > > > > >> > > > > >
> > > > >> > > > > > >> > > > >
> > > > >> > > > > > >> > > >
> > > > >> > > > > > >> > >
> > > > >> > > > > > >> >
> > > > >> > > > > > >>
> > > > >> > > > > > >
> > > > >> > > > > >
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Justine Olshan <jo...@confluent.io>]

Hi Jason,

Thanks for your comments.

1. Yes, the directory will still be based on the topic names.
LeaderAndIsrRequest is one of the few requests that will still contain the
topic name. So I think we have this covered. Sorry for confusion.

2. Part of the usage of the file is to have persistent storage of the topic
ID and use it to compare with the ID supplied in the LeaderAndIsr Request.
There is some discussion in the KIP about changes to the directory
structure, but I believe directory changes were considered to be out of
scope when the KIP was written.

3. I think this is a good point, but I again I wonder about the scope of
the KIP. Most of the changes mentioned in the KIP are for supporting topic
deletion and I believe that is why the produce request was listed under
future work.

4. This sounds like it might be a good solution, but I will need to discuss
more with KIP-500 folks to get the details right.

Thanks,
Justine

On Thu, Sep 24, 2020 at 12:30 PM Jason Gustafson <ja...@confluent.io> wrote:

> Hi Justine,
>
> Thanks for picking up this work. I have a few questions/comments:
>
> 1. It sounds like the directory structure is still going to be based on
> topic names. Do I have that right? One complication is that the
> LeaderAndIsr request does not include the topic name any longer. This means
> that a replica must wait for the UpdateMetadata request to arrive with the
> topic name mapping before it can create the directory. I wonder if it would
> be simpler to avoid assumptions on the ordering of UpdateMetadata and let
> LeaderAndIsr include the topic name as well. Feels like we are not saving
> that much by excluding it.
>
> 2. On a related note, it seems that the reason we have the partition
> metadata file is because we are not changing the directory structure. We
> need it so that we remember which directories map to which topic id. I
> think it would be useful to add some detail about why changing the
> directory layout was rejected. Basically trying to understand how much
> effort we are saving by skipping this step if we have to implement this
> additional bookkeeping. One concern I have for example is that the
> partition metadata file gets corrupt and then all bets are off as far as
> topic consistency.
>
> 3. Is it worth adding support now for topic ids in the `Produce` request
> now? Overhead is mentioned as one of the motivations and the best APIs to
> get savings from are `Produce` and `Fetch`.
>
> 4. When it comes to bootstrapping the metadata topic with respect to
> KIP-500, one suggestion would be to reserve a sentinel topic ID which can
> be used initially by new brokers. When the first controller is elected, it
> can assign a topicId to the metadata topic. This is a bit similar to how we
> were planning to generate the clusterId.
>
> Thanks,
> Jason
>
>
> On Thu, Sep 24, 2020 at 11:10 AM Jun Rao <ju...@confluent.io> wrote:
>
> > Hi, Justine,
> >
> > Thanks for the updated KIP. A few more comments below.
> >
> > 30.  {"name": "id", "type": "string", "doc": "version id"}}: The doc
> should
> > say UUID.
> >
> > 31. LeaderAndIsrResponse v5 and StopReplicaResponse v4 : It seems there
> is
> > no need to add topic_id at partitions level.
> >
> > 32. Regarding partition metadata file. Perhaps the key can be a single
> > word, sth like the following.
> > version: 0
> > topic_id: 46bdb63f-9e8d-4a38-bf7b-ee4eb2a794e4
> >
> > 33. Another tricky thing that I realized is how to support the metadata
> > topic introduced in KIP-595. It's a bit tricky to assign a UUID to the
> > metadata topic since we have a chicken-and-egg problem. The controller
> > needs to persist the UUID in the metadata topic in order to assign one
> > successfully, but the metadata topic is needed to elect a controller
> > (KIP-631). So, this probably needs a bit more thought.
> >
> > Jun
> >
> > On Thu, Sep 24, 2020 at 3:04 AM Ismael Juma <is...@juma.me.uk> wrote:
> >
> > > Also, can we provide more details on how the Partition Metadata file
> will
> > > be used?
> > >
> > > Ismael
> > >
> > > On Thu, Sep 24, 2020 at 3:01 AM Ismael Juma <is...@juma.me.uk> wrote:
> > >
> > > > Hi Justine,
> > > >
> > > > I think we need to update the "Rejected Alternatives" section to take
> > > into
> > > > account that the proposal now removes the topic name from the fetch
> > > > request. Also, if we are removing it from the Fetch request, does it
> > make
> > > > sense not to remove it from similar requests like ListOffsetRequest?
> > > >
> > > > Ismael
> > > >
> > > > On Thu, Sep 24, 2020 at 2:46 AM David Jacot <dj...@confluent.io>
> > wrote:
> > > >
> > > >> Hi Justine,
> > > >>
> > > >> Thanks for the KIP. I finally had time to read it :). It is a great
> > > >> improvement.
> > > >>
> > > >> I have a few comments/questions:
> > > >>
> > > >> 1. It seems that the schema of the StopReplicaRequest is slightly
> > > >> outdated.
> > > >> We
> > > >> did some changes as part of KIP-570. V3 is already organized by
> > topics.
> > > >>
> > > >> 2. I just want to make sure that I understand the reconciliation
> > > >> logic correctly. When
> > > >> an "INCREMENTAL" LeaderAndIsr Request is received, the broker will
> > also
> > > >> reconcile
> > > >> when the local uuid does not match the uuid in the request, right?
> In
> > > this
> > > >> case, the
> > > >> local log is staged for deletion.
> > > >>
> > > >> 3. In the documentation of the `delete.stale.topic.delay.ms`
> config,
> > it
> > > >> says "When a
> > > >> FULL LeaderAndIsrRequest is received..." but I suppose it applies to
> > > both
> > > >> types.
> > > >>
> > > >> Best,
> > > >> David
> > > >>
> > > >> On Thu, Sep 24, 2020 at 1:40 AM Justine Olshan <
> jolshan@confluent.io>
> > > >> wrote:
> > > >>
> > > >> > Hi Jun,
> > > >> >
> > > >> > Thanks for the comments. I apologize for some of the typos and
> > > >> confusion.
> > > >> > I’ve updated the KIP to fix some of the issues you mentioned.
> > > >> >
> > > >> > 20.2 I’ve changed the type to String.
> > > >> > 20.1/3 I’ve updated the TopicZNode to fix formatting and reflect
> the
> > > >> latest
> > > >> > version before this change.
> > > >> >
> > > >> > 21. You are correct and I’ve removed this line. I’ve also added a
> > line
> > > >> > mentioning an IBP bump is necessary for migration
> > > >> >
> > > >> > 22. I think the wording was unclear but your summary is what was
> > > >> intended
> > > >> > by this line. I’ve updated to make this point more clear. “Remove
> > > >> deleted
> > > >> > topics from replicas by sending StopReplicaRequest V3 before the
> IBP
> > > >> bump
> > > >> > using the old logic, and using V4 and the new logic with topic IDs
> > > after
> > > >> > the IBP bump.”
> > > >> >
> > > >> > 23. I’ve removed the unspecified type from the KIP and mention
> that
> > > IBP
> > > >> > will be used to handle this request. “IBP will be used to
> determine
> > > >> whether
> > > >> > this new form of the request will be used. For older requests, we
> > will
> > > >> > ignore this field and default to previous behavior.”
> > > >> >
> > > >> > 24. I’ve fixed this typo.
> > > >> >
> > > >> > 25. I've added a topics at a higher level for LeaderAndIsrResponse
> > v5,
> > > >> > StopReplicaResponse v4. I've also changed StopReplicaRequest v4 to
> > > have
> > > >> > topics at a higher level.
> > > >> >
> > > >> > 26. I’ve updated forgotten_topics_data--added the topic ID and
> > removed
> > > >> the
> > > >> > topic name
> > > >> >
> > > >> > 27. I’ve decided on plain text, and I’ve added an example.
> > > >> >
> > > >> > 28. I’ve added this idea to future work.
> > > >> >
> > > >> > Thanks again for taking a look,
> > > >> >
> > > >> > Justine
> > > >> >
> > > >> > On Wed, Sep 23, 2020 at 10:28 AM Jun Rao <ju...@confluent.io>
> wrote:
> > > >> >
> > > >> > > Hi, Justine,
> > > >> > >
> > > >> > > Thanks for the response. Made another pass. A few more comments
> > > below.
> > > >> > >
> > > >> > > 20. znode schema:
> > > >> > > 20.1 It seems that {"name": "version", "type": "int", "id":
> > "UUID",
> > > >> > "doc":
> > > >> > > "version id"} should be {"name": "version", "type": "int"},
> > {"name":
> > > >> > "id",
> > > >> > > "type": "UUID", "doc": "version id"}.
> > > >> > > 20.2 The znode format is JSON which doesn't have UUID type. So
> the
> > > >> type
> > > >> > > probably should be string?
> > > >> > > 20.3 Also, the existing format used seems outdated. It should
> have
> > > the
> > > >> > > following format.
> > > >> > >     Json.encodeAsBytes(Map(
> > > >> > >       "version" -> 2,
> > > >> > >       "partitions" -> replicaAssignmentJson.asJava,
> > > >> > >       "adding_replicas" -> addingReplicasAssignmentJson.asJava,
> > > >> > >       "removing_replicas" ->
> removingReplicasAssignmentJson.asJava
> > > >> > >     ).asJava)
> > > >> > >   }
> > > >> > >
> > > >> > > 21. Migration: The KIP says "The migration process can take
> place
> > > >> without
> > > >> > > an inter-broker protocol bump, as the format stored in
> > > >> > > /brokers/topics/[topic] will be compatible with older broker
> > > >> versions."
> > > >> > > However, since we are bumping up the version of inter-broker
> > > >> requests, it
> > > >> > > seems that we need to use IBP for migration.
> > > >> > >
> > > >> > > 22. The KIP says "Remove deleted topics from replicas by sending
> > > >> > > StopReplicaRequest V3 for any topics which do not contain a
> topic
> > > ID,
> > > >> and
> > > >> > > V4 for any topics which do contain a topic ID.". However, if we
> > use
> > > >> IBP,
> > > >> > it
> > > >> > > seems that the controller will either send StopReplicaRequest V3
> > > >> > > or StopReplicaRequest V4, but never mixed V3 and V4 for
> different
> > > >> topics.
> > > >> > > Basically, before the IBP bump, V3 will be used. After the IBP
> > bump,
> > > >> > > topicId will be created and V4 will be used.
> > > >> > >
> > > >> > > 23. Given that we depend on IBP, do we still need "0
> UNSPECIFIED"
> > > >> > > in LeaderAndIsr?
> > > >> > >
> > > >> > > 24. LeaderAndIsrResponse v5 : It still has the topic field.
> > > >> > >
> > > >> > > 25. LeaderAndIsrResponse v5, StopReplicaResponse v4: Could we
> use
> > > this
> > > >> > > opportunity to organize the response in 2 levels, first by
> topic,
> > > >> then by
> > > >> > > partition, as most other requests/responses?
> > > >> > >
> > > >> > > 26. FetchRequest v13 : Should forgotten_topics_data use topicId
> > too?
> > > >> > >
> > > >> > > 27. "This file can either be plain text (key/value pairs) or
> > JSON."
> > > >> Have
> > > >> > we
> > > >> > > decided which one to use? Also, it would be helpful to provide
> an
> > > >> > example.
> > > >> > >
> > > >> > > 28. Future improvement: Another future improvement opportunity
> is
> > to
> > > >> use
> > > >> > > topicId in GroupMetadataManager.offsetCommitKey in the
> > offset_commit
> > > >> > topic.
> > > >> > > This may save some space.
> > > >> > >
> > > >> > > Thanks,
> > > >> > >
> > > >> > > Jun
> > > >> > >
> > > >> > > On Wed, Sep 23, 2020 at 8:50 AM Justine Olshan <
> > > jolshan@confluent.io>
> > > >> > > wrote:
> > > >> > >
> > > >> > > > Hi Tom,
> > > >> > > >
> > > >> > > > Thanks for the comment. I think this is a really good idea and
> > it
> > > >> has
> > > >> > > been
> > > >> > > > added to the KIP under the newly added tooling section.
> > > >> > > >
> > > >> > > > Thanks again,
> > > >> > > > Justine
> > > >> > > >
> > > >> > > > On Wed, Sep 23, 2020 at 3:17 AM Tom Bentley <
> > tbentley@redhat.com>
> > > >> > wrote:
> > > >> > > >
> > > >> > > > > Hi Justine,
> > > >> > > > >
> > > >> > > > > I know you started the vote thread, but on re-reading the
> KIP
> > I
> > > >> > noticed
> > > >> > > > > that although the topic id is included in the
> MetadataResponse
> > > >> it's
> > > >> > not
> > > >> > > > > surfaced in the output from `kafka-topics.sh --describe`.
> > Maybe
> > > >> that
> > > >> > > was
> > > >> > > > > intentional because ids are intentionally not really
> something
> > > the
> > > >> > user
> > > >> > > > > should care deeply about, but it would also make life harder
> > for
> > > >> > anyone
> > > >> > > > > debugging Kafka and this would likely get worse the more
> topic
> > > ids
> > > >> > got
> > > >> > > > > rolled out across the protocols, clients etc. It seems
> likely
> > > that
> > > >> > > > > `kafka-topics.sh` will eventually need the ability to show
> the
> > > id
> > > >> of
> > > >> > a
> > > >> > > > > topic and perhaps find a topic name given an id. Is there
> any
> > > >> reason
> > > >> > > not
> > > >> > > > to
> > > >> > > > > implement that in this KIP?
> > > >> > > > >
> > > >> > > > > Many thanks,
> > > >> > > > >
> > > >> > > > > Tom
> > > >> > > > >
> > > >> > > > > On Mon, Sep 21, 2020 at 9:54 PM Justine Olshan <
> > > >> jolshan@confluent.io
> > > >> > >
> > > >> > > > > wrote:
> > > >> > > > >
> > > >> > > > > > Hi all,
> > > >> > > > > >
> > > >> > > > > > After thinking about it, I've decided to remove the topic
> > name
> > > >> from
> > > >> > > the
> > > >> > > > > > Fetch Request and Response after all. Since there are so
> > many
> > > of
> > > >> > > these
> > > >> > > > > > requests per second, it is worth removing the extra
> > > information.
> > > >> > I've
> > > >> > > > > > updated the KIP to reflect this change.
> > > >> > > > > >
> > > >> > > > > > Please let me know if there is anything else we should
> > discuss
> > > >> > before
> > > >> > > > > > voting.
> > > >> > > > > >
> > > >> > > > > > Thank you,
> > > >> > > > > > Justine
> > > >> > > > > >
> > > >> > > > > > On Fri, Sep 18, 2020 at 9:46 AM Justine Olshan <
> > > >> > jolshan@confluent.io
> > > >> > > >
> > > >> > > > > > wrote:
> > > >> > > > > >
> > > >> > > > > > > Hi Jun,
> > > >> > > > > > >
> > > >> > > > > > > I see what you are saying. For now we can remove the
> extra
> > > >> > > > information.
> > > >> > > > > > > I'll leave the option to add more fields to the file in
> > the
> > > >> > future.
> > > >> > > > The
> > > >> > > > > > KIP
> > > >> > > > > > > has been updated to reflect this change.
> > > >> > > > > > >
> > > >> > > > > > > Thanks,
> > > >> > > > > > > Justine
> > > >> > > > > > >
> > > >> > > > > > > On Fri, Sep 18, 2020 at 8:46 AM Jun Rao <
> jun@confluent.io
> > >
> > > >> > wrote:
> > > >> > > > > > >
> > > >> > > > > > >> Hi, Justine,
> > > >> > > > > > >>
> > > >> > > > > > >> Thanks for the reply.
> > > >> > > > > > >>
> > > >> > > > > > >> 13. If the log directory is the source of truth, it
> means
> > > >> that
> > > >> > the
> > > >> > > > > > >> redundant info in the metadata file will be ignored.
> Then
> > > the
> > > >> > > > question
> > > >> > > > > > is
> > > >> > > > > > >> why do we need to put the redundant info in the
> metadata
> > > file
> > > >> > now?
> > > >> > > > > > >>
> > > >> > > > > > >> Thanks,
> > > >> > > > > > >>
> > > >> > > > > > >> Jun
> > > >> > > > > > >>
> > > >> > > > > > >> On Thu, Sep 17, 2020 at 5:07 PM Justine Olshan <
> > > >> > > > jolshan@confluent.io>
> > > >> > > > > > >> wrote:
> > > >> > > > > > >>
> > > >> > > > > > >> > Hi Jun,
> > > >> > > > > > >> > Thanks for the quick response!
> > > >> > > > > > >> >
> > > >> > > > > > >> > 12. I've decided to bump up the versions on the
> > requests
> > > >> and
> > > >> > > > updated
> > > >> > > > > > the
> > > >> > > > > > >> > KIP. I think it's good we thoroughly discussed the
> > > options
> > > >> > here,
> > > >> > > > so
> > > >> > > > > we
> > > >> > > > > > >> know
> > > >> > > > > > >> > we made a good choice. :)
> > > >> > > > > > >> >
> > > >> > > > > > >> > 13. This is an interesting situation. I think if this
> > > does
> > > >> > occur
> > > >> > > > we
> > > >> > > > > > >> should
> > > >> > > > > > >> > give a warning. I agree that it's hard to know the
> > source
> > > >> of
> > > >> > > truth
> > > >> > > > > for
> > > >> > > > > > >> sure
> > > >> > > > > > >> > since the directory or the file could be manually
> > > >> modified. I
> > > >> > > > guess
> > > >> > > > > > the
> > > >> > > > > > >> > directory could be used as the source of truth. To be
> > > >> honest,
> > > >> > > I'm
> > > >> > > > > not
> > > >> > > > > > >> > really sure what happens in kafka when the log
> > directory
> > > is
> > > >> > > > renamed
> > > >> > > > > > >> > manually in such a way. I'm also wondering if the
> > > >> situation is
> > > >> > > > > > >> recoverable
> > > >> > > > > > >> > in this scenario.
> > > >> > > > > > >> >
> > > >> > > > > > >> > Thanks,
> > > >> > > > > > >> > Justine
> > > >> > > > > > >> >
> > > >> > > > > > >> > On Thu, Sep 17, 2020 at 4:28 PM Jun Rao <
> > > jun@confluent.io>
> > > >> > > wrote:
> > > >> > > > > > >> >
> > > >> > > > > > >> > > Hi, Justine,
> > > >> > > > > > >> > >
> > > >> > > > > > >> > > Thanks for the reply.
> > > >> > > > > > >> > >
> > > >> > > > > > >> > > 12. I don't have a strong preference either.
> However,
> > > if
> > > >> we
> > > >> > > need
> > > >> > > > > IBP
> > > >> > > > > > >> > > anyway, maybe it's easier to just bump up the
> version
> > > for
> > > >> > all
> > > >> > > > > inter
> > > >> > > > > > >> > broker
> > > >> > > > > > >> > > requests and add the topic id field as a regular
> > > field. A
> > > >> > > > regular
> > > >> > > > > > >> field
> > > >> > > > > > >> > is
> > > >> > > > > > >> > > a bit more concise in wire transfer than a flexible
> > > >> field.
> > > >> > > > > > >> > >
> > > >> > > > > > >> > > 13. The confusion that I was referring to is
> between
> > > the
> > > >> > topic
> > > >> > > > > name
> > > >> > > > > > >> and
> > > >> > > > > > >> > > partition number between the log dir and the
> metadata
> > > >> file.
> > > >> > > For
> > > >> > > > > > >> example,
> > > >> > > > > > >> > if
> > > >> > > > > > >> > > the log dir is topicA-1 and the metadata file in it
> > has
> > > >> > topicB
> > > >> > > > and
> > > >> > > > > > >> > > partition 0 (say due to a bug or manual
> > modification),
> > > >> which
> > > >> > > one
> > > >> > > > > do
> > > >> > > > > > we
> > > >> > > > > > >> > use
> > > >> > > > > > >> > > as the source of truth?
> > > >> > > > > > >> > >
> > > >> > > > > > >> > > Jun
> > > >> > > > > > >> > >
> > > >> > > > > > >> > > On Thu, Sep 17, 2020 at 3:43 PM Justine Olshan <
> > > >> > > > > > jolshan@confluent.io>
> > > >> > > > > > >> > > wrote:
> > > >> > > > > > >> > >
> > > >> > > > > > >> > > > Hi Jun,
> > > >> > > > > > >> > > > Thanks for the comments.
> > > >> > > > > > >> > > >
> > > >> > > > > > >> > > > 12. I bumped the LeaderAndIsrRequest because I
> > > removed
> > > >> the
> > > >> > > > topic
> > > >> > > > > > >> name
> > > >> > > > > > >> > > field
> > > >> > > > > > >> > > > in the response. It may be possible to avoid
> > bumping
> > > >> the
> > > >> > > > version
> > > >> > > > > > >> > without
> > > >> > > > > > >> > > > that change, but I may be missing something.
> > > >> > > > > > >> > > > I believe StopReplica is actually on version 3
> now,
> > > but
> > > >> > > > because
> > > >> > > > > > >> > version 2
> > > >> > > > > > >> > > > is flexible, I kept that listed as version 2 on
> the
> > > KIP
> > > >> > > page.
> > > >> > > > > > >> However,
> > > >> > > > > > >> > > you
> > > >> > > > > > >> > > > may be right in that we may need to bump the
> > version
> > > on
> > > >> > > > > > StopReplica
> > > >> > > > > > >> to
> > > >> > > > > > >> > > deal
> > > >> > > > > > >> > > > with deletion differently as mentioned above. I
> > don't
> > > >> know
> > > >> > > if
> > > >> > > > I
> > > >> > > > > > >> have a
> > > >> > > > > > >> > > big
> > > >> > > > > > >> > > > preference over used tagged fields or not.
> > > >> > > > > > >> > > >
> > > >> > > > > > >> > > > 13. I was thinking that in the case where the
> file
> > > and
> > > >> the
> > > >> > > > > request
> > > >> > > > > > >> > topic
> > > >> > > > > > >> > > > ids don't match, it means that the broker's
> > topic/the
> > > >> one
> > > >> > in
> > > >> > > > the
> > > >> > > > > > >> file
> > > >> > > > > > >> > has
> > > >> > > > > > >> > > > been deleted. In that case, we would need to
> delete
> > > the
> > > >> > old
> > > >> > > > > topic
> > > >> > > > > > >> and
> > > >> > > > > > >> > > start
> > > >> > > > > > >> > > > receiving the new version. If the topic name were
> > to
> > > >> > change,
> > > >> > > > but
> > > >> > > > > > the
> > > >> > > > > > >> > ids
> > > >> > > > > > >> > > > still matched, the file would also need to
> update.
> > > Am I
> > > >> > > > missing
> > > >> > > > > a
> > > >> > > > > > >> case
> > > >> > > > > > >> > > > where the file would be correct and not the
> > request?
> > > >> > > > > > >> > > >
> > > >> > > > > > >> > > > Thanks,
> > > >> > > > > > >> > > > Justine
> > > >> > > > > > >> > > >
> > > >> > > > > > >> > > > On Thu, Sep 17, 2020 at 3:18 PM Jun Rao <
> > > >> jun@confluent.io
> > > >> > >
> > > >> > > > > wrote:
> > > >> > > > > > >> > > >
> > > >> > > > > > >> > > > > Hi, Justine,
> > > >> > > > > > >> > > > >
> > > >> > > > > > >> > > > > Thanks for the reply. A couple of more comments
> > > >> below.
> > > >> > > > > > >> > > > >
> > > >> > > > > > >> > > > > 12. ListOffset and OffsetForLeader currently
> > don't
> > > >> > support
> > > >> > > > > > >> flexible
> > > >> > > > > > >> > > > fields.
> > > >> > > > > > >> > > > > So, we have to bump up the version number and
> use
> > > >> IBP at
> > > >> > > > least
> > > >> > > > > > for
> > > >> > > > > > >> > > these
> > > >> > > > > > >> > > > > two requests. Note that it seems 2.7.0 will
> > require
> > > >> IBP
> > > >> > > > anyway
> > > >> > > > > > >> > because
> > > >> > > > > > >> > > of
> > > >> > > > > > >> > > > > changes in KAFKA-10435. Also, it seems that the
> > > >> version
> > > >> > > for
> > > >> > > > > > >> > > > > LeaderAndIsrRequest and StopReplica are bumped
> > even
> > > >> > though
> > > >> > > > we
> > > >> > > > > > only
> > > >> > > > > > >> > > added
> > > >> > > > > > >> > > > a
> > > >> > > > > > >> > > > > tagged field. But since IBP is needed anyway,
> we
> > > may
> > > >> > want
> > > >> > > to
> > > >> > > > > > >> revisit
> > > >> > > > > > >> > > the
> > > >> > > > > > >> > > > > overall tagged field choice.
> > > >> > > > > > >> > > > >
> > > >> > > > > > >> > > > > 13. The only downside is the potential
> confusion
> > on
> > > >> > which
> > > >> > > > one
> > > >> > > > > is
> > > >> > > > > > >> the
> > > >> > > > > > >> > > > source
> > > >> > > > > > >> > > > > of truth if they don't match. Another option is
> > to
> > > >> > include
> > > >> > > > > those
> > > >> > > > > > >> > fields
> > > >> > > > > > >> > > > in
> > > >> > > > > > >> > > > > the metadata file when we actually change the
> > > >> directory
> > > >> > > > > > structure.
> > > >> > > > > > >> > > > >
> > > >> > > > > > >> > > > > Thanks,
> > > >> > > > > > >> > > > >
> > > >> > > > > > >> > > > > Jun
> > > >> > > > > > >> > > > >
> > > >> > > > > > >> > > > > On Thu, Sep 17, 2020 at 2:01 PM Justine Olshan
> <
> > > >> > > > > > >> jolshan@confluent.io
> > > >> > > > > > >> > >
> > > >> > > > > > >> > > > > wrote:
> > > >> > > > > > >> > > > >
> > > >> > > > > > >> > > > > > Hello all,
> > > >> > > > > > >> > > > > >
> > > >> > > > > > >> > > > > > I've thought some more about removing the
> topic
> > > >> name
> > > >> > > field
> > > >> > > > > > from
> > > >> > > > > > >> > some
> > > >> > > > > > >> > > of
> > > >> > > > > > >> > > > > the
> > > >> > > > > > >> > > > > > requests. On closer inspection of the
> > > >> > > requests/responses,
> > > >> > > > it
> > > >> > > > > > >> seems
> > > >> > > > > > >> > > that
> > > >> > > > > > >> > > > > the
> > > >> > > > > > >> > > > > > internal changes would be much larger than I
> > > >> expected.
> > > >> > > > Some
> > > >> > > > > > >> > protocols
> > > >> > > > > > >> > > > > > involve clients, so they would require
> changes
> > > too.
> > > >> > I'm
> > > >> > > > > > thinking
> > > >> > > > > > >> > that
> > > >> > > > > > >> > > > for
> > > >> > > > > > >> > > > > > now, removing the topic name from these
> > requests
> > > >> and
> > > >> > > > > responses
> > > >> > > > > > >> are
> > > >> > > > > > >> > > out
> > > >> > > > > > >> > > > of
> > > >> > > > > > >> > > > > > scope.
> > > >> > > > > > >> > > > > >
> > > >> > > > > > >> > > > > > I have decided to just keep the change
> > > >> > > > LeaderAndIsrResponse
> > > >> > > > > to
> > > >> > > > > > >> > remove
> > > >> > > > > > >> > > > the
> > > >> > > > > > >> > > > > > topic name, and have updated the KIP to
> reflect
> > > >> this
> > > >> > > > > change. I
> > > >> > > > > > >> have
> > > >> > > > > > >> > > > also
> > > >> > > > > > >> > > > > > mentioned the other requests and responses in
> > > >> future
> > > >> > > work.
> > > >> > > > > > >> > > > > >
> > > >> > > > > > >> > > > > > I'm hoping to start the voting process soon,
> so
> > > >> let me
> > > >> > > > know
> > > >> > > > > if
> > > >> > > > > > >> > there
> > > >> > > > > > >> > > is
> > > >> > > > > > >> > > > > > anything else we should discuss.
> > > >> > > > > > >> > > > > >
> > > >> > > > > > >> > > > > > Thank you,
> > > >> > > > > > >> > > > > > Justine
> > > >> > > > > > >> > > > > >
> > > >> > > > > > >> > > > > > On Tue, Sep 15, 2020 at 3:57 PM Justine
> Olshan
> > <
> > > >> > > > > > >> > jolshan@confluent.io
> > > >> > > > > > >> > > >
> > > >> > > > > > >> > > > > > wrote:
> > > >> > > > > > >> > > > > >
> > > >> > > > > > >> > > > > > > Hello again,
> > > >> > > > > > >> > > > > > > To follow up on some of the other comments:
> > > >> > > > > > >> > > > > > >
> > > >> > > > > > >> > > > > > > 10/11) We can remove the topic name from
> > these
> > > >> > > > > > >> > requests/responses,
> > > >> > > > > > >> > > > and
> > > >> > > > > > >> > > > > > > that means we will just have to make a few
> > > >> internal
> > > >> > > > > changes
> > > >> > > > > > to
> > > >> > > > > > >> > make
> > > >> > > > > > >> > > > > > > partitions accessible by topic id and
> > > partition.
> > > >> I
> > > >> > can
> > > >> > > > > > update
> > > >> > > > > > >> the
> > > >> > > > > > >> > > KIP
> > > >> > > > > > >> > > > > to
> > > >> > > > > > >> > > > > > > remove them unless anyone thinks they
> should
> > > >> stay.
> > > >> > > > > > >> > > > > > >
> > > >> > > > > > >> > > > > > > 12) Addressed in the previous email. I've
> > > updated
> > > >> > the
> > > >> > > > KIP
> > > >> > > > > to
> > > >> > > > > > >> > > include
> > > >> > > > > > >> > > > > > > tagged fields for the requests and
> responses.
> > > >> (More
> > > >> > on
> > > >> > > > > that
> > > >> > > > > > >> > below)
> > > >> > > > > > >> > > > > > >
> > > >> > > > > > >> > > > > > > 13) I think part of the idea for including
> > this
> > > >> > > > > information
> > > >> > > > > > >> is to
> > > >> > > > > > >> > > > > prepare
> > > >> > > > > > >> > > > > > > for future changes. Perhaps the directory
> > > >> structure
> > > >> > > > might
> > > >> > > > > > >> change
> > > >> > > > > > >> > > from
> > > >> > > > > > >> > > > > > > topicName_partitionNumber to something like
> > > >> > > > > > >> > > topicID_partitionNumber.
> > > >> > > > > > >> > > > > Then
> > > >> > > > > > >> > > > > > > it would be useful to have the topic name
> in
> > > the
> > > >> > file
> > > >> > > > > since
> > > >> > > > > > it
> > > >> > > > > > >> > > would
> > > >> > > > > > >> > > > > not
> > > >> > > > > > >> > > > > > be
> > > >> > > > > > >> > > > > > > in the directory structure. Supporting
> topic
> > > >> renames
> > > >> > > > might
> > > >> > > > > > be
> > > >> > > > > > >> > > easier
> > > >> > > > > > >> > > > if
> > > >> > > > > > >> > > > > > the
> > > >> > > > > > >> > > > > > > other fields are included. Would there be
> any
> > > >> > > downsides
> > > >> > > > to
> > > >> > > > > > >> > > including
> > > >> > > > > > >> > > > > this
> > > >> > > > > > >> > > > > > > information?
> > > >> > > > > > >> > > > > > >
> > > >> > > > > > >> > > > > > > 14)  Yes, we would need to copy the
> partition
> > > >> > metadata
> > > >> > > > > file
> > > >> > > > > > in
> > > >> > > > > > >> > this
> > > >> > > > > > >> > > > > > > process. I've updated the KIP to include
> > this.
> > > >> > > > > > >> > > > > > >
> > > >> > > > > > >> > > > > > > 15) I believe Lucas meant v1 and v2 here.
> He
> > > was
> > > >> > > > referring
> > > >> > > > > > to
> > > >> > > > > > >> how
> > > >> > > > > > >> > > the
> > > >> > > > > > >> > > > > > > requests would fall under different IBP and
> > > meant
> > > >> > that
> > > >> > > > > older
> > > >> > > > > > >> > > brokers
> > > >> > > > > > >> > > > > > would
> > > >> > > > > > >> > > > > > > have to use the older version of the
> request
> > > and
> > > >> the
> > > >> > > > > > existing
> > > >> > > > > > >> > topic
> > > >> > > > > > >> > > > > > > deletion process. At first, it seemed like
> > > tagged
> > > >> > > fields
> > > >> > > > > > would
> > > >> > > > > > >> > > > resolve
> > > >> > > > > > >> > > > > > > the IBP issue. However, we may need IBP for
> > > this
> > > >> > > request
> > > >> > > > > > after
> > > >> > > > > > >> > all
> > > >> > > > > > >> > > > > since
> > > >> > > > > > >> > > > > > > the controller handles the topic deletion
> > > >> > differently
> > > >> > > > > > >> depending
> > > >> > > > > > >> > on
> > > >> > > > > > >> > > > the
> > > >> > > > > > >> > > > > > IBP
> > > >> > > > > > >> > > > > > > version. In an older version, we can't just
> > > send
> > > >> a
> > > >> > > > > > StopReplica
> > > >> > > > > > >> > > delete
> > > >> > > > > > >> > > > > the
> > > >> > > > > > >> > > > > > > topic immediately like we'd want to for
> this
> > > KIP.
> > > >> > > > > > >> > > > > > >
> > > >> > > > > > >> > > > > > > This makes me wonder if we want tagged
> fields
> > > on
> > > >> all
> > > >> > > the
> > > >> > > > > > >> requests
> > > >> > > > > > >> > > > after
> > > >> > > > > > >> > > > > > > all. Let me know your thoughts!
> > > >> > > > > > >> > > > > > >
> > > >> > > > > > >> > > > > > > Justine
> > > >> > > > > > >> > > > > > >
> > > >> > > > > > >> > > > > > > On Tue, Sep 15, 2020 at 1:03 PM Justine
> > Olshan
> > > <
> > > >> > > > > > >> > > jolshan@confluent.io
> > > >> > > > > > >> > > > >
> > > >> > > > > > >> > > > > > > wrote:
> > > >> > > > > > >> > > > > > >
> > > >> > > > > > >> > > > > > >> Hi all,
> > > >> > > > > > >> > > > > > >> Jun brought up a good point in his last
> > email
> > > >> about
> > > >> > > > > tagged
> > > >> > > > > > >> > fields,
> > > >> > > > > > >> > > > and
> > > >> > > > > > >> > > > > > >> I've updated the KIP to reflect that the
> > > >> changes to
> > > >> > > > > > requests
> > > >> > > > > > >> and
> > > >> > > > > > >> > > > > > responses
> > > >> > > > > > >> > > > > > >> will be in the form of tagged fields to
> > avoid
> > > >> > > changing
> > > >> > > > > IBP.
> > > >> > > > > > >> > > > > > >>
> > > >> > > > > > >> > > > > > >> Jun: I plan on sending a followup email to
> > > >> address
> > > >> > > some
> > > >> > > > > of
> > > >> > > > > > >> the
> > > >> > > > > > >> > > other
> > > >> > > > > > >> > > > > > >> points.
> > > >> > > > > > >> > > > > > >>
> > > >> > > > > > >> > > > > > >> Thanks,
> > > >> > > > > > >> > > > > > >> Justine
> > > >> > > > > > >> > > > > > >>
> > > >> > > > > > >> > > > > > >> On Mon, Sep 14, 2020 at 4:25 PM Jun Rao <
> > > >> > > > > jun@confluent.io>
> > > >> > > > > > >> > wrote:
> > > >> > > > > > >> > > > > > >>
> > > >> > > > > > >> > > > > > >>> Hi, Justine,
> > > >> > > > > > >> > > > > > >>>
> > > >> > > > > > >> > > > > > >>> Thanks for the updated KIP. A few
> comments
> > > >> below.
> > > >> > > > > > >> > > > > > >>>
> > > >> > > > > > >> > > > > > >>> 10. LeaderAndIsr Response: Do we need the
> > > topic
> > > >> > > name?
> > > >> > > > > > >> > > > > > >>>
> > > >> > > > > > >> > > > > > >>> 11. For the changed request/response,
> other
> > > >> than
> > > >> > > > > > >> LeaderAndIsr,
> > > >> > > > > > >> > > > > > >>> UpdateMetadata, Metadata, do we need to
> > > include
> > > >> > the
> > > >> > > > > topic
> > > >> > > > > > >> name?
> > > >> > > > > > >> > > > > > >>>
> > > >> > > > > > >> > > > > > >>> 12. It seems that upgrades don't require
> > IBP.
> > > >> Does
> > > >> > > > that
> > > >> > > > > > mean
> > > >> > > > > > >> > the
> > > >> > > > > > >> > > > new
> > > >> > > > > > >> > > > > > >>> fields
> > > >> > > > > > >> > > > > > >>> in all the request/response are added as
> > > tagged
> > > >> > > fields
> > > >> > > > > > >> without
> > > >> > > > > > >> > > > > bumping
> > > >> > > > > > >> > > > > > up
> > > >> > > > > > >> > > > > > >>> the request version? It would be useful
> to
> > > make
> > > >> > that
> > > >> > > > > > clear.
> > > >> > > > > > >> > > > > > >>>
> > > >> > > > > > >> > > > > > >>> 13. Partition Metadata file: Do we need
> to
> > > >> include
> > > >> > > the
> > > >> > > > > > topic
> > > >> > > > > > >> > name
> > > >> > > > > > >> > > > and
> > > >> > > > > > >> > > > > > the
> > > >> > > > > > >> > > > > > >>> partition id since they are implied in
> the
> > > >> > directory
> > > >> > > > > name?
> > > >> > > > > > >> > > > > > >>>
> > > >> > > > > > >> > > > > > >>> 14. In the JBOD mode, we support moving a
> > > >> > > partition's
> > > >> > > > > data
> > > >> > > > > > >> from
> > > >> > > > > > >> > > one
> > > >> > > > > > >> > > > > > disk
> > > >> > > > > > >> > > > > > >>> to
> > > >> > > > > > >> > > > > > >>> another. Will the new partition metadata
> > file
> > > >> be
> > > >> > > > copied
> > > >> > > > > > >> during
> > > >> > > > > > >> > > that
> > > >> > > > > > >> > > > > > >>> process?
> > > >> > > > > > >> > > > > > >>>
> > > >> > > > > > >> > > > > > >>> 15. The KIP says "Remove deleted topics
> > from
> > > >> > > replicas
> > > >> > > > by
> > > >> > > > > > >> > sending
> > > >> > > > > > >> > > > > > >>> StopReplicaRequest V2 for any topics
> which
> > do
> > > >> not
> > > >> > > > > contain
> > > >> > > > > > a
> > > >> > > > > > >> > topic
> > > >> > > > > > >> > > > ID,
> > > >> > > > > > >> > > > > > and
> > > >> > > > > > >> > > > > > >>> V3 for any topics which do contain a
> topic
> > > >> ID.".
> > > >> > > > > However,
> > > >> > > > > > it
> > > >> > > > > > >> > > seems
> > > >> > > > > > >> > > > > the
> > > >> > > > > > >> > > > > > >>> updated controller will create all
> missing
> > > >> topic
> > > >> > IDs
> > > >> > > > > first
> > > >> > > > > > >> > before
> > > >> > > > > > >> > > > > doing
> > > >> > > > > > >> > > > > > >>> other actions. So, is StopReplicaRequest
> V2
> > > >> > needed?
> > > >> > > > > > >> > > > > > >>>
> > > >> > > > > > >> > > > > > >>> Jun
> > > >> > > > > > >> > > > > > >>>
> > > >> > > > > > >> > > > > > >>> On Fri, Sep 11, 2020 at 10:31 AM John
> > > Roesler <
> > > >> > > > > > >> > > vvcephei@apache.org
> > > >> > > > > > >> > > > >
> > > >> > > > > > >> > > > > > >>> wrote:
> > > >> > > > > > >> > > > > > >>>
> > > >> > > > > > >> > > > > > >>> > Thanks, Justine!
> > > >> > > > > > >> > > > > > >>> >
> > > >> > > > > > >> > > > > > >>> > Your response seems compelling to me.
> > > >> > > > > > >> > > > > > >>> >
> > > >> > > > > > >> > > > > > >>> > -John
> > > >> > > > > > >> > > > > > >>> >
> > > >> > > > > > >> > > > > > >>> > On Fri, 2020-09-11 at 10:17 -0700,
> > Justine
> > > >> > Olshan
> > > >> > > > > wrote:
> > > >> > > > > > >> > > > > > >>> > > Hello all,
> > > >> > > > > > >> > > > > > >>> > > Thanks for continuing the
> discussion! I
> > > >> have a
> > > >> > > few
> > > >> > > > > > >> > responses
> > > >> > > > > > >> > > to
> > > >> > > > > > >> > > > > > your
> > > >> > > > > > >> > > > > > >>> > points.
> > > >> > > > > > >> > > > > > >>> > >
> > > >> > > > > > >> > > > > > >>> > > Tom: You are correct in that this KIP
> > has
> > > >> not
> > > >> > > > > > mentioned
> > > >> > > > > > >> the
> > > >> > > > > > >> > > > > > >>> > > DeleteTopicsRequest. I think that
> this
> > > >> would
> > > >> > be
> > > >> > > > out
> > > >> > > > > of
> > > >> > > > > > >> > scope
> > > >> > > > > > >> > > > for
> > > >> > > > > > >> > > > > > >>> now, but
> > > >> > > > > > >> > > > > > >>> > > may be something worth adding in the
> > > >> future.
> > > >> > > > > > >> > > > > > >>> > >
> > > >> > > > > > >> > > > > > >>> > > John: We did consider sequence ids,
> but
> > > >> there
> > > >> > > are
> > > >> > > > a
> > > >> > > > > > few
> > > >> > > > > > >> > > reasons
> > > >> > > > > > >> > > > > to
> > > >> > > > > > >> > > > > > >>> favor
> > > >> > > > > > >> > > > > > >>> > > UUIDs. There are several cases where
> > > topics
> > > >> > from
> > > >> > > > > > >> different
> > > >> > > > > > >> > > > > clusters
> > > >> > > > > > >> > > > > > >>> may
> > > >> > > > > > >> > > > > > >>> > > interact now and in the future. For
> > > >> example,
> > > >> > > > Mirror
> > > >> > > > > > >> Maker 2
> > > >> > > > > > >> > > may
> > > >> > > > > > >> > > > > > >>> benefit
> > > >> > > > > > >> > > > > > >>> > > from being able to detect when a
> > cluster
> > > >> being
> > > >> > > > > > mirrored
> > > >> > > > > > >> is
> > > >> > > > > > >> > > > > deleted
> > > >> > > > > > >> > > > > > >>> and
> > > >> > > > > > >> > > > > > >>> > > recreated and globally unique
> > identifiers
> > > >> > would
> > > >> > > > make
> > > >> > > > > > >> > > resolving
> > > >> > > > > > >> > > > > > issues
> > > >> > > > > > >> > > > > > >>> > > easier than sequence IDs which may
> > > collide
> > > >> > > between
> > > >> > > > > > >> > clusters.
> > > >> > > > > > >> > > > > > KIP-405
> > > >> > > > > > >> > > > > > >>> > > (tiered storage) will also benefit
> from
> > > >> > globally
> > > >> > > > > > unique
> > > >> > > > > > >> IDs
> > > >> > > > > > >> > > as
> > > >> > > > > > >> > > > > > shared
> > > >> > > > > > >> > > > > > >>> > > buckets may be used between clusters.
> > > >> > > > > > >> > > > > > >>> > >
> > > >> > > > > > >> > > > > > >>> > > Globally unique IDs would also make
> > > >> > > functionality
> > > >> > > > > like
> > > >> > > > > > >> > moving
> > > >> > > > > > >> > > > > > topics
> > > >> > > > > > >> > > > > > >>> > > between disparate clusters easier in
> > the
> > > >> > future,
> > > >> > > > > > >> simplify
> > > >> > > > > > >> > any
> > > >> > > > > > >> > > > > > future
> > > >> > > > > > >> > > > > > >>> > > implementations of backups and
> > restores,
> > > >> and
> > > >> > > more.
> > > >> > > > > In
> > > >> > > > > > >> > > general,
> > > >> > > > > > >> > > > > > >>> unique IDs
> > > >> > > > > > >> > > > > > >>> > > would ensure that the source cluster
> > > >> topics do
> > > >> > > not
> > > >> > > > > > >> conflict
> > > >> > > > > > >> > > > with
> > > >> > > > > > >> > > > > > the
> > > >> > > > > > >> > > > > > >>> > > destination cluster topics.
> > > >> > > > > > >> > > > > > >>> > >
> > > >> > > > > > >> > > > > > >>> > > If we were to use sequence ids, we
> > would
> > > >> need
> > > >> > > > > > >> sufficiently
> > > >> > > > > > >> > > > large
> > > >> > > > > > >> > > > > > >>> cluster
> > > >> > > > > > >> > > > > > >>> > > ids to be stored with the topic
> > > >> identifiers or
> > > >> > > we
> > > >> > > > > run
> > > >> > > > > > >> the
> > > >> > > > > > >> > > risk
> > > >> > > > > > >> > > > of
> > > >> > > > > > >> > > > > > >>> > > collisions. This will give up any
> > > >> advantage in
> > > >> > > > > > >> compactness
> > > >> > > > > > >> > > that
> > > >> > > > > > >> > > > > > >>> sequence
> > > >> > > > > > >> > > > > > >>> > > numbers may bring. Given these
> > > advantages I
> > > >> > > think
> > > >> > > > it
> > > >> > > > > > >> makes
> > > >> > > > > > >> > > > sense
> > > >> > > > > > >> > > > > to
> > > >> > > > > > >> > > > > > >>> use
> > > >> > > > > > >> > > > > > >>> > > UUIDs.
> > > >> > > > > > >> > > > > > >>> > >
> > > >> > > > > > >> > > > > > >>> > > Gokul: This is an interesting idea,
> but
> > > >> this
> > > >> > is
> > > >> > > a
> > > >> > > > > > >> breaking
> > > >> > > > > > >> > > > > change.
> > > >> > > > > > >> > > > > > >>> Out of
> > > >> > > > > > >> > > > > > >>> > > scope for now, but maybe worth
> > discussing
> > > >> in
> > > >> > the
> > > >> > > > > > future.
> > > >> > > > > > >> > > > > > >>> > >
> > > >> > > > > > >> > > > > > >>> > > Hope this explains some of the
> > decisions,
> > > >> > > > > > >> > > > > > >>> > >
> > > >> > > > > > >> > > > > > >>> > > Justine
> > > >> > > > > > >> > > > > > >>> > >
> > > >> > > > > > >> > > > > > >>> > >
> > > >> > > > > > >> > > > > > >>> > >
> > > >> > > > > > >> > > > > > >>> > > On Fri, Sep 11, 2020 at 8:27 AM Gokul
> > > >> Ramanan
> > > >> > > > > > >> Subramanian <
> > > >> > > > > > >> > > > > > >>> > > gokul2411s@gmail.com> wrote:
> > > >> > > > > > >> > > > > > >>> > >
> > > >> > > > > > >> > > > > > >>> > > > Hi.
> > > >> > > > > > >> > > > > > >>> > > >
> > > >> > > > > > >> > > > > > >>> > > > Thanks for the KIP.
> > > >> > > > > > >> > > > > > >>> > > >
> > > >> > > > > > >> > > > > > >>> > > > Have you thought about whether it
> > makes
> > > >> > sense
> > > >> > > to
> > > >> > > > > > >> support
> > > >> > > > > > >> > > > > > >>> authorizing a
> > > >> > > > > > >> > > > > > >>> > > > principal for a topic ID rather
> than
> > a
> > > >> topic
> > > >> > > > name
> > > >> > > > > to
> > > >> > > > > > >> > > achieve
> > > >> > > > > > >> > > > > > >>> tighter
> > > >> > > > > > >> > > > > > >>> > > > security?
> > > >> > > > > > >> > > > > > >>> > > >
> > > >> > > > > > >> > > > > > >>> > > > Or is the topic ID fundamentally an
> > > >> internal
> > > >> > > > > detail
> > > >> > > > > > >> > similar
> > > >> > > > > > >> > > > to
> > > >> > > > > > >> > > > > > >>> epochs
> > > >> > > > > > >> > > > > > >>> > used
> > > >> > > > > > >> > > > > > >>> > > > in a bunch of other places in
> Kafka?
> > > >> > > > > > >> > > > > > >>> > > >
> > > >> > > > > > >> > > > > > >>> > > > Thanks.
> > > >> > > > > > >> > > > > > >>> > > >
> > > >> > > > > > >> > > > > > >>> > > > On Fri, Sep 11, 2020 at 4:06 PM
> John
> > > >> > Roesler <
> > > >> > > > > > >> > > > > > vvcephei@apache.org>
> > > >> > > > > > >> > > > > > >>> > wrote:
> > > >> > > > > > >> > > > > > >>> > > >
> > > >> > > > > > >> > > > > > >>> > > > > Hello Justine,
> > > >> > > > > > >> > > > > > >>> > > > >
> > > >> > > > > > >> > > > > > >>> > > > > Thanks for the KIP!
> > > >> > > > > > >> > > > > > >>> > > > >
> > > >> > > > > > >> > > > > > >>> > > > > I happen to have been confronted
> > > >> recently
> > > >> > > with
> > > >> > > > > the
> > > >> > > > > > >> need
> > > >> > > > > > >> > > to
> > > >> > > > > > >> > > > > keep
> > > >> > > > > > >> > > > > > >>> > track of
> > > >> > > > > > >> > > > > > >>> > > > a
> > > >> > > > > > >> > > > > > >>> > > > > large number of topics as
> compactly
> > > as
> > > >> > > > > possible. I
> > > >> > > > > > >> was
> > > >> > > > > > >> > > > going
> > > >> > > > > > >> > > > > to
> > > >> > > > > > >> > > > > > >>> come
> > > >> > > > > > >> > > > > > >>> > up
> > > >> > > > > > >> > > > > > >>> > > > > with some way to dictionary
> encode
> > > the
> > > >> > topic
> > > >> > > > > names
> > > >> > > > > > >> as
> > > >> > > > > > >> > > > > integers,
> > > >> > > > > > >> > > > > > >>> but
> > > >> > > > > > >> > > > > > >>> > this
> > > >> > > > > > >> > > > > > >>> > > > > seems much better!
> > > >> > > > > > >> > > > > > >>> > > > >
> > > >> > > > > > >> > > > > > >>> > > > > Apologies if this has been raised
> > > >> before,
> > > >> > > but
> > > >> > > > > I’m
> > > >> > > > > > >> > > wondering
> > > >> > > > > > >> > > > > > >>> about the
> > > >> > > > > > >> > > > > > >>> > > > > choice of UUID vs sequence number
> > for
> > > >> the
> > > >> > > ids.
> > > >> > > > > > >> > Typically,
> > > >> > > > > > >> > > > > I’ve
> > > >> > > > > > >> > > > > > >>> seen
> > > >> > > > > > >> > > > > > >>> > UUIDs
> > > >> > > > > > >> > > > > > >>> > > > > in two situations:
> > > >> > > > > > >> > > > > > >>> > > > > 1. When processes need to
> generate
> > > >> > > > non-colliding
> > > >> > > > > > >> > > > identifiers
> > > >> > > > > > >> > > > > > >>> without
> > > >> > > > > > >> > > > > > >>> > > > > coordination.
> > > >> > > > > > >> > > > > > >>> > > > > 2. When the identifier needs to
> be
> > > >> > > > “universally
> > > >> > > > > > >> > unique”;
> > > >> > > > > > >> > > > > I.e.,
> > > >> > > > > > >> > > > > > >>> the
> > > >> > > > > > >> > > > > > >>> > > > > identifier needs to distinguish
> the
> > > >> entity
> > > >> > > > from
> > > >> > > > > > all
> > > >> > > > > > >> > other
> > > >> > > > > > >> > > > > > >>> entities
> > > >> > > > > > >> > > > > > >>> > that
> > > >> > > > > > >> > > > > > >>> > > > > could ever exist. This is useful
> in
> > > >> cases
> > > >> > > > where
> > > >> > > > > > >> > entities
> > > >> > > > > > >> > > > from
> > > >> > > > > > >> > > > > > all
> > > >> > > > > > >> > > > > > >>> > kinds
> > > >> > > > > > >> > > > > > >>> > > > of
> > > >> > > > > > >> > > > > > >>> > > > > systems get mixed together, such
> as
> > > >> when
> > > >> > > > dumping
> > > >> > > > > > >> logs
> > > >> > > > > > >> > > from
> > > >> > > > > > >> > > > > all
> > > >> > > > > > >> > > > > > >>> > processes
> > > >> > > > > > >> > > > > > >>> > > > in
> > > >> > > > > > >> > > > > > >>> > > > > a company into a common system.
> > > >> > > > > > >> > > > > > >>> > > > >
> > > >> > > > > > >> > > > > > >>> > > > > Maybe I’m being short-sighted,
> but
> > it
> > > >> > > doesn’t
> > > >> > > > > seem
> > > >> > > > > > >> like
> > > >> > > > > > >> > > > > either
> > > >> > > > > > >> > > > > > >>> really
> > > >> > > > > > >> > > > > > >>> > > > > applies here. It seems like the
> > > brokers
> > > >> > > could
> > > >> > > > > and
> > > >> > > > > > >> would
> > > >> > > > > > >> > > > > achieve
> > > >> > > > > > >> > > > > > >>> > consensus
> > > >> > > > > > >> > > > > > >>> > > > > when creating a topic anyway,
> which
> > > is
> > > >> all
> > > >> > > > > that’s
> > > >> > > > > > >> > > required
> > > >> > > > > > >> > > > to
> > > >> > > > > > >> > > > > > >>> > generate
> > > >> > > > > > >> > > > > > >>> > > > > non-colliding sequence ids. For
> the
> > > >> > second,
> > > >> > > as
> > > >> > > > > you
> > > >> > > > > > >> > > mention,
> > > >> > > > > > >> > > > > we
> > > >> > > > > > >> > > > > > >>> could
> > > >> > > > > > >> > > > > > >>> > > > assign
> > > >> > > > > > >> > > > > > >>> > > > > a UUID to the cluster as a whole,
> > > which
> > > >> > > would
> > > >> > > > > > render
> > > >> > > > > > >> > any
> > > >> > > > > > >> > > > > > resource
> > > >> > > > > > >> > > > > > >>> > scoped
> > > >> > > > > > >> > > > > > >>> > > > to
> > > >> > > > > > >> > > > > > >>> > > > > the broker universally unique as
> > > well.
> > > >> > > > > > >> > > > > > >>> > > > >
> > > >> > > > > > >> > > > > > >>> > > > > The reason I mention this is
> that,
> > > >> > although
> > > >> > > a
> > > >> > > > > UUID
> > > >> > > > > > >> is
> > > >> > > > > > >> > way
> > > >> > > > > > >> > > > > more
> > > >> > > > > > >> > > > > > >>> > compact
> > > >> > > > > > >> > > > > > >>> > > > > than topic names, it’s still 16
> > > bytes.
> > > >> In
> > > >> > > > > > contrast,
> > > >> > > > > > >> a
> > > >> > > > > > >> > > > 4-byte
> > > >> > > > > > >> > > > > > >>> integer
> > > >> > > > > > >> > > > > > >>> > > > > sequence id would give us 4
> billion
> > > >> unique
> > > >> > > > > topics
> > > >> > > > > > >> per
> > > >> > > > > > >> > > > > cluster,
> > > >> > > > > > >> > > > > > >>> which
> > > >> > > > > > >> > > > > > >>> > > > seems
> > > >> > > > > > >> > > > > > >>> > > > > like enough ;)
> > > >> > > > > > >> > > > > > >>> > > > >
> > > >> > > > > > >> > > > > > >>> > > > > Considering the number of
> different
> > > >> times
> > > >> > > > these
> > > >> > > > > > >> topic
> > > >> > > > > > >> > > > > > >>> identifiers are
> > > >> > > > > > >> > > > > > >>> > > > sent
> > > >> > > > > > >> > > > > > >>> > > > > over the wire or stored in
> memory,
> > it
> > > >> > seems
> > > >> > > > like
> > > >> > > > > > it
> > > >> > > > > > >> > might
> > > >> > > > > > >> > > > be
> > > >> > > > > > >> > > > > > >>> worth
> > > >> > > > > > >> > > > > > >>> > the
> > > >> > > > > > >> > > > > > >>> > > > > additional 4x space savings.
> > > >> > > > > > >> > > > > > >>> > > > >
> > > >> > > > > > >> > > > > > >>> > > > > What do you think about this?
> > > >> > > > > > >> > > > > > >>> > > > >
> > > >> > > > > > >> > > > > > >>> > > > > Thanks,
> > > >> > > > > > >> > > > > > >>> > > > > John
> > > >> > > > > > >> > > > > > >>> > > > >
> > > >> > > > > > >> > > > > > >>> > > > > On Fri, Sep 11, 2020, at 03:20,
> Tom
> > > >> > Bentley
> > > >> > > > > wrote:
> > > >> > > > > > >> > > > > > >>> > > > > > Hi Justine,
> > > >> > > > > > >> > > > > > >>> > > > > >
> > > >> > > > > > >> > > > > > >>> > > > > > This looks like a very welcome
> > > >> > > improvement.
> > > >> > > > > > >> Thanks!
> > > >> > > > > > >> > > > > > >>> > > > > >
> > > >> > > > > > >> > > > > > >>> > > > > > Maybe I missed it, but the KIP
> > > >> doesn't
> > > >> > > seem
> > > >> > > > to
> > > >> > > > > > >> > mention
> > > >> > > > > > >> > > > > > changing
> > > >> > > > > > >> > > > > > >>> > > > > > DeleteTopicsRequest to identify
> > the
> > > >> > topic
> > > >> > > > > using
> > > >> > > > > > an
> > > >> > > > > > >> > id.
> > > >> > > > > > >> > > > > Maybe
> > > >> > > > > > >> > > > > > >>> > that's out
> > > >> > > > > > >> > > > > > >>> > > > > of
> > > >> > > > > > >> > > > > > >>> > > > > > scope, but DeleteTopicsRequest
> is
> > > not
> > > >> > > listed
> > > >> > > > > > among
> > > >> > > > > > >> > the
> > > >> > > > > > >> > > > > Future
> > > >> > > > > > >> > > > > > >>> Work
> > > >> > > > > > >> > > > > > >>> > APIs
> > > >> > > > > > >> > > > > > >>> > > > > > either.
> > > >> > > > > > >> > > > > > >>> > > > > >
> > > >> > > > > > >> > > > > > >>> > > > > > Kind regards,
> > > >> > > > > > >> > > > > > >>> > > > > >
> > > >> > > > > > >> > > > > > >>> > > > > > Tom
> > > >> > > > > > >> > > > > > >>> > > > > >
> > > >> > > > > > >> > > > > > >>> > > > > > On Thu, Sep 10, 2020 at 3:59 PM
> > > >> Satish
> > > >> > > > > Duggana <
> > > >> > > > > > >> > > > > > >>> > > > satish.duggana@gmail.com
> > > >> > > > > > >> > > > > > >>> > > > > > wrote:
> > > >> > > > > > >> > > > > > >>> > > > > >
> > > >> > > > > > >> > > > > > >>> > > > > > > Thanks Lucas/Justine for the
> > nice
> > > >> KIP.
> > > >> > > > > > >> > > > > > >>> > > > > > >
> > > >> > > > > > >> > > > > > >>> > > > > > > It has several benefits which
> > > also
> > > >> > > include
> > > >> > > > > > >> > > simplifying
> > > >> > > > > > >> > > > > the
> > > >> > > > > > >> > > > > > >>> topic
> > > >> > > > > > >> > > > > > >>> > > > > > > deletion process by
> controller
> > > and
> > > >> > logs
> > > >> > > > > > cleanup
> > > >> > > > > > >> by
> > > >> > > > > > >> > > > > brokers
> > > >> > > > > > >> > > > > > in
> > > >> > > > > > >> > > > > > >>> > corner
> > > >> > > > > > >> > > > > > >>> > > > > > > cases.
> > > >> > > > > > >> > > > > > >>> > > > > > >
> > > >> > > > > > >> > > > > > >>> > > > > > > Best,
> > > >> > > > > > >> > > > > > >>> > > > > > > Satish.
> > > >> > > > > > >> > > > > > >>> > > > > > >
> > > >> > > > > > >> > > > > > >>> > > > > > > On Wed, Sep 9, 2020 at 10:07
> PM
> > > >> > Justine
> > > >> > > > > > Olshan <
> > > >> > > > > > >> > > > > > >>> > jolshan@confluent.io
> > > >> > > > > > >> > > > > > >>> > > > > > > wrote:
> > > >> > > > > > >> > > > > > >>> > > > > > > > Hello all, it's been
> almost a
> > > >> year!
> > > >> > > I've
> > > >> > > > > > made
> > > >> > > > > > >> > some
> > > >> > > > > > >> > > > > > changes
> > > >> > > > > > >> > > > > > >>> to
> > > >> > > > > > >> > > > > > >>> > this
> > > >> > > > > > >> > > > > > >>> > > > > KIP
> > > >> > > > > > >> > > > > > >>> > > > > > > and hope to continue the
> > > >> discussion.
> > > >> > > > > > >> > > > > > >>> > > > > > > > One of the main changes
> I've
> > > >> added
> > > >> > is
> > > >> > > > now
> > > >> > > > > > the
> > > >> > > > > > >> > > > metadata
> > > >> > > > > > >> > > > > > >>> response
> > > >> > > > > > >> > > > > > >>> > > > will
> > > >> > > > > > >> > > > > > >>> > > > > > > include the topic ID (as
> Colin
> > > >> > > suggested).
> > > >> > > > > > >> Clients
> > > >> > > > > > >> > > can
> > > >> > > > > > >> > > > > > >>> obtain the
> > > >> > > > > > >> > > > > > >>> > > > > topicID
> > > >> > > > > > >> > > > > > >>> > > > > > > of a given topic through a
> > > >> > > > TopicDescription.
> > > >> > > > > > The
> > > >> > > > > > >> > > > topicId
> > > >> > > > > > >> > > > > > will
> > > >> > > > > > >> > > > > > >>> > also be
> > > >> > > > > > >> > > > > > >>> > > > > > > included with the
> > UpdateMetadata
> > > >> > > request.
> > > >> > > > > > >> > > > > > >>> > > > > > > > Let me know what you all
> > think.
> > > >> > > > > > >> > > > > > >>> > > > > > > > Thank you,
> > > >> > > > > > >> > > > > > >>> > > > > > > > Justine
> > > >> > > > > > >> > > > > > >>> > > > > > > >
> > > >> > > > > > >> > > > > > >>> > > > > > > > On 2019/09/13 16:38:26,
> > "Colin
> > > >> > > McCabe" <
> > > >> > > > > > >> > > > > > cmccabe@apache.org
> > > >> > > > > > >> > > > > > >>> >
> > > >> > > > > > >> > > > > > >>> > wrote:
> > > >> > > > > > >> > > > > > >>> > > > > > > > > Hi Lucas,
> > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > >> > > > > > >> > > > > > >>> > > > > > > > > Thanks for tackling this.
> > > >> Topic
> > > >> > IDs
> > > >> > > > > are a
> > > >> > > > > > >> > great
> > > >> > > > > > >> > > > > idea,
> > > >> > > > > > >> > > > > > >>> and
> > > >> > > > > > >> > > > > > >>> > this
> > > >> > > > > > >> > > > > > >>> > > > is
> > > >> > > > > > >> > > > > > >>> > > > > a
> > > >> > > > > > >> > > > > > >>> > > > > > > really good writeup.
> > > >> > > > > > >> > > > > > >>> > > > > > > > > For
> > /brokers/topics/[topic],
> > > >> the
> > > >> > > > schema
> > > >> > > > > > >> version
> > > >> > > > > > >> > > > > should
> > > >> > > > > > >> > > > > > be
> > > >> > > > > > >> > > > > > >>> > bumped
> > > >> > > > > > >> > > > > > >>> > > > to
> > > >> > > > > > >> > > > > > >>> > > > > > > version 3, rather than 2.
> > > KIP-455
> > > >> > > bumped
> > > >> > > > > the
> > > >> > > > > > >> > version
> > > >> > > > > > >> > > > of
> > > >> > > > > > >> > > > > > this
> > > >> > > > > > >> > > > > > >>> > znode
> > > >> > > > > > >> > > > > > >>> > > > to
> > > >> > > > > > >> > > > > > >>> > > > > 2
> > > >> > > > > > >> > > > > > >>> > > > > > > already :)
> > > >> > > > > > >> > > > > > >>> > > > > > > > > Given that we're going to
> > be
> > > >> > seeing
> > > >> > > > > these
> > > >> > > > > > >> > things
> > > >> > > > > > >> > > as
> > > >> > > > > > >> > > > > > >>> strings
> > > >> > > > > > >> > > > > > >>> > as
> > > >> > > > > > >> > > > > > >>> > > > lot
> > > >> > > > > > >> > > > > > >>> > > > > (in
> > > >> > > > > > >> > > > > > >>> > > > > > > logs, in ZooKeeper, on the
> > > >> > command-line,
> > > >> > > > > > etc.),
> > > >> > > > > > >> > does
> > > >> > > > > > >> > > it
> > > >> > > > > > >> > > > > > make
> > > >> > > > > > >> > > > > > >>> > sense to
> > > >> > > > > > >> > > > > > >>> > > > > use
> > > >> > > > > > >> > > > > > >>> > > > > > > base64 when converting them
> to
> > > >> > strings?
> > > >> > > > > > >> > > > > > >>> > > > > > > > > Here is an example of the
> > hex
> > > >> > > > > > >> representation:
> > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > >> > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > >> > > > > > >> > > > > > >>> > > > > > > > > And here is an example in
> > > >> base64.
> > > >> > > > > > >> > > > > > >>> > > > > > > > > b8tRS7h4TJ2Vt43Dp85v2A
> > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > >> > > > > > >> > > > > > >>> > > > > > > > > The base64 version saves
> 15
> > > >> > letters
> > > >> > > > (to
> > > >> > > > > be
> > > >> > > > > > >> > fair,
> > > >> > > > > > >> > > 4
> > > >> > > > > > >> > > > of
> > > >> > > > > > >> > > > > > >>> those
> > > >> > > > > > >> > > > > > >>> > were
> > > >> > > > > > >> > > > > > >>> > > > > > > dashes that we could have
> > elided
> > > in
> > > >> > the
> > > >> > > > hex
> > > >> > > > > > >> > > > > > representation.)
> > > >> > > > > > >> > > > > > >>> > > > > > > > > Another thing to consider
> > is
> > > >> that
> > > >> > we
> > > >> > > > > > should
> > > >> > > > > > >> > > specify
> > > >> > > > > > >> > > > > > that
> > > >> > > > > > >> > > > > > >>> the
> > > >> > > > > > >> > > > > > >>> > > > > > > all-zeroes UUID is not a
> valid
> > > >> topic
> > > >> > > UUID.
> > > >> > > > > >  We
> > > >> > > > > > >> > can't
> > > >> > > > > > >> > > > use
> > > >> > > > > > >> > > > > > >>> null
> > > >> > > > > > >> > > > > > >>> > for
> > > >> > > > > > >> > > > > > >>> > > > this
> > > >> > > > > > >> > > > > > >>> > > > > > > because we can't pass a null
> > UUID
> > > >> over
> > > >> > > the
> > > >> > > > > RPC
> > > >> > > > > > >> > > protocol
> > > >> > > > > > >> > > > > > >>> (there
> > > >> > > > > > >> > > > > > >>> > is no
> > > >> > > > > > >> > > > > > >>> > > > > > > special pattern for null, nor
> > do
> > > we
> > > >> > want
> > > >> > > > to
> > > >> > > > > > >> waste
> > > >> > > > > > >> > > space
> > > >> > > > > > >> > > > > > >>> reserving
> > > >> > > > > > >> > > > > > >>> > > > such
> > > >> > > > > > >> > > > > > >>> > > > > a
> > > >> > > > > > >> > > > > > >>> > > > > > > pattern.)
> > > >> > > > > > >> > > > > > >>> > > > > > > > > Maybe I missed it, but
> did
> > > you
> > > >> > > > describe
> > > >> > > > > > >> > > "migration
> > > >> > > > > > >> > > > > > of...
> > > >> > > > > > >> > > > > > >>> > existing
> > > >> > > > > > >> > > > > > >>> > > > > > > topic[s] without topic IDs"
> in
> > > >> detail
> > > >> > in
> > > >> > > > any
> > > >> > > > > > >> > section?
> > > >> > > > > > >> > > > It
> > > >> > > > > > >> > > > > > >>> seems
> > > >> > > > > > >> > > > > > >>> > like
> > > >> > > > > > >> > > > > > >>> > > > > when
> > > >> > > > > > >> > > > > > >>> > > > > > > the new controller becomes
> > > active,
> > > >> it
> > > >> > > > should
> > > >> > > > > > >> just
> > > >> > > > > > >> > > > > generate
> > > >> > > > > > >> > > > > > >>> random
> > > >> > > > > > >> > > > > > >>> > > > > UUIDs for
> > > >> > > > > > >> > > > > > >>> > > > > > > these, and write the random
> > UUIDs
> > > >> back
> > > >> > > to
> > > >> > > > > > >> > ZooKeeper.
> > > >> > > > > > >> > > > It
> > > >> > > > > > >> > > > > > >>> would be
> > > >> > > > > > >> > > > > > >>> > > > good
> > > >> > > > > > >> > > > > > >>> > > > > to
> > > >> > > > > > >> > > > > > >>> > > > > > > spell that out.  We should
> make
> > > it
> > > >> > clear
> > > >> > > > > that
> > > >> > > > > > >> this
> > > >> > > > > > >> > > > > happens
> > > >> > > > > > >> > > > > > >>> > regardless
> > > >> > > > > > >> > > > > > >>> > > > > of
> > > >> > > > > > >> > > > > > >>> > > > > > > the inter-broker protocol
> > version
> > > >> > (it's
> > > >> > > a
> > > >> > > > > > >> > compatible
> > > >> > > > > > >> > > > > > change).
> > > >> > > > > > >> > > > > > >>> > > > > > > > > "LeaderAndIsrRequests
> > > >> including an
> > > >> > > > > > >> > > > is_every_partition
> > > >> > > > > > >> > > > > > >>> flag"
> > > >> > > > > > >> > > > > > >>> > > > seems a
> > > >> > > > > > >> > > > > > >>> > > > > > > bit wordy.  Can we just call
> > > these
> > > >> > "full
> > > >> > > > > > >> > > > > > >>> LeaderAndIsrRequests"?
> > > >> > > > > > >> > > > > > >>> > Then
> > > >> > > > > > >> > > > > > >>> > > > > the
> > > >> > > > > > >> > > > > > >>> > > > > > > RPC field could be named
> > "full".
> > > >> > Also,
> > > >> > > it
> > > >> > > > > > would
> > > >> > > > > > >> > > > probably
> > > >> > > > > > >> > > > > > be
> > > >> > > > > > >> > > > > > >>> > better
> > > >> > > > > > >> > > > > > >>> > > > > for the
> > > >> > > > > > >> > > > > > >>> > > > > > > RPC field to be an enum of {
> > > >> > > UNSPECIFIED,
> > > >> > > > > > >> > > INCREMENTAL,
> > > >> > > > > > >> > > > > FULL
> > > >> > > > > > >> > > > > > >>> }, so
> > > >> > > > > > >> > > > > > >>> > > > that
> > > >> > > > > > >> > > > > > >>> > > > > we
> > > >> > > > > > >> > > > > > >>> > > > > > > can cleanly handle old
> versions
> > > (by
> > > >> > > > treating
> > > >> > > > > > >> them
> > > >> > > > > > >> > as
> > > >> > > > > > >> > > > > > >>> UNSPECIFIED)
> > > >> > > > > > >> > > > > > >>> > > > > > > > > In the
> LeaderAndIsrRequest
> > > >> > section,
> > > >> > > > you
> > > >> > > > > > >> write
> > > >> > > > > > >> > "A
> > > >> > > > > > >> > > > > final
> > > >> > > > > > >> > > > > > >>> > deletion
> > > >> > > > > > >> > > > > > >>> > > > > event
> > > >> > > > > > >> > > > > > >>> > > > > > > will be secheduled for X ms
> > after
> > > >> the
> > > >> > > > > > >> > > > LeaderAndIsrRequest
> > > >> > > > > > >> > > > > > was
> > > >> > > > > > >> > > > > > >>> > first
> > > >> > > > > > >> > > > > > >>> > > > > > > received..."  I guess the X
> > was a
> > > >> > > > > placeholder
> > > >> > > > > > >> that
> > > >> > > > > > >> > > you
> > > >> > > > > > >> > > > > > >>> intended
> > > >> > > > > > >> > > > > > >>> > to
> > > >> > > > > > >> > > > > > >>> > > > > replace
> > > >> > > > > > >> > > > > > >>> > > > > > > before posting? :)  In any
> > case,
> > > >> this
> > > >> > > > seems
> > > >> > > > > > like
> > > >> > > > > > >> > the
> > > >> > > > > > >> > > > kind
> > > >> > > > > > >> > > > > > of
> > > >> > > > > > >> > > > > > >>> > thing
> > > >> > > > > > >> > > > > > >>> > > > we'd
> > > >> > > > > > >> > > > > > >>> > > > > > > want a configuration for.
> > Let's
> > > >> > > describe
> > > >> > > > > that
> > > >> > > > > > >> > > > > > configuration
> > > >> > > > > > >> > > > > > >>> key
> > > >> > > > > > >> > > > > > >>> > > > > somewhere
> > > >> > > > > > >> > > > > > >>> > > > > > > in this KIP, including what
> its
> > > >> > default
> > > >> > > > > value
> > > >> > > > > > >> is.
> > > >> > > > > > >> > > > > > >>> > > > > > > > > We should probably also
> > log a
> > > >> > bunch
> > > >> > > of
> > > >> > > > > > >> messages
> > > >> > > > > > >> > > at
> > > >> > > > > > >> > > > > WARN
> > > >> > > > > > >> > > > > > >>> level
> > > >> > > > > > >> > > > > > >>> > > > when
> > > >> > > > > > >> > > > > > >>> > > > > > > something is scheduled for
> > > >> deletion,
> > > >> > as
> > > >> > > > > well.
> > > >> > > > > > >> > (Maybe
> > > >> > > > > > >> > > > > this
> > > >> > > > > > >> > > > > > >>> was
> > > >> > > > > > >> > > > > > >>> > > > > assumed, but
> > > >> > > > > > >> > > > > > >>> > > > > > > it would be good to mention
> > it).
> > > >> > > > > > >> > > > > > >>> > > > > > > > > I feel like there are a
> few
> > > >> > sections
> > > >> > > > > that
> > > >> > > > > > >> > should
> > > >> > > > > > >> > > be
> > > >> > > > > > >> > > > > > >>> moved to
> > > >> > > > > > >> > > > > > >>> > > > > "rejected
> > > >> > > > > > >> > > > > > >>> > > > > > > alternatives."  For example,
> in
> > > the
> > > >> > > > > > DeleteTopics
> > > >> > > > > > >> > > > section,
> > > >> > > > > > >> > > > > > >>> since
> > > >> > > > > > >> > > > > > >>> > we're
> > > >> > > > > > >> > > > > > >>> > > > > not
> > > >> > > > > > >> > > > > > >>> > > > > > > going to do option 1 or 2,
> > these
> > > >> > should
> > > >> > > be
> > > >> > > > > > moved
> > > >> > > > > > >> > into
> > > >> > > > > > >> > > > > > >>> "rejected
> > > >> > > > > > >> > > > > > >>> > > > > > > alternatives,"  rather than
> > > >> appearing
> > > >> > > > > inline.
> > > >> > > > > > >> > > Another
> > > >> > > > > > >> > > > > case
> > > >> > > > > > >> > > > > > >>> is
> > > >> > > > > > >> > > > > > >>> > the
> > > >> > > > > > >> > > > > > >>> > > > > "Should
> > > >> > > > > > >> > > > > > >>> > > > > > > we remove topic name from the
> > > >> protocol
> > > >> > > > where
> > > >> > > > > > >> > > possible"
> > > >> > > > > > >> > > > > > >>> section.
> > > >> > > > > > >> > > > > > >>> > This
> > > >> > > > > > >> > > > > > >>> > > > > is
> > > >> > > > > > >> > > > > > >>> > > > > > > clearly discussing a design
> > > >> > alternative
> > > >> > > > that
> > > >> > > > > > >> we're
> > > >> > > > > > >> > > not
> > > >> > > > > > >> > > > > > >>> proposing
> > > >> > > > > > >> > > > > > >>> > to
> > > >> > > > > > >> > > > > > >>> > > > > > > implement: removing the topic
> > > name
> > > >> > from
> > > >> > > > > those
> > > >> > > > > > >> > > > protocols.
> > > >> > > > > > >> > > > > > >>> > > > > > > > > Is it really necessary to
> > > have
> > > >> a
> > > >> > new
> > > >> > > > > > >> > > > > > >>> > /admin/delete_topics_by_id
> > > >> > > > > > >> > > > > > >>> > > > > path
> > > >> > > > > > >> > > > > > >>> > > > > > > in ZooKeeper?  It seems like
> we
> > > >> don't
> > > >> > > > really
> > > >> > > > > > >> need
> > > >> > > > > > >> > > this.
> > > >> > > > > > >> > > > > > >>> Whenever
> > > >> > > > > > >> > > > > > >>> > > > > there is
> > > >> > > > > > >> > > > > > >>> > > > > > > a new controller, we'll send
> > out
> > > >> full
> > > >> > > > > > >> > > > > LeaderAndIsrRequests
> > > >> > > > > > >> > > > > > >>> which
> > > >> > > > > > >> > > > > > >>> > will
> > > >> > > > > > >> > > > > > >>> > > > > > > trigger the stale topics to
> be
> > > >> cleaned
> > > >> > > up.
> > > >> > > > > >  The
> > > >> > > > > > >> > > active
> > > >> > > > > > >> > > > > > >>> > controller
> > > >> > > > > > >> > > > > > >>> > > > will
> > > >> > > > > > >> > > > > > >>> > > > > > > also send the full
> > > >> LeaderAndIsrRequest
> > > >> > > to
> > > >> > > > > > >> brokers
> > > >> > > > > > >> > > that
> > > >> > > > > > >> > > > > are
> > > >> > > > > > >> > > > > > >>> just
> > > >> > > > > > >> > > > > > >>> > > > > starting
> > > >> > > > > > >> > > > > > >>> > > > > > > up.    So we don't really
> need
> > > this
> > > >> > kind
> > > >> > > > of
> > > >> > > > > > >> > two-phase
> > > >> > > > > > >> > > > > > commit
> > > >> > > > > > >> > > > > > >>> > (send
> > > >> > > > > > >> > > > > > >>> > > > out
> > > >> > > > > > >> > > > > > >>> > > > > > > StopReplicasRequest, get ACKs
> > > from
> > > >> all
> > > >> > > > > nodes,
> > > >> > > > > > >> > commit
> > > >> > > > > > >> > > by
> > > >> > > > > > >> > > > > > >>> removing
> > > >> > > > > > >> > > > > > >>> > > > > > > /admin/delete_topics node)
> any
> > > >> more.
> > > >> > > > > > >> > > > > > >>> > > > > > > > > You mention that
> > FetchRequest
> > > >> will
> > > >> > > now
> > > >> > > > > > >> include
> > > >> > > > > > >> > > UUID
> > > >> > > > > > >> > > > > to
> > > >> > > > > > >> > > > > > >>> avoid
> > > >> > > > > > >> > > > > > >>> > > > issues
> > > >> > > > > > >> > > > > > >>> > > > > > > where requests are made to
> > stale
> > > >> > > > partitions.
> > > >> > > > > > >> > > However,
> > > >> > > > > > >> > > > > > >>> adding a
> > > >> > > > > > >> > > > > > >>> > UUID
> > > >> > > > > > >> > > > > > >>> > > > to
> > > >> > > > > > >> > > > > > >>> > > > > > > MetadataRequest is listed as
> > > future
> > > >> > > work,
> > > >> > > > > out
> > > >> > > > > > of
> > > >> > > > > > >> > > scope
> > > >> > > > > > >> > > > > for
> > > >> > > > > > >> > > > > > >>> this
> > > >> > > > > > >> > > > > > >>> > KIP.
> > > >> > > > > > >> > > > > > >>> > > > > How
> > > >> > > > > > >> > > > > > >>> > > > > > > will the client learn what
> the
> > > >> topic
> > > >> > > UUID
> > > >> > > > > is,
> > > >> > > > > > if
> > > >> > > > > > >> > the
> > > >> > > > > > >> > > > > > metadata
> > > >> > > > > > >> > > > > > >>> > > > response
> > > >> > > > > > >> > > > > > >>> > > > > > > doesn't include that
> > information?
> > > >> It
> > > >> > > > seems
> > > >> > > > > > like
> > > >> > > > > > >> > > adding
> > > >> > > > > > >> > > > > the
> > > >> > > > > > >> > > > > > >>> UUID
> > > >> > > > > > >> > > > > > >>> > to
> > > >> > > > > > >> > > > > > >>> > > > > > > MetadataResponse would be an
> > > >> > improvement
> > > >> > > > > here
> > > >> > > > > > >> that
> > > >> > > > > > >> > > > might
> > > >> > > > > > >> > > > > > not
> > > >> > > > > > >> > > > > > >>> be
> > > >> > > > > > >> > > > > > >>> > too
> > > >> > > > > > >> > > > > > >>> > > > > hard to
> > > >> > > > > > >> > > > > > >>> > > > > > > make.
> > > >> > > > > > >> > > > > > >>> > > > > > > > > best,
> > > >> > > > > > >> > > > > > >>> > > > > > > > > Colin
> > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > >> > > > > > >> > > > > > >>> > > > > > > > >
> > > >> > > > > > >> > > > > > >>> > > > > > > > > On Mon, Sep 9, 2019, at
> > > 17:48,
> > > >> > > Ryanne
> > > >> > > > > > Dolan
> > > >> > > > > > >> > > wrote:
> > > >> > > > > > >> > > > > > >>> > > > > > > > > > Lucas, this would be
> > great.
> > > >> I've
> > > >> > > run
> > > >> > > > > > into
> > > >> > > > > > >> > > issues
> > > >> > > > > > >> > > > > with
> > > >> > > > > > >> > > > > > >>> > topics
> > > >> > > > > > >> > > > > > >>> > > > > being
> > > >> > > > > > >> > > > > > >>> > > > > > > > > > resurrected
> accidentally,
> > > >> since
> > > >> > a
> > > >> > > > > client
> > > >> > > > > > >> > cannot
> > > >> > > > > > >> > > > > > easily
> > > >> > > > > > >> > > > > > >>> > > > > distinguish
> > > >> > > > > > >> > > > > > >>> > > > > > > between
> > > >> > > > > > >> > > > > > >>> > > > > > > > > > a deleted topic and a
> new
> > > >> topic
> > > >> > > with
> > > >> > > > > the
> > > >> > > > > > >> same
> > > >> > > > > > >> > > > name.
> > > >> > > > > > >> > > > > > I'd
> > > >> > > > > > >> > > > > > >>> > need
> > > >> > > > > > >> > > > > > >>> > > > the
> > > >> > > > > > >> > > > > > >>> > > > > ID
> > > >> > > > > > >> > > > > > >>> > > > > > > > > > accessible from the
> > client
> > > to
> > > >> > > solve
> > > >> > > > > that
> > > >> > > > > > >> > issue,
> > > >> > > > > > >> > > > but
> > > >> > > > > > >> > > > > > >>> this
> > > >> > > > > > >> > > > > > >>> > is a
> > > >> > > > > > >> > > > > > >>> > > > > good
> > > >> > > > > > >> > > > > > >>> > > > > > > first
> > > >> > > > > > >> > > > > > >>> > > > > > > > > > step.
> > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> > > >> > > > > > >> > > > > > >>> > > > > > > > > > Ryanne
> > > >> > > > > > >> > > > > > >>> > > > > > > > > >
> > > >> > > > > > >> > > > > > >>> > > > > > > > > > On Wed, Sep 4, 2019 at
> > 1:41
> > > >> PM
> > > >> > > Lucas
> > > >> > > > > > >> > > Bradstreet <
> > > >> > > > > > >> > > > > > >>> > > > > lucas@confluent.io>
> > > >> > > > > > >> > > > > > >>> > > > > > > wrote:
> > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Hi all,
> > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > >> > > > > > >> > > > > > >>> > > > > > > > > > > I would like to kick
> > off
> > > >> > > > discussion
> > > >> > > > > of
> > > >> > > > > > >> > > KIP-516,
> > > >> > > > > > >> > > > > an
> > > >> > > > > > >> > > > > > >>> > > > > implementation
> > > >> > > > > > >> > > > > > >>> > > > > > > of topic
> > > >> > > > > > >> > > > > > >>> > > > > > > > > > > IDs for Kafka. Topic
> > IDs
> > > >> aim
> > > >> > to
> > > >> > > > > solve
> > > >> > > > > > >> topic
> > > >> > > > > > >> > > > > > >>> uniqueness
> > > >> > > > > > >> > > > > > >>> > > > > problems in
> > > >> > > > > > >> > > > > > >>> > > > > > > Kafka,
> > > >> > > > > > >> > > > > > >>> > > > > > > > > > > where referring to a
> > > topic
> > > >> by
> > > >> > > name
> > > >> > > > > > >> alone is
> > > >> > > > > > >> > > > > > >>> insufficient.
> > > >> > > > > > >> > > > > > >>> > > > Such
> > > >> > > > > > >> > > > > > >>> > > > > > > cases
> > > >> > > > > > >> > > > > > >>> > > > > > > > > > > include when a topic
> > has
> > > >> been
> > > >> > > > > deleted
> > > >> > > > > > >> and
> > > >> > > > > > >> > > > > recreated
> > > >> > > > > > >> > > > > > >>> with
> > > >> > > > > > >> > > > > > >>> > the
> > > >> > > > > > >> > > > > > >>> > > > > same
> > > >> > > > > > >> > > > > > >>> > > > > > > name.
> > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Unique identifiers
> will
> > > >> help
> > > >> > > > > simplify
> > > >> > > > > > >> and
> > > >> > > > > > >> > > > improve
> > > >> > > > > > >> > > > > > >>> Kafka's
> > > >> > > > > > >> > > > > > >>> > > > topic
> > > >> > > > > > >> > > > > > >>> > > > > > > deletion
> > > >> > > > > > >> > > > > > >>> > > > > > > > > > > process, as well as
> > > prevent
> > > >> > > cases
> > > >> > > > > > where
> > > >> > > > > > >> > > brokers
> > > >> > > > > > >> > > > > may
> > > >> > > > > > >> > > > > > >>> > > > incorrectly
> > > >> > > > > > >> > > > > > >>> > > > > > > interact
> > > >> > > > > > >> > > > > > >>> > > > > > > > > > > with stale versions
> of
> > > >> topics.
> > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > >> > > > > > >> > > > > > >>> > > >
> > > >> > > > > > >> > > > > > >>> >
> > > >> > > > > > >> > > > > > >>>
> > > >> > > > > > >> > > > > >
> > > >> > > > > > >> > > > >
> > > >> > > > > > >> > > >
> > > >> > > > > > >> > >
> > > >> > > > > > >> >
> > > >> > > > > > >>
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Looking forward to
> your
> > > >> > > thoughts.
> > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Lucas
> > > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > >> > > > > > >> > > > > > >>> >
> > > >> > > > > > >> > > > > > >>> >
> > > >> > > > > > >> > > > > > >>>
> > > >> > > > > > >> > > > > > >>
> > > >> > > > > > >> > > > > >
> > > >> > > > > > >> > > > >
> > > >> > > > > > >> > > >
> > > >> > > > > > >> > >
> > > >> > > > > > >> >
> > > >> > > > > > >>
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > > >
> > >
> >
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Jason Gustafson <ja...@confluent.io>]

Hi Justine,

Thanks for picking up this work. I have a few questions/comments:

1. It sounds like the directory structure is still going to be based on
topic names. Do I have that right? One complication is that the
LeaderAndIsr request does not include the topic name any longer. This means
that a replica must wait for the UpdateMetadata request to arrive with the
topic name mapping before it can create the directory. I wonder if it would
be simpler to avoid assumptions on the ordering of UpdateMetadata and let
LeaderAndIsr include the topic name as well. Feels like we are not saving
that much by excluding it.

2. On a related note, it seems that the reason we have the partition
metadata file is because we are not changing the directory structure. We
need it so that we remember which directories map to which topic id. I
think it would be useful to add some detail about why changing the
directory layout was rejected. Basically trying to understand how much
effort we are saving by skipping this step if we have to implement this
additional bookkeeping. One concern I have for example is that the
partition metadata file gets corrupt and then all bets are off as far as
topic consistency.

3. Is it worth adding support now for topic ids in the `Produce` request
now? Overhead is mentioned as one of the motivations and the best APIs to
get savings from are `Produce` and `Fetch`.

4. When it comes to bootstrapping the metadata topic with respect to
KIP-500, one suggestion would be to reserve a sentinel topic ID which can
be used initially by new brokers. When the first controller is elected, it
can assign a topicId to the metadata topic. This is a bit similar to how we
were planning to generate the clusterId.

Thanks,
Jason


On Thu, Sep 24, 2020 at 11:10 AM Jun Rao <ju...@confluent.io> wrote:

> Hi, Justine,
>
> Thanks for the updated KIP. A few more comments below.
>
> 30.  {"name": "id", "type": "string", "doc": "version id"}}: The doc should
> say UUID.
>
> 31. LeaderAndIsrResponse v5 and StopReplicaResponse v4 : It seems there is
> no need to add topic_id at partitions level.
>
> 32. Regarding partition metadata file. Perhaps the key can be a single
> word, sth like the following.
> version: 0
> topic_id: 46bdb63f-9e8d-4a38-bf7b-ee4eb2a794e4
>
> 33. Another tricky thing that I realized is how to support the metadata
> topic introduced in KIP-595. It's a bit tricky to assign a UUID to the
> metadata topic since we have a chicken-and-egg problem. The controller
> needs to persist the UUID in the metadata topic in order to assign one
> successfully, but the metadata topic is needed to elect a controller
> (KIP-631). So, this probably needs a bit more thought.
>
> Jun
>
> On Thu, Sep 24, 2020 at 3:04 AM Ismael Juma <is...@juma.me.uk> wrote:
>
> > Also, can we provide more details on how the Partition Metadata file will
> > be used?
> >
> > Ismael
> >
> > On Thu, Sep 24, 2020 at 3:01 AM Ismael Juma <is...@juma.me.uk> wrote:
> >
> > > Hi Justine,
> > >
> > > I think we need to update the "Rejected Alternatives" section to take
> > into
> > > account that the proposal now removes the topic name from the fetch
> > > request. Also, if we are removing it from the Fetch request, does it
> make
> > > sense not to remove it from similar requests like ListOffsetRequest?
> > >
> > > Ismael
> > >
> > > On Thu, Sep 24, 2020 at 2:46 AM David Jacot <dj...@confluent.io>
> wrote:
> > >
> > >> Hi Justine,
> > >>
> > >> Thanks for the KIP. I finally had time to read it :). It is a great
> > >> improvement.
> > >>
> > >> I have a few comments/questions:
> > >>
> > >> 1. It seems that the schema of the StopReplicaRequest is slightly
> > >> outdated.
> > >> We
> > >> did some changes as part of KIP-570. V3 is already organized by
> topics.
> > >>
> > >> 2. I just want to make sure that I understand the reconciliation
> > >> logic correctly. When
> > >> an "INCREMENTAL" LeaderAndIsr Request is received, the broker will
> also
> > >> reconcile
> > >> when the local uuid does not match the uuid in the request, right? In
> > this
> > >> case, the
> > >> local log is staged for deletion.
> > >>
> > >> 3. In the documentation of the `delete.stale.topic.delay.ms` config,
> it
> > >> says "When a
> > >> FULL LeaderAndIsrRequest is received..." but I suppose it applies to
> > both
> > >> types.
> > >>
> > >> Best,
> > >> David
> > >>
> > >> On Thu, Sep 24, 2020 at 1:40 AM Justine Olshan <jo...@confluent.io>
> > >> wrote:
> > >>
> > >> > Hi Jun,
> > >> >
> > >> > Thanks for the comments. I apologize for some of the typos and
> > >> confusion.
> > >> > I’ve updated the KIP to fix some of the issues you mentioned.
> > >> >
> > >> > 20.2 I’ve changed the type to String.
> > >> > 20.1/3 I’ve updated the TopicZNode to fix formatting and reflect the
> > >> latest
> > >> > version before this change.
> > >> >
> > >> > 21. You are correct and I’ve removed this line. I’ve also added a
> line
> > >> > mentioning an IBP bump is necessary for migration
> > >> >
> > >> > 22. I think the wording was unclear but your summary is what was
> > >> intended
> > >> > by this line. I’ve updated to make this point more clear. “Remove
> > >> deleted
> > >> > topics from replicas by sending StopReplicaRequest V3 before the IBP
> > >> bump
> > >> > using the old logic, and using V4 and the new logic with topic IDs
> > after
> > >> > the IBP bump.”
> > >> >
> > >> > 23. I’ve removed the unspecified type from the KIP and mention that
> > IBP
> > >> > will be used to handle this request. “IBP will be used to determine
> > >> whether
> > >> > this new form of the request will be used. For older requests, we
> will
> > >> > ignore this field and default to previous behavior.”
> > >> >
> > >> > 24. I’ve fixed this typo.
> > >> >
> > >> > 25. I've added a topics at a higher level for LeaderAndIsrResponse
> v5,
> > >> > StopReplicaResponse v4. I've also changed StopReplicaRequest v4 to
> > have
> > >> > topics at a higher level.
> > >> >
> > >> > 26. I’ve updated forgotten_topics_data--added the topic ID and
> removed
> > >> the
> > >> > topic name
> > >> >
> > >> > 27. I’ve decided on plain text, and I’ve added an example.
> > >> >
> > >> > 28. I’ve added this idea to future work.
> > >> >
> > >> > Thanks again for taking a look,
> > >> >
> > >> > Justine
> > >> >
> > >> > On Wed, Sep 23, 2020 at 10:28 AM Jun Rao <ju...@confluent.io> wrote:
> > >> >
> > >> > > Hi, Justine,
> > >> > >
> > >> > > Thanks for the response. Made another pass. A few more comments
> > below.
> > >> > >
> > >> > > 20. znode schema:
> > >> > > 20.1 It seems that {"name": "version", "type": "int", "id":
> "UUID",
> > >> > "doc":
> > >> > > "version id"} should be {"name": "version", "type": "int"},
> {"name":
> > >> > "id",
> > >> > > "type": "UUID", "doc": "version id"}.
> > >> > > 20.2 The znode format is JSON which doesn't have UUID type. So the
> > >> type
> > >> > > probably should be string?
> > >> > > 20.3 Also, the existing format used seems outdated. It should have
> > the
> > >> > > following format.
> > >> > >     Json.encodeAsBytes(Map(
> > >> > >       "version" -> 2,
> > >> > >       "partitions" -> replicaAssignmentJson.asJava,
> > >> > >       "adding_replicas" -> addingReplicasAssignmentJson.asJava,
> > >> > >       "removing_replicas" -> removingReplicasAssignmentJson.asJava
> > >> > >     ).asJava)
> > >> > >   }
> > >> > >
> > >> > > 21. Migration: The KIP says "The migration process can take place
> > >> without
> > >> > > an inter-broker protocol bump, as the format stored in
> > >> > > /brokers/topics/[topic] will be compatible with older broker
> > >> versions."
> > >> > > However, since we are bumping up the version of inter-broker
> > >> requests, it
> > >> > > seems that we need to use IBP for migration.
> > >> > >
> > >> > > 22. The KIP says "Remove deleted topics from replicas by sending
> > >> > > StopReplicaRequest V3 for any topics which do not contain a topic
> > ID,
> > >> and
> > >> > > V4 for any topics which do contain a topic ID.". However, if we
> use
> > >> IBP,
> > >> > it
> > >> > > seems that the controller will either send StopReplicaRequest V3
> > >> > > or StopReplicaRequest V4, but never mixed V3 and V4 for different
> > >> topics.
> > >> > > Basically, before the IBP bump, V3 will be used. After the IBP
> bump,
> > >> > > topicId will be created and V4 will be used.
> > >> > >
> > >> > > 23. Given that we depend on IBP, do we still need "0 UNSPECIFIED"
> > >> > > in LeaderAndIsr?
> > >> > >
> > >> > > 24. LeaderAndIsrResponse v5 : It still has the topic field.
> > >> > >
> > >> > > 25. LeaderAndIsrResponse v5, StopReplicaResponse v4: Could we use
> > this
> > >> > > opportunity to organize the response in 2 levels, first by topic,
> > >> then by
> > >> > > partition, as most other requests/responses?
> > >> > >
> > >> > > 26. FetchRequest v13 : Should forgotten_topics_data use topicId
> too?
> > >> > >
> > >> > > 27. "This file can either be plain text (key/value pairs) or
> JSON."
> > >> Have
> > >> > we
> > >> > > decided which one to use? Also, it would be helpful to provide an
> > >> > example.
> > >> > >
> > >> > > 28. Future improvement: Another future improvement opportunity is
> to
> > >> use
> > >> > > topicId in GroupMetadataManager.offsetCommitKey in the
> offset_commit
> > >> > topic.
> > >> > > This may save some space.
> > >> > >
> > >> > > Thanks,
> > >> > >
> > >> > > Jun
> > >> > >
> > >> > > On Wed, Sep 23, 2020 at 8:50 AM Justine Olshan <
> > jolshan@confluent.io>
> > >> > > wrote:
> > >> > >
> > >> > > > Hi Tom,
> > >> > > >
> > >> > > > Thanks for the comment. I think this is a really good idea and
> it
> > >> has
> > >> > > been
> > >> > > > added to the KIP under the newly added tooling section.
> > >> > > >
> > >> > > > Thanks again,
> > >> > > > Justine
> > >> > > >
> > >> > > > On Wed, Sep 23, 2020 at 3:17 AM Tom Bentley <
> tbentley@redhat.com>
> > >> > wrote:
> > >> > > >
> > >> > > > > Hi Justine,
> > >> > > > >
> > >> > > > > I know you started the vote thread, but on re-reading the KIP
> I
> > >> > noticed
> > >> > > > > that although the topic id is included in the MetadataResponse
> > >> it's
> > >> > not
> > >> > > > > surfaced in the output from `kafka-topics.sh --describe`.
> Maybe
> > >> that
> > >> > > was
> > >> > > > > intentional because ids are intentionally not really something
> > the
> > >> > user
> > >> > > > > should care deeply about, but it would also make life harder
> for
> > >> > anyone
> > >> > > > > debugging Kafka and this would likely get worse the more topic
> > ids
> > >> > got
> > >> > > > > rolled out across the protocols, clients etc. It seems likely
> > that
> > >> > > > > `kafka-topics.sh` will eventually need the ability to show the
> > id
> > >> of
> > >> > a
> > >> > > > > topic and perhaps find a topic name given an id. Is there any
> > >> reason
> > >> > > not
> > >> > > > to
> > >> > > > > implement that in this KIP?
> > >> > > > >
> > >> > > > > Many thanks,
> > >> > > > >
> > >> > > > > Tom
> > >> > > > >
> > >> > > > > On Mon, Sep 21, 2020 at 9:54 PM Justine Olshan <
> > >> jolshan@confluent.io
> > >> > >
> > >> > > > > wrote:
> > >> > > > >
> > >> > > > > > Hi all,
> > >> > > > > >
> > >> > > > > > After thinking about it, I've decided to remove the topic
> name
> > >> from
> > >> > > the
> > >> > > > > > Fetch Request and Response after all. Since there are so
> many
> > of
> > >> > > these
> > >> > > > > > requests per second, it is worth removing the extra
> > information.
> > >> > I've
> > >> > > > > > updated the KIP to reflect this change.
> > >> > > > > >
> > >> > > > > > Please let me know if there is anything else we should
> discuss
> > >> > before
> > >> > > > > > voting.
> > >> > > > > >
> > >> > > > > > Thank you,
> > >> > > > > > Justine
> > >> > > > > >
> > >> > > > > > On Fri, Sep 18, 2020 at 9:46 AM Justine Olshan <
> > >> > jolshan@confluent.io
> > >> > > >
> > >> > > > > > wrote:
> > >> > > > > >
> > >> > > > > > > Hi Jun,
> > >> > > > > > >
> > >> > > > > > > I see what you are saying. For now we can remove the extra
> > >> > > > information.
> > >> > > > > > > I'll leave the option to add more fields to the file in
> the
> > >> > future.
> > >> > > > The
> > >> > > > > > KIP
> > >> > > > > > > has been updated to reflect this change.
> > >> > > > > > >
> > >> > > > > > > Thanks,
> > >> > > > > > > Justine
> > >> > > > > > >
> > >> > > > > > > On Fri, Sep 18, 2020 at 8:46 AM Jun Rao <jun@confluent.io
> >
> > >> > wrote:
> > >> > > > > > >
> > >> > > > > > >> Hi, Justine,
> > >> > > > > > >>
> > >> > > > > > >> Thanks for the reply.
> > >> > > > > > >>
> > >> > > > > > >> 13. If the log directory is the source of truth, it means
> > >> that
> > >> > the
> > >> > > > > > >> redundant info in the metadata file will be ignored. Then
> > the
> > >> > > > question
> > >> > > > > > is
> > >> > > > > > >> why do we need to put the redundant info in the metadata
> > file
> > >> > now?
> > >> > > > > > >>
> > >> > > > > > >> Thanks,
> > >> > > > > > >>
> > >> > > > > > >> Jun
> > >> > > > > > >>
> > >> > > > > > >> On Thu, Sep 17, 2020 at 5:07 PM Justine Olshan <
> > >> > > > jolshan@confluent.io>
> > >> > > > > > >> wrote:
> > >> > > > > > >>
> > >> > > > > > >> > Hi Jun,
> > >> > > > > > >> > Thanks for the quick response!
> > >> > > > > > >> >
> > >> > > > > > >> > 12. I've decided to bump up the versions on the
> requests
> > >> and
> > >> > > > updated
> > >> > > > > > the
> > >> > > > > > >> > KIP. I think it's good we thoroughly discussed the
> > options
> > >> > here,
> > >> > > > so
> > >> > > > > we
> > >> > > > > > >> know
> > >> > > > > > >> > we made a good choice. :)
> > >> > > > > > >> >
> > >> > > > > > >> > 13. This is an interesting situation. I think if this
> > does
> > >> > occur
> > >> > > > we
> > >> > > > > > >> should
> > >> > > > > > >> > give a warning. I agree that it's hard to know the
> source
> > >> of
> > >> > > truth
> > >> > > > > for
> > >> > > > > > >> sure
> > >> > > > > > >> > since the directory or the file could be manually
> > >> modified. I
> > >> > > > guess
> > >> > > > > > the
> > >> > > > > > >> > directory could be used as the source of truth. To be
> > >> honest,
> > >> > > I'm
> > >> > > > > not
> > >> > > > > > >> > really sure what happens in kafka when the log
> directory
> > is
> > >> > > > renamed
> > >> > > > > > >> > manually in such a way. I'm also wondering if the
> > >> situation is
> > >> > > > > > >> recoverable
> > >> > > > > > >> > in this scenario.
> > >> > > > > > >> >
> > >> > > > > > >> > Thanks,
> > >> > > > > > >> > Justine
> > >> > > > > > >> >
> > >> > > > > > >> > On Thu, Sep 17, 2020 at 4:28 PM Jun Rao <
> > jun@confluent.io>
> > >> > > wrote:
> > >> > > > > > >> >
> > >> > > > > > >> > > Hi, Justine,
> > >> > > > > > >> > >
> > >> > > > > > >> > > Thanks for the reply.
> > >> > > > > > >> > >
> > >> > > > > > >> > > 12. I don't have a strong preference either. However,
> > if
> > >> we
> > >> > > need
> > >> > > > > IBP
> > >> > > > > > >> > > anyway, maybe it's easier to just bump up the version
> > for
> > >> > all
> > >> > > > > inter
> > >> > > > > > >> > broker
> > >> > > > > > >> > > requests and add the topic id field as a regular
> > field. A
> > >> > > > regular
> > >> > > > > > >> field
> > >> > > > > > >> > is
> > >> > > > > > >> > > a bit more concise in wire transfer than a flexible
> > >> field.
> > >> > > > > > >> > >
> > >> > > > > > >> > > 13. The confusion that I was referring to is between
> > the
> > >> > topic
> > >> > > > > name
> > >> > > > > > >> and
> > >> > > > > > >> > > partition number between the log dir and the metadata
> > >> file.
> > >> > > For
> > >> > > > > > >> example,
> > >> > > > > > >> > if
> > >> > > > > > >> > > the log dir is topicA-1 and the metadata file in it
> has
> > >> > topicB
> > >> > > > and
> > >> > > > > > >> > > partition 0 (say due to a bug or manual
> modification),
> > >> which
> > >> > > one
> > >> > > > > do
> > >> > > > > > we
> > >> > > > > > >> > use
> > >> > > > > > >> > > as the source of truth?
> > >> > > > > > >> > >
> > >> > > > > > >> > > Jun
> > >> > > > > > >> > >
> > >> > > > > > >> > > On Thu, Sep 17, 2020 at 3:43 PM Justine Olshan <
> > >> > > > > > jolshan@confluent.io>
> > >> > > > > > >> > > wrote:
> > >> > > > > > >> > >
> > >> > > > > > >> > > > Hi Jun,
> > >> > > > > > >> > > > Thanks for the comments.
> > >> > > > > > >> > > >
> > >> > > > > > >> > > > 12. I bumped the LeaderAndIsrRequest because I
> > removed
> > >> the
> > >> > > > topic
> > >> > > > > > >> name
> > >> > > > > > >> > > field
> > >> > > > > > >> > > > in the response. It may be possible to avoid
> bumping
> > >> the
> > >> > > > version
> > >> > > > > > >> > without
> > >> > > > > > >> > > > that change, but I may be missing something.
> > >> > > > > > >> > > > I believe StopReplica is actually on version 3 now,
> > but
> > >> > > > because
> > >> > > > > > >> > version 2
> > >> > > > > > >> > > > is flexible, I kept that listed as version 2 on the
> > KIP
> > >> > > page.
> > >> > > > > > >> However,
> > >> > > > > > >> > > you
> > >> > > > > > >> > > > may be right in that we may need to bump the
> version
> > on
> > >> > > > > > StopReplica
> > >> > > > > > >> to
> > >> > > > > > >> > > deal
> > >> > > > > > >> > > > with deletion differently as mentioned above. I
> don't
> > >> know
> > >> > > if
> > >> > > > I
> > >> > > > > > >> have a
> > >> > > > > > >> > > big
> > >> > > > > > >> > > > preference over used tagged fields or not.
> > >> > > > > > >> > > >
> > >> > > > > > >> > > > 13. I was thinking that in the case where the file
> > and
> > >> the
> > >> > > > > request
> > >> > > > > > >> > topic
> > >> > > > > > >> > > > ids don't match, it means that the broker's
> topic/the
> > >> one
> > >> > in
> > >> > > > the
> > >> > > > > > >> file
> > >> > > > > > >> > has
> > >> > > > > > >> > > > been deleted. In that case, we would need to delete
> > the
> > >> > old
> > >> > > > > topic
> > >> > > > > > >> and
> > >> > > > > > >> > > start
> > >> > > > > > >> > > > receiving the new version. If the topic name were
> to
> > >> > change,
> > >> > > > but
> > >> > > > > > the
> > >> > > > > > >> > ids
> > >> > > > > > >> > > > still matched, the file would also need to update.
> > Am I
> > >> > > > missing
> > >> > > > > a
> > >> > > > > > >> case
> > >> > > > > > >> > > > where the file would be correct and not the
> request?
> > >> > > > > > >> > > >
> > >> > > > > > >> > > > Thanks,
> > >> > > > > > >> > > > Justine
> > >> > > > > > >> > > >
> > >> > > > > > >> > > > On Thu, Sep 17, 2020 at 3:18 PM Jun Rao <
> > >> jun@confluent.io
> > >> > >
> > >> > > > > wrote:
> > >> > > > > > >> > > >
> > >> > > > > > >> > > > > Hi, Justine,
> > >> > > > > > >> > > > >
> > >> > > > > > >> > > > > Thanks for the reply. A couple of more comments
> > >> below.
> > >> > > > > > >> > > > >
> > >> > > > > > >> > > > > 12. ListOffset and OffsetForLeader currently
> don't
> > >> > support
> > >> > > > > > >> flexible
> > >> > > > > > >> > > > fields.
> > >> > > > > > >> > > > > So, we have to bump up the version number and use
> > >> IBP at
> > >> > > > least
> > >> > > > > > for
> > >> > > > > > >> > > these
> > >> > > > > > >> > > > > two requests. Note that it seems 2.7.0 will
> require
> > >> IBP
> > >> > > > anyway
> > >> > > > > > >> > because
> > >> > > > > > >> > > of
> > >> > > > > > >> > > > > changes in KAFKA-10435. Also, it seems that the
> > >> version
> > >> > > for
> > >> > > > > > >> > > > > LeaderAndIsrRequest and StopReplica are bumped
> even
> > >> > though
> > >> > > > we
> > >> > > > > > only
> > >> > > > > > >> > > added
> > >> > > > > > >> > > > a
> > >> > > > > > >> > > > > tagged field. But since IBP is needed anyway, we
> > may
> > >> > want
> > >> > > to
> > >> > > > > > >> revisit
> > >> > > > > > >> > > the
> > >> > > > > > >> > > > > overall tagged field choice.
> > >> > > > > > >> > > > >
> > >> > > > > > >> > > > > 13. The only downside is the potential confusion
> on
> > >> > which
> > >> > > > one
> > >> > > > > is
> > >> > > > > > >> the
> > >> > > > > > >> > > > source
> > >> > > > > > >> > > > > of truth if they don't match. Another option is
> to
> > >> > include
> > >> > > > > those
> > >> > > > > > >> > fields
> > >> > > > > > >> > > > in
> > >> > > > > > >> > > > > the metadata file when we actually change the
> > >> directory
> > >> > > > > > structure.
> > >> > > > > > >> > > > >
> > >> > > > > > >> > > > > Thanks,
> > >> > > > > > >> > > > >
> > >> > > > > > >> > > > > Jun
> > >> > > > > > >> > > > >
> > >> > > > > > >> > > > > On Thu, Sep 17, 2020 at 2:01 PM Justine Olshan <
> > >> > > > > > >> jolshan@confluent.io
> > >> > > > > > >> > >
> > >> > > > > > >> > > > > wrote:
> > >> > > > > > >> > > > >
> > >> > > > > > >> > > > > > Hello all,
> > >> > > > > > >> > > > > >
> > >> > > > > > >> > > > > > I've thought some more about removing the topic
> > >> name
> > >> > > field
> > >> > > > > > from
> > >> > > > > > >> > some
> > >> > > > > > >> > > of
> > >> > > > > > >> > > > > the
> > >> > > > > > >> > > > > > requests. On closer inspection of the
> > >> > > requests/responses,
> > >> > > > it
> > >> > > > > > >> seems
> > >> > > > > > >> > > that
> > >> > > > > > >> > > > > the
> > >> > > > > > >> > > > > > internal changes would be much larger than I
> > >> expected.
> > >> > > > Some
> > >> > > > > > >> > protocols
> > >> > > > > > >> > > > > > involve clients, so they would require changes
> > too.
> > >> > I'm
> > >> > > > > > thinking
> > >> > > > > > >> > that
> > >> > > > > > >> > > > for
> > >> > > > > > >> > > > > > now, removing the topic name from these
> requests
> > >> and
> > >> > > > > responses
> > >> > > > > > >> are
> > >> > > > > > >> > > out
> > >> > > > > > >> > > > of
> > >> > > > > > >> > > > > > scope.
> > >> > > > > > >> > > > > >
> > >> > > > > > >> > > > > > I have decided to just keep the change
> > >> > > > LeaderAndIsrResponse
> > >> > > > > to
> > >> > > > > > >> > remove
> > >> > > > > > >> > > > the
> > >> > > > > > >> > > > > > topic name, and have updated the KIP to reflect
> > >> this
> > >> > > > > change. I
> > >> > > > > > >> have
> > >> > > > > > >> > > > also
> > >> > > > > > >> > > > > > mentioned the other requests and responses in
> > >> future
> > >> > > work.
> > >> > > > > > >> > > > > >
> > >> > > > > > >> > > > > > I'm hoping to start the voting process soon, so
> > >> let me
> > >> > > > know
> > >> > > > > if
> > >> > > > > > >> > there
> > >> > > > > > >> > > is
> > >> > > > > > >> > > > > > anything else we should discuss.
> > >> > > > > > >> > > > > >
> > >> > > > > > >> > > > > > Thank you,
> > >> > > > > > >> > > > > > Justine
> > >> > > > > > >> > > > > >
> > >> > > > > > >> > > > > > On Tue, Sep 15, 2020 at 3:57 PM Justine Olshan
> <
> > >> > > > > > >> > jolshan@confluent.io
> > >> > > > > > >> > > >
> > >> > > > > > >> > > > > > wrote:
> > >> > > > > > >> > > > > >
> > >> > > > > > >> > > > > > > Hello again,
> > >> > > > > > >> > > > > > > To follow up on some of the other comments:
> > >> > > > > > >> > > > > > >
> > >> > > > > > >> > > > > > > 10/11) We can remove the topic name from
> these
> > >> > > > > > >> > requests/responses,
> > >> > > > > > >> > > > and
> > >> > > > > > >> > > > > > > that means we will just have to make a few
> > >> internal
> > >> > > > > changes
> > >> > > > > > to
> > >> > > > > > >> > make
> > >> > > > > > >> > > > > > > partitions accessible by topic id and
> > partition.
> > >> I
> > >> > can
> > >> > > > > > update
> > >> > > > > > >> the
> > >> > > > > > >> > > KIP
> > >> > > > > > >> > > > > to
> > >> > > > > > >> > > > > > > remove them unless anyone thinks they should
> > >> stay.
> > >> > > > > > >> > > > > > >
> > >> > > > > > >> > > > > > > 12) Addressed in the previous email. I've
> > updated
> > >> > the
> > >> > > > KIP
> > >> > > > > to
> > >> > > > > > >> > > include
> > >> > > > > > >> > > > > > > tagged fields for the requests and responses.
> > >> (More
> > >> > on
> > >> > > > > that
> > >> > > > > > >> > below)
> > >> > > > > > >> > > > > > >
> > >> > > > > > >> > > > > > > 13) I think part of the idea for including
> this
> > >> > > > > information
> > >> > > > > > >> is to
> > >> > > > > > >> > > > > prepare
> > >> > > > > > >> > > > > > > for future changes. Perhaps the directory
> > >> structure
> > >> > > > might
> > >> > > > > > >> change
> > >> > > > > > >> > > from
> > >> > > > > > >> > > > > > > topicName_partitionNumber to something like
> > >> > > > > > >> > > topicID_partitionNumber.
> > >> > > > > > >> > > > > Then
> > >> > > > > > >> > > > > > > it would be useful to have the topic name in
> > the
> > >> > file
> > >> > > > > since
> > >> > > > > > it
> > >> > > > > > >> > > would
> > >> > > > > > >> > > > > not
> > >> > > > > > >> > > > > > be
> > >> > > > > > >> > > > > > > in the directory structure. Supporting topic
> > >> renames
> > >> > > > might
> > >> > > > > > be
> > >> > > > > > >> > > easier
> > >> > > > > > >> > > > if
> > >> > > > > > >> > > > > > the
> > >> > > > > > >> > > > > > > other fields are included. Would there be any
> > >> > > downsides
> > >> > > > to
> > >> > > > > > >> > > including
> > >> > > > > > >> > > > > this
> > >> > > > > > >> > > > > > > information?
> > >> > > > > > >> > > > > > >
> > >> > > > > > >> > > > > > > 14)  Yes, we would need to copy the partition
> > >> > metadata
> > >> > > > > file
> > >> > > > > > in
> > >> > > > > > >> > this
> > >> > > > > > >> > > > > > > process. I've updated the KIP to include
> this.
> > >> > > > > > >> > > > > > >
> > >> > > > > > >> > > > > > > 15) I believe Lucas meant v1 and v2 here. He
> > was
> > >> > > > referring
> > >> > > > > > to
> > >> > > > > > >> how
> > >> > > > > > >> > > the
> > >> > > > > > >> > > > > > > requests would fall under different IBP and
> > meant
> > >> > that
> > >> > > > > older
> > >> > > > > > >> > > brokers
> > >> > > > > > >> > > > > > would
> > >> > > > > > >> > > > > > > have to use the older version of the request
> > and
> > >> the
> > >> > > > > > existing
> > >> > > > > > >> > topic
> > >> > > > > > >> > > > > > > deletion process. At first, it seemed like
> > tagged
> > >> > > fields
> > >> > > > > > would
> > >> > > > > > >> > > > resolve
> > >> > > > > > >> > > > > > > the IBP issue. However, we may need IBP for
> > this
> > >> > > request
> > >> > > > > > after
> > >> > > > > > >> > all
> > >> > > > > > >> > > > > since
> > >> > > > > > >> > > > > > > the controller handles the topic deletion
> > >> > differently
> > >> > > > > > >> depending
> > >> > > > > > >> > on
> > >> > > > > > >> > > > the
> > >> > > > > > >> > > > > > IBP
> > >> > > > > > >> > > > > > > version. In an older version, we can't just
> > send
> > >> a
> > >> > > > > > StopReplica
> > >> > > > > > >> > > delete
> > >> > > > > > >> > > > > the
> > >> > > > > > >> > > > > > > topic immediately like we'd want to for this
> > KIP.
> > >> > > > > > >> > > > > > >
> > >> > > > > > >> > > > > > > This makes me wonder if we want tagged fields
> > on
> > >> all
> > >> > > the
> > >> > > > > > >> requests
> > >> > > > > > >> > > > after
> > >> > > > > > >> > > > > > > all. Let me know your thoughts!
> > >> > > > > > >> > > > > > >
> > >> > > > > > >> > > > > > > Justine
> > >> > > > > > >> > > > > > >
> > >> > > > > > >> > > > > > > On Tue, Sep 15, 2020 at 1:03 PM Justine
> Olshan
> > <
> > >> > > > > > >> > > jolshan@confluent.io
> > >> > > > > > >> > > > >
> > >> > > > > > >> > > > > > > wrote:
> > >> > > > > > >> > > > > > >
> > >> > > > > > >> > > > > > >> Hi all,
> > >> > > > > > >> > > > > > >> Jun brought up a good point in his last
> email
> > >> about
> > >> > > > > tagged
> > >> > > > > > >> > fields,
> > >> > > > > > >> > > > and
> > >> > > > > > >> > > > > > >> I've updated the KIP to reflect that the
> > >> changes to
> > >> > > > > > requests
> > >> > > > > > >> and
> > >> > > > > > >> > > > > > responses
> > >> > > > > > >> > > > > > >> will be in the form of tagged fields to
> avoid
> > >> > > changing
> > >> > > > > IBP.
> > >> > > > > > >> > > > > > >>
> > >> > > > > > >> > > > > > >> Jun: I plan on sending a followup email to
> > >> address
> > >> > > some
> > >> > > > > of
> > >> > > > > > >> the
> > >> > > > > > >> > > other
> > >> > > > > > >> > > > > > >> points.
> > >> > > > > > >> > > > > > >>
> > >> > > > > > >> > > > > > >> Thanks,
> > >> > > > > > >> > > > > > >> Justine
> > >> > > > > > >> > > > > > >>
> > >> > > > > > >> > > > > > >> On Mon, Sep 14, 2020 at 4:25 PM Jun Rao <
> > >> > > > > jun@confluent.io>
> > >> > > > > > >> > wrote:
> > >> > > > > > >> > > > > > >>
> > >> > > > > > >> > > > > > >>> Hi, Justine,
> > >> > > > > > >> > > > > > >>>
> > >> > > > > > >> > > > > > >>> Thanks for the updated KIP. A few comments
> > >> below.
> > >> > > > > > >> > > > > > >>>
> > >> > > > > > >> > > > > > >>> 10. LeaderAndIsr Response: Do we need the
> > topic
> > >> > > name?
> > >> > > > > > >> > > > > > >>>
> > >> > > > > > >> > > > > > >>> 11. For the changed request/response, other
> > >> than
> > >> > > > > > >> LeaderAndIsr,
> > >> > > > > > >> > > > > > >>> UpdateMetadata, Metadata, do we need to
> > include
> > >> > the
> > >> > > > > topic
> > >> > > > > > >> name?
> > >> > > > > > >> > > > > > >>>
> > >> > > > > > >> > > > > > >>> 12. It seems that upgrades don't require
> IBP.
> > >> Does
> > >> > > > that
> > >> > > > > > mean
> > >> > > > > > >> > the
> > >> > > > > > >> > > > new
> > >> > > > > > >> > > > > > >>> fields
> > >> > > > > > >> > > > > > >>> in all the request/response are added as
> > tagged
> > >> > > fields
> > >> > > > > > >> without
> > >> > > > > > >> > > > > bumping
> > >> > > > > > >> > > > > > up
> > >> > > > > > >> > > > > > >>> the request version? It would be useful to
> > make
> > >> > that
> > >> > > > > > clear.
> > >> > > > > > >> > > > > > >>>
> > >> > > > > > >> > > > > > >>> 13. Partition Metadata file: Do we need to
> > >> include
> > >> > > the
> > >> > > > > > topic
> > >> > > > > > >> > name
> > >> > > > > > >> > > > and
> > >> > > > > > >> > > > > > the
> > >> > > > > > >> > > > > > >>> partition id since they are implied in the
> > >> > directory
> > >> > > > > name?
> > >> > > > > > >> > > > > > >>>
> > >> > > > > > >> > > > > > >>> 14. In the JBOD mode, we support moving a
> > >> > > partition's
> > >> > > > > data
> > >> > > > > > >> from
> > >> > > > > > >> > > one
> > >> > > > > > >> > > > > > disk
> > >> > > > > > >> > > > > > >>> to
> > >> > > > > > >> > > > > > >>> another. Will the new partition metadata
> file
> > >> be
> > >> > > > copied
> > >> > > > > > >> during
> > >> > > > > > >> > > that
> > >> > > > > > >> > > > > > >>> process?
> > >> > > > > > >> > > > > > >>>
> > >> > > > > > >> > > > > > >>> 15. The KIP says "Remove deleted topics
> from
> > >> > > replicas
> > >> > > > by
> > >> > > > > > >> > sending
> > >> > > > > > >> > > > > > >>> StopReplicaRequest V2 for any topics which
> do
> > >> not
> > >> > > > > contain
> > >> > > > > > a
> > >> > > > > > >> > topic
> > >> > > > > > >> > > > ID,
> > >> > > > > > >> > > > > > and
> > >> > > > > > >> > > > > > >>> V3 for any topics which do contain a topic
> > >> ID.".
> > >> > > > > However,
> > >> > > > > > it
> > >> > > > > > >> > > seems
> > >> > > > > > >> > > > > the
> > >> > > > > > >> > > > > > >>> updated controller will create all missing
> > >> topic
> > >> > IDs
> > >> > > > > first
> > >> > > > > > >> > before
> > >> > > > > > >> > > > > doing
> > >> > > > > > >> > > > > > >>> other actions. So, is StopReplicaRequest V2
> > >> > needed?
> > >> > > > > > >> > > > > > >>>
> > >> > > > > > >> > > > > > >>> Jun
> > >> > > > > > >> > > > > > >>>
> > >> > > > > > >> > > > > > >>> On Fri, Sep 11, 2020 at 10:31 AM John
> > Roesler <
> > >> > > > > > >> > > vvcephei@apache.org
> > >> > > > > > >> > > > >
> > >> > > > > > >> > > > > > >>> wrote:
> > >> > > > > > >> > > > > > >>>
> > >> > > > > > >> > > > > > >>> > Thanks, Justine!
> > >> > > > > > >> > > > > > >>> >
> > >> > > > > > >> > > > > > >>> > Your response seems compelling to me.
> > >> > > > > > >> > > > > > >>> >
> > >> > > > > > >> > > > > > >>> > -John
> > >> > > > > > >> > > > > > >>> >
> > >> > > > > > >> > > > > > >>> > On Fri, 2020-09-11 at 10:17 -0700,
> Justine
> > >> > Olshan
> > >> > > > > wrote:
> > >> > > > > > >> > > > > > >>> > > Hello all,
> > >> > > > > > >> > > > > > >>> > > Thanks for continuing the discussion! I
> > >> have a
> > >> > > few
> > >> > > > > > >> > responses
> > >> > > > > > >> > > to
> > >> > > > > > >> > > > > > your
> > >> > > > > > >> > > > > > >>> > points.
> > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > >> > > > > > >>> > > Tom: You are correct in that this KIP
> has
> > >> not
> > >> > > > > > mentioned
> > >> > > > > > >> the
> > >> > > > > > >> > > > > > >>> > > DeleteTopicsRequest. I think that this
> > >> would
> > >> > be
> > >> > > > out
> > >> > > > > of
> > >> > > > > > >> > scope
> > >> > > > > > >> > > > for
> > >> > > > > > >> > > > > > >>> now, but
> > >> > > > > > >> > > > > > >>> > > may be something worth adding in the
> > >> future.
> > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > >> > > > > > >>> > > John: We did consider sequence ids, but
> > >> there
> > >> > > are
> > >> > > > a
> > >> > > > > > few
> > >> > > > > > >> > > reasons
> > >> > > > > > >> > > > > to
> > >> > > > > > >> > > > > > >>> favor
> > >> > > > > > >> > > > > > >>> > > UUIDs. There are several cases where
> > topics
> > >> > from
> > >> > > > > > >> different
> > >> > > > > > >> > > > > clusters
> > >> > > > > > >> > > > > > >>> may
> > >> > > > > > >> > > > > > >>> > > interact now and in the future. For
> > >> example,
> > >> > > > Mirror
> > >> > > > > > >> Maker 2
> > >> > > > > > >> > > may
> > >> > > > > > >> > > > > > >>> benefit
> > >> > > > > > >> > > > > > >>> > > from being able to detect when a
> cluster
> > >> being
> > >> > > > > > mirrored
> > >> > > > > > >> is
> > >> > > > > > >> > > > > deleted
> > >> > > > > > >> > > > > > >>> and
> > >> > > > > > >> > > > > > >>> > > recreated and globally unique
> identifiers
> > >> > would
> > >> > > > make
> > >> > > > > > >> > > resolving
> > >> > > > > > >> > > > > > issues
> > >> > > > > > >> > > > > > >>> > > easier than sequence IDs which may
> > collide
> > >> > > between
> > >> > > > > > >> > clusters.
> > >> > > > > > >> > > > > > KIP-405
> > >> > > > > > >> > > > > > >>> > > (tiered storage) will also benefit from
> > >> > globally
> > >> > > > > > unique
> > >> > > > > > >> IDs
> > >> > > > > > >> > > as
> > >> > > > > > >> > > > > > shared
> > >> > > > > > >> > > > > > >>> > > buckets may be used between clusters.
> > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > >> > > > > > >>> > > Globally unique IDs would also make
> > >> > > functionality
> > >> > > > > like
> > >> > > > > > >> > moving
> > >> > > > > > >> > > > > > topics
> > >> > > > > > >> > > > > > >>> > > between disparate clusters easier in
> the
> > >> > future,
> > >> > > > > > >> simplify
> > >> > > > > > >> > any
> > >> > > > > > >> > > > > > future
> > >> > > > > > >> > > > > > >>> > > implementations of backups and
> restores,
> > >> and
> > >> > > more.
> > >> > > > > In
> > >> > > > > > >> > > general,
> > >> > > > > > >> > > > > > >>> unique IDs
> > >> > > > > > >> > > > > > >>> > > would ensure that the source cluster
> > >> topics do
> > >> > > not
> > >> > > > > > >> conflict
> > >> > > > > > >> > > > with
> > >> > > > > > >> > > > > > the
> > >> > > > > > >> > > > > > >>> > > destination cluster topics.
> > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > >> > > > > > >>> > > If we were to use sequence ids, we
> would
> > >> need
> > >> > > > > > >> sufficiently
> > >> > > > > > >> > > > large
> > >> > > > > > >> > > > > > >>> cluster
> > >> > > > > > >> > > > > > >>> > > ids to be stored with the topic
> > >> identifiers or
> > >> > > we
> > >> > > > > run
> > >> > > > > > >> the
> > >> > > > > > >> > > risk
> > >> > > > > > >> > > > of
> > >> > > > > > >> > > > > > >>> > > collisions. This will give up any
> > >> advantage in
> > >> > > > > > >> compactness
> > >> > > > > > >> > > that
> > >> > > > > > >> > > > > > >>> sequence
> > >> > > > > > >> > > > > > >>> > > numbers may bring. Given these
> > advantages I
> > >> > > think
> > >> > > > it
> > >> > > > > > >> makes
> > >> > > > > > >> > > > sense
> > >> > > > > > >> > > > > to
> > >> > > > > > >> > > > > > >>> use
> > >> > > > > > >> > > > > > >>> > > UUIDs.
> > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > >> > > > > > >>> > > Gokul: This is an interesting idea, but
> > >> this
> > >> > is
> > >> > > a
> > >> > > > > > >> breaking
> > >> > > > > > >> > > > > change.
> > >> > > > > > >> > > > > > >>> Out of
> > >> > > > > > >> > > > > > >>> > > scope for now, but maybe worth
> discussing
> > >> in
> > >> > the
> > >> > > > > > future.
> > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > >> > > > > > >>> > > Hope this explains some of the
> decisions,
> > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > >> > > > > > >>> > > Justine
> > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > >> > > > > > >>> > > On Fri, Sep 11, 2020 at 8:27 AM Gokul
> > >> Ramanan
> > >> > > > > > >> Subramanian <
> > >> > > > > > >> > > > > > >>> > > gokul2411s@gmail.com> wrote:
> > >> > > > > > >> > > > > > >>> > >
> > >> > > > > > >> > > > > > >>> > > > Hi.
> > >> > > > > > >> > > > > > >>> > > >
> > >> > > > > > >> > > > > > >>> > > > Thanks for the KIP.
> > >> > > > > > >> > > > > > >>> > > >
> > >> > > > > > >> > > > > > >>> > > > Have you thought about whether it
> makes
> > >> > sense
> > >> > > to
> > >> > > > > > >> support
> > >> > > > > > >> > > > > > >>> authorizing a
> > >> > > > > > >> > > > > > >>> > > > principal for a topic ID rather than
> a
> > >> topic
> > >> > > > name
> > >> > > > > to
> > >> > > > > > >> > > achieve
> > >> > > > > > >> > > > > > >>> tighter
> > >> > > > > > >> > > > > > >>> > > > security?
> > >> > > > > > >> > > > > > >>> > > >
> > >> > > > > > >> > > > > > >>> > > > Or is the topic ID fundamentally an
> > >> internal
> > >> > > > > detail
> > >> > > > > > >> > similar
> > >> > > > > > >> > > > to
> > >> > > > > > >> > > > > > >>> epochs
> > >> > > > > > >> > > > > > >>> > used
> > >> > > > > > >> > > > > > >>> > > > in a bunch of other places in Kafka?
> > >> > > > > > >> > > > > > >>> > > >
> > >> > > > > > >> > > > > > >>> > > > Thanks.
> > >> > > > > > >> > > > > > >>> > > >
> > >> > > > > > >> > > > > > >>> > > > On Fri, Sep 11, 2020 at 4:06 PM John
> > >> > Roesler <
> > >> > > > > > >> > > > > > vvcephei@apache.org>
> > >> > > > > > >> > > > > > >>> > wrote:
> > >> > > > > > >> > > > > > >>> > > >
> > >> > > > > > >> > > > > > >>> > > > > Hello Justine,
> > >> > > > > > >> > > > > > >>> > > > >
> > >> > > > > > >> > > > > > >>> > > > > Thanks for the KIP!
> > >> > > > > > >> > > > > > >>> > > > >
> > >> > > > > > >> > > > > > >>> > > > > I happen to have been confronted
> > >> recently
> > >> > > with
> > >> > > > > the
> > >> > > > > > >> need
> > >> > > > > > >> > > to
> > >> > > > > > >> > > > > keep
> > >> > > > > > >> > > > > > >>> > track of
> > >> > > > > > >> > > > > > >>> > > > a
> > >> > > > > > >> > > > > > >>> > > > > large number of topics as compactly
> > as
> > >> > > > > possible. I
> > >> > > > > > >> was
> > >> > > > > > >> > > > going
> > >> > > > > > >> > > > > to
> > >> > > > > > >> > > > > > >>> come
> > >> > > > > > >> > > > > > >>> > up
> > >> > > > > > >> > > > > > >>> > > > > with some way to dictionary encode
> > the
> > >> > topic
> > >> > > > > names
> > >> > > > > > >> as
> > >> > > > > > >> > > > > integers,
> > >> > > > > > >> > > > > > >>> but
> > >> > > > > > >> > > > > > >>> > this
> > >> > > > > > >> > > > > > >>> > > > > seems much better!
> > >> > > > > > >> > > > > > >>> > > > >
> > >> > > > > > >> > > > > > >>> > > > > Apologies if this has been raised
> > >> before,
> > >> > > but
> > >> > > > > I’m
> > >> > > > > > >> > > wondering
> > >> > > > > > >> > > > > > >>> about the
> > >> > > > > > >> > > > > > >>> > > > > choice of UUID vs sequence number
> for
> > >> the
> > >> > > ids.
> > >> > > > > > >> > Typically,
> > >> > > > > > >> > > > > I’ve
> > >> > > > > > >> > > > > > >>> seen
> > >> > > > > > >> > > > > > >>> > UUIDs
> > >> > > > > > >> > > > > > >>> > > > > in two situations:
> > >> > > > > > >> > > > > > >>> > > > > 1. When processes need to generate
> > >> > > > non-colliding
> > >> > > > > > >> > > > identifiers
> > >> > > > > > >> > > > > > >>> without
> > >> > > > > > >> > > > > > >>> > > > > coordination.
> > >> > > > > > >> > > > > > >>> > > > > 2. When the identifier needs to be
> > >> > > > “universally
> > >> > > > > > >> > unique”;
> > >> > > > > > >> > > > > I.e.,
> > >> > > > > > >> > > > > > >>> the
> > >> > > > > > >> > > > > > >>> > > > > identifier needs to distinguish the
> > >> entity
> > >> > > > from
> > >> > > > > > all
> > >> > > > > > >> > other
> > >> > > > > > >> > > > > > >>> entities
> > >> > > > > > >> > > > > > >>> > that
> > >> > > > > > >> > > > > > >>> > > > > could ever exist. This is useful in
> > >> cases
> > >> > > > where
> > >> > > > > > >> > entities
> > >> > > > > > >> > > > from
> > >> > > > > > >> > > > > > all
> > >> > > > > > >> > > > > > >>> > kinds
> > >> > > > > > >> > > > > > >>> > > > of
> > >> > > > > > >> > > > > > >>> > > > > systems get mixed together, such as
> > >> when
> > >> > > > dumping
> > >> > > > > > >> logs
> > >> > > > > > >> > > from
> > >> > > > > > >> > > > > all
> > >> > > > > > >> > > > > > >>> > processes
> > >> > > > > > >> > > > > > >>> > > > in
> > >> > > > > > >> > > > > > >>> > > > > a company into a common system.
> > >> > > > > > >> > > > > > >>> > > > >
> > >> > > > > > >> > > > > > >>> > > > > Maybe I’m being short-sighted, but
> it
> > >> > > doesn’t
> > >> > > > > seem
> > >> > > > > > >> like
> > >> > > > > > >> > > > > either
> > >> > > > > > >> > > > > > >>> really
> > >> > > > > > >> > > > > > >>> > > > > applies here. It seems like the
> > brokers
> > >> > > could
> > >> > > > > and
> > >> > > > > > >> would
> > >> > > > > > >> > > > > achieve
> > >> > > > > > >> > > > > > >>> > consensus
> > >> > > > > > >> > > > > > >>> > > > > when creating a topic anyway, which
> > is
> > >> all
> > >> > > > > that’s
> > >> > > > > > >> > > required
> > >> > > > > > >> > > > to
> > >> > > > > > >> > > > > > >>> > generate
> > >> > > > > > >> > > > > > >>> > > > > non-colliding sequence ids. For the
> > >> > second,
> > >> > > as
> > >> > > > > you
> > >> > > > > > >> > > mention,
> > >> > > > > > >> > > > > we
> > >> > > > > > >> > > > > > >>> could
> > >> > > > > > >> > > > > > >>> > > > assign
> > >> > > > > > >> > > > > > >>> > > > > a UUID to the cluster as a whole,
> > which
> > >> > > would
> > >> > > > > > render
> > >> > > > > > >> > any
> > >> > > > > > >> > > > > > resource
> > >> > > > > > >> > > > > > >>> > scoped
> > >> > > > > > >> > > > > > >>> > > > to
> > >> > > > > > >> > > > > > >>> > > > > the broker universally unique as
> > well.
> > >> > > > > > >> > > > > > >>> > > > >
> > >> > > > > > >> > > > > > >>> > > > > The reason I mention this is that,
> > >> > although
> > >> > > a
> > >> > > > > UUID
> > >> > > > > > >> is
> > >> > > > > > >> > way
> > >> > > > > > >> > > > > more
> > >> > > > > > >> > > > > > >>> > compact
> > >> > > > > > >> > > > > > >>> > > > > than topic names, it’s still 16
> > bytes.
> > >> In
> > >> > > > > > contrast,
> > >> > > > > > >> a
> > >> > > > > > >> > > > 4-byte
> > >> > > > > > >> > > > > > >>> integer
> > >> > > > > > >> > > > > > >>> > > > > sequence id would give us 4 billion
> > >> unique
> > >> > > > > topics
> > >> > > > > > >> per
> > >> > > > > > >> > > > > cluster,
> > >> > > > > > >> > > > > > >>> which
> > >> > > > > > >> > > > > > >>> > > > seems
> > >> > > > > > >> > > > > > >>> > > > > like enough ;)
> > >> > > > > > >> > > > > > >>> > > > >
> > >> > > > > > >> > > > > > >>> > > > > Considering the number of different
> > >> times
> > >> > > > these
> > >> > > > > > >> topic
> > >> > > > > > >> > > > > > >>> identifiers are
> > >> > > > > > >> > > > > > >>> > > > sent
> > >> > > > > > >> > > > > > >>> > > > > over the wire or stored in memory,
> it
> > >> > seems
> > >> > > > like
> > >> > > > > > it
> > >> > > > > > >> > might
> > >> > > > > > >> > > > be
> > >> > > > > > >> > > > > > >>> worth
> > >> > > > > > >> > > > > > >>> > the
> > >> > > > > > >> > > > > > >>> > > > > additional 4x space savings.
> > >> > > > > > >> > > > > > >>> > > > >
> > >> > > > > > >> > > > > > >>> > > > > What do you think about this?
> > >> > > > > > >> > > > > > >>> > > > >
> > >> > > > > > >> > > > > > >>> > > > > Thanks,
> > >> > > > > > >> > > > > > >>> > > > > John
> > >> > > > > > >> > > > > > >>> > > > >
> > >> > > > > > >> > > > > > >>> > > > > On Fri, Sep 11, 2020, at 03:20, Tom
> > >> > Bentley
> > >> > > > > wrote:
> > >> > > > > > >> > > > > > >>> > > > > > Hi Justine,
> > >> > > > > > >> > > > > > >>> > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > This looks like a very welcome
> > >> > > improvement.
> > >> > > > > > >> Thanks!
> > >> > > > > > >> > > > > > >>> > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > Maybe I missed it, but the KIP
> > >> doesn't
> > >> > > seem
> > >> > > > to
> > >> > > > > > >> > mention
> > >> > > > > > >> > > > > > changing
> > >> > > > > > >> > > > > > >>> > > > > > DeleteTopicsRequest to identify
> the
> > >> > topic
> > >> > > > > using
> > >> > > > > > an
> > >> > > > > > >> > id.
> > >> > > > > > >> > > > > Maybe
> > >> > > > > > >> > > > > > >>> > that's out
> > >> > > > > > >> > > > > > >>> > > > > of
> > >> > > > > > >> > > > > > >>> > > > > > scope, but DeleteTopicsRequest is
> > not
> > >> > > listed
> > >> > > > > > among
> > >> > > > > > >> > the
> > >> > > > > > >> > > > > Future
> > >> > > > > > >> > > > > > >>> Work
> > >> > > > > > >> > > > > > >>> > APIs
> > >> > > > > > >> > > > > > >>> > > > > > either.
> > >> > > > > > >> > > > > > >>> > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > Kind regards,
> > >> > > > > > >> > > > > > >>> > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > Tom
> > >> > > > > > >> > > > > > >>> > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > On Thu, Sep 10, 2020 at 3:59 PM
> > >> Satish
> > >> > > > > Duggana <
> > >> > > > > > >> > > > > > >>> > > > satish.duggana@gmail.com
> > >> > > > > > >> > > > > > >>> > > > > > wrote:
> > >> > > > > > >> > > > > > >>> > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > > Thanks Lucas/Justine for the
> nice
> > >> KIP.
> > >> > > > > > >> > > > > > >>> > > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > > It has several benefits which
> > also
> > >> > > include
> > >> > > > > > >> > > simplifying
> > >> > > > > > >> > > > > the
> > >> > > > > > >> > > > > > >>> topic
> > >> > > > > > >> > > > > > >>> > > > > > > deletion process by controller
> > and
> > >> > logs
> > >> > > > > > cleanup
> > >> > > > > > >> by
> > >> > > > > > >> > > > > brokers
> > >> > > > > > >> > > > > > in
> > >> > > > > > >> > > > > > >>> > corner
> > >> > > > > > >> > > > > > >>> > > > > > > cases.
> > >> > > > > > >> > > > > > >>> > > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > > Best,
> > >> > > > > > >> > > > > > >>> > > > > > > Satish.
> > >> > > > > > >> > > > > > >>> > > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > > On Wed, Sep 9, 2020 at 10:07 PM
> > >> > Justine
> > >> > > > > > Olshan <
> > >> > > > > > >> > > > > > >>> > jolshan@confluent.io
> > >> > > > > > >> > > > > > >>> > > > > > > wrote:
> > >> > > > > > >> > > > > > >>> > > > > > > > Hello all, it's been almost a
> > >> year!
> > >> > > I've
> > >> > > > > > made
> > >> > > > > > >> > some
> > >> > > > > > >> > > > > > changes
> > >> > > > > > >> > > > > > >>> to
> > >> > > > > > >> > > > > > >>> > this
> > >> > > > > > >> > > > > > >>> > > > > KIP
> > >> > > > > > >> > > > > > >>> > > > > > > and hope to continue the
> > >> discussion.
> > >> > > > > > >> > > > > > >>> > > > > > > > One of the main changes I've
> > >> added
> > >> > is
> > >> > > > now
> > >> > > > > > the
> > >> > > > > > >> > > > metadata
> > >> > > > > > >> > > > > > >>> response
> > >> > > > > > >> > > > > > >>> > > > will
> > >> > > > > > >> > > > > > >>> > > > > > > include the topic ID (as Colin
> > >> > > suggested).
> > >> > > > > > >> Clients
> > >> > > > > > >> > > can
> > >> > > > > > >> > > > > > >>> obtain the
> > >> > > > > > >> > > > > > >>> > > > > topicID
> > >> > > > > > >> > > > > > >>> > > > > > > of a given topic through a
> > >> > > > TopicDescription.
> > >> > > > > > The
> > >> > > > > > >> > > > topicId
> > >> > > > > > >> > > > > > will
> > >> > > > > > >> > > > > > >>> > also be
> > >> > > > > > >> > > > > > >>> > > > > > > included with the
> UpdateMetadata
> > >> > > request.
> > >> > > > > > >> > > > > > >>> > > > > > > > Let me know what you all
> think.
> > >> > > > > > >> > > > > > >>> > > > > > > > Thank you,
> > >> > > > > > >> > > > > > >>> > > > > > > > Justine
> > >> > > > > > >> > > > > > >>> > > > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > > > On 2019/09/13 16:38:26,
> "Colin
> > >> > > McCabe" <
> > >> > > > > > >> > > > > > cmccabe@apache.org
> > >> > > > > > >> > > > > > >>> >
> > >> > > > > > >> > > > > > >>> > wrote:
> > >> > > > > > >> > > > > > >>> > > > > > > > > Hi Lucas,
> > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > > > > Thanks for tackling this.
> > >> Topic
> > >> > IDs
> > >> > > > > are a
> > >> > > > > > >> > great
> > >> > > > > > >> > > > > idea,
> > >> > > > > > >> > > > > > >>> and
> > >> > > > > > >> > > > > > >>> > this
> > >> > > > > > >> > > > > > >>> > > > is
> > >> > > > > > >> > > > > > >>> > > > > a
> > >> > > > > > >> > > > > > >>> > > > > > > really good writeup.
> > >> > > > > > >> > > > > > >>> > > > > > > > > For
> /brokers/topics/[topic],
> > >> the
> > >> > > > schema
> > >> > > > > > >> version
> > >> > > > > > >> > > > > should
> > >> > > > > > >> > > > > > be
> > >> > > > > > >> > > > > > >>> > bumped
> > >> > > > > > >> > > > > > >>> > > > to
> > >> > > > > > >> > > > > > >>> > > > > > > version 3, rather than 2.
> > KIP-455
> > >> > > bumped
> > >> > > > > the
> > >> > > > > > >> > version
> > >> > > > > > >> > > > of
> > >> > > > > > >> > > > > > this
> > >> > > > > > >> > > > > > >>> > znode
> > >> > > > > > >> > > > > > >>> > > > to
> > >> > > > > > >> > > > > > >>> > > > > 2
> > >> > > > > > >> > > > > > >>> > > > > > > already :)
> > >> > > > > > >> > > > > > >>> > > > > > > > > Given that we're going to
> be
> > >> > seeing
> > >> > > > > these
> > >> > > > > > >> > things
> > >> > > > > > >> > > as
> > >> > > > > > >> > > > > > >>> strings
> > >> > > > > > >> > > > > > >>> > as
> > >> > > > > > >> > > > > > >>> > > > lot
> > >> > > > > > >> > > > > > >>> > > > > (in
> > >> > > > > > >> > > > > > >>> > > > > > > logs, in ZooKeeper, on the
> > >> > command-line,
> > >> > > > > > etc.),
> > >> > > > > > >> > does
> > >> > > > > > >> > > it
> > >> > > > > > >> > > > > > make
> > >> > > > > > >> > > > > > >>> > sense to
> > >> > > > > > >> > > > > > >>> > > > > use
> > >> > > > > > >> > > > > > >>> > > > > > > base64 when converting them to
> > >> > strings?
> > >> > > > > > >> > > > > > >>> > > > > > > > > Here is an example of the
> hex
> > >> > > > > > >> representation:
> > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >> > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > > > > And here is an example in
> > >> base64.
> > >> > > > > > >> > > > > > >>> > > > > > > > > b8tRS7h4TJ2Vt43Dp85v2A
> > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > > > > The base64 version saves 15
> > >> > letters
> > >> > > > (to
> > >> > > > > be
> > >> > > > > > >> > fair,
> > >> > > > > > >> > > 4
> > >> > > > > > >> > > > of
> > >> > > > > > >> > > > > > >>> those
> > >> > > > > > >> > > > > > >>> > were
> > >> > > > > > >> > > > > > >>> > > > > > > dashes that we could have
> elided
> > in
> > >> > the
> > >> > > > hex
> > >> > > > > > >> > > > > > representation.)
> > >> > > > > > >> > > > > > >>> > > > > > > > > Another thing to consider
> is
> > >> that
> > >> > we
> > >> > > > > > should
> > >> > > > > > >> > > specify
> > >> > > > > > >> > > > > > that
> > >> > > > > > >> > > > > > >>> the
> > >> > > > > > >> > > > > > >>> > > > > > > all-zeroes UUID is not a valid
> > >> topic
> > >> > > UUID.
> > >> > > > > >  We
> > >> > > > > > >> > can't
> > >> > > > > > >> > > > use
> > >> > > > > > >> > > > > > >>> null
> > >> > > > > > >> > > > > > >>> > for
> > >> > > > > > >> > > > > > >>> > > > this
> > >> > > > > > >> > > > > > >>> > > > > > > because we can't pass a null
> UUID
> > >> over
> > >> > > the
> > >> > > > > RPC
> > >> > > > > > >> > > protocol
> > >> > > > > > >> > > > > > >>> (there
> > >> > > > > > >> > > > > > >>> > is no
> > >> > > > > > >> > > > > > >>> > > > > > > special pattern for null, nor
> do
> > we
> > >> > want
> > >> > > > to
> > >> > > > > > >> waste
> > >> > > > > > >> > > space
> > >> > > > > > >> > > > > > >>> reserving
> > >> > > > > > >> > > > > > >>> > > > such
> > >> > > > > > >> > > > > > >>> > > > > a
> > >> > > > > > >> > > > > > >>> > > > > > > pattern.)
> > >> > > > > > >> > > > > > >>> > > > > > > > > Maybe I missed it, but did
> > you
> > >> > > > describe
> > >> > > > > > >> > > "migration
> > >> > > > > > >> > > > > > of...
> > >> > > > > > >> > > > > > >>> > existing
> > >> > > > > > >> > > > > > >>> > > > > > > topic[s] without topic IDs" in
> > >> detail
> > >> > in
> > >> > > > any
> > >> > > > > > >> > section?
> > >> > > > > > >> > > > It
> > >> > > > > > >> > > > > > >>> seems
> > >> > > > > > >> > > > > > >>> > like
> > >> > > > > > >> > > > > > >>> > > > > when
> > >> > > > > > >> > > > > > >>> > > > > > > the new controller becomes
> > active,
> > >> it
> > >> > > > should
> > >> > > > > > >> just
> > >> > > > > > >> > > > > generate
> > >> > > > > > >> > > > > > >>> random
> > >> > > > > > >> > > > > > >>> > > > > UUIDs for
> > >> > > > > > >> > > > > > >>> > > > > > > these, and write the random
> UUIDs
> > >> back
> > >> > > to
> > >> > > > > > >> > ZooKeeper.
> > >> > > > > > >> > > > It
> > >> > > > > > >> > > > > > >>> would be
> > >> > > > > > >> > > > > > >>> > > > good
> > >> > > > > > >> > > > > > >>> > > > > to
> > >> > > > > > >> > > > > > >>> > > > > > > spell that out.  We should make
> > it
> > >> > clear
> > >> > > > > that
> > >> > > > > > >> this
> > >> > > > > > >> > > > > happens
> > >> > > > > > >> > > > > > >>> > regardless
> > >> > > > > > >> > > > > > >>> > > > > of
> > >> > > > > > >> > > > > > >>> > > > > > > the inter-broker protocol
> version
> > >> > (it's
> > >> > > a
> > >> > > > > > >> > compatible
> > >> > > > > > >> > > > > > change).
> > >> > > > > > >> > > > > > >>> > > > > > > > > "LeaderAndIsrRequests
> > >> including an
> > >> > > > > > >> > > > is_every_partition
> > >> > > > > > >> > > > > > >>> flag"
> > >> > > > > > >> > > > > > >>> > > > seems a
> > >> > > > > > >> > > > > > >>> > > > > > > bit wordy.  Can we just call
> > these
> > >> > "full
> > >> > > > > > >> > > > > > >>> LeaderAndIsrRequests"?
> > >> > > > > > >> > > > > > >>> > Then
> > >> > > > > > >> > > > > > >>> > > > > the
> > >> > > > > > >> > > > > > >>> > > > > > > RPC field could be named
> "full".
> > >> > Also,
> > >> > > it
> > >> > > > > > would
> > >> > > > > > >> > > > probably
> > >> > > > > > >> > > > > > be
> > >> > > > > > >> > > > > > >>> > better
> > >> > > > > > >> > > > > > >>> > > > > for the
> > >> > > > > > >> > > > > > >>> > > > > > > RPC field to be an enum of {
> > >> > > UNSPECIFIED,
> > >> > > > > > >> > > INCREMENTAL,
> > >> > > > > > >> > > > > FULL
> > >> > > > > > >> > > > > > >>> }, so
> > >> > > > > > >> > > > > > >>> > > > that
> > >> > > > > > >> > > > > > >>> > > > > we
> > >> > > > > > >> > > > > > >>> > > > > > > can cleanly handle old versions
> > (by
> > >> > > > treating
> > >> > > > > > >> them
> > >> > > > > > >> > as
> > >> > > > > > >> > > > > > >>> UNSPECIFIED)
> > >> > > > > > >> > > > > > >>> > > > > > > > > In the LeaderAndIsrRequest
> > >> > section,
> > >> > > > you
> > >> > > > > > >> write
> > >> > > > > > >> > "A
> > >> > > > > > >> > > > > final
> > >> > > > > > >> > > > > > >>> > deletion
> > >> > > > > > >> > > > > > >>> > > > > event
> > >> > > > > > >> > > > > > >>> > > > > > > will be secheduled for X ms
> after
> > >> the
> > >> > > > > > >> > > > LeaderAndIsrRequest
> > >> > > > > > >> > > > > > was
> > >> > > > > > >> > > > > > >>> > first
> > >> > > > > > >> > > > > > >>> > > > > > > received..."  I guess the X
> was a
> > >> > > > > placeholder
> > >> > > > > > >> that
> > >> > > > > > >> > > you
> > >> > > > > > >> > > > > > >>> intended
> > >> > > > > > >> > > > > > >>> > to
> > >> > > > > > >> > > > > > >>> > > > > replace
> > >> > > > > > >> > > > > > >>> > > > > > > before posting? :)  In any
> case,
> > >> this
> > >> > > > seems
> > >> > > > > > like
> > >> > > > > > >> > the
> > >> > > > > > >> > > > kind
> > >> > > > > > >> > > > > > of
> > >> > > > > > >> > > > > > >>> > thing
> > >> > > > > > >> > > > > > >>> > > > we'd
> > >> > > > > > >> > > > > > >>> > > > > > > want a configuration for.
> Let's
> > >> > > describe
> > >> > > > > that
> > >> > > > > > >> > > > > > configuration
> > >> > > > > > >> > > > > > >>> key
> > >> > > > > > >> > > > > > >>> > > > > somewhere
> > >> > > > > > >> > > > > > >>> > > > > > > in this KIP, including what its
> > >> > default
> > >> > > > > value
> > >> > > > > > >> is.
> > >> > > > > > >> > > > > > >>> > > > > > > > > We should probably also
> log a
> > >> > bunch
> > >> > > of
> > >> > > > > > >> messages
> > >> > > > > > >> > > at
> > >> > > > > > >> > > > > WARN
> > >> > > > > > >> > > > > > >>> level
> > >> > > > > > >> > > > > > >>> > > > when
> > >> > > > > > >> > > > > > >>> > > > > > > something is scheduled for
> > >> deletion,
> > >> > as
> > >> > > > > well.
> > >> > > > > > >> > (Maybe
> > >> > > > > > >> > > > > this
> > >> > > > > > >> > > > > > >>> was
> > >> > > > > > >> > > > > > >>> > > > > assumed, but
> > >> > > > > > >> > > > > > >>> > > > > > > it would be good to mention
> it).
> > >> > > > > > >> > > > > > >>> > > > > > > > > I feel like there are a few
> > >> > sections
> > >> > > > > that
> > >> > > > > > >> > should
> > >> > > > > > >> > > be
> > >> > > > > > >> > > > > > >>> moved to
> > >> > > > > > >> > > > > > >>> > > > > "rejected
> > >> > > > > > >> > > > > > >>> > > > > > > alternatives."  For example, in
> > the
> > >> > > > > > DeleteTopics
> > >> > > > > > >> > > > section,
> > >> > > > > > >> > > > > > >>> since
> > >> > > > > > >> > > > > > >>> > we're
> > >> > > > > > >> > > > > > >>> > > > > not
> > >> > > > > > >> > > > > > >>> > > > > > > going to do option 1 or 2,
> these
> > >> > should
> > >> > > be
> > >> > > > > > moved
> > >> > > > > > >> > into
> > >> > > > > > >> > > > > > >>> "rejected
> > >> > > > > > >> > > > > > >>> > > > > > > alternatives,"  rather than
> > >> appearing
> > >> > > > > inline.
> > >> > > > > > >> > > Another
> > >> > > > > > >> > > > > case
> > >> > > > > > >> > > > > > >>> is
> > >> > > > > > >> > > > > > >>> > the
> > >> > > > > > >> > > > > > >>> > > > > "Should
> > >> > > > > > >> > > > > > >>> > > > > > > we remove topic name from the
> > >> protocol
> > >> > > > where
> > >> > > > > > >> > > possible"
> > >> > > > > > >> > > > > > >>> section.
> > >> > > > > > >> > > > > > >>> > This
> > >> > > > > > >> > > > > > >>> > > > > is
> > >> > > > > > >> > > > > > >>> > > > > > > clearly discussing a design
> > >> > alternative
> > >> > > > that
> > >> > > > > > >> we're
> > >> > > > > > >> > > not
> > >> > > > > > >> > > > > > >>> proposing
> > >> > > > > > >> > > > > > >>> > to
> > >> > > > > > >> > > > > > >>> > > > > > > implement: removing the topic
> > name
> > >> > from
> > >> > > > > those
> > >> > > > > > >> > > > protocols.
> > >> > > > > > >> > > > > > >>> > > > > > > > > Is it really necessary to
> > have
> > >> a
> > >> > new
> > >> > > > > > >> > > > > > >>> > /admin/delete_topics_by_id
> > >> > > > > > >> > > > > > >>> > > > > path
> > >> > > > > > >> > > > > > >>> > > > > > > in ZooKeeper?  It seems like we
> > >> don't
> > >> > > > really
> > >> > > > > > >> need
> > >> > > > > > >> > > this.
> > >> > > > > > >> > > > > > >>> Whenever
> > >> > > > > > >> > > > > > >>> > > > > there is
> > >> > > > > > >> > > > > > >>> > > > > > > a new controller, we'll send
> out
> > >> full
> > >> > > > > > >> > > > > LeaderAndIsrRequests
> > >> > > > > > >> > > > > > >>> which
> > >> > > > > > >> > > > > > >>> > will
> > >> > > > > > >> > > > > > >>> > > > > > > trigger the stale topics to be
> > >> cleaned
> > >> > > up.
> > >> > > > > >  The
> > >> > > > > > >> > > active
> > >> > > > > > >> > > > > > >>> > controller
> > >> > > > > > >> > > > > > >>> > > > will
> > >> > > > > > >> > > > > > >>> > > > > > > also send the full
> > >> LeaderAndIsrRequest
> > >> > > to
> > >> > > > > > >> brokers
> > >> > > > > > >> > > that
> > >> > > > > > >> > > > > are
> > >> > > > > > >> > > > > > >>> just
> > >> > > > > > >> > > > > > >>> > > > > starting
> > >> > > > > > >> > > > > > >>> > > > > > > up.    So we don't really need
> > this
> > >> > kind
> > >> > > > of
> > >> > > > > > >> > two-phase
> > >> > > > > > >> > > > > > commit
> > >> > > > > > >> > > > > > >>> > (send
> > >> > > > > > >> > > > > > >>> > > > out
> > >> > > > > > >> > > > > > >>> > > > > > > StopReplicasRequest, get ACKs
> > from
> > >> all
> > >> > > > > nodes,
> > >> > > > > > >> > commit
> > >> > > > > > >> > > by
> > >> > > > > > >> > > > > > >>> removing
> > >> > > > > > >> > > > > > >>> > > > > > > /admin/delete_topics node) any
> > >> more.
> > >> > > > > > >> > > > > > >>> > > > > > > > > You mention that
> FetchRequest
> > >> will
> > >> > > now
> > >> > > > > > >> include
> > >> > > > > > >> > > UUID
> > >> > > > > > >> > > > > to
> > >> > > > > > >> > > > > > >>> avoid
> > >> > > > > > >> > > > > > >>> > > > issues
> > >> > > > > > >> > > > > > >>> > > > > > > where requests are made to
> stale
> > >> > > > partitions.
> > >> > > > > > >> > > However,
> > >> > > > > > >> > > > > > >>> adding a
> > >> > > > > > >> > > > > > >>> > UUID
> > >> > > > > > >> > > > > > >>> > > > to
> > >> > > > > > >> > > > > > >>> > > > > > > MetadataRequest is listed as
> > future
> > >> > > work,
> > >> > > > > out
> > >> > > > > > of
> > >> > > > > > >> > > scope
> > >> > > > > > >> > > > > for
> > >> > > > > > >> > > > > > >>> this
> > >> > > > > > >> > > > > > >>> > KIP.
> > >> > > > > > >> > > > > > >>> > > > > How
> > >> > > > > > >> > > > > > >>> > > > > > > will the client learn what the
> > >> topic
> > >> > > UUID
> > >> > > > > is,
> > >> > > > > > if
> > >> > > > > > >> > the
> > >> > > > > > >> > > > > > metadata
> > >> > > > > > >> > > > > > >>> > > > response
> > >> > > > > > >> > > > > > >>> > > > > > > doesn't include that
> information?
> > >> It
> > >> > > > seems
> > >> > > > > > like
> > >> > > > > > >> > > adding
> > >> > > > > > >> > > > > the
> > >> > > > > > >> > > > > > >>> UUID
> > >> > > > > > >> > > > > > >>> > to
> > >> > > > > > >> > > > > > >>> > > > > > > MetadataResponse would be an
> > >> > improvement
> > >> > > > > here
> > >> > > > > > >> that
> > >> > > > > > >> > > > might
> > >> > > > > > >> > > > > > not
> > >> > > > > > >> > > > > > >>> be
> > >> > > > > > >> > > > > > >>> > too
> > >> > > > > > >> > > > > > >>> > > > > hard to
> > >> > > > > > >> > > > > > >>> > > > > > > make.
> > >> > > > > > >> > > > > > >>> > > > > > > > > best,
> > >> > > > > > >> > > > > > >>> > > > > > > > > Colin
> > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > > > > On Mon, Sep 9, 2019, at
> > 17:48,
> > >> > > Ryanne
> > >> > > > > > Dolan
> > >> > > > > > >> > > wrote:
> > >> > > > > > >> > > > > > >>> > > > > > > > > > Lucas, this would be
> great.
> > >> I've
> > >> > > run
> > >> > > > > > into
> > >> > > > > > >> > > issues
> > >> > > > > > >> > > > > with
> > >> > > > > > >> > > > > > >>> > topics
> > >> > > > > > >> > > > > > >>> > > > > being
> > >> > > > > > >> > > > > > >>> > > > > > > > > > resurrected accidentally,
> > >> since
> > >> > a
> > >> > > > > client
> > >> > > > > > >> > cannot
> > >> > > > > > >> > > > > > easily
> > >> > > > > > >> > > > > > >>> > > > > distinguish
> > >> > > > > > >> > > > > > >>> > > > > > > between
> > >> > > > > > >> > > > > > >>> > > > > > > > > > a deleted topic and a new
> > >> topic
> > >> > > with
> > >> > > > > the
> > >> > > > > > >> same
> > >> > > > > > >> > > > name.
> > >> > > > > > >> > > > > > I'd
> > >> > > > > > >> > > > > > >>> > need
> > >> > > > > > >> > > > > > >>> > > > the
> > >> > > > > > >> > > > > > >>> > > > > ID
> > >> > > > > > >> > > > > > >>> > > > > > > > > > accessible from the
> client
> > to
> > >> > > solve
> > >> > > > > that
> > >> > > > > > >> > issue,
> > >> > > > > > >> > > > but
> > >> > > > > > >> > > > > > >>> this
> > >> > > > > > >> > > > > > >>> > is a
> > >> > > > > > >> > > > > > >>> > > > > good
> > >> > > > > > >> > > > > > >>> > > > > > > first
> > >> > > > > > >> > > > > > >>> > > > > > > > > > step.
> > >> > > > > > >> > > > > > >>> > > > > > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > > > > > Ryanne
> > >> > > > > > >> > > > > > >>> > > > > > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > > > > > On Wed, Sep 4, 2019 at
> 1:41
> > >> PM
> > >> > > Lucas
> > >> > > > > > >> > > Bradstreet <
> > >> > > > > > >> > > > > > >>> > > > > lucas@confluent.io>
> > >> > > > > > >> > > > > > >>> > > > > > > wrote:
> > >> > > > > > >> > > > > > >>> > > > > > > > > > > Hi all,
> > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > > > > > > I would like to kick
> off
> > >> > > > discussion
> > >> > > > > of
> > >> > > > > > >> > > KIP-516,
> > >> > > > > > >> > > > > an
> > >> > > > > > >> > > > > > >>> > > > > implementation
> > >> > > > > > >> > > > > > >>> > > > > > > of topic
> > >> > > > > > >> > > > > > >>> > > > > > > > > > > IDs for Kafka. Topic
> IDs
> > >> aim
> > >> > to
> > >> > > > > solve
> > >> > > > > > >> topic
> > >> > > > > > >> > > > > > >>> uniqueness
> > >> > > > > > >> > > > > > >>> > > > > problems in
> > >> > > > > > >> > > > > > >>> > > > > > > Kafka,
> > >> > > > > > >> > > > > > >>> > > > > > > > > > > where referring to a
> > topic
> > >> by
> > >> > > name
> > >> > > > > > >> alone is
> > >> > > > > > >> > > > > > >>> insufficient.
> > >> > > > > > >> > > > > > >>> > > > Such
> > >> > > > > > >> > > > > > >>> > > > > > > cases
> > >> > > > > > >> > > > > > >>> > > > > > > > > > > include when a topic
> has
> > >> been
> > >> > > > > deleted
> > >> > > > > > >> and
> > >> > > > > > >> > > > > recreated
> > >> > > > > > >> > > > > > >>> with
> > >> > > > > > >> > > > > > >>> > the
> > >> > > > > > >> > > > > > >>> > > > > same
> > >> > > > > > >> > > > > > >>> > > > > > > name.
> > >> > > > > > >> > > > > > >>> > > > > > > > > > > Unique identifiers will
> > >> help
> > >> > > > > simplify
> > >> > > > > > >> and
> > >> > > > > > >> > > > improve
> > >> > > > > > >> > > > > > >>> Kafka's
> > >> > > > > > >> > > > > > >>> > > > topic
> > >> > > > > > >> > > > > > >>> > > > > > > deletion
> > >> > > > > > >> > > > > > >>> > > > > > > > > > > process, as well as
> > prevent
> > >> > > cases
> > >> > > > > > where
> > >> > > > > > >> > > brokers
> > >> > > > > > >> > > > > may
> > >> > > > > > >> > > > > > >>> > > > incorrectly
> > >> > > > > > >> > > > > > >>> > > > > > > interact
> > >> > > > > > >> > > > > > >>> > > > > > > > > > > with stale versions of
> > >> topics.
> > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > >> > > > > > >> > > > > > >>> > > >
> > >> > > > > > >> > > > > > >>> >
> > >> > > > > > >> > > > > > >>>
> > >> > > > > > >> > > > > >
> > >> > > > > > >> > > > >
> > >> > > > > > >> > > >
> > >> > > > > > >> > >
> > >> > > > > > >> >
> > >> > > > > > >>
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > >> > > > > > >> > > > > > >>> > > > > > > > > > > Looking forward to your
> > >> > > thoughts.
> > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > >> > > > > > >> > > > > > >>> > > > > > > > > > > Lucas
> > >> > > > > > >> > > > > > >>> > > > > > > > > > >
> > >> > > > > > >> > > > > > >>> >
> > >> > > > > > >> > > > > > >>> >
> > >> > > > > > >> > > > > > >>>
> > >> > > > > > >> > > > > > >>
> > >> > > > > > >> > > > > >
> > >> > > > > > >> > > > >
> > >> > > > > > >> > > >
> > >> > > > > > >> > >
> > >> > > > > > >> >
> > >> > > > > > >>
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> >
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Jun Rao <ju...@confluent.io>]

Hi, Justine,

Thanks for the updated KIP. A few more comments below.

30.  {"name": "id", "type": "string", "doc": "version id"}}: The doc should
say UUID.

31. LeaderAndIsrResponse v5 and StopReplicaResponse v4 : It seems there is
no need to add topic_id at partitions level.

32. Regarding partition metadata file. Perhaps the key can be a single
word, sth like the following.
version: 0
topic_id: 46bdb63f-9e8d-4a38-bf7b-ee4eb2a794e4

33. Another tricky thing that I realized is how to support the metadata
topic introduced in KIP-595. It's a bit tricky to assign a UUID to the
metadata topic since we have a chicken-and-egg problem. The controller
needs to persist the UUID in the metadata topic in order to assign one
successfully, but the metadata topic is needed to elect a controller
(KIP-631). So, this probably needs a bit more thought.

Jun

On Thu, Sep 24, 2020 at 3:04 AM Ismael Juma <is...@juma.me.uk> wrote:

> Also, can we provide more details on how the Partition Metadata file will
> be used?
>
> Ismael
>
> On Thu, Sep 24, 2020 at 3:01 AM Ismael Juma <is...@juma.me.uk> wrote:
>
> > Hi Justine,
> >
> > I think we need to update the "Rejected Alternatives" section to take
> into
> > account that the proposal now removes the topic name from the fetch
> > request. Also, if we are removing it from the Fetch request, does it make
> > sense not to remove it from similar requests like ListOffsetRequest?
> >
> > Ismael
> >
> > On Thu, Sep 24, 2020 at 2:46 AM David Jacot <dj...@confluent.io> wrote:
> >
> >> Hi Justine,
> >>
> >> Thanks for the KIP. I finally had time to read it :). It is a great
> >> improvement.
> >>
> >> I have a few comments/questions:
> >>
> >> 1. It seems that the schema of the StopReplicaRequest is slightly
> >> outdated.
> >> We
> >> did some changes as part of KIP-570. V3 is already organized by topics.
> >>
> >> 2. I just want to make sure that I understand the reconciliation
> >> logic correctly. When
> >> an "INCREMENTAL" LeaderAndIsr Request is received, the broker will also
> >> reconcile
> >> when the local uuid does not match the uuid in the request, right? In
> this
> >> case, the
> >> local log is staged for deletion.
> >>
> >> 3. In the documentation of the `delete.stale.topic.delay.ms` config, it
> >> says "When a
> >> FULL LeaderAndIsrRequest is received..." but I suppose it applies to
> both
> >> types.
> >>
> >> Best,
> >> David
> >>
> >> On Thu, Sep 24, 2020 at 1:40 AM Justine Olshan <jo...@confluent.io>
> >> wrote:
> >>
> >> > Hi Jun,
> >> >
> >> > Thanks for the comments. I apologize for some of the typos and
> >> confusion.
> >> > I’ve updated the KIP to fix some of the issues you mentioned.
> >> >
> >> > 20.2 I’ve changed the type to String.
> >> > 20.1/3 I’ve updated the TopicZNode to fix formatting and reflect the
> >> latest
> >> > version before this change.
> >> >
> >> > 21. You are correct and I’ve removed this line. I’ve also added a line
> >> > mentioning an IBP bump is necessary for migration
> >> >
> >> > 22. I think the wording was unclear but your summary is what was
> >> intended
> >> > by this line. I’ve updated to make this point more clear. “Remove
> >> deleted
> >> > topics from replicas by sending StopReplicaRequest V3 before the IBP
> >> bump
> >> > using the old logic, and using V4 and the new logic with topic IDs
> after
> >> > the IBP bump.”
> >> >
> >> > 23. I’ve removed the unspecified type from the KIP and mention that
> IBP
> >> > will be used to handle this request. “IBP will be used to determine
> >> whether
> >> > this new form of the request will be used. For older requests, we will
> >> > ignore this field and default to previous behavior.”
> >> >
> >> > 24. I’ve fixed this typo.
> >> >
> >> > 25. I've added a topics at a higher level for LeaderAndIsrResponse v5,
> >> > StopReplicaResponse v4. I've also changed StopReplicaRequest v4 to
> have
> >> > topics at a higher level.
> >> >
> >> > 26. I’ve updated forgotten_topics_data--added the topic ID and removed
> >> the
> >> > topic name
> >> >
> >> > 27. I’ve decided on plain text, and I’ve added an example.
> >> >
> >> > 28. I’ve added this idea to future work.
> >> >
> >> > Thanks again for taking a look,
> >> >
> >> > Justine
> >> >
> >> > On Wed, Sep 23, 2020 at 10:28 AM Jun Rao <ju...@confluent.io> wrote:
> >> >
> >> > > Hi, Justine,
> >> > >
> >> > > Thanks for the response. Made another pass. A few more comments
> below.
> >> > >
> >> > > 20. znode schema:
> >> > > 20.1 It seems that {"name": "version", "type": "int", "id": "UUID",
> >> > "doc":
> >> > > "version id"} should be {"name": "version", "type": "int"}, {"name":
> >> > "id",
> >> > > "type": "UUID", "doc": "version id"}.
> >> > > 20.2 The znode format is JSON which doesn't have UUID type. So the
> >> type
> >> > > probably should be string?
> >> > > 20.3 Also, the existing format used seems outdated. It should have
> the
> >> > > following format.
> >> > >     Json.encodeAsBytes(Map(
> >> > >       "version" -> 2,
> >> > >       "partitions" -> replicaAssignmentJson.asJava,
> >> > >       "adding_replicas" -> addingReplicasAssignmentJson.asJava,
> >> > >       "removing_replicas" -> removingReplicasAssignmentJson.asJava
> >> > >     ).asJava)
> >> > >   }
> >> > >
> >> > > 21. Migration: The KIP says "The migration process can take place
> >> without
> >> > > an inter-broker protocol bump, as the format stored in
> >> > > /brokers/topics/[topic] will be compatible with older broker
> >> versions."
> >> > > However, since we are bumping up the version of inter-broker
> >> requests, it
> >> > > seems that we need to use IBP for migration.
> >> > >
> >> > > 22. The KIP says "Remove deleted topics from replicas by sending
> >> > > StopReplicaRequest V3 for any topics which do not contain a topic
> ID,
> >> and
> >> > > V4 for any topics which do contain a topic ID.". However, if we use
> >> IBP,
> >> > it
> >> > > seems that the controller will either send StopReplicaRequest V3
> >> > > or StopReplicaRequest V4, but never mixed V3 and V4 for different
> >> topics.
> >> > > Basically, before the IBP bump, V3 will be used. After the IBP bump,
> >> > > topicId will be created and V4 will be used.
> >> > >
> >> > > 23. Given that we depend on IBP, do we still need "0 UNSPECIFIED"
> >> > > in LeaderAndIsr?
> >> > >
> >> > > 24. LeaderAndIsrResponse v5 : It still has the topic field.
> >> > >
> >> > > 25. LeaderAndIsrResponse v5, StopReplicaResponse v4: Could we use
> this
> >> > > opportunity to organize the response in 2 levels, first by topic,
> >> then by
> >> > > partition, as most other requests/responses?
> >> > >
> >> > > 26. FetchRequest v13 : Should forgotten_topics_data use topicId too?
> >> > >
> >> > > 27. "This file can either be plain text (key/value pairs) or JSON."
> >> Have
> >> > we
> >> > > decided which one to use? Also, it would be helpful to provide an
> >> > example.
> >> > >
> >> > > 28. Future improvement: Another future improvement opportunity is to
> >> use
> >> > > topicId in GroupMetadataManager.offsetCommitKey in the offset_commit
> >> > topic.
> >> > > This may save some space.
> >> > >
> >> > > Thanks,
> >> > >
> >> > > Jun
> >> > >
> >> > > On Wed, Sep 23, 2020 at 8:50 AM Justine Olshan <
> jolshan@confluent.io>
> >> > > wrote:
> >> > >
> >> > > > Hi Tom,
> >> > > >
> >> > > > Thanks for the comment. I think this is a really good idea and it
> >> has
> >> > > been
> >> > > > added to the KIP under the newly added tooling section.
> >> > > >
> >> > > > Thanks again,
> >> > > > Justine
> >> > > >
> >> > > > On Wed, Sep 23, 2020 at 3:17 AM Tom Bentley <tb...@redhat.com>
> >> > wrote:
> >> > > >
> >> > > > > Hi Justine,
> >> > > > >
> >> > > > > I know you started the vote thread, but on re-reading the KIP I
> >> > noticed
> >> > > > > that although the topic id is included in the MetadataResponse
> >> it's
> >> > not
> >> > > > > surfaced in the output from `kafka-topics.sh --describe`. Maybe
> >> that
> >> > > was
> >> > > > > intentional because ids are intentionally not really something
> the
> >> > user
> >> > > > > should care deeply about, but it would also make life harder for
> >> > anyone
> >> > > > > debugging Kafka and this would likely get worse the more topic
> ids
> >> > got
> >> > > > > rolled out across the protocols, clients etc. It seems likely
> that
> >> > > > > `kafka-topics.sh` will eventually need the ability to show the
> id
> >> of
> >> > a
> >> > > > > topic and perhaps find a topic name given an id. Is there any
> >> reason
> >> > > not
> >> > > > to
> >> > > > > implement that in this KIP?
> >> > > > >
> >> > > > > Many thanks,
> >> > > > >
> >> > > > > Tom
> >> > > > >
> >> > > > > On Mon, Sep 21, 2020 at 9:54 PM Justine Olshan <
> >> jolshan@confluent.io
> >> > >
> >> > > > > wrote:
> >> > > > >
> >> > > > > > Hi all,
> >> > > > > >
> >> > > > > > After thinking about it, I've decided to remove the topic name
> >> from
> >> > > the
> >> > > > > > Fetch Request and Response after all. Since there are so many
> of
> >> > > these
> >> > > > > > requests per second, it is worth removing the extra
> information.
> >> > I've
> >> > > > > > updated the KIP to reflect this change.
> >> > > > > >
> >> > > > > > Please let me know if there is anything else we should discuss
> >> > before
> >> > > > > > voting.
> >> > > > > >
> >> > > > > > Thank you,
> >> > > > > > Justine
> >> > > > > >
> >> > > > > > On Fri, Sep 18, 2020 at 9:46 AM Justine Olshan <
> >> > jolshan@confluent.io
> >> > > >
> >> > > > > > wrote:
> >> > > > > >
> >> > > > > > > Hi Jun,
> >> > > > > > >
> >> > > > > > > I see what you are saying. For now we can remove the extra
> >> > > > information.
> >> > > > > > > I'll leave the option to add more fields to the file in the
> >> > future.
> >> > > > The
> >> > > > > > KIP
> >> > > > > > > has been updated to reflect this change.
> >> > > > > > >
> >> > > > > > > Thanks,
> >> > > > > > > Justine
> >> > > > > > >
> >> > > > > > > On Fri, Sep 18, 2020 at 8:46 AM Jun Rao <ju...@confluent.io>
> >> > wrote:
> >> > > > > > >
> >> > > > > > >> Hi, Justine,
> >> > > > > > >>
> >> > > > > > >> Thanks for the reply.
> >> > > > > > >>
> >> > > > > > >> 13. If the log directory is the source of truth, it means
> >> that
> >> > the
> >> > > > > > >> redundant info in the metadata file will be ignored. Then
> the
> >> > > > question
> >> > > > > > is
> >> > > > > > >> why do we need to put the redundant info in the metadata
> file
> >> > now?
> >> > > > > > >>
> >> > > > > > >> Thanks,
> >> > > > > > >>
> >> > > > > > >> Jun
> >> > > > > > >>
> >> > > > > > >> On Thu, Sep 17, 2020 at 5:07 PM Justine Olshan <
> >> > > > jolshan@confluent.io>
> >> > > > > > >> wrote:
> >> > > > > > >>
> >> > > > > > >> > Hi Jun,
> >> > > > > > >> > Thanks for the quick response!
> >> > > > > > >> >
> >> > > > > > >> > 12. I've decided to bump up the versions on the requests
> >> and
> >> > > > updated
> >> > > > > > the
> >> > > > > > >> > KIP. I think it's good we thoroughly discussed the
> options
> >> > here,
> >> > > > so
> >> > > > > we
> >> > > > > > >> know
> >> > > > > > >> > we made a good choice. :)
> >> > > > > > >> >
> >> > > > > > >> > 13. This is an interesting situation. I think if this
> does
> >> > occur
> >> > > > we
> >> > > > > > >> should
> >> > > > > > >> > give a warning. I agree that it's hard to know the source
> >> of
> >> > > truth
> >> > > > > for
> >> > > > > > >> sure
> >> > > > > > >> > since the directory or the file could be manually
> >> modified. I
> >> > > > guess
> >> > > > > > the
> >> > > > > > >> > directory could be used as the source of truth. To be
> >> honest,
> >> > > I'm
> >> > > > > not
> >> > > > > > >> > really sure what happens in kafka when the log directory
> is
> >> > > > renamed
> >> > > > > > >> > manually in such a way. I'm also wondering if the
> >> situation is
> >> > > > > > >> recoverable
> >> > > > > > >> > in this scenario.
> >> > > > > > >> >
> >> > > > > > >> > Thanks,
> >> > > > > > >> > Justine
> >> > > > > > >> >
> >> > > > > > >> > On Thu, Sep 17, 2020 at 4:28 PM Jun Rao <
> jun@confluent.io>
> >> > > wrote:
> >> > > > > > >> >
> >> > > > > > >> > > Hi, Justine,
> >> > > > > > >> > >
> >> > > > > > >> > > Thanks for the reply.
> >> > > > > > >> > >
> >> > > > > > >> > > 12. I don't have a strong preference either. However,
> if
> >> we
> >> > > need
> >> > > > > IBP
> >> > > > > > >> > > anyway, maybe it's easier to just bump up the version
> for
> >> > all
> >> > > > > inter
> >> > > > > > >> > broker
> >> > > > > > >> > > requests and add the topic id field as a regular
> field. A
> >> > > > regular
> >> > > > > > >> field
> >> > > > > > >> > is
> >> > > > > > >> > > a bit more concise in wire transfer than a flexible
> >> field.
> >> > > > > > >> > >
> >> > > > > > >> > > 13. The confusion that I was referring to is between
> the
> >> > topic
> >> > > > > name
> >> > > > > > >> and
> >> > > > > > >> > > partition number between the log dir and the metadata
> >> file.
> >> > > For
> >> > > > > > >> example,
> >> > > > > > >> > if
> >> > > > > > >> > > the log dir is topicA-1 and the metadata file in it has
> >> > topicB
> >> > > > and
> >> > > > > > >> > > partition 0 (say due to a bug or manual modification),
> >> which
> >> > > one
> >> > > > > do
> >> > > > > > we
> >> > > > > > >> > use
> >> > > > > > >> > > as the source of truth?
> >> > > > > > >> > >
> >> > > > > > >> > > Jun
> >> > > > > > >> > >
> >> > > > > > >> > > On Thu, Sep 17, 2020 at 3:43 PM Justine Olshan <
> >> > > > > > jolshan@confluent.io>
> >> > > > > > >> > > wrote:
> >> > > > > > >> > >
> >> > > > > > >> > > > Hi Jun,
> >> > > > > > >> > > > Thanks for the comments.
> >> > > > > > >> > > >
> >> > > > > > >> > > > 12. I bumped the LeaderAndIsrRequest because I
> removed
> >> the
> >> > > > topic
> >> > > > > > >> name
> >> > > > > > >> > > field
> >> > > > > > >> > > > in the response. It may be possible to avoid bumping
> >> the
> >> > > > version
> >> > > > > > >> > without
> >> > > > > > >> > > > that change, but I may be missing something.
> >> > > > > > >> > > > I believe StopReplica is actually on version 3 now,
> but
> >> > > > because
> >> > > > > > >> > version 2
> >> > > > > > >> > > > is flexible, I kept that listed as version 2 on the
> KIP
> >> > > page.
> >> > > > > > >> However,
> >> > > > > > >> > > you
> >> > > > > > >> > > > may be right in that we may need to bump the version
> on
> >> > > > > > StopReplica
> >> > > > > > >> to
> >> > > > > > >> > > deal
> >> > > > > > >> > > > with deletion differently as mentioned above. I don't
> >> know
> >> > > if
> >> > > > I
> >> > > > > > >> have a
> >> > > > > > >> > > big
> >> > > > > > >> > > > preference over used tagged fields or not.
> >> > > > > > >> > > >
> >> > > > > > >> > > > 13. I was thinking that in the case where the file
> and
> >> the
> >> > > > > request
> >> > > > > > >> > topic
> >> > > > > > >> > > > ids don't match, it means that the broker's topic/the
> >> one
> >> > in
> >> > > > the
> >> > > > > > >> file
> >> > > > > > >> > has
> >> > > > > > >> > > > been deleted. In that case, we would need to delete
> the
> >> > old
> >> > > > > topic
> >> > > > > > >> and
> >> > > > > > >> > > start
> >> > > > > > >> > > > receiving the new version. If the topic name were to
> >> > change,
> >> > > > but
> >> > > > > > the
> >> > > > > > >> > ids
> >> > > > > > >> > > > still matched, the file would also need to update.
> Am I
> >> > > > missing
> >> > > > > a
> >> > > > > > >> case
> >> > > > > > >> > > > where the file would be correct and not the request?
> >> > > > > > >> > > >
> >> > > > > > >> > > > Thanks,
> >> > > > > > >> > > > Justine
> >> > > > > > >> > > >
> >> > > > > > >> > > > On Thu, Sep 17, 2020 at 3:18 PM Jun Rao <
> >> jun@confluent.io
> >> > >
> >> > > > > wrote:
> >> > > > > > >> > > >
> >> > > > > > >> > > > > Hi, Justine,
> >> > > > > > >> > > > >
> >> > > > > > >> > > > > Thanks for the reply. A couple of more comments
> >> below.
> >> > > > > > >> > > > >
> >> > > > > > >> > > > > 12. ListOffset and OffsetForLeader currently don't
> >> > support
> >> > > > > > >> flexible
> >> > > > > > >> > > > fields.
> >> > > > > > >> > > > > So, we have to bump up the version number and use
> >> IBP at
> >> > > > least
> >> > > > > > for
> >> > > > > > >> > > these
> >> > > > > > >> > > > > two requests. Note that it seems 2.7.0 will require
> >> IBP
> >> > > > anyway
> >> > > > > > >> > because
> >> > > > > > >> > > of
> >> > > > > > >> > > > > changes in KAFKA-10435. Also, it seems that the
> >> version
> >> > > for
> >> > > > > > >> > > > > LeaderAndIsrRequest and StopReplica are bumped even
> >> > though
> >> > > > we
> >> > > > > > only
> >> > > > > > >> > > added
> >> > > > > > >> > > > a
> >> > > > > > >> > > > > tagged field. But since IBP is needed anyway, we
> may
> >> > want
> >> > > to
> >> > > > > > >> revisit
> >> > > > > > >> > > the
> >> > > > > > >> > > > > overall tagged field choice.
> >> > > > > > >> > > > >
> >> > > > > > >> > > > > 13. The only downside is the potential confusion on
> >> > which
> >> > > > one
> >> > > > > is
> >> > > > > > >> the
> >> > > > > > >> > > > source
> >> > > > > > >> > > > > of truth if they don't match. Another option is to
> >> > include
> >> > > > > those
> >> > > > > > >> > fields
> >> > > > > > >> > > > in
> >> > > > > > >> > > > > the metadata file when we actually change the
> >> directory
> >> > > > > > structure.
> >> > > > > > >> > > > >
> >> > > > > > >> > > > > Thanks,
> >> > > > > > >> > > > >
> >> > > > > > >> > > > > Jun
> >> > > > > > >> > > > >
> >> > > > > > >> > > > > On Thu, Sep 17, 2020 at 2:01 PM Justine Olshan <
> >> > > > > > >> jolshan@confluent.io
> >> > > > > > >> > >
> >> > > > > > >> > > > > wrote:
> >> > > > > > >> > > > >
> >> > > > > > >> > > > > > Hello all,
> >> > > > > > >> > > > > >
> >> > > > > > >> > > > > > I've thought some more about removing the topic
> >> name
> >> > > field
> >> > > > > > from
> >> > > > > > >> > some
> >> > > > > > >> > > of
> >> > > > > > >> > > > > the
> >> > > > > > >> > > > > > requests. On closer inspection of the
> >> > > requests/responses,
> >> > > > it
> >> > > > > > >> seems
> >> > > > > > >> > > that
> >> > > > > > >> > > > > the
> >> > > > > > >> > > > > > internal changes would be much larger than I
> >> expected.
> >> > > > Some
> >> > > > > > >> > protocols
> >> > > > > > >> > > > > > involve clients, so they would require changes
> too.
> >> > I'm
> >> > > > > > thinking
> >> > > > > > >> > that
> >> > > > > > >> > > > for
> >> > > > > > >> > > > > > now, removing the topic name from these requests
> >> and
> >> > > > > responses
> >> > > > > > >> are
> >> > > > > > >> > > out
> >> > > > > > >> > > > of
> >> > > > > > >> > > > > > scope.
> >> > > > > > >> > > > > >
> >> > > > > > >> > > > > > I have decided to just keep the change
> >> > > > LeaderAndIsrResponse
> >> > > > > to
> >> > > > > > >> > remove
> >> > > > > > >> > > > the
> >> > > > > > >> > > > > > topic name, and have updated the KIP to reflect
> >> this
> >> > > > > change. I
> >> > > > > > >> have
> >> > > > > > >> > > > also
> >> > > > > > >> > > > > > mentioned the other requests and responses in
> >> future
> >> > > work.
> >> > > > > > >> > > > > >
> >> > > > > > >> > > > > > I'm hoping to start the voting process soon, so
> >> let me
> >> > > > know
> >> > > > > if
> >> > > > > > >> > there
> >> > > > > > >> > > is
> >> > > > > > >> > > > > > anything else we should discuss.
> >> > > > > > >> > > > > >
> >> > > > > > >> > > > > > Thank you,
> >> > > > > > >> > > > > > Justine
> >> > > > > > >> > > > > >
> >> > > > > > >> > > > > > On Tue, Sep 15, 2020 at 3:57 PM Justine Olshan <
> >> > > > > > >> > jolshan@confluent.io
> >> > > > > > >> > > >
> >> > > > > > >> > > > > > wrote:
> >> > > > > > >> > > > > >
> >> > > > > > >> > > > > > > Hello again,
> >> > > > > > >> > > > > > > To follow up on some of the other comments:
> >> > > > > > >> > > > > > >
> >> > > > > > >> > > > > > > 10/11) We can remove the topic name from these
> >> > > > > > >> > requests/responses,
> >> > > > > > >> > > > and
> >> > > > > > >> > > > > > > that means we will just have to make a few
> >> internal
> >> > > > > changes
> >> > > > > > to
> >> > > > > > >> > make
> >> > > > > > >> > > > > > > partitions accessible by topic id and
> partition.
> >> I
> >> > can
> >> > > > > > update
> >> > > > > > >> the
> >> > > > > > >> > > KIP
> >> > > > > > >> > > > > to
> >> > > > > > >> > > > > > > remove them unless anyone thinks they should
> >> stay.
> >> > > > > > >> > > > > > >
> >> > > > > > >> > > > > > > 12) Addressed in the previous email. I've
> updated
> >> > the
> >> > > > KIP
> >> > > > > to
> >> > > > > > >> > > include
> >> > > > > > >> > > > > > > tagged fields for the requests and responses.
> >> (More
> >> > on
> >> > > > > that
> >> > > > > > >> > below)
> >> > > > > > >> > > > > > >
> >> > > > > > >> > > > > > > 13) I think part of the idea for including this
> >> > > > > information
> >> > > > > > >> is to
> >> > > > > > >> > > > > prepare
> >> > > > > > >> > > > > > > for future changes. Perhaps the directory
> >> structure
> >> > > > might
> >> > > > > > >> change
> >> > > > > > >> > > from
> >> > > > > > >> > > > > > > topicName_partitionNumber to something like
> >> > > > > > >> > > topicID_partitionNumber.
> >> > > > > > >> > > > > Then
> >> > > > > > >> > > > > > > it would be useful to have the topic name in
> the
> >> > file
> >> > > > > since
> >> > > > > > it
> >> > > > > > >> > > would
> >> > > > > > >> > > > > not
> >> > > > > > >> > > > > > be
> >> > > > > > >> > > > > > > in the directory structure. Supporting topic
> >> renames
> >> > > > might
> >> > > > > > be
> >> > > > > > >> > > easier
> >> > > > > > >> > > > if
> >> > > > > > >> > > > > > the
> >> > > > > > >> > > > > > > other fields are included. Would there be any
> >> > > downsides
> >> > > > to
> >> > > > > > >> > > including
> >> > > > > > >> > > > > this
> >> > > > > > >> > > > > > > information?
> >> > > > > > >> > > > > > >
> >> > > > > > >> > > > > > > 14)  Yes, we would need to copy the partition
> >> > metadata
> >> > > > > file
> >> > > > > > in
> >> > > > > > >> > this
> >> > > > > > >> > > > > > > process. I've updated the KIP to include this.
> >> > > > > > >> > > > > > >
> >> > > > > > >> > > > > > > 15) I believe Lucas meant v1 and v2 here. He
> was
> >> > > > referring
> >> > > > > > to
> >> > > > > > >> how
> >> > > > > > >> > > the
> >> > > > > > >> > > > > > > requests would fall under different IBP and
> meant
> >> > that
> >> > > > > older
> >> > > > > > >> > > brokers
> >> > > > > > >> > > > > > would
> >> > > > > > >> > > > > > > have to use the older version of the request
> and
> >> the
> >> > > > > > existing
> >> > > > > > >> > topic
> >> > > > > > >> > > > > > > deletion process. At first, it seemed like
> tagged
> >> > > fields
> >> > > > > > would
> >> > > > > > >> > > > resolve
> >> > > > > > >> > > > > > > the IBP issue. However, we may need IBP for
> this
> >> > > request
> >> > > > > > after
> >> > > > > > >> > all
> >> > > > > > >> > > > > since
> >> > > > > > >> > > > > > > the controller handles the topic deletion
> >> > differently
> >> > > > > > >> depending
> >> > > > > > >> > on
> >> > > > > > >> > > > the
> >> > > > > > >> > > > > > IBP
> >> > > > > > >> > > > > > > version. In an older version, we can't just
> send
> >> a
> >> > > > > > StopReplica
> >> > > > > > >> > > delete
> >> > > > > > >> > > > > the
> >> > > > > > >> > > > > > > topic immediately like we'd want to for this
> KIP.
> >> > > > > > >> > > > > > >
> >> > > > > > >> > > > > > > This makes me wonder if we want tagged fields
> on
> >> all
> >> > > the
> >> > > > > > >> requests
> >> > > > > > >> > > > after
> >> > > > > > >> > > > > > > all. Let me know your thoughts!
> >> > > > > > >> > > > > > >
> >> > > > > > >> > > > > > > Justine
> >> > > > > > >> > > > > > >
> >> > > > > > >> > > > > > > On Tue, Sep 15, 2020 at 1:03 PM Justine Olshan
> <
> >> > > > > > >> > > jolshan@confluent.io
> >> > > > > > >> > > > >
> >> > > > > > >> > > > > > > wrote:
> >> > > > > > >> > > > > > >
> >> > > > > > >> > > > > > >> Hi all,
> >> > > > > > >> > > > > > >> Jun brought up a good point in his last email
> >> about
> >> > > > > tagged
> >> > > > > > >> > fields,
> >> > > > > > >> > > > and
> >> > > > > > >> > > > > > >> I've updated the KIP to reflect that the
> >> changes to
> >> > > > > > requests
> >> > > > > > >> and
> >> > > > > > >> > > > > > responses
> >> > > > > > >> > > > > > >> will be in the form of tagged fields to avoid
> >> > > changing
> >> > > > > IBP.
> >> > > > > > >> > > > > > >>
> >> > > > > > >> > > > > > >> Jun: I plan on sending a followup email to
> >> address
> >> > > some
> >> > > > > of
> >> > > > > > >> the
> >> > > > > > >> > > other
> >> > > > > > >> > > > > > >> points.
> >> > > > > > >> > > > > > >>
> >> > > > > > >> > > > > > >> Thanks,
> >> > > > > > >> > > > > > >> Justine
> >> > > > > > >> > > > > > >>
> >> > > > > > >> > > > > > >> On Mon, Sep 14, 2020 at 4:25 PM Jun Rao <
> >> > > > > jun@confluent.io>
> >> > > > > > >> > wrote:
> >> > > > > > >> > > > > > >>
> >> > > > > > >> > > > > > >>> Hi, Justine,
> >> > > > > > >> > > > > > >>>
> >> > > > > > >> > > > > > >>> Thanks for the updated KIP. A few comments
> >> below.
> >> > > > > > >> > > > > > >>>
> >> > > > > > >> > > > > > >>> 10. LeaderAndIsr Response: Do we need the
> topic
> >> > > name?
> >> > > > > > >> > > > > > >>>
> >> > > > > > >> > > > > > >>> 11. For the changed request/response, other
> >> than
> >> > > > > > >> LeaderAndIsr,
> >> > > > > > >> > > > > > >>> UpdateMetadata, Metadata, do we need to
> include
> >> > the
> >> > > > > topic
> >> > > > > > >> name?
> >> > > > > > >> > > > > > >>>
> >> > > > > > >> > > > > > >>> 12. It seems that upgrades don't require IBP.
> >> Does
> >> > > > that
> >> > > > > > mean
> >> > > > > > >> > the
> >> > > > > > >> > > > new
> >> > > > > > >> > > > > > >>> fields
> >> > > > > > >> > > > > > >>> in all the request/response are added as
> tagged
> >> > > fields
> >> > > > > > >> without
> >> > > > > > >> > > > > bumping
> >> > > > > > >> > > > > > up
> >> > > > > > >> > > > > > >>> the request version? It would be useful to
> make
> >> > that
> >> > > > > > clear.
> >> > > > > > >> > > > > > >>>
> >> > > > > > >> > > > > > >>> 13. Partition Metadata file: Do we need to
> >> include
> >> > > the
> >> > > > > > topic
> >> > > > > > >> > name
> >> > > > > > >> > > > and
> >> > > > > > >> > > > > > the
> >> > > > > > >> > > > > > >>> partition id since they are implied in the
> >> > directory
> >> > > > > name?
> >> > > > > > >> > > > > > >>>
> >> > > > > > >> > > > > > >>> 14. In the JBOD mode, we support moving a
> >> > > partition's
> >> > > > > data
> >> > > > > > >> from
> >> > > > > > >> > > one
> >> > > > > > >> > > > > > disk
> >> > > > > > >> > > > > > >>> to
> >> > > > > > >> > > > > > >>> another. Will the new partition metadata file
> >> be
> >> > > > copied
> >> > > > > > >> during
> >> > > > > > >> > > that
> >> > > > > > >> > > > > > >>> process?
> >> > > > > > >> > > > > > >>>
> >> > > > > > >> > > > > > >>> 15. The KIP says "Remove deleted topics from
> >> > > replicas
> >> > > > by
> >> > > > > > >> > sending
> >> > > > > > >> > > > > > >>> StopReplicaRequest V2 for any topics which do
> >> not
> >> > > > > contain
> >> > > > > > a
> >> > > > > > >> > topic
> >> > > > > > >> > > > ID,
> >> > > > > > >> > > > > > and
> >> > > > > > >> > > > > > >>> V3 for any topics which do contain a topic
> >> ID.".
> >> > > > > However,
> >> > > > > > it
> >> > > > > > >> > > seems
> >> > > > > > >> > > > > the
> >> > > > > > >> > > > > > >>> updated controller will create all missing
> >> topic
> >> > IDs
> >> > > > > first
> >> > > > > > >> > before
> >> > > > > > >> > > > > doing
> >> > > > > > >> > > > > > >>> other actions. So, is StopReplicaRequest V2
> >> > needed?
> >> > > > > > >> > > > > > >>>
> >> > > > > > >> > > > > > >>> Jun
> >> > > > > > >> > > > > > >>>
> >> > > > > > >> > > > > > >>> On Fri, Sep 11, 2020 at 10:31 AM John
> Roesler <
> >> > > > > > >> > > vvcephei@apache.org
> >> > > > > > >> > > > >
> >> > > > > > >> > > > > > >>> wrote:
> >> > > > > > >> > > > > > >>>
> >> > > > > > >> > > > > > >>> > Thanks, Justine!
> >> > > > > > >> > > > > > >>> >
> >> > > > > > >> > > > > > >>> > Your response seems compelling to me.
> >> > > > > > >> > > > > > >>> >
> >> > > > > > >> > > > > > >>> > -John
> >> > > > > > >> > > > > > >>> >
> >> > > > > > >> > > > > > >>> > On Fri, 2020-09-11 at 10:17 -0700, Justine
> >> > Olshan
> >> > > > > wrote:
> >> > > > > > >> > > > > > >>> > > Hello all,
> >> > > > > > >> > > > > > >>> > > Thanks for continuing the discussion! I
> >> have a
> >> > > few
> >> > > > > > >> > responses
> >> > > > > > >> > > to
> >> > > > > > >> > > > > > your
> >> > > > > > >> > > > > > >>> > points.
> >> > > > > > >> > > > > > >>> > >
> >> > > > > > >> > > > > > >>> > > Tom: You are correct in that this KIP has
> >> not
> >> > > > > > mentioned
> >> > > > > > >> the
> >> > > > > > >> > > > > > >>> > > DeleteTopicsRequest. I think that this
> >> would
> >> > be
> >> > > > out
> >> > > > > of
> >> > > > > > >> > scope
> >> > > > > > >> > > > for
> >> > > > > > >> > > > > > >>> now, but
> >> > > > > > >> > > > > > >>> > > may be something worth adding in the
> >> future.
> >> > > > > > >> > > > > > >>> > >
> >> > > > > > >> > > > > > >>> > > John: We did consider sequence ids, but
> >> there
> >> > > are
> >> > > > a
> >> > > > > > few
> >> > > > > > >> > > reasons
> >> > > > > > >> > > > > to
> >> > > > > > >> > > > > > >>> favor
> >> > > > > > >> > > > > > >>> > > UUIDs. There are several cases where
> topics
> >> > from
> >> > > > > > >> different
> >> > > > > > >> > > > > clusters
> >> > > > > > >> > > > > > >>> may
> >> > > > > > >> > > > > > >>> > > interact now and in the future. For
> >> example,
> >> > > > Mirror
> >> > > > > > >> Maker 2
> >> > > > > > >> > > may
> >> > > > > > >> > > > > > >>> benefit
> >> > > > > > >> > > > > > >>> > > from being able to detect when a cluster
> >> being
> >> > > > > > mirrored
> >> > > > > > >> is
> >> > > > > > >> > > > > deleted
> >> > > > > > >> > > > > > >>> and
> >> > > > > > >> > > > > > >>> > > recreated and globally unique identifiers
> >> > would
> >> > > > make
> >> > > > > > >> > > resolving
> >> > > > > > >> > > > > > issues
> >> > > > > > >> > > > > > >>> > > easier than sequence IDs which may
> collide
> >> > > between
> >> > > > > > >> > clusters.
> >> > > > > > >> > > > > > KIP-405
> >> > > > > > >> > > > > > >>> > > (tiered storage) will also benefit from
> >> > globally
> >> > > > > > unique
> >> > > > > > >> IDs
> >> > > > > > >> > > as
> >> > > > > > >> > > > > > shared
> >> > > > > > >> > > > > > >>> > > buckets may be used between clusters.
> >> > > > > > >> > > > > > >>> > >
> >> > > > > > >> > > > > > >>> > > Globally unique IDs would also make
> >> > > functionality
> >> > > > > like
> >> > > > > > >> > moving
> >> > > > > > >> > > > > > topics
> >> > > > > > >> > > > > > >>> > > between disparate clusters easier in the
> >> > future,
> >> > > > > > >> simplify
> >> > > > > > >> > any
> >> > > > > > >> > > > > > future
> >> > > > > > >> > > > > > >>> > > implementations of backups and restores,
> >> and
> >> > > more.
> >> > > > > In
> >> > > > > > >> > > general,
> >> > > > > > >> > > > > > >>> unique IDs
> >> > > > > > >> > > > > > >>> > > would ensure that the source cluster
> >> topics do
> >> > > not
> >> > > > > > >> conflict
> >> > > > > > >> > > > with
> >> > > > > > >> > > > > > the
> >> > > > > > >> > > > > > >>> > > destination cluster topics.
> >> > > > > > >> > > > > > >>> > >
> >> > > > > > >> > > > > > >>> > > If we were to use sequence ids, we would
> >> need
> >> > > > > > >> sufficiently
> >> > > > > > >> > > > large
> >> > > > > > >> > > > > > >>> cluster
> >> > > > > > >> > > > > > >>> > > ids to be stored with the topic
> >> identifiers or
> >> > > we
> >> > > > > run
> >> > > > > > >> the
> >> > > > > > >> > > risk
> >> > > > > > >> > > > of
> >> > > > > > >> > > > > > >>> > > collisions. This will give up any
> >> advantage in
> >> > > > > > >> compactness
> >> > > > > > >> > > that
> >> > > > > > >> > > > > > >>> sequence
> >> > > > > > >> > > > > > >>> > > numbers may bring. Given these
> advantages I
> >> > > think
> >> > > > it
> >> > > > > > >> makes
> >> > > > > > >> > > > sense
> >> > > > > > >> > > > > to
> >> > > > > > >> > > > > > >>> use
> >> > > > > > >> > > > > > >>> > > UUIDs.
> >> > > > > > >> > > > > > >>> > >
> >> > > > > > >> > > > > > >>> > > Gokul: This is an interesting idea, but
> >> this
> >> > is
> >> > > a
> >> > > > > > >> breaking
> >> > > > > > >> > > > > change.
> >> > > > > > >> > > > > > >>> Out of
> >> > > > > > >> > > > > > >>> > > scope for now, but maybe worth discussing
> >> in
> >> > the
> >> > > > > > future.
> >> > > > > > >> > > > > > >>> > >
> >> > > > > > >> > > > > > >>> > > Hope this explains some of the decisions,
> >> > > > > > >> > > > > > >>> > >
> >> > > > > > >> > > > > > >>> > > Justine
> >> > > > > > >> > > > > > >>> > >
> >> > > > > > >> > > > > > >>> > >
> >> > > > > > >> > > > > > >>> > >
> >> > > > > > >> > > > > > >>> > > On Fri, Sep 11, 2020 at 8:27 AM Gokul
> >> Ramanan
> >> > > > > > >> Subramanian <
> >> > > > > > >> > > > > > >>> > > gokul2411s@gmail.com> wrote:
> >> > > > > > >> > > > > > >>> > >
> >> > > > > > >> > > > > > >>> > > > Hi.
> >> > > > > > >> > > > > > >>> > > >
> >> > > > > > >> > > > > > >>> > > > Thanks for the KIP.
> >> > > > > > >> > > > > > >>> > > >
> >> > > > > > >> > > > > > >>> > > > Have you thought about whether it makes
> >> > sense
> >> > > to
> >> > > > > > >> support
> >> > > > > > >> > > > > > >>> authorizing a
> >> > > > > > >> > > > > > >>> > > > principal for a topic ID rather than a
> >> topic
> >> > > > name
> >> > > > > to
> >> > > > > > >> > > achieve
> >> > > > > > >> > > > > > >>> tighter
> >> > > > > > >> > > > > > >>> > > > security?
> >> > > > > > >> > > > > > >>> > > >
> >> > > > > > >> > > > > > >>> > > > Or is the topic ID fundamentally an
> >> internal
> >> > > > > detail
> >> > > > > > >> > similar
> >> > > > > > >> > > > to
> >> > > > > > >> > > > > > >>> epochs
> >> > > > > > >> > > > > > >>> > used
> >> > > > > > >> > > > > > >>> > > > in a bunch of other places in Kafka?
> >> > > > > > >> > > > > > >>> > > >
> >> > > > > > >> > > > > > >>> > > > Thanks.
> >> > > > > > >> > > > > > >>> > > >
> >> > > > > > >> > > > > > >>> > > > On Fri, Sep 11, 2020 at 4:06 PM John
> >> > Roesler <
> >> > > > > > >> > > > > > vvcephei@apache.org>
> >> > > > > > >> > > > > > >>> > wrote:
> >> > > > > > >> > > > > > >>> > > >
> >> > > > > > >> > > > > > >>> > > > > Hello Justine,
> >> > > > > > >> > > > > > >>> > > > >
> >> > > > > > >> > > > > > >>> > > > > Thanks for the KIP!
> >> > > > > > >> > > > > > >>> > > > >
> >> > > > > > >> > > > > > >>> > > > > I happen to have been confronted
> >> recently
> >> > > with
> >> > > > > the
> >> > > > > > >> need
> >> > > > > > >> > > to
> >> > > > > > >> > > > > keep
> >> > > > > > >> > > > > > >>> > track of
> >> > > > > > >> > > > > > >>> > > > a
> >> > > > > > >> > > > > > >>> > > > > large number of topics as compactly
> as
> >> > > > > possible. I
> >> > > > > > >> was
> >> > > > > > >> > > > going
> >> > > > > > >> > > > > to
> >> > > > > > >> > > > > > >>> come
> >> > > > > > >> > > > > > >>> > up
> >> > > > > > >> > > > > > >>> > > > > with some way to dictionary encode
> the
> >> > topic
> >> > > > > names
> >> > > > > > >> as
> >> > > > > > >> > > > > integers,
> >> > > > > > >> > > > > > >>> but
> >> > > > > > >> > > > > > >>> > this
> >> > > > > > >> > > > > > >>> > > > > seems much better!
> >> > > > > > >> > > > > > >>> > > > >
> >> > > > > > >> > > > > > >>> > > > > Apologies if this has been raised
> >> before,
> >> > > but
> >> > > > > I’m
> >> > > > > > >> > > wondering
> >> > > > > > >> > > > > > >>> about the
> >> > > > > > >> > > > > > >>> > > > > choice of UUID vs sequence number for
> >> the
> >> > > ids.
> >> > > > > > >> > Typically,
> >> > > > > > >> > > > > I’ve
> >> > > > > > >> > > > > > >>> seen
> >> > > > > > >> > > > > > >>> > UUIDs
> >> > > > > > >> > > > > > >>> > > > > in two situations:
> >> > > > > > >> > > > > > >>> > > > > 1. When processes need to generate
> >> > > > non-colliding
> >> > > > > > >> > > > identifiers
> >> > > > > > >> > > > > > >>> without
> >> > > > > > >> > > > > > >>> > > > > coordination.
> >> > > > > > >> > > > > > >>> > > > > 2. When the identifier needs to be
> >> > > > “universally
> >> > > > > > >> > unique”;
> >> > > > > > >> > > > > I.e.,
> >> > > > > > >> > > > > > >>> the
> >> > > > > > >> > > > > > >>> > > > > identifier needs to distinguish the
> >> entity
> >> > > > from
> >> > > > > > all
> >> > > > > > >> > other
> >> > > > > > >> > > > > > >>> entities
> >> > > > > > >> > > > > > >>> > that
> >> > > > > > >> > > > > > >>> > > > > could ever exist. This is useful in
> >> cases
> >> > > > where
> >> > > > > > >> > entities
> >> > > > > > >> > > > from
> >> > > > > > >> > > > > > all
> >> > > > > > >> > > > > > >>> > kinds
> >> > > > > > >> > > > > > >>> > > > of
> >> > > > > > >> > > > > > >>> > > > > systems get mixed together, such as
> >> when
> >> > > > dumping
> >> > > > > > >> logs
> >> > > > > > >> > > from
> >> > > > > > >> > > > > all
> >> > > > > > >> > > > > > >>> > processes
> >> > > > > > >> > > > > > >>> > > > in
> >> > > > > > >> > > > > > >>> > > > > a company into a common system.
> >> > > > > > >> > > > > > >>> > > > >
> >> > > > > > >> > > > > > >>> > > > > Maybe I’m being short-sighted, but it
> >> > > doesn’t
> >> > > > > seem
> >> > > > > > >> like
> >> > > > > > >> > > > > either
> >> > > > > > >> > > > > > >>> really
> >> > > > > > >> > > > > > >>> > > > > applies here. It seems like the
> brokers
> >> > > could
> >> > > > > and
> >> > > > > > >> would
> >> > > > > > >> > > > > achieve
> >> > > > > > >> > > > > > >>> > consensus
> >> > > > > > >> > > > > > >>> > > > > when creating a topic anyway, which
> is
> >> all
> >> > > > > that’s
> >> > > > > > >> > > required
> >> > > > > > >> > > > to
> >> > > > > > >> > > > > > >>> > generate
> >> > > > > > >> > > > > > >>> > > > > non-colliding sequence ids. For the
> >> > second,
> >> > > as
> >> > > > > you
> >> > > > > > >> > > mention,
> >> > > > > > >> > > > > we
> >> > > > > > >> > > > > > >>> could
> >> > > > > > >> > > > > > >>> > > > assign
> >> > > > > > >> > > > > > >>> > > > > a UUID to the cluster as a whole,
> which
> >> > > would
> >> > > > > > render
> >> > > > > > >> > any
> >> > > > > > >> > > > > > resource
> >> > > > > > >> > > > > > >>> > scoped
> >> > > > > > >> > > > > > >>> > > > to
> >> > > > > > >> > > > > > >>> > > > > the broker universally unique as
> well.
> >> > > > > > >> > > > > > >>> > > > >
> >> > > > > > >> > > > > > >>> > > > > The reason I mention this is that,
> >> > although
> >> > > a
> >> > > > > UUID
> >> > > > > > >> is
> >> > > > > > >> > way
> >> > > > > > >> > > > > more
> >> > > > > > >> > > > > > >>> > compact
> >> > > > > > >> > > > > > >>> > > > > than topic names, it’s still 16
> bytes.
> >> In
> >> > > > > > contrast,
> >> > > > > > >> a
> >> > > > > > >> > > > 4-byte
> >> > > > > > >> > > > > > >>> integer
> >> > > > > > >> > > > > > >>> > > > > sequence id would give us 4 billion
> >> unique
> >> > > > > topics
> >> > > > > > >> per
> >> > > > > > >> > > > > cluster,
> >> > > > > > >> > > > > > >>> which
> >> > > > > > >> > > > > > >>> > > > seems
> >> > > > > > >> > > > > > >>> > > > > like enough ;)
> >> > > > > > >> > > > > > >>> > > > >
> >> > > > > > >> > > > > > >>> > > > > Considering the number of different
> >> times
> >> > > > these
> >> > > > > > >> topic
> >> > > > > > >> > > > > > >>> identifiers are
> >> > > > > > >> > > > > > >>> > > > sent
> >> > > > > > >> > > > > > >>> > > > > over the wire or stored in memory, it
> >> > seems
> >> > > > like
> >> > > > > > it
> >> > > > > > >> > might
> >> > > > > > >> > > > be
> >> > > > > > >> > > > > > >>> worth
> >> > > > > > >> > > > > > >>> > the
> >> > > > > > >> > > > > > >>> > > > > additional 4x space savings.
> >> > > > > > >> > > > > > >>> > > > >
> >> > > > > > >> > > > > > >>> > > > > What do you think about this?
> >> > > > > > >> > > > > > >>> > > > >
> >> > > > > > >> > > > > > >>> > > > > Thanks,
> >> > > > > > >> > > > > > >>> > > > > John
> >> > > > > > >> > > > > > >>> > > > >
> >> > > > > > >> > > > > > >>> > > > > On Fri, Sep 11, 2020, at 03:20, Tom
> >> > Bentley
> >> > > > > wrote:
> >> > > > > > >> > > > > > >>> > > > > > Hi Justine,
> >> > > > > > >> > > > > > >>> > > > > >
> >> > > > > > >> > > > > > >>> > > > > > This looks like a very welcome
> >> > > improvement.
> >> > > > > > >> Thanks!
> >> > > > > > >> > > > > > >>> > > > > >
> >> > > > > > >> > > > > > >>> > > > > > Maybe I missed it, but the KIP
> >> doesn't
> >> > > seem
> >> > > > to
> >> > > > > > >> > mention
> >> > > > > > >> > > > > > changing
> >> > > > > > >> > > > > > >>> > > > > > DeleteTopicsRequest to identify the
> >> > topic
> >> > > > > using
> >> > > > > > an
> >> > > > > > >> > id.
> >> > > > > > >> > > > > Maybe
> >> > > > > > >> > > > > > >>> > that's out
> >> > > > > > >> > > > > > >>> > > > > of
> >> > > > > > >> > > > > > >>> > > > > > scope, but DeleteTopicsRequest is
> not
> >> > > listed
> >> > > > > > among
> >> > > > > > >> > the
> >> > > > > > >> > > > > Future
> >> > > > > > >> > > > > > >>> Work
> >> > > > > > >> > > > > > >>> > APIs
> >> > > > > > >> > > > > > >>> > > > > > either.
> >> > > > > > >> > > > > > >>> > > > > >
> >> > > > > > >> > > > > > >>> > > > > > Kind regards,
> >> > > > > > >> > > > > > >>> > > > > >
> >> > > > > > >> > > > > > >>> > > > > > Tom
> >> > > > > > >> > > > > > >>> > > > > >
> >> > > > > > >> > > > > > >>> > > > > > On Thu, Sep 10, 2020 at 3:59 PM
> >> Satish
> >> > > > > Duggana <
> >> > > > > > >> > > > > > >>> > > > satish.duggana@gmail.com
> >> > > > > > >> > > > > > >>> > > > > > wrote:
> >> > > > > > >> > > > > > >>> > > > > >
> >> > > > > > >> > > > > > >>> > > > > > > Thanks Lucas/Justine for the nice
> >> KIP.
> >> > > > > > >> > > > > > >>> > > > > > >
> >> > > > > > >> > > > > > >>> > > > > > > It has several benefits which
> also
> >> > > include
> >> > > > > > >> > > simplifying
> >> > > > > > >> > > > > the
> >> > > > > > >> > > > > > >>> topic
> >> > > > > > >> > > > > > >>> > > > > > > deletion process by controller
> and
> >> > logs
> >> > > > > > cleanup
> >> > > > > > >> by
> >> > > > > > >> > > > > brokers
> >> > > > > > >> > > > > > in
> >> > > > > > >> > > > > > >>> > corner
> >> > > > > > >> > > > > > >>> > > > > > > cases.
> >> > > > > > >> > > > > > >>> > > > > > >
> >> > > > > > >> > > > > > >>> > > > > > > Best,
> >> > > > > > >> > > > > > >>> > > > > > > Satish.
> >> > > > > > >> > > > > > >>> > > > > > >
> >> > > > > > >> > > > > > >>> > > > > > > On Wed, Sep 9, 2020 at 10:07 PM
> >> > Justine
> >> > > > > > Olshan <
> >> > > > > > >> > > > > > >>> > jolshan@confluent.io
> >> > > > > > >> > > > > > >>> > > > > > > wrote:
> >> > > > > > >> > > > > > >>> > > > > > > > Hello all, it's been almost a
> >> year!
> >> > > I've
> >> > > > > > made
> >> > > > > > >> > some
> >> > > > > > >> > > > > > changes
> >> > > > > > >> > > > > > >>> to
> >> > > > > > >> > > > > > >>> > this
> >> > > > > > >> > > > > > >>> > > > > KIP
> >> > > > > > >> > > > > > >>> > > > > > > and hope to continue the
> >> discussion.
> >> > > > > > >> > > > > > >>> > > > > > > > One of the main changes I've
> >> added
> >> > is
> >> > > > now
> >> > > > > > the
> >> > > > > > >> > > > metadata
> >> > > > > > >> > > > > > >>> response
> >> > > > > > >> > > > > > >>> > > > will
> >> > > > > > >> > > > > > >>> > > > > > > include the topic ID (as Colin
> >> > > suggested).
> >> > > > > > >> Clients
> >> > > > > > >> > > can
> >> > > > > > >> > > > > > >>> obtain the
> >> > > > > > >> > > > > > >>> > > > > topicID
> >> > > > > > >> > > > > > >>> > > > > > > of a given topic through a
> >> > > > TopicDescription.
> >> > > > > > The
> >> > > > > > >> > > > topicId
> >> > > > > > >> > > > > > will
> >> > > > > > >> > > > > > >>> > also be
> >> > > > > > >> > > > > > >>> > > > > > > included with the UpdateMetadata
> >> > > request.
> >> > > > > > >> > > > > > >>> > > > > > > > Let me know what you all think.
> >> > > > > > >> > > > > > >>> > > > > > > > Thank you,
> >> > > > > > >> > > > > > >>> > > > > > > > Justine
> >> > > > > > >> > > > > > >>> > > > > > > >
> >> > > > > > >> > > > > > >>> > > > > > > > On 2019/09/13 16:38:26, "Colin
> >> > > McCabe" <
> >> > > > > > >> > > > > > cmccabe@apache.org
> >> > > > > > >> > > > > > >>> >
> >> > > > > > >> > > > > > >>> > wrote:
> >> > > > > > >> > > > > > >>> > > > > > > > > Hi Lucas,
> >> > > > > > >> > > > > > >>> > > > > > > > >
> >> > > > > > >> > > > > > >>> > > > > > > > > Thanks for tackling this.
> >> Topic
> >> > IDs
> >> > > > > are a
> >> > > > > > >> > great
> >> > > > > > >> > > > > idea,
> >> > > > > > >> > > > > > >>> and
> >> > > > > > >> > > > > > >>> > this
> >> > > > > > >> > > > > > >>> > > > is
> >> > > > > > >> > > > > > >>> > > > > a
> >> > > > > > >> > > > > > >>> > > > > > > really good writeup.
> >> > > > > > >> > > > > > >>> > > > > > > > > For /brokers/topics/[topic],
> >> the
> >> > > > schema
> >> > > > > > >> version
> >> > > > > > >> > > > > should
> >> > > > > > >> > > > > > be
> >> > > > > > >> > > > > > >>> > bumped
> >> > > > > > >> > > > > > >>> > > > to
> >> > > > > > >> > > > > > >>> > > > > > > version 3, rather than 2.
> KIP-455
> >> > > bumped
> >> > > > > the
> >> > > > > > >> > version
> >> > > > > > >> > > > of
> >> > > > > > >> > > > > > this
> >> > > > > > >> > > > > > >>> > znode
> >> > > > > > >> > > > > > >>> > > > to
> >> > > > > > >> > > > > > >>> > > > > 2
> >> > > > > > >> > > > > > >>> > > > > > > already :)
> >> > > > > > >> > > > > > >>> > > > > > > > > Given that we're going to be
> >> > seeing
> >> > > > > these
> >> > > > > > >> > things
> >> > > > > > >> > > as
> >> > > > > > >> > > > > > >>> strings
> >> > > > > > >> > > > > > >>> > as
> >> > > > > > >> > > > > > >>> > > > lot
> >> > > > > > >> > > > > > >>> > > > > (in
> >> > > > > > >> > > > > > >>> > > > > > > logs, in ZooKeeper, on the
> >> > command-line,
> >> > > > > > etc.),
> >> > > > > > >> > does
> >> > > > > > >> > > it
> >> > > > > > >> > > > > > make
> >> > > > > > >> > > > > > >>> > sense to
> >> > > > > > >> > > > > > >>> > > > > use
> >> > > > > > >> > > > > > >>> > > > > > > base64 when converting them to
> >> > strings?
> >> > > > > > >> > > > > > >>> > > > > > > > > Here is an example of the hex
> >> > > > > > >> representation:
> >> > > > > > >> > > > > > >>> > > > > > > > >
> >> > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> >> > > > > > >> > > > > > >>> > > > > > > > >
> >> > > > > > >> > > > > > >>> > > > > > > > > And here is an example in
> >> base64.
> >> > > > > > >> > > > > > >>> > > > > > > > > b8tRS7h4TJ2Vt43Dp85v2A
> >> > > > > > >> > > > > > >>> > > > > > > > >
> >> > > > > > >> > > > > > >>> > > > > > > > > The base64 version saves 15
> >> > letters
> >> > > > (to
> >> > > > > be
> >> > > > > > >> > fair,
> >> > > > > > >> > > 4
> >> > > > > > >> > > > of
> >> > > > > > >> > > > > > >>> those
> >> > > > > > >> > > > > > >>> > were
> >> > > > > > >> > > > > > >>> > > > > > > dashes that we could have elided
> in
> >> > the
> >> > > > hex
> >> > > > > > >> > > > > > representation.)
> >> > > > > > >> > > > > > >>> > > > > > > > > Another thing to consider is
> >> that
> >> > we
> >> > > > > > should
> >> > > > > > >> > > specify
> >> > > > > > >> > > > > > that
> >> > > > > > >> > > > > > >>> the
> >> > > > > > >> > > > > > >>> > > > > > > all-zeroes UUID is not a valid
> >> topic
> >> > > UUID.
> >> > > > > >  We
> >> > > > > > >> > can't
> >> > > > > > >> > > > use
> >> > > > > > >> > > > > > >>> null
> >> > > > > > >> > > > > > >>> > for
> >> > > > > > >> > > > > > >>> > > > this
> >> > > > > > >> > > > > > >>> > > > > > > because we can't pass a null UUID
> >> over
> >> > > the
> >> > > > > RPC
> >> > > > > > >> > > protocol
> >> > > > > > >> > > > > > >>> (there
> >> > > > > > >> > > > > > >>> > is no
> >> > > > > > >> > > > > > >>> > > > > > > special pattern for null, nor do
> we
> >> > want
> >> > > > to
> >> > > > > > >> waste
> >> > > > > > >> > > space
> >> > > > > > >> > > > > > >>> reserving
> >> > > > > > >> > > > > > >>> > > > such
> >> > > > > > >> > > > > > >>> > > > > a
> >> > > > > > >> > > > > > >>> > > > > > > pattern.)
> >> > > > > > >> > > > > > >>> > > > > > > > > Maybe I missed it, but did
> you
> >> > > > describe
> >> > > > > > >> > > "migration
> >> > > > > > >> > > > > > of...
> >> > > > > > >> > > > > > >>> > existing
> >> > > > > > >> > > > > > >>> > > > > > > topic[s] without topic IDs" in
> >> detail
> >> > in
> >> > > > any
> >> > > > > > >> > section?
> >> > > > > > >> > > > It
> >> > > > > > >> > > > > > >>> seems
> >> > > > > > >> > > > > > >>> > like
> >> > > > > > >> > > > > > >>> > > > > when
> >> > > > > > >> > > > > > >>> > > > > > > the new controller becomes
> active,
> >> it
> >> > > > should
> >> > > > > > >> just
> >> > > > > > >> > > > > generate
> >> > > > > > >> > > > > > >>> random
> >> > > > > > >> > > > > > >>> > > > > UUIDs for
> >> > > > > > >> > > > > > >>> > > > > > > these, and write the random UUIDs
> >> back
> >> > > to
> >> > > > > > >> > ZooKeeper.
> >> > > > > > >> > > > It
> >> > > > > > >> > > > > > >>> would be
> >> > > > > > >> > > > > > >>> > > > good
> >> > > > > > >> > > > > > >>> > > > > to
> >> > > > > > >> > > > > > >>> > > > > > > spell that out.  We should make
> it
> >> > clear
> >> > > > > that
> >> > > > > > >> this
> >> > > > > > >> > > > > happens
> >> > > > > > >> > > > > > >>> > regardless
> >> > > > > > >> > > > > > >>> > > > > of
> >> > > > > > >> > > > > > >>> > > > > > > the inter-broker protocol version
> >> > (it's
> >> > > a
> >> > > > > > >> > compatible
> >> > > > > > >> > > > > > change).
> >> > > > > > >> > > > > > >>> > > > > > > > > "LeaderAndIsrRequests
> >> including an
> >> > > > > > >> > > > is_every_partition
> >> > > > > > >> > > > > > >>> flag"
> >> > > > > > >> > > > > > >>> > > > seems a
> >> > > > > > >> > > > > > >>> > > > > > > bit wordy.  Can we just call
> these
> >> > "full
> >> > > > > > >> > > > > > >>> LeaderAndIsrRequests"?
> >> > > > > > >> > > > > > >>> > Then
> >> > > > > > >> > > > > > >>> > > > > the
> >> > > > > > >> > > > > > >>> > > > > > > RPC field could be named "full".
> >> > Also,
> >> > > it
> >> > > > > > would
> >> > > > > > >> > > > probably
> >> > > > > > >> > > > > > be
> >> > > > > > >> > > > > > >>> > better
> >> > > > > > >> > > > > > >>> > > > > for the
> >> > > > > > >> > > > > > >>> > > > > > > RPC field to be an enum of {
> >> > > UNSPECIFIED,
> >> > > > > > >> > > INCREMENTAL,
> >> > > > > > >> > > > > FULL
> >> > > > > > >> > > > > > >>> }, so
> >> > > > > > >> > > > > > >>> > > > that
> >> > > > > > >> > > > > > >>> > > > > we
> >> > > > > > >> > > > > > >>> > > > > > > can cleanly handle old versions
> (by
> >> > > > treating
> >> > > > > > >> them
> >> > > > > > >> > as
> >> > > > > > >> > > > > > >>> UNSPECIFIED)
> >> > > > > > >> > > > > > >>> > > > > > > > > In the LeaderAndIsrRequest
> >> > section,
> >> > > > you
> >> > > > > > >> write
> >> > > > > > >> > "A
> >> > > > > > >> > > > > final
> >> > > > > > >> > > > > > >>> > deletion
> >> > > > > > >> > > > > > >>> > > > > event
> >> > > > > > >> > > > > > >>> > > > > > > will be secheduled for X ms after
> >> the
> >> > > > > > >> > > > LeaderAndIsrRequest
> >> > > > > > >> > > > > > was
> >> > > > > > >> > > > > > >>> > first
> >> > > > > > >> > > > > > >>> > > > > > > received..."  I guess the X was a
> >> > > > > placeholder
> >> > > > > > >> that
> >> > > > > > >> > > you
> >> > > > > > >> > > > > > >>> intended
> >> > > > > > >> > > > > > >>> > to
> >> > > > > > >> > > > > > >>> > > > > replace
> >> > > > > > >> > > > > > >>> > > > > > > before posting? :)  In any case,
> >> this
> >> > > > seems
> >> > > > > > like
> >> > > > > > >> > the
> >> > > > > > >> > > > kind
> >> > > > > > >> > > > > > of
> >> > > > > > >> > > > > > >>> > thing
> >> > > > > > >> > > > > > >>> > > > we'd
> >> > > > > > >> > > > > > >>> > > > > > > want a configuration for.  Let's
> >> > > describe
> >> > > > > that
> >> > > > > > >> > > > > > configuration
> >> > > > > > >> > > > > > >>> key
> >> > > > > > >> > > > > > >>> > > > > somewhere
> >> > > > > > >> > > > > > >>> > > > > > > in this KIP, including what its
> >> > default
> >> > > > > value
> >> > > > > > >> is.
> >> > > > > > >> > > > > > >>> > > > > > > > > We should probably also log a
> >> > bunch
> >> > > of
> >> > > > > > >> messages
> >> > > > > > >> > > at
> >> > > > > > >> > > > > WARN
> >> > > > > > >> > > > > > >>> level
> >> > > > > > >> > > > > > >>> > > > when
> >> > > > > > >> > > > > > >>> > > > > > > something is scheduled for
> >> deletion,
> >> > as
> >> > > > > well.
> >> > > > > > >> > (Maybe
> >> > > > > > >> > > > > this
> >> > > > > > >> > > > > > >>> was
> >> > > > > > >> > > > > > >>> > > > > assumed, but
> >> > > > > > >> > > > > > >>> > > > > > > it would be good to mention it).
> >> > > > > > >> > > > > > >>> > > > > > > > > I feel like there are a few
> >> > sections
> >> > > > > that
> >> > > > > > >> > should
> >> > > > > > >> > > be
> >> > > > > > >> > > > > > >>> moved to
> >> > > > > > >> > > > > > >>> > > > > "rejected
> >> > > > > > >> > > > > > >>> > > > > > > alternatives."  For example, in
> the
> >> > > > > > DeleteTopics
> >> > > > > > >> > > > section,
> >> > > > > > >> > > > > > >>> since
> >> > > > > > >> > > > > > >>> > we're
> >> > > > > > >> > > > > > >>> > > > > not
> >> > > > > > >> > > > > > >>> > > > > > > going to do option 1 or 2, these
> >> > should
> >> > > be
> >> > > > > > moved
> >> > > > > > >> > into
> >> > > > > > >> > > > > > >>> "rejected
> >> > > > > > >> > > > > > >>> > > > > > > alternatives,"  rather than
> >> appearing
> >> > > > > inline.
> >> > > > > > >> > > Another
> >> > > > > > >> > > > > case
> >> > > > > > >> > > > > > >>> is
> >> > > > > > >> > > > > > >>> > the
> >> > > > > > >> > > > > > >>> > > > > "Should
> >> > > > > > >> > > > > > >>> > > > > > > we remove topic name from the
> >> protocol
> >> > > > where
> >> > > > > > >> > > possible"
> >> > > > > > >> > > > > > >>> section.
> >> > > > > > >> > > > > > >>> > This
> >> > > > > > >> > > > > > >>> > > > > is
> >> > > > > > >> > > > > > >>> > > > > > > clearly discussing a design
> >> > alternative
> >> > > > that
> >> > > > > > >> we're
> >> > > > > > >> > > not
> >> > > > > > >> > > > > > >>> proposing
> >> > > > > > >> > > > > > >>> > to
> >> > > > > > >> > > > > > >>> > > > > > > implement: removing the topic
> name
> >> > from
> >> > > > > those
> >> > > > > > >> > > > protocols.
> >> > > > > > >> > > > > > >>> > > > > > > > > Is it really necessary to
> have
> >> a
> >> > new
> >> > > > > > >> > > > > > >>> > /admin/delete_topics_by_id
> >> > > > > > >> > > > > > >>> > > > > path
> >> > > > > > >> > > > > > >>> > > > > > > in ZooKeeper?  It seems like we
> >> don't
> >> > > > really
> >> > > > > > >> need
> >> > > > > > >> > > this.
> >> > > > > > >> > > > > > >>> Whenever
> >> > > > > > >> > > > > > >>> > > > > there is
> >> > > > > > >> > > > > > >>> > > > > > > a new controller, we'll send out
> >> full
> >> > > > > > >> > > > > LeaderAndIsrRequests
> >> > > > > > >> > > > > > >>> which
> >> > > > > > >> > > > > > >>> > will
> >> > > > > > >> > > > > > >>> > > > > > > trigger the stale topics to be
> >> cleaned
> >> > > up.
> >> > > > > >  The
> >> > > > > > >> > > active
> >> > > > > > >> > > > > > >>> > controller
> >> > > > > > >> > > > > > >>> > > > will
> >> > > > > > >> > > > > > >>> > > > > > > also send the full
> >> LeaderAndIsrRequest
> >> > > to
> >> > > > > > >> brokers
> >> > > > > > >> > > that
> >> > > > > > >> > > > > are
> >> > > > > > >> > > > > > >>> just
> >> > > > > > >> > > > > > >>> > > > > starting
> >> > > > > > >> > > > > > >>> > > > > > > up.    So we don't really need
> this
> >> > kind
> >> > > > of
> >> > > > > > >> > two-phase
> >> > > > > > >> > > > > > commit
> >> > > > > > >> > > > > > >>> > (send
> >> > > > > > >> > > > > > >>> > > > out
> >> > > > > > >> > > > > > >>> > > > > > > StopReplicasRequest, get ACKs
> from
> >> all
> >> > > > > nodes,
> >> > > > > > >> > commit
> >> > > > > > >> > > by
> >> > > > > > >> > > > > > >>> removing
> >> > > > > > >> > > > > > >>> > > > > > > /admin/delete_topics node) any
> >> more.
> >> > > > > > >> > > > > > >>> > > > > > > > > You mention that FetchRequest
> >> will
> >> > > now
> >> > > > > > >> include
> >> > > > > > >> > > UUID
> >> > > > > > >> > > > > to
> >> > > > > > >> > > > > > >>> avoid
> >> > > > > > >> > > > > > >>> > > > issues
> >> > > > > > >> > > > > > >>> > > > > > > where requests are made to stale
> >> > > > partitions.
> >> > > > > > >> > > However,
> >> > > > > > >> > > > > > >>> adding a
> >> > > > > > >> > > > > > >>> > UUID
> >> > > > > > >> > > > > > >>> > > > to
> >> > > > > > >> > > > > > >>> > > > > > > MetadataRequest is listed as
> future
> >> > > work,
> >> > > > > out
> >> > > > > > of
> >> > > > > > >> > > scope
> >> > > > > > >> > > > > for
> >> > > > > > >> > > > > > >>> this
> >> > > > > > >> > > > > > >>> > KIP.
> >> > > > > > >> > > > > > >>> > > > > How
> >> > > > > > >> > > > > > >>> > > > > > > will the client learn what the
> >> topic
> >> > > UUID
> >> > > > > is,
> >> > > > > > if
> >> > > > > > >> > the
> >> > > > > > >> > > > > > metadata
> >> > > > > > >> > > > > > >>> > > > response
> >> > > > > > >> > > > > > >>> > > > > > > doesn't include that information?
> >> It
> >> > > > seems
> >> > > > > > like
> >> > > > > > >> > > adding
> >> > > > > > >> > > > > the
> >> > > > > > >> > > > > > >>> UUID
> >> > > > > > >> > > > > > >>> > to
> >> > > > > > >> > > > > > >>> > > > > > > MetadataResponse would be an
> >> > improvement
> >> > > > > here
> >> > > > > > >> that
> >> > > > > > >> > > > might
> >> > > > > > >> > > > > > not
> >> > > > > > >> > > > > > >>> be
> >> > > > > > >> > > > > > >>> > too
> >> > > > > > >> > > > > > >>> > > > > hard to
> >> > > > > > >> > > > > > >>> > > > > > > make.
> >> > > > > > >> > > > > > >>> > > > > > > > > best,
> >> > > > > > >> > > > > > >>> > > > > > > > > Colin
> >> > > > > > >> > > > > > >>> > > > > > > > >
> >> > > > > > >> > > > > > >>> > > > > > > > >
> >> > > > > > >> > > > > > >>> > > > > > > > > On Mon, Sep 9, 2019, at
> 17:48,
> >> > > Ryanne
> >> > > > > > Dolan
> >> > > > > > >> > > wrote:
> >> > > > > > >> > > > > > >>> > > > > > > > > > Lucas, this would be great.
> >> I've
> >> > > run
> >> > > > > > into
> >> > > > > > >> > > issues
> >> > > > > > >> > > > > with
> >> > > > > > >> > > > > > >>> > topics
> >> > > > > > >> > > > > > >>> > > > > being
> >> > > > > > >> > > > > > >>> > > > > > > > > > resurrected accidentally,
> >> since
> >> > a
> >> > > > > client
> >> > > > > > >> > cannot
> >> > > > > > >> > > > > > easily
> >> > > > > > >> > > > > > >>> > > > > distinguish
> >> > > > > > >> > > > > > >>> > > > > > > between
> >> > > > > > >> > > > > > >>> > > > > > > > > > a deleted topic and a new
> >> topic
> >> > > with
> >> > > > > the
> >> > > > > > >> same
> >> > > > > > >> > > > name.
> >> > > > > > >> > > > > > I'd
> >> > > > > > >> > > > > > >>> > need
> >> > > > > > >> > > > > > >>> > > > the
> >> > > > > > >> > > > > > >>> > > > > ID
> >> > > > > > >> > > > > > >>> > > > > > > > > > accessible from the client
> to
> >> > > solve
> >> > > > > that
> >> > > > > > >> > issue,
> >> > > > > > >> > > > but
> >> > > > > > >> > > > > > >>> this
> >> > > > > > >> > > > > > >>> > is a
> >> > > > > > >> > > > > > >>> > > > > good
> >> > > > > > >> > > > > > >>> > > > > > > first
> >> > > > > > >> > > > > > >>> > > > > > > > > > step.
> >> > > > > > >> > > > > > >>> > > > > > > > > >
> >> > > > > > >> > > > > > >>> > > > > > > > > > Ryanne
> >> > > > > > >> > > > > > >>> > > > > > > > > >
> >> > > > > > >> > > > > > >>> > > > > > > > > > On Wed, Sep 4, 2019 at 1:41
> >> PM
> >> > > Lucas
> >> > > > > > >> > > Bradstreet <
> >> > > > > > >> > > > > > >>> > > > > lucas@confluent.io>
> >> > > > > > >> > > > > > >>> > > > > > > wrote:
> >> > > > > > >> > > > > > >>> > > > > > > > > > > Hi all,
> >> > > > > > >> > > > > > >>> > > > > > > > > > >
> >> > > > > > >> > > > > > >>> > > > > > > > > > > I would like to kick off
> >> > > > discussion
> >> > > > > of
> >> > > > > > >> > > KIP-516,
> >> > > > > > >> > > > > an
> >> > > > > > >> > > > > > >>> > > > > implementation
> >> > > > > > >> > > > > > >>> > > > > > > of topic
> >> > > > > > >> > > > > > >>> > > > > > > > > > > IDs for Kafka. Topic IDs
> >> aim
> >> > to
> >> > > > > solve
> >> > > > > > >> topic
> >> > > > > > >> > > > > > >>> uniqueness
> >> > > > > > >> > > > > > >>> > > > > problems in
> >> > > > > > >> > > > > > >>> > > > > > > Kafka,
> >> > > > > > >> > > > > > >>> > > > > > > > > > > where referring to a
> topic
> >> by
> >> > > name
> >> > > > > > >> alone is
> >> > > > > > >> > > > > > >>> insufficient.
> >> > > > > > >> > > > > > >>> > > > Such
> >> > > > > > >> > > > > > >>> > > > > > > cases
> >> > > > > > >> > > > > > >>> > > > > > > > > > > include when a topic has
> >> been
> >> > > > > deleted
> >> > > > > > >> and
> >> > > > > > >> > > > > recreated
> >> > > > > > >> > > > > > >>> with
> >> > > > > > >> > > > > > >>> > the
> >> > > > > > >> > > > > > >>> > > > > same
> >> > > > > > >> > > > > > >>> > > > > > > name.
> >> > > > > > >> > > > > > >>> > > > > > > > > > > Unique identifiers will
> >> help
> >> > > > > simplify
> >> > > > > > >> and
> >> > > > > > >> > > > improve
> >> > > > > > >> > > > > > >>> Kafka's
> >> > > > > > >> > > > > > >>> > > > topic
> >> > > > > > >> > > > > > >>> > > > > > > deletion
> >> > > > > > >> > > > > > >>> > > > > > > > > > > process, as well as
> prevent
> >> > > cases
> >> > > > > > where
> >> > > > > > >> > > brokers
> >> > > > > > >> > > > > may
> >> > > > > > >> > > > > > >>> > > > incorrectly
> >> > > > > > >> > > > > > >>> > > > > > > interact
> >> > > > > > >> > > > > > >>> > > > > > > > > > > with stale versions of
> >> topics.
> >> > > > > > >> > > > > > >>> > > > > > > > > > >
> >> > > > > > >> > > > > > >>> > > > > > > > > > >
> >> > > > > > >> > > > > > >>> > > > > > > > > > >
> >> > > > > > >> > > > > > >>> > > >
> >> > > > > > >> > > > > > >>> >
> >> > > > > > >> > > > > > >>>
> >> > > > > > >> > > > > >
> >> > > > > > >> > > > >
> >> > > > > > >> > > >
> >> > > > > > >> > >
> >> > > > > > >> >
> >> > > > > > >>
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> >> > > > > > >> > > > > > >>> > > > > > > > > > > Looking forward to your
> >> > > thoughts.
> >> > > > > > >> > > > > > >>> > > > > > > > > > >
> >> > > > > > >> > > > > > >>> > > > > > > > > > > Lucas
> >> > > > > > >> > > > > > >>> > > > > > > > > > >
> >> > > > > > >> > > > > > >>> >
> >> > > > > > >> > > > > > >>> >
> >> > > > > > >> > > > > > >>>
> >> > > > > > >> > > > > > >>
> >> > > > > > >> > > > > >
> >> > > > > > >> > > > >
> >> > > > > > >> > > >
> >> > > > > > >> > >
> >> > > > > > >> >
> >> > > > > > >>
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Ismael Juma <is...@juma.me.uk>]

Also, can we provide more details on how the Partition Metadata file will
be used?

Ismael

On Thu, Sep 24, 2020 at 3:01 AM Ismael Juma <is...@juma.me.uk> wrote:

> Hi Justine,
>
> I think we need to update the "Rejected Alternatives" section to take into
> account that the proposal now removes the topic name from the fetch
> request. Also, if we are removing it from the Fetch request, does it make
> sense not to remove it from similar requests like ListOffsetRequest?
>
> Ismael
>
> On Thu, Sep 24, 2020 at 2:46 AM David Jacot <dj...@confluent.io> wrote:
>
>> Hi Justine,
>>
>> Thanks for the KIP. I finally had time to read it :). It is a great
>> improvement.
>>
>> I have a few comments/questions:
>>
>> 1. It seems that the schema of the StopReplicaRequest is slightly
>> outdated.
>> We
>> did some changes as part of KIP-570. V3 is already organized by topics.
>>
>> 2. I just want to make sure that I understand the reconciliation
>> logic correctly. When
>> an "INCREMENTAL" LeaderAndIsr Request is received, the broker will also
>> reconcile
>> when the local uuid does not match the uuid in the request, right? In this
>> case, the
>> local log is staged for deletion.
>>
>> 3. In the documentation of the `delete.stale.topic.delay.ms` config, it
>> says "When a
>> FULL LeaderAndIsrRequest is received..." but I suppose it applies to both
>> types.
>>
>> Best,
>> David
>>
>> On Thu, Sep 24, 2020 at 1:40 AM Justine Olshan <jo...@confluent.io>
>> wrote:
>>
>> > Hi Jun,
>> >
>> > Thanks for the comments. I apologize for some of the typos and
>> confusion.
>> > I’ve updated the KIP to fix some of the issues you mentioned.
>> >
>> > 20.2 I’ve changed the type to String.
>> > 20.1/3 I’ve updated the TopicZNode to fix formatting and reflect the
>> latest
>> > version before this change.
>> >
>> > 21. You are correct and I’ve removed this line. I’ve also added a line
>> > mentioning an IBP bump is necessary for migration
>> >
>> > 22. I think the wording was unclear but your summary is what was
>> intended
>> > by this line. I’ve updated to make this point more clear. “Remove
>> deleted
>> > topics from replicas by sending StopReplicaRequest V3 before the IBP
>> bump
>> > using the old logic, and using V4 and the new logic with topic IDs after
>> > the IBP bump.”
>> >
>> > 23. I’ve removed the unspecified type from the KIP and mention that IBP
>> > will be used to handle this request. “IBP will be used to determine
>> whether
>> > this new form of the request will be used. For older requests, we will
>> > ignore this field and default to previous behavior.”
>> >
>> > 24. I’ve fixed this typo.
>> >
>> > 25. I've added a topics at a higher level for LeaderAndIsrResponse v5,
>> > StopReplicaResponse v4. I've also changed StopReplicaRequest v4 to have
>> > topics at a higher level.
>> >
>> > 26. I’ve updated forgotten_topics_data--added the topic ID and removed
>> the
>> > topic name
>> >
>> > 27. I’ve decided on plain text, and I’ve added an example.
>> >
>> > 28. I’ve added this idea to future work.
>> >
>> > Thanks again for taking a look,
>> >
>> > Justine
>> >
>> > On Wed, Sep 23, 2020 at 10:28 AM Jun Rao <ju...@confluent.io> wrote:
>> >
>> > > Hi, Justine,
>> > >
>> > > Thanks for the response. Made another pass. A few more comments below.
>> > >
>> > > 20. znode schema:
>> > > 20.1 It seems that {"name": "version", "type": "int", "id": "UUID",
>> > "doc":
>> > > "version id"} should be {"name": "version", "type": "int"}, {"name":
>> > "id",
>> > > "type": "UUID", "doc": "version id"}.
>> > > 20.2 The znode format is JSON which doesn't have UUID type. So the
>> type
>> > > probably should be string?
>> > > 20.3 Also, the existing format used seems outdated. It should have the
>> > > following format.
>> > >     Json.encodeAsBytes(Map(
>> > >       "version" -> 2,
>> > >       "partitions" -> replicaAssignmentJson.asJava,
>> > >       "adding_replicas" -> addingReplicasAssignmentJson.asJava,
>> > >       "removing_replicas" -> removingReplicasAssignmentJson.asJava
>> > >     ).asJava)
>> > >   }
>> > >
>> > > 21. Migration: The KIP says "The migration process can take place
>> without
>> > > an inter-broker protocol bump, as the format stored in
>> > > /brokers/topics/[topic] will be compatible with older broker
>> versions."
>> > > However, since we are bumping up the version of inter-broker
>> requests, it
>> > > seems that we need to use IBP for migration.
>> > >
>> > > 22. The KIP says "Remove deleted topics from replicas by sending
>> > > StopReplicaRequest V3 for any topics which do not contain a topic ID,
>> and
>> > > V4 for any topics which do contain a topic ID.". However, if we use
>> IBP,
>> > it
>> > > seems that the controller will either send StopReplicaRequest V3
>> > > or StopReplicaRequest V4, but never mixed V3 and V4 for different
>> topics.
>> > > Basically, before the IBP bump, V3 will be used. After the IBP bump,
>> > > topicId will be created and V4 will be used.
>> > >
>> > > 23. Given that we depend on IBP, do we still need "0 UNSPECIFIED"
>> > > in LeaderAndIsr?
>> > >
>> > > 24. LeaderAndIsrResponse v5 : It still has the topic field.
>> > >
>> > > 25. LeaderAndIsrResponse v5, StopReplicaResponse v4: Could we use this
>> > > opportunity to organize the response in 2 levels, first by topic,
>> then by
>> > > partition, as most other requests/responses?
>> > >
>> > > 26. FetchRequest v13 : Should forgotten_topics_data use topicId too?
>> > >
>> > > 27. "This file can either be plain text (key/value pairs) or JSON."
>> Have
>> > we
>> > > decided which one to use? Also, it would be helpful to provide an
>> > example.
>> > >
>> > > 28. Future improvement: Another future improvement opportunity is to
>> use
>> > > topicId in GroupMetadataManager.offsetCommitKey in the offset_commit
>> > topic.
>> > > This may save some space.
>> > >
>> > > Thanks,
>> > >
>> > > Jun
>> > >
>> > > On Wed, Sep 23, 2020 at 8:50 AM Justine Olshan <jo...@confluent.io>
>> > > wrote:
>> > >
>> > > > Hi Tom,
>> > > >
>> > > > Thanks for the comment. I think this is a really good idea and it
>> has
>> > > been
>> > > > added to the KIP under the newly added tooling section.
>> > > >
>> > > > Thanks again,
>> > > > Justine
>> > > >
>> > > > On Wed, Sep 23, 2020 at 3:17 AM Tom Bentley <tb...@redhat.com>
>> > wrote:
>> > > >
>> > > > > Hi Justine,
>> > > > >
>> > > > > I know you started the vote thread, but on re-reading the KIP I
>> > noticed
>> > > > > that although the topic id is included in the MetadataResponse
>> it's
>> > not
>> > > > > surfaced in the output from `kafka-topics.sh --describe`. Maybe
>> that
>> > > was
>> > > > > intentional because ids are intentionally not really something the
>> > user
>> > > > > should care deeply about, but it would also make life harder for
>> > anyone
>> > > > > debugging Kafka and this would likely get worse the more topic ids
>> > got
>> > > > > rolled out across the protocols, clients etc. It seems likely that
>> > > > > `kafka-topics.sh` will eventually need the ability to show the id
>> of
>> > a
>> > > > > topic and perhaps find a topic name given an id. Is there any
>> reason
>> > > not
>> > > > to
>> > > > > implement that in this KIP?
>> > > > >
>> > > > > Many thanks,
>> > > > >
>> > > > > Tom
>> > > > >
>> > > > > On Mon, Sep 21, 2020 at 9:54 PM Justine Olshan <
>> jolshan@confluent.io
>> > >
>> > > > > wrote:
>> > > > >
>> > > > > > Hi all,
>> > > > > >
>> > > > > > After thinking about it, I've decided to remove the topic name
>> from
>> > > the
>> > > > > > Fetch Request and Response after all. Since there are so many of
>> > > these
>> > > > > > requests per second, it is worth removing the extra information.
>> > I've
>> > > > > > updated the KIP to reflect this change.
>> > > > > >
>> > > > > > Please let me know if there is anything else we should discuss
>> > before
>> > > > > > voting.
>> > > > > >
>> > > > > > Thank you,
>> > > > > > Justine
>> > > > > >
>> > > > > > On Fri, Sep 18, 2020 at 9:46 AM Justine Olshan <
>> > jolshan@confluent.io
>> > > >
>> > > > > > wrote:
>> > > > > >
>> > > > > > > Hi Jun,
>> > > > > > >
>> > > > > > > I see what you are saying. For now we can remove the extra
>> > > > information.
>> > > > > > > I'll leave the option to add more fields to the file in the
>> > future.
>> > > > The
>> > > > > > KIP
>> > > > > > > has been updated to reflect this change.
>> > > > > > >
>> > > > > > > Thanks,
>> > > > > > > Justine
>> > > > > > >
>> > > > > > > On Fri, Sep 18, 2020 at 8:46 AM Jun Rao <ju...@confluent.io>
>> > wrote:
>> > > > > > >
>> > > > > > >> Hi, Justine,
>> > > > > > >>
>> > > > > > >> Thanks for the reply.
>> > > > > > >>
>> > > > > > >> 13. If the log directory is the source of truth, it means
>> that
>> > the
>> > > > > > >> redundant info in the metadata file will be ignored. Then the
>> > > > question
>> > > > > > is
>> > > > > > >> why do we need to put the redundant info in the metadata file
>> > now?
>> > > > > > >>
>> > > > > > >> Thanks,
>> > > > > > >>
>> > > > > > >> Jun
>> > > > > > >>
>> > > > > > >> On Thu, Sep 17, 2020 at 5:07 PM Justine Olshan <
>> > > > jolshan@confluent.io>
>> > > > > > >> wrote:
>> > > > > > >>
>> > > > > > >> > Hi Jun,
>> > > > > > >> > Thanks for the quick response!
>> > > > > > >> >
>> > > > > > >> > 12. I've decided to bump up the versions on the requests
>> and
>> > > > updated
>> > > > > > the
>> > > > > > >> > KIP. I think it's good we thoroughly discussed the options
>> > here,
>> > > > so
>> > > > > we
>> > > > > > >> know
>> > > > > > >> > we made a good choice. :)
>> > > > > > >> >
>> > > > > > >> > 13. This is an interesting situation. I think if this does
>> > occur
>> > > > we
>> > > > > > >> should
>> > > > > > >> > give a warning. I agree that it's hard to know the source
>> of
>> > > truth
>> > > > > for
>> > > > > > >> sure
>> > > > > > >> > since the directory or the file could be manually
>> modified. I
>> > > > guess
>> > > > > > the
>> > > > > > >> > directory could be used as the source of truth. To be
>> honest,
>> > > I'm
>> > > > > not
>> > > > > > >> > really sure what happens in kafka when the log directory is
>> > > > renamed
>> > > > > > >> > manually in such a way. I'm also wondering if the
>> situation is
>> > > > > > >> recoverable
>> > > > > > >> > in this scenario.
>> > > > > > >> >
>> > > > > > >> > Thanks,
>> > > > > > >> > Justine
>> > > > > > >> >
>> > > > > > >> > On Thu, Sep 17, 2020 at 4:28 PM Jun Rao <ju...@confluent.io>
>> > > wrote:
>> > > > > > >> >
>> > > > > > >> > > Hi, Justine,
>> > > > > > >> > >
>> > > > > > >> > > Thanks for the reply.
>> > > > > > >> > >
>> > > > > > >> > > 12. I don't have a strong preference either. However, if
>> we
>> > > need
>> > > > > IBP
>> > > > > > >> > > anyway, maybe it's easier to just bump up the version for
>> > all
>> > > > > inter
>> > > > > > >> > broker
>> > > > > > >> > > requests and add the topic id field as a regular field. A
>> > > > regular
>> > > > > > >> field
>> > > > > > >> > is
>> > > > > > >> > > a bit more concise in wire transfer than a flexible
>> field.
>> > > > > > >> > >
>> > > > > > >> > > 13. The confusion that I was referring to is between the
>> > topic
>> > > > > name
>> > > > > > >> and
>> > > > > > >> > > partition number between the log dir and the metadata
>> file.
>> > > For
>> > > > > > >> example,
>> > > > > > >> > if
>> > > > > > >> > > the log dir is topicA-1 and the metadata file in it has
>> > topicB
>> > > > and
>> > > > > > >> > > partition 0 (say due to a bug or manual modification),
>> which
>> > > one
>> > > > > do
>> > > > > > we
>> > > > > > >> > use
>> > > > > > >> > > as the source of truth?
>> > > > > > >> > >
>> > > > > > >> > > Jun
>> > > > > > >> > >
>> > > > > > >> > > On Thu, Sep 17, 2020 at 3:43 PM Justine Olshan <
>> > > > > > jolshan@confluent.io>
>> > > > > > >> > > wrote:
>> > > > > > >> > >
>> > > > > > >> > > > Hi Jun,
>> > > > > > >> > > > Thanks for the comments.
>> > > > > > >> > > >
>> > > > > > >> > > > 12. I bumped the LeaderAndIsrRequest because I removed
>> the
>> > > > topic
>> > > > > > >> name
>> > > > > > >> > > field
>> > > > > > >> > > > in the response. It may be possible to avoid bumping
>> the
>> > > > version
>> > > > > > >> > without
>> > > > > > >> > > > that change, but I may be missing something.
>> > > > > > >> > > > I believe StopReplica is actually on version 3 now, but
>> > > > because
>> > > > > > >> > version 2
>> > > > > > >> > > > is flexible, I kept that listed as version 2 on the KIP
>> > > page.
>> > > > > > >> However,
>> > > > > > >> > > you
>> > > > > > >> > > > may be right in that we may need to bump the version on
>> > > > > > StopReplica
>> > > > > > >> to
>> > > > > > >> > > deal
>> > > > > > >> > > > with deletion differently as mentioned above. I don't
>> know
>> > > if
>> > > > I
>> > > > > > >> have a
>> > > > > > >> > > big
>> > > > > > >> > > > preference over used tagged fields or not.
>> > > > > > >> > > >
>> > > > > > >> > > > 13. I was thinking that in the case where the file and
>> the
>> > > > > request
>> > > > > > >> > topic
>> > > > > > >> > > > ids don't match, it means that the broker's topic/the
>> one
>> > in
>> > > > the
>> > > > > > >> file
>> > > > > > >> > has
>> > > > > > >> > > > been deleted. In that case, we would need to delete the
>> > old
>> > > > > topic
>> > > > > > >> and
>> > > > > > >> > > start
>> > > > > > >> > > > receiving the new version. If the topic name were to
>> > change,
>> > > > but
>> > > > > > the
>> > > > > > >> > ids
>> > > > > > >> > > > still matched, the file would also need to update. Am I
>> > > > missing
>> > > > > a
>> > > > > > >> case
>> > > > > > >> > > > where the file would be correct and not the request?
>> > > > > > >> > > >
>> > > > > > >> > > > Thanks,
>> > > > > > >> > > > Justine
>> > > > > > >> > > >
>> > > > > > >> > > > On Thu, Sep 17, 2020 at 3:18 PM Jun Rao <
>> jun@confluent.io
>> > >
>> > > > > wrote:
>> > > > > > >> > > >
>> > > > > > >> > > > > Hi, Justine,
>> > > > > > >> > > > >
>> > > > > > >> > > > > Thanks for the reply. A couple of more comments
>> below.
>> > > > > > >> > > > >
>> > > > > > >> > > > > 12. ListOffset and OffsetForLeader currently don't
>> > support
>> > > > > > >> flexible
>> > > > > > >> > > > fields.
>> > > > > > >> > > > > So, we have to bump up the version number and use
>> IBP at
>> > > > least
>> > > > > > for
>> > > > > > >> > > these
>> > > > > > >> > > > > two requests. Note that it seems 2.7.0 will require
>> IBP
>> > > > anyway
>> > > > > > >> > because
>> > > > > > >> > > of
>> > > > > > >> > > > > changes in KAFKA-10435. Also, it seems that the
>> version
>> > > for
>> > > > > > >> > > > > LeaderAndIsrRequest and StopReplica are bumped even
>> > though
>> > > > we
>> > > > > > only
>> > > > > > >> > > added
>> > > > > > >> > > > a
>> > > > > > >> > > > > tagged field. But since IBP is needed anyway, we may
>> > want
>> > > to
>> > > > > > >> revisit
>> > > > > > >> > > the
>> > > > > > >> > > > > overall tagged field choice.
>> > > > > > >> > > > >
>> > > > > > >> > > > > 13. The only downside is the potential confusion on
>> > which
>> > > > one
>> > > > > is
>> > > > > > >> the
>> > > > > > >> > > > source
>> > > > > > >> > > > > of truth if they don't match. Another option is to
>> > include
>> > > > > those
>> > > > > > >> > fields
>> > > > > > >> > > > in
>> > > > > > >> > > > > the metadata file when we actually change the
>> directory
>> > > > > > structure.
>> > > > > > >> > > > >
>> > > > > > >> > > > > Thanks,
>> > > > > > >> > > > >
>> > > > > > >> > > > > Jun
>> > > > > > >> > > > >
>> > > > > > >> > > > > On Thu, Sep 17, 2020 at 2:01 PM Justine Olshan <
>> > > > > > >> jolshan@confluent.io
>> > > > > > >> > >
>> > > > > > >> > > > > wrote:
>> > > > > > >> > > > >
>> > > > > > >> > > > > > Hello all,
>> > > > > > >> > > > > >
>> > > > > > >> > > > > > I've thought some more about removing the topic
>> name
>> > > field
>> > > > > > from
>> > > > > > >> > some
>> > > > > > >> > > of
>> > > > > > >> > > > > the
>> > > > > > >> > > > > > requests. On closer inspection of the
>> > > requests/responses,
>> > > > it
>> > > > > > >> seems
>> > > > > > >> > > that
>> > > > > > >> > > > > the
>> > > > > > >> > > > > > internal changes would be much larger than I
>> expected.
>> > > > Some
>> > > > > > >> > protocols
>> > > > > > >> > > > > > involve clients, so they would require changes too.
>> > I'm
>> > > > > > thinking
>> > > > > > >> > that
>> > > > > > >> > > > for
>> > > > > > >> > > > > > now, removing the topic name from these requests
>> and
>> > > > > responses
>> > > > > > >> are
>> > > > > > >> > > out
>> > > > > > >> > > > of
>> > > > > > >> > > > > > scope.
>> > > > > > >> > > > > >
>> > > > > > >> > > > > > I have decided to just keep the change
>> > > > LeaderAndIsrResponse
>> > > > > to
>> > > > > > >> > remove
>> > > > > > >> > > > the
>> > > > > > >> > > > > > topic name, and have updated the KIP to reflect
>> this
>> > > > > change. I
>> > > > > > >> have
>> > > > > > >> > > > also
>> > > > > > >> > > > > > mentioned the other requests and responses in
>> future
>> > > work.
>> > > > > > >> > > > > >
>> > > > > > >> > > > > > I'm hoping to start the voting process soon, so
>> let me
>> > > > know
>> > > > > if
>> > > > > > >> > there
>> > > > > > >> > > is
>> > > > > > >> > > > > > anything else we should discuss.
>> > > > > > >> > > > > >
>> > > > > > >> > > > > > Thank you,
>> > > > > > >> > > > > > Justine
>> > > > > > >> > > > > >
>> > > > > > >> > > > > > On Tue, Sep 15, 2020 at 3:57 PM Justine Olshan <
>> > > > > > >> > jolshan@confluent.io
>> > > > > > >> > > >
>> > > > > > >> > > > > > wrote:
>> > > > > > >> > > > > >
>> > > > > > >> > > > > > > Hello again,
>> > > > > > >> > > > > > > To follow up on some of the other comments:
>> > > > > > >> > > > > > >
>> > > > > > >> > > > > > > 10/11) We can remove the topic name from these
>> > > > > > >> > requests/responses,
>> > > > > > >> > > > and
>> > > > > > >> > > > > > > that means we will just have to make a few
>> internal
>> > > > > changes
>> > > > > > to
>> > > > > > >> > make
>> > > > > > >> > > > > > > partitions accessible by topic id and partition.
>> I
>> > can
>> > > > > > update
>> > > > > > >> the
>> > > > > > >> > > KIP
>> > > > > > >> > > > > to
>> > > > > > >> > > > > > > remove them unless anyone thinks they should
>> stay.
>> > > > > > >> > > > > > >
>> > > > > > >> > > > > > > 12) Addressed in the previous email. I've updated
>> > the
>> > > > KIP
>> > > > > to
>> > > > > > >> > > include
>> > > > > > >> > > > > > > tagged fields for the requests and responses.
>> (More
>> > on
>> > > > > that
>> > > > > > >> > below)
>> > > > > > >> > > > > > >
>> > > > > > >> > > > > > > 13) I think part of the idea for including this
>> > > > > information
>> > > > > > >> is to
>> > > > > > >> > > > > prepare
>> > > > > > >> > > > > > > for future changes. Perhaps the directory
>> structure
>> > > > might
>> > > > > > >> change
>> > > > > > >> > > from
>> > > > > > >> > > > > > > topicName_partitionNumber to something like
>> > > > > > >> > > topicID_partitionNumber.
>> > > > > > >> > > > > Then
>> > > > > > >> > > > > > > it would be useful to have the topic name in the
>> > file
>> > > > > since
>> > > > > > it
>> > > > > > >> > > would
>> > > > > > >> > > > > not
>> > > > > > >> > > > > > be
>> > > > > > >> > > > > > > in the directory structure. Supporting topic
>> renames
>> > > > might
>> > > > > > be
>> > > > > > >> > > easier
>> > > > > > >> > > > if
>> > > > > > >> > > > > > the
>> > > > > > >> > > > > > > other fields are included. Would there be any
>> > > downsides
>> > > > to
>> > > > > > >> > > including
>> > > > > > >> > > > > this
>> > > > > > >> > > > > > > information?
>> > > > > > >> > > > > > >
>> > > > > > >> > > > > > > 14)  Yes, we would need to copy the partition
>> > metadata
>> > > > > file
>> > > > > > in
>> > > > > > >> > this
>> > > > > > >> > > > > > > process. I've updated the KIP to include this.
>> > > > > > >> > > > > > >
>> > > > > > >> > > > > > > 15) I believe Lucas meant v1 and v2 here. He was
>> > > > referring
>> > > > > > to
>> > > > > > >> how
>> > > > > > >> > > the
>> > > > > > >> > > > > > > requests would fall under different IBP and meant
>> > that
>> > > > > older
>> > > > > > >> > > brokers
>> > > > > > >> > > > > > would
>> > > > > > >> > > > > > > have to use the older version of the request and
>> the
>> > > > > > existing
>> > > > > > >> > topic
>> > > > > > >> > > > > > > deletion process. At first, it seemed like tagged
>> > > fields
>> > > > > > would
>> > > > > > >> > > > resolve
>> > > > > > >> > > > > > > the IBP issue. However, we may need IBP for this
>> > > request
>> > > > > > after
>> > > > > > >> > all
>> > > > > > >> > > > > since
>> > > > > > >> > > > > > > the controller handles the topic deletion
>> > differently
>> > > > > > >> depending
>> > > > > > >> > on
>> > > > > > >> > > > the
>> > > > > > >> > > > > > IBP
>> > > > > > >> > > > > > > version. In an older version, we can't just send
>> a
>> > > > > > StopReplica
>> > > > > > >> > > delete
>> > > > > > >> > > > > the
>> > > > > > >> > > > > > > topic immediately like we'd want to for this KIP.
>> > > > > > >> > > > > > >
>> > > > > > >> > > > > > > This makes me wonder if we want tagged fields on
>> all
>> > > the
>> > > > > > >> requests
>> > > > > > >> > > > after
>> > > > > > >> > > > > > > all. Let me know your thoughts!
>> > > > > > >> > > > > > >
>> > > > > > >> > > > > > > Justine
>> > > > > > >> > > > > > >
>> > > > > > >> > > > > > > On Tue, Sep 15, 2020 at 1:03 PM Justine Olshan <
>> > > > > > >> > > jolshan@confluent.io
>> > > > > > >> > > > >
>> > > > > > >> > > > > > > wrote:
>> > > > > > >> > > > > > >
>> > > > > > >> > > > > > >> Hi all,
>> > > > > > >> > > > > > >> Jun brought up a good point in his last email
>> about
>> > > > > tagged
>> > > > > > >> > fields,
>> > > > > > >> > > > and
>> > > > > > >> > > > > > >> I've updated the KIP to reflect that the
>> changes to
>> > > > > > requests
>> > > > > > >> and
>> > > > > > >> > > > > > responses
>> > > > > > >> > > > > > >> will be in the form of tagged fields to avoid
>> > > changing
>> > > > > IBP.
>> > > > > > >> > > > > > >>
>> > > > > > >> > > > > > >> Jun: I plan on sending a followup email to
>> address
>> > > some
>> > > > > of
>> > > > > > >> the
>> > > > > > >> > > other
>> > > > > > >> > > > > > >> points.
>> > > > > > >> > > > > > >>
>> > > > > > >> > > > > > >> Thanks,
>> > > > > > >> > > > > > >> Justine
>> > > > > > >> > > > > > >>
>> > > > > > >> > > > > > >> On Mon, Sep 14, 2020 at 4:25 PM Jun Rao <
>> > > > > jun@confluent.io>
>> > > > > > >> > wrote:
>> > > > > > >> > > > > > >>
>> > > > > > >> > > > > > >>> Hi, Justine,
>> > > > > > >> > > > > > >>>
>> > > > > > >> > > > > > >>> Thanks for the updated KIP. A few comments
>> below.
>> > > > > > >> > > > > > >>>
>> > > > > > >> > > > > > >>> 10. LeaderAndIsr Response: Do we need the topic
>> > > name?
>> > > > > > >> > > > > > >>>
>> > > > > > >> > > > > > >>> 11. For the changed request/response, other
>> than
>> > > > > > >> LeaderAndIsr,
>> > > > > > >> > > > > > >>> UpdateMetadata, Metadata, do we need to include
>> > the
>> > > > > topic
>> > > > > > >> name?
>> > > > > > >> > > > > > >>>
>> > > > > > >> > > > > > >>> 12. It seems that upgrades don't require IBP.
>> Does
>> > > > that
>> > > > > > mean
>> > > > > > >> > the
>> > > > > > >> > > > new
>> > > > > > >> > > > > > >>> fields
>> > > > > > >> > > > > > >>> in all the request/response are added as tagged
>> > > fields
>> > > > > > >> without
>> > > > > > >> > > > > bumping
>> > > > > > >> > > > > > up
>> > > > > > >> > > > > > >>> the request version? It would be useful to make
>> > that
>> > > > > > clear.
>> > > > > > >> > > > > > >>>
>> > > > > > >> > > > > > >>> 13. Partition Metadata file: Do we need to
>> include
>> > > the
>> > > > > > topic
>> > > > > > >> > name
>> > > > > > >> > > > and
>> > > > > > >> > > > > > the
>> > > > > > >> > > > > > >>> partition id since they are implied in the
>> > directory
>> > > > > name?
>> > > > > > >> > > > > > >>>
>> > > > > > >> > > > > > >>> 14. In the JBOD mode, we support moving a
>> > > partition's
>> > > > > data
>> > > > > > >> from
>> > > > > > >> > > one
>> > > > > > >> > > > > > disk
>> > > > > > >> > > > > > >>> to
>> > > > > > >> > > > > > >>> another. Will the new partition metadata file
>> be
>> > > > copied
>> > > > > > >> during
>> > > > > > >> > > that
>> > > > > > >> > > > > > >>> process?
>> > > > > > >> > > > > > >>>
>> > > > > > >> > > > > > >>> 15. The KIP says "Remove deleted topics from
>> > > replicas
>> > > > by
>> > > > > > >> > sending
>> > > > > > >> > > > > > >>> StopReplicaRequest V2 for any topics which do
>> not
>> > > > > contain
>> > > > > > a
>> > > > > > >> > topic
>> > > > > > >> > > > ID,
>> > > > > > >> > > > > > and
>> > > > > > >> > > > > > >>> V3 for any topics which do contain a topic
>> ID.".
>> > > > > However,
>> > > > > > it
>> > > > > > >> > > seems
>> > > > > > >> > > > > the
>> > > > > > >> > > > > > >>> updated controller will create all missing
>> topic
>> > IDs
>> > > > > first
>> > > > > > >> > before
>> > > > > > >> > > > > doing
>> > > > > > >> > > > > > >>> other actions. So, is StopReplicaRequest V2
>> > needed?
>> > > > > > >> > > > > > >>>
>> > > > > > >> > > > > > >>> Jun
>> > > > > > >> > > > > > >>>
>> > > > > > >> > > > > > >>> On Fri, Sep 11, 2020 at 10:31 AM John Roesler <
>> > > > > > >> > > vvcephei@apache.org
>> > > > > > >> > > > >
>> > > > > > >> > > > > > >>> wrote:
>> > > > > > >> > > > > > >>>
>> > > > > > >> > > > > > >>> > Thanks, Justine!
>> > > > > > >> > > > > > >>> >
>> > > > > > >> > > > > > >>> > Your response seems compelling to me.
>> > > > > > >> > > > > > >>> >
>> > > > > > >> > > > > > >>> > -John
>> > > > > > >> > > > > > >>> >
>> > > > > > >> > > > > > >>> > On Fri, 2020-09-11 at 10:17 -0700, Justine
>> > Olshan
>> > > > > wrote:
>> > > > > > >> > > > > > >>> > > Hello all,
>> > > > > > >> > > > > > >>> > > Thanks for continuing the discussion! I
>> have a
>> > > few
>> > > > > > >> > responses
>> > > > > > >> > > to
>> > > > > > >> > > > > > your
>> > > > > > >> > > > > > >>> > points.
>> > > > > > >> > > > > > >>> > >
>> > > > > > >> > > > > > >>> > > Tom: You are correct in that this KIP has
>> not
>> > > > > > mentioned
>> > > > > > >> the
>> > > > > > >> > > > > > >>> > > DeleteTopicsRequest. I think that this
>> would
>> > be
>> > > > out
>> > > > > of
>> > > > > > >> > scope
>> > > > > > >> > > > for
>> > > > > > >> > > > > > >>> now, but
>> > > > > > >> > > > > > >>> > > may be something worth adding in the
>> future.
>> > > > > > >> > > > > > >>> > >
>> > > > > > >> > > > > > >>> > > John: We did consider sequence ids, but
>> there
>> > > are
>> > > > a
>> > > > > > few
>> > > > > > >> > > reasons
>> > > > > > >> > > > > to
>> > > > > > >> > > > > > >>> favor
>> > > > > > >> > > > > > >>> > > UUIDs. There are several cases where topics
>> > from
>> > > > > > >> different
>> > > > > > >> > > > > clusters
>> > > > > > >> > > > > > >>> may
>> > > > > > >> > > > > > >>> > > interact now and in the future. For
>> example,
>> > > > Mirror
>> > > > > > >> Maker 2
>> > > > > > >> > > may
>> > > > > > >> > > > > > >>> benefit
>> > > > > > >> > > > > > >>> > > from being able to detect when a cluster
>> being
>> > > > > > mirrored
>> > > > > > >> is
>> > > > > > >> > > > > deleted
>> > > > > > >> > > > > > >>> and
>> > > > > > >> > > > > > >>> > > recreated and globally unique identifiers
>> > would
>> > > > make
>> > > > > > >> > > resolving
>> > > > > > >> > > > > > issues
>> > > > > > >> > > > > > >>> > > easier than sequence IDs which may collide
>> > > between
>> > > > > > >> > clusters.
>> > > > > > >> > > > > > KIP-405
>> > > > > > >> > > > > > >>> > > (tiered storage) will also benefit from
>> > globally
>> > > > > > unique
>> > > > > > >> IDs
>> > > > > > >> > > as
>> > > > > > >> > > > > > shared
>> > > > > > >> > > > > > >>> > > buckets may be used between clusters.
>> > > > > > >> > > > > > >>> > >
>> > > > > > >> > > > > > >>> > > Globally unique IDs would also make
>> > > functionality
>> > > > > like
>> > > > > > >> > moving
>> > > > > > >> > > > > > topics
>> > > > > > >> > > > > > >>> > > between disparate clusters easier in the
>> > future,
>> > > > > > >> simplify
>> > > > > > >> > any
>> > > > > > >> > > > > > future
>> > > > > > >> > > > > > >>> > > implementations of backups and restores,
>> and
>> > > more.
>> > > > > In
>> > > > > > >> > > general,
>> > > > > > >> > > > > > >>> unique IDs
>> > > > > > >> > > > > > >>> > > would ensure that the source cluster
>> topics do
>> > > not
>> > > > > > >> conflict
>> > > > > > >> > > > with
>> > > > > > >> > > > > > the
>> > > > > > >> > > > > > >>> > > destination cluster topics.
>> > > > > > >> > > > > > >>> > >
>> > > > > > >> > > > > > >>> > > If we were to use sequence ids, we would
>> need
>> > > > > > >> sufficiently
>> > > > > > >> > > > large
>> > > > > > >> > > > > > >>> cluster
>> > > > > > >> > > > > > >>> > > ids to be stored with the topic
>> identifiers or
>> > > we
>> > > > > run
>> > > > > > >> the
>> > > > > > >> > > risk
>> > > > > > >> > > > of
>> > > > > > >> > > > > > >>> > > collisions. This will give up any
>> advantage in
>> > > > > > >> compactness
>> > > > > > >> > > that
>> > > > > > >> > > > > > >>> sequence
>> > > > > > >> > > > > > >>> > > numbers may bring. Given these advantages I
>> > > think
>> > > > it
>> > > > > > >> makes
>> > > > > > >> > > > sense
>> > > > > > >> > > > > to
>> > > > > > >> > > > > > >>> use
>> > > > > > >> > > > > > >>> > > UUIDs.
>> > > > > > >> > > > > > >>> > >
>> > > > > > >> > > > > > >>> > > Gokul: This is an interesting idea, but
>> this
>> > is
>> > > a
>> > > > > > >> breaking
>> > > > > > >> > > > > change.
>> > > > > > >> > > > > > >>> Out of
>> > > > > > >> > > > > > >>> > > scope for now, but maybe worth discussing
>> in
>> > the
>> > > > > > future.
>> > > > > > >> > > > > > >>> > >
>> > > > > > >> > > > > > >>> > > Hope this explains some of the decisions,
>> > > > > > >> > > > > > >>> > >
>> > > > > > >> > > > > > >>> > > Justine
>> > > > > > >> > > > > > >>> > >
>> > > > > > >> > > > > > >>> > >
>> > > > > > >> > > > > > >>> > >
>> > > > > > >> > > > > > >>> > > On Fri, Sep 11, 2020 at 8:27 AM Gokul
>> Ramanan
>> > > > > > >> Subramanian <
>> > > > > > >> > > > > > >>> > > gokul2411s@gmail.com> wrote:
>> > > > > > >> > > > > > >>> > >
>> > > > > > >> > > > > > >>> > > > Hi.
>> > > > > > >> > > > > > >>> > > >
>> > > > > > >> > > > > > >>> > > > Thanks for the KIP.
>> > > > > > >> > > > > > >>> > > >
>> > > > > > >> > > > > > >>> > > > Have you thought about whether it makes
>> > sense
>> > > to
>> > > > > > >> support
>> > > > > > >> > > > > > >>> authorizing a
>> > > > > > >> > > > > > >>> > > > principal for a topic ID rather than a
>> topic
>> > > > name
>> > > > > to
>> > > > > > >> > > achieve
>> > > > > > >> > > > > > >>> tighter
>> > > > > > >> > > > > > >>> > > > security?
>> > > > > > >> > > > > > >>> > > >
>> > > > > > >> > > > > > >>> > > > Or is the topic ID fundamentally an
>> internal
>> > > > > detail
>> > > > > > >> > similar
>> > > > > > >> > > > to
>> > > > > > >> > > > > > >>> epochs
>> > > > > > >> > > > > > >>> > used
>> > > > > > >> > > > > > >>> > > > in a bunch of other places in Kafka?
>> > > > > > >> > > > > > >>> > > >
>> > > > > > >> > > > > > >>> > > > Thanks.
>> > > > > > >> > > > > > >>> > > >
>> > > > > > >> > > > > > >>> > > > On Fri, Sep 11, 2020 at 4:06 PM John
>> > Roesler <
>> > > > > > >> > > > > > vvcephei@apache.org>
>> > > > > > >> > > > > > >>> > wrote:
>> > > > > > >> > > > > > >>> > > >
>> > > > > > >> > > > > > >>> > > > > Hello Justine,
>> > > > > > >> > > > > > >>> > > > >
>> > > > > > >> > > > > > >>> > > > > Thanks for the KIP!
>> > > > > > >> > > > > > >>> > > > >
>> > > > > > >> > > > > > >>> > > > > I happen to have been confronted
>> recently
>> > > with
>> > > > > the
>> > > > > > >> need
>> > > > > > >> > > to
>> > > > > > >> > > > > keep
>> > > > > > >> > > > > > >>> > track of
>> > > > > > >> > > > > > >>> > > > a
>> > > > > > >> > > > > > >>> > > > > large number of topics as compactly as
>> > > > > possible. I
>> > > > > > >> was
>> > > > > > >> > > > going
>> > > > > > >> > > > > to
>> > > > > > >> > > > > > >>> come
>> > > > > > >> > > > > > >>> > up
>> > > > > > >> > > > > > >>> > > > > with some way to dictionary encode the
>> > topic
>> > > > > names
>> > > > > > >> as
>> > > > > > >> > > > > integers,
>> > > > > > >> > > > > > >>> but
>> > > > > > >> > > > > > >>> > this
>> > > > > > >> > > > > > >>> > > > > seems much better!
>> > > > > > >> > > > > > >>> > > > >
>> > > > > > >> > > > > > >>> > > > > Apologies if this has been raised
>> before,
>> > > but
>> > > > > I’m
>> > > > > > >> > > wondering
>> > > > > > >> > > > > > >>> about the
>> > > > > > >> > > > > > >>> > > > > choice of UUID vs sequence number for
>> the
>> > > ids.
>> > > > > > >> > Typically,
>> > > > > > >> > > > > I’ve
>> > > > > > >> > > > > > >>> seen
>> > > > > > >> > > > > > >>> > UUIDs
>> > > > > > >> > > > > > >>> > > > > in two situations:
>> > > > > > >> > > > > > >>> > > > > 1. When processes need to generate
>> > > > non-colliding
>> > > > > > >> > > > identifiers
>> > > > > > >> > > > > > >>> without
>> > > > > > >> > > > > > >>> > > > > coordination.
>> > > > > > >> > > > > > >>> > > > > 2. When the identifier needs to be
>> > > > “universally
>> > > > > > >> > unique”;
>> > > > > > >> > > > > I.e.,
>> > > > > > >> > > > > > >>> the
>> > > > > > >> > > > > > >>> > > > > identifier needs to distinguish the
>> entity
>> > > > from
>> > > > > > all
>> > > > > > >> > other
>> > > > > > >> > > > > > >>> entities
>> > > > > > >> > > > > > >>> > that
>> > > > > > >> > > > > > >>> > > > > could ever exist. This is useful in
>> cases
>> > > > where
>> > > > > > >> > entities
>> > > > > > >> > > > from
>> > > > > > >> > > > > > all
>> > > > > > >> > > > > > >>> > kinds
>> > > > > > >> > > > > > >>> > > > of
>> > > > > > >> > > > > > >>> > > > > systems get mixed together, such as
>> when
>> > > > dumping
>> > > > > > >> logs
>> > > > > > >> > > from
>> > > > > > >> > > > > all
>> > > > > > >> > > > > > >>> > processes
>> > > > > > >> > > > > > >>> > > > in
>> > > > > > >> > > > > > >>> > > > > a company into a common system.
>> > > > > > >> > > > > > >>> > > > >
>> > > > > > >> > > > > > >>> > > > > Maybe I’m being short-sighted, but it
>> > > doesn’t
>> > > > > seem
>> > > > > > >> like
>> > > > > > >> > > > > either
>> > > > > > >> > > > > > >>> really
>> > > > > > >> > > > > > >>> > > > > applies here. It seems like the brokers
>> > > could
>> > > > > and
>> > > > > > >> would
>> > > > > > >> > > > > achieve
>> > > > > > >> > > > > > >>> > consensus
>> > > > > > >> > > > > > >>> > > > > when creating a topic anyway, which is
>> all
>> > > > > that’s
>> > > > > > >> > > required
>> > > > > > >> > > > to
>> > > > > > >> > > > > > >>> > generate
>> > > > > > >> > > > > > >>> > > > > non-colliding sequence ids. For the
>> > second,
>> > > as
>> > > > > you
>> > > > > > >> > > mention,
>> > > > > > >> > > > > we
>> > > > > > >> > > > > > >>> could
>> > > > > > >> > > > > > >>> > > > assign
>> > > > > > >> > > > > > >>> > > > > a UUID to the cluster as a whole, which
>> > > would
>> > > > > > render
>> > > > > > >> > any
>> > > > > > >> > > > > > resource
>> > > > > > >> > > > > > >>> > scoped
>> > > > > > >> > > > > > >>> > > > to
>> > > > > > >> > > > > > >>> > > > > the broker universally unique as well.
>> > > > > > >> > > > > > >>> > > > >
>> > > > > > >> > > > > > >>> > > > > The reason I mention this is that,
>> > although
>> > > a
>> > > > > UUID
>> > > > > > >> is
>> > > > > > >> > way
>> > > > > > >> > > > > more
>> > > > > > >> > > > > > >>> > compact
>> > > > > > >> > > > > > >>> > > > > than topic names, it’s still 16 bytes.
>> In
>> > > > > > contrast,
>> > > > > > >> a
>> > > > > > >> > > > 4-byte
>> > > > > > >> > > > > > >>> integer
>> > > > > > >> > > > > > >>> > > > > sequence id would give us 4 billion
>> unique
>> > > > > topics
>> > > > > > >> per
>> > > > > > >> > > > > cluster,
>> > > > > > >> > > > > > >>> which
>> > > > > > >> > > > > > >>> > > > seems
>> > > > > > >> > > > > > >>> > > > > like enough ;)
>> > > > > > >> > > > > > >>> > > > >
>> > > > > > >> > > > > > >>> > > > > Considering the number of different
>> times
>> > > > these
>> > > > > > >> topic
>> > > > > > >> > > > > > >>> identifiers are
>> > > > > > >> > > > > > >>> > > > sent
>> > > > > > >> > > > > > >>> > > > > over the wire or stored in memory, it
>> > seems
>> > > > like
>> > > > > > it
>> > > > > > >> > might
>> > > > > > >> > > > be
>> > > > > > >> > > > > > >>> worth
>> > > > > > >> > > > > > >>> > the
>> > > > > > >> > > > > > >>> > > > > additional 4x space savings.
>> > > > > > >> > > > > > >>> > > > >
>> > > > > > >> > > > > > >>> > > > > What do you think about this?
>> > > > > > >> > > > > > >>> > > > >
>> > > > > > >> > > > > > >>> > > > > Thanks,
>> > > > > > >> > > > > > >>> > > > > John
>> > > > > > >> > > > > > >>> > > > >
>> > > > > > >> > > > > > >>> > > > > On Fri, Sep 11, 2020, at 03:20, Tom
>> > Bentley
>> > > > > wrote:
>> > > > > > >> > > > > > >>> > > > > > Hi Justine,
>> > > > > > >> > > > > > >>> > > > > >
>> > > > > > >> > > > > > >>> > > > > > This looks like a very welcome
>> > > improvement.
>> > > > > > >> Thanks!
>> > > > > > >> > > > > > >>> > > > > >
>> > > > > > >> > > > > > >>> > > > > > Maybe I missed it, but the KIP
>> doesn't
>> > > seem
>> > > > to
>> > > > > > >> > mention
>> > > > > > >> > > > > > changing
>> > > > > > >> > > > > > >>> > > > > > DeleteTopicsRequest to identify the
>> > topic
>> > > > > using
>> > > > > > an
>> > > > > > >> > id.
>> > > > > > >> > > > > Maybe
>> > > > > > >> > > > > > >>> > that's out
>> > > > > > >> > > > > > >>> > > > > of
>> > > > > > >> > > > > > >>> > > > > > scope, but DeleteTopicsRequest is not
>> > > listed
>> > > > > > among
>> > > > > > >> > the
>> > > > > > >> > > > > Future
>> > > > > > >> > > > > > >>> Work
>> > > > > > >> > > > > > >>> > APIs
>> > > > > > >> > > > > > >>> > > > > > either.
>> > > > > > >> > > > > > >>> > > > > >
>> > > > > > >> > > > > > >>> > > > > > Kind regards,
>> > > > > > >> > > > > > >>> > > > > >
>> > > > > > >> > > > > > >>> > > > > > Tom
>> > > > > > >> > > > > > >>> > > > > >
>> > > > > > >> > > > > > >>> > > > > > On Thu, Sep 10, 2020 at 3:59 PM
>> Satish
>> > > > > Duggana <
>> > > > > > >> > > > > > >>> > > > satish.duggana@gmail.com
>> > > > > > >> > > > > > >>> > > > > > wrote:
>> > > > > > >> > > > > > >>> > > > > >
>> > > > > > >> > > > > > >>> > > > > > > Thanks Lucas/Justine for the nice
>> KIP.
>> > > > > > >> > > > > > >>> > > > > > >
>> > > > > > >> > > > > > >>> > > > > > > It has several benefits which also
>> > > include
>> > > > > > >> > > simplifying
>> > > > > > >> > > > > the
>> > > > > > >> > > > > > >>> topic
>> > > > > > >> > > > > > >>> > > > > > > deletion process by controller and
>> > logs
>> > > > > > cleanup
>> > > > > > >> by
>> > > > > > >> > > > > brokers
>> > > > > > >> > > > > > in
>> > > > > > >> > > > > > >>> > corner
>> > > > > > >> > > > > > >>> > > > > > > cases.
>> > > > > > >> > > > > > >>> > > > > > >
>> > > > > > >> > > > > > >>> > > > > > > Best,
>> > > > > > >> > > > > > >>> > > > > > > Satish.
>> > > > > > >> > > > > > >>> > > > > > >
>> > > > > > >> > > > > > >>> > > > > > > On Wed, Sep 9, 2020 at 10:07 PM
>> > Justine
>> > > > > > Olshan <
>> > > > > > >> > > > > > >>> > jolshan@confluent.io
>> > > > > > >> > > > > > >>> > > > > > > wrote:
>> > > > > > >> > > > > > >>> > > > > > > > Hello all, it's been almost a
>> year!
>> > > I've
>> > > > > > made
>> > > > > > >> > some
>> > > > > > >> > > > > > changes
>> > > > > > >> > > > > > >>> to
>> > > > > > >> > > > > > >>> > this
>> > > > > > >> > > > > > >>> > > > > KIP
>> > > > > > >> > > > > > >>> > > > > > > and hope to continue the
>> discussion.
>> > > > > > >> > > > > > >>> > > > > > > > One of the main changes I've
>> added
>> > is
>> > > > now
>> > > > > > the
>> > > > > > >> > > > metadata
>> > > > > > >> > > > > > >>> response
>> > > > > > >> > > > > > >>> > > > will
>> > > > > > >> > > > > > >>> > > > > > > include the topic ID (as Colin
>> > > suggested).
>> > > > > > >> Clients
>> > > > > > >> > > can
>> > > > > > >> > > > > > >>> obtain the
>> > > > > > >> > > > > > >>> > > > > topicID
>> > > > > > >> > > > > > >>> > > > > > > of a given topic through a
>> > > > TopicDescription.
>> > > > > > The
>> > > > > > >> > > > topicId
>> > > > > > >> > > > > > will
>> > > > > > >> > > > > > >>> > also be
>> > > > > > >> > > > > > >>> > > > > > > included with the UpdateMetadata
>> > > request.
>> > > > > > >> > > > > > >>> > > > > > > > Let me know what you all think.
>> > > > > > >> > > > > > >>> > > > > > > > Thank you,
>> > > > > > >> > > > > > >>> > > > > > > > Justine
>> > > > > > >> > > > > > >>> > > > > > > >
>> > > > > > >> > > > > > >>> > > > > > > > On 2019/09/13 16:38:26, "Colin
>> > > McCabe" <
>> > > > > > >> > > > > > cmccabe@apache.org
>> > > > > > >> > > > > > >>> >
>> > > > > > >> > > > > > >>> > wrote:
>> > > > > > >> > > > > > >>> > > > > > > > > Hi Lucas,
>> > > > > > >> > > > > > >>> > > > > > > > >
>> > > > > > >> > > > > > >>> > > > > > > > > Thanks for tackling this.
>> Topic
>> > IDs
>> > > > > are a
>> > > > > > >> > great
>> > > > > > >> > > > > idea,
>> > > > > > >> > > > > > >>> and
>> > > > > > >> > > > > > >>> > this
>> > > > > > >> > > > > > >>> > > > is
>> > > > > > >> > > > > > >>> > > > > a
>> > > > > > >> > > > > > >>> > > > > > > really good writeup.
>> > > > > > >> > > > > > >>> > > > > > > > > For /brokers/topics/[topic],
>> the
>> > > > schema
>> > > > > > >> version
>> > > > > > >> > > > > should
>> > > > > > >> > > > > > be
>> > > > > > >> > > > > > >>> > bumped
>> > > > > > >> > > > > > >>> > > > to
>> > > > > > >> > > > > > >>> > > > > > > version 3, rather than 2.  KIP-455
>> > > bumped
>> > > > > the
>> > > > > > >> > version
>> > > > > > >> > > > of
>> > > > > > >> > > > > > this
>> > > > > > >> > > > > > >>> > znode
>> > > > > > >> > > > > > >>> > > > to
>> > > > > > >> > > > > > >>> > > > > 2
>> > > > > > >> > > > > > >>> > > > > > > already :)
>> > > > > > >> > > > > > >>> > > > > > > > > Given that we're going to be
>> > seeing
>> > > > > these
>> > > > > > >> > things
>> > > > > > >> > > as
>> > > > > > >> > > > > > >>> strings
>> > > > > > >> > > > > > >>> > as
>> > > > > > >> > > > > > >>> > > > lot
>> > > > > > >> > > > > > >>> > > > > (in
>> > > > > > >> > > > > > >>> > > > > > > logs, in ZooKeeper, on the
>> > command-line,
>> > > > > > etc.),
>> > > > > > >> > does
>> > > > > > >> > > it
>> > > > > > >> > > > > > make
>> > > > > > >> > > > > > >>> > sense to
>> > > > > > >> > > > > > >>> > > > > use
>> > > > > > >> > > > > > >>> > > > > > > base64 when converting them to
>> > strings?
>> > > > > > >> > > > > > >>> > > > > > > > > Here is an example of the hex
>> > > > > > >> representation:
>> > > > > > >> > > > > > >>> > > > > > > > >
>> > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
>> > > > > > >> > > > > > >>> > > > > > > > >
>> > > > > > >> > > > > > >>> > > > > > > > > And here is an example in
>> base64.
>> > > > > > >> > > > > > >>> > > > > > > > > b8tRS7h4TJ2Vt43Dp85v2A
>> > > > > > >> > > > > > >>> > > > > > > > >
>> > > > > > >> > > > > > >>> > > > > > > > > The base64 version saves 15
>> > letters
>> > > > (to
>> > > > > be
>> > > > > > >> > fair,
>> > > > > > >> > > 4
>> > > > > > >> > > > of
>> > > > > > >> > > > > > >>> those
>> > > > > > >> > > > > > >>> > were
>> > > > > > >> > > > > > >>> > > > > > > dashes that we could have elided in
>> > the
>> > > > hex
>> > > > > > >> > > > > > representation.)
>> > > > > > >> > > > > > >>> > > > > > > > > Another thing to consider is
>> that
>> > we
>> > > > > > should
>> > > > > > >> > > specify
>> > > > > > >> > > > > > that
>> > > > > > >> > > > > > >>> the
>> > > > > > >> > > > > > >>> > > > > > > all-zeroes UUID is not a valid
>> topic
>> > > UUID.
>> > > > > >  We
>> > > > > > >> > can't
>> > > > > > >> > > > use
>> > > > > > >> > > > > > >>> null
>> > > > > > >> > > > > > >>> > for
>> > > > > > >> > > > > > >>> > > > this
>> > > > > > >> > > > > > >>> > > > > > > because we can't pass a null UUID
>> over
>> > > the
>> > > > > RPC
>> > > > > > >> > > protocol
>> > > > > > >> > > > > > >>> (there
>> > > > > > >> > > > > > >>> > is no
>> > > > > > >> > > > > > >>> > > > > > > special pattern for null, nor do we
>> > want
>> > > > to
>> > > > > > >> waste
>> > > > > > >> > > space
>> > > > > > >> > > > > > >>> reserving
>> > > > > > >> > > > > > >>> > > > such
>> > > > > > >> > > > > > >>> > > > > a
>> > > > > > >> > > > > > >>> > > > > > > pattern.)
>> > > > > > >> > > > > > >>> > > > > > > > > Maybe I missed it, but did you
>> > > > describe
>> > > > > > >> > > "migration
>> > > > > > >> > > > > > of...
>> > > > > > >> > > > > > >>> > existing
>> > > > > > >> > > > > > >>> > > > > > > topic[s] without topic IDs" in
>> detail
>> > in
>> > > > any
>> > > > > > >> > section?
>> > > > > > >> > > > It
>> > > > > > >> > > > > > >>> seems
>> > > > > > >> > > > > > >>> > like
>> > > > > > >> > > > > > >>> > > > > when
>> > > > > > >> > > > > > >>> > > > > > > the new controller becomes active,
>> it
>> > > > should
>> > > > > > >> just
>> > > > > > >> > > > > generate
>> > > > > > >> > > > > > >>> random
>> > > > > > >> > > > > > >>> > > > > UUIDs for
>> > > > > > >> > > > > > >>> > > > > > > these, and write the random UUIDs
>> back
>> > > to
>> > > > > > >> > ZooKeeper.
>> > > > > > >> > > > It
>> > > > > > >> > > > > > >>> would be
>> > > > > > >> > > > > > >>> > > > good
>> > > > > > >> > > > > > >>> > > > > to
>> > > > > > >> > > > > > >>> > > > > > > spell that out.  We should make it
>> > clear
>> > > > > that
>> > > > > > >> this
>> > > > > > >> > > > > happens
>> > > > > > >> > > > > > >>> > regardless
>> > > > > > >> > > > > > >>> > > > > of
>> > > > > > >> > > > > > >>> > > > > > > the inter-broker protocol version
>> > (it's
>> > > a
>> > > > > > >> > compatible
>> > > > > > >> > > > > > change).
>> > > > > > >> > > > > > >>> > > > > > > > > "LeaderAndIsrRequests
>> including an
>> > > > > > >> > > > is_every_partition
>> > > > > > >> > > > > > >>> flag"
>> > > > > > >> > > > > > >>> > > > seems a
>> > > > > > >> > > > > > >>> > > > > > > bit wordy.  Can we just call these
>> > "full
>> > > > > > >> > > > > > >>> LeaderAndIsrRequests"?
>> > > > > > >> > > > > > >>> > Then
>> > > > > > >> > > > > > >>> > > > > the
>> > > > > > >> > > > > > >>> > > > > > > RPC field could be named "full".
>> > Also,
>> > > it
>> > > > > > would
>> > > > > > >> > > > probably
>> > > > > > >> > > > > > be
>> > > > > > >> > > > > > >>> > better
>> > > > > > >> > > > > > >>> > > > > for the
>> > > > > > >> > > > > > >>> > > > > > > RPC field to be an enum of {
>> > > UNSPECIFIED,
>> > > > > > >> > > INCREMENTAL,
>> > > > > > >> > > > > FULL
>> > > > > > >> > > > > > >>> }, so
>> > > > > > >> > > > > > >>> > > > that
>> > > > > > >> > > > > > >>> > > > > we
>> > > > > > >> > > > > > >>> > > > > > > can cleanly handle old versions (by
>> > > > treating
>> > > > > > >> them
>> > > > > > >> > as
>> > > > > > >> > > > > > >>> UNSPECIFIED)
>> > > > > > >> > > > > > >>> > > > > > > > > In the LeaderAndIsrRequest
>> > section,
>> > > > you
>> > > > > > >> write
>> > > > > > >> > "A
>> > > > > > >> > > > > final
>> > > > > > >> > > > > > >>> > deletion
>> > > > > > >> > > > > > >>> > > > > event
>> > > > > > >> > > > > > >>> > > > > > > will be secheduled for X ms after
>> the
>> > > > > > >> > > > LeaderAndIsrRequest
>> > > > > > >> > > > > > was
>> > > > > > >> > > > > > >>> > first
>> > > > > > >> > > > > > >>> > > > > > > received..."  I guess the X was a
>> > > > > placeholder
>> > > > > > >> that
>> > > > > > >> > > you
>> > > > > > >> > > > > > >>> intended
>> > > > > > >> > > > > > >>> > to
>> > > > > > >> > > > > > >>> > > > > replace
>> > > > > > >> > > > > > >>> > > > > > > before posting? :)  In any case,
>> this
>> > > > seems
>> > > > > > like
>> > > > > > >> > the
>> > > > > > >> > > > kind
>> > > > > > >> > > > > > of
>> > > > > > >> > > > > > >>> > thing
>> > > > > > >> > > > > > >>> > > > we'd
>> > > > > > >> > > > > > >>> > > > > > > want a configuration for.  Let's
>> > > describe
>> > > > > that
>> > > > > > >> > > > > > configuration
>> > > > > > >> > > > > > >>> key
>> > > > > > >> > > > > > >>> > > > > somewhere
>> > > > > > >> > > > > > >>> > > > > > > in this KIP, including what its
>> > default
>> > > > > value
>> > > > > > >> is.
>> > > > > > >> > > > > > >>> > > > > > > > > We should probably also log a
>> > bunch
>> > > of
>> > > > > > >> messages
>> > > > > > >> > > at
>> > > > > > >> > > > > WARN
>> > > > > > >> > > > > > >>> level
>> > > > > > >> > > > > > >>> > > > when
>> > > > > > >> > > > > > >>> > > > > > > something is scheduled for
>> deletion,
>> > as
>> > > > > well.
>> > > > > > >> > (Maybe
>> > > > > > >> > > > > this
>> > > > > > >> > > > > > >>> was
>> > > > > > >> > > > > > >>> > > > > assumed, but
>> > > > > > >> > > > > > >>> > > > > > > it would be good to mention it).
>> > > > > > >> > > > > > >>> > > > > > > > > I feel like there are a few
>> > sections
>> > > > > that
>> > > > > > >> > should
>> > > > > > >> > > be
>> > > > > > >> > > > > > >>> moved to
>> > > > > > >> > > > > > >>> > > > > "rejected
>> > > > > > >> > > > > > >>> > > > > > > alternatives."  For example, in the
>> > > > > > DeleteTopics
>> > > > > > >> > > > section,
>> > > > > > >> > > > > > >>> since
>> > > > > > >> > > > > > >>> > we're
>> > > > > > >> > > > > > >>> > > > > not
>> > > > > > >> > > > > > >>> > > > > > > going to do option 1 or 2, these
>> > should
>> > > be
>> > > > > > moved
>> > > > > > >> > into
>> > > > > > >> > > > > > >>> "rejected
>> > > > > > >> > > > > > >>> > > > > > > alternatives,"  rather than
>> appearing
>> > > > > inline.
>> > > > > > >> > > Another
>> > > > > > >> > > > > case
>> > > > > > >> > > > > > >>> is
>> > > > > > >> > > > > > >>> > the
>> > > > > > >> > > > > > >>> > > > > "Should
>> > > > > > >> > > > > > >>> > > > > > > we remove topic name from the
>> protocol
>> > > > where
>> > > > > > >> > > possible"
>> > > > > > >> > > > > > >>> section.
>> > > > > > >> > > > > > >>> > This
>> > > > > > >> > > > > > >>> > > > > is
>> > > > > > >> > > > > > >>> > > > > > > clearly discussing a design
>> > alternative
>> > > > that
>> > > > > > >> we're
>> > > > > > >> > > not
>> > > > > > >> > > > > > >>> proposing
>> > > > > > >> > > > > > >>> > to
>> > > > > > >> > > > > > >>> > > > > > > implement: removing the topic name
>> > from
>> > > > > those
>> > > > > > >> > > > protocols.
>> > > > > > >> > > > > > >>> > > > > > > > > Is it really necessary to have
>> a
>> > new
>> > > > > > >> > > > > > >>> > /admin/delete_topics_by_id
>> > > > > > >> > > > > > >>> > > > > path
>> > > > > > >> > > > > > >>> > > > > > > in ZooKeeper?  It seems like we
>> don't
>> > > > really
>> > > > > > >> need
>> > > > > > >> > > this.
>> > > > > > >> > > > > > >>> Whenever
>> > > > > > >> > > > > > >>> > > > > there is
>> > > > > > >> > > > > > >>> > > > > > > a new controller, we'll send out
>> full
>> > > > > > >> > > > > LeaderAndIsrRequests
>> > > > > > >> > > > > > >>> which
>> > > > > > >> > > > > > >>> > will
>> > > > > > >> > > > > > >>> > > > > > > trigger the stale topics to be
>> cleaned
>> > > up.
>> > > > > >  The
>> > > > > > >> > > active
>> > > > > > >> > > > > > >>> > controller
>> > > > > > >> > > > > > >>> > > > will
>> > > > > > >> > > > > > >>> > > > > > > also send the full
>> LeaderAndIsrRequest
>> > > to
>> > > > > > >> brokers
>> > > > > > >> > > that
>> > > > > > >> > > > > are
>> > > > > > >> > > > > > >>> just
>> > > > > > >> > > > > > >>> > > > > starting
>> > > > > > >> > > > > > >>> > > > > > > up.    So we don't really need this
>> > kind
>> > > > of
>> > > > > > >> > two-phase
>> > > > > > >> > > > > > commit
>> > > > > > >> > > > > > >>> > (send
>> > > > > > >> > > > > > >>> > > > out
>> > > > > > >> > > > > > >>> > > > > > > StopReplicasRequest, get ACKs from
>> all
>> > > > > nodes,
>> > > > > > >> > commit
>> > > > > > >> > > by
>> > > > > > >> > > > > > >>> removing
>> > > > > > >> > > > > > >>> > > > > > > /admin/delete_topics node) any
>> more.
>> > > > > > >> > > > > > >>> > > > > > > > > You mention that FetchRequest
>> will
>> > > now
>> > > > > > >> include
>> > > > > > >> > > UUID
>> > > > > > >> > > > > to
>> > > > > > >> > > > > > >>> avoid
>> > > > > > >> > > > > > >>> > > > issues
>> > > > > > >> > > > > > >>> > > > > > > where requests are made to stale
>> > > > partitions.
>> > > > > > >> > > However,
>> > > > > > >> > > > > > >>> adding a
>> > > > > > >> > > > > > >>> > UUID
>> > > > > > >> > > > > > >>> > > > to
>> > > > > > >> > > > > > >>> > > > > > > MetadataRequest is listed as future
>> > > work,
>> > > > > out
>> > > > > > of
>> > > > > > >> > > scope
>> > > > > > >> > > > > for
>> > > > > > >> > > > > > >>> this
>> > > > > > >> > > > > > >>> > KIP.
>> > > > > > >> > > > > > >>> > > > > How
>> > > > > > >> > > > > > >>> > > > > > > will the client learn what the
>> topic
>> > > UUID
>> > > > > is,
>> > > > > > if
>> > > > > > >> > the
>> > > > > > >> > > > > > metadata
>> > > > > > >> > > > > > >>> > > > response
>> > > > > > >> > > > > > >>> > > > > > > doesn't include that information?
>> It
>> > > > seems
>> > > > > > like
>> > > > > > >> > > adding
>> > > > > > >> > > > > the
>> > > > > > >> > > > > > >>> UUID
>> > > > > > >> > > > > > >>> > to
>> > > > > > >> > > > > > >>> > > > > > > MetadataResponse would be an
>> > improvement
>> > > > > here
>> > > > > > >> that
>> > > > > > >> > > > might
>> > > > > > >> > > > > > not
>> > > > > > >> > > > > > >>> be
>> > > > > > >> > > > > > >>> > too
>> > > > > > >> > > > > > >>> > > > > hard to
>> > > > > > >> > > > > > >>> > > > > > > make.
>> > > > > > >> > > > > > >>> > > > > > > > > best,
>> > > > > > >> > > > > > >>> > > > > > > > > Colin
>> > > > > > >> > > > > > >>> > > > > > > > >
>> > > > > > >> > > > > > >>> > > > > > > > >
>> > > > > > >> > > > > > >>> > > > > > > > > On Mon, Sep 9, 2019, at 17:48,
>> > > Ryanne
>> > > > > > Dolan
>> > > > > > >> > > wrote:
>> > > > > > >> > > > > > >>> > > > > > > > > > Lucas, this would be great.
>> I've
>> > > run
>> > > > > > into
>> > > > > > >> > > issues
>> > > > > > >> > > > > with
>> > > > > > >> > > > > > >>> > topics
>> > > > > > >> > > > > > >>> > > > > being
>> > > > > > >> > > > > > >>> > > > > > > > > > resurrected accidentally,
>> since
>> > a
>> > > > > client
>> > > > > > >> > cannot
>> > > > > > >> > > > > > easily
>> > > > > > >> > > > > > >>> > > > > distinguish
>> > > > > > >> > > > > > >>> > > > > > > between
>> > > > > > >> > > > > > >>> > > > > > > > > > a deleted topic and a new
>> topic
>> > > with
>> > > > > the
>> > > > > > >> same
>> > > > > > >> > > > name.
>> > > > > > >> > > > > > I'd
>> > > > > > >> > > > > > >>> > need
>> > > > > > >> > > > > > >>> > > > the
>> > > > > > >> > > > > > >>> > > > > ID
>> > > > > > >> > > > > > >>> > > > > > > > > > accessible from the client to
>> > > solve
>> > > > > that
>> > > > > > >> > issue,
>> > > > > > >> > > > but
>> > > > > > >> > > > > > >>> this
>> > > > > > >> > > > > > >>> > is a
>> > > > > > >> > > > > > >>> > > > > good
>> > > > > > >> > > > > > >>> > > > > > > first
>> > > > > > >> > > > > > >>> > > > > > > > > > step.
>> > > > > > >> > > > > > >>> > > > > > > > > >
>> > > > > > >> > > > > > >>> > > > > > > > > > Ryanne
>> > > > > > >> > > > > > >>> > > > > > > > > >
>> > > > > > >> > > > > > >>> > > > > > > > > > On Wed, Sep 4, 2019 at 1:41
>> PM
>> > > Lucas
>> > > > > > >> > > Bradstreet <
>> > > > > > >> > > > > > >>> > > > > lucas@confluent.io>
>> > > > > > >> > > > > > >>> > > > > > > wrote:
>> > > > > > >> > > > > > >>> > > > > > > > > > > Hi all,
>> > > > > > >> > > > > > >>> > > > > > > > > > >
>> > > > > > >> > > > > > >>> > > > > > > > > > > I would like to kick off
>> > > > discussion
>> > > > > of
>> > > > > > >> > > KIP-516,
>> > > > > > >> > > > > an
>> > > > > > >> > > > > > >>> > > > > implementation
>> > > > > > >> > > > > > >>> > > > > > > of topic
>> > > > > > >> > > > > > >>> > > > > > > > > > > IDs for Kafka. Topic IDs
>> aim
>> > to
>> > > > > solve
>> > > > > > >> topic
>> > > > > > >> > > > > > >>> uniqueness
>> > > > > > >> > > > > > >>> > > > > problems in
>> > > > > > >> > > > > > >>> > > > > > > Kafka,
>> > > > > > >> > > > > > >>> > > > > > > > > > > where referring to a topic
>> by
>> > > name
>> > > > > > >> alone is
>> > > > > > >> > > > > > >>> insufficient.
>> > > > > > >> > > > > > >>> > > > Such
>> > > > > > >> > > > > > >>> > > > > > > cases
>> > > > > > >> > > > > > >>> > > > > > > > > > > include when a topic has
>> been
>> > > > > deleted
>> > > > > > >> and
>> > > > > > >> > > > > recreated
>> > > > > > >> > > > > > >>> with
>> > > > > > >> > > > > > >>> > the
>> > > > > > >> > > > > > >>> > > > > same
>> > > > > > >> > > > > > >>> > > > > > > name.
>> > > > > > >> > > > > > >>> > > > > > > > > > > Unique identifiers will
>> help
>> > > > > simplify
>> > > > > > >> and
>> > > > > > >> > > > improve
>> > > > > > >> > > > > > >>> Kafka's
>> > > > > > >> > > > > > >>> > > > topic
>> > > > > > >> > > > > > >>> > > > > > > deletion
>> > > > > > >> > > > > > >>> > > > > > > > > > > process, as well as prevent
>> > > cases
>> > > > > > where
>> > > > > > >> > > brokers
>> > > > > > >> > > > > may
>> > > > > > >> > > > > > >>> > > > incorrectly
>> > > > > > >> > > > > > >>> > > > > > > interact
>> > > > > > >> > > > > > >>> > > > > > > > > > > with stale versions of
>> topics.
>> > > > > > >> > > > > > >>> > > > > > > > > > >
>> > > > > > >> > > > > > >>> > > > > > > > > > >
>> > > > > > >> > > > > > >>> > > > > > > > > > >
>> > > > > > >> > > > > > >>> > > >
>> > > > > > >> > > > > > >>> >
>> > > > > > >> > > > > > >>>
>> > > > > > >> > > > > >
>> > > > > > >> > > > >
>> > > > > > >> > > >
>> > > > > > >> > >
>> > > > > > >> >
>> > > > > > >>
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
>> > > > > > >> > > > > > >>> > > > > > > > > > > Looking forward to your
>> > > thoughts.
>> > > > > > >> > > > > > >>> > > > > > > > > > >
>> > > > > > >> > > > > > >>> > > > > > > > > > > Lucas
>> > > > > > >> > > > > > >>> > > > > > > > > > >
>> > > > > > >> > > > > > >>> >
>> > > > > > >> > > > > > >>> >
>> > > > > > >> > > > > > >>>
>> > > > > > >> > > > > > >>
>> > > > > > >> > > > > >
>> > > > > > >> > > > >
>> > > > > > >> > > >
>> > > > > > >> > >
>> > > > > > >> >
>> > > > > > >>
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Ismael Juma <is...@juma.me.uk>]

Hi Justine,

I think we need to update the "Rejected Alternatives" section to take into
account that the proposal now removes the topic name from the fetch
request. Also, if we are removing it from the Fetch request, does it make
sense not to remove it from similar requests like ListOffsetRequest?

Ismael

On Thu, Sep 24, 2020 at 2:46 AM David Jacot <dj...@confluent.io> wrote:

> Hi Justine,
>
> Thanks for the KIP. I finally had time to read it :). It is a great
> improvement.
>
> I have a few comments/questions:
>
> 1. It seems that the schema of the StopReplicaRequest is slightly outdated.
> We
> did some changes as part of KIP-570. V3 is already organized by topics.
>
> 2. I just want to make sure that I understand the reconciliation
> logic correctly. When
> an "INCREMENTAL" LeaderAndIsr Request is received, the broker will also
> reconcile
> when the local uuid does not match the uuid in the request, right? In this
> case, the
> local log is staged for deletion.
>
> 3. In the documentation of the `delete.stale.topic.delay.ms` config, it
> says "When a
> FULL LeaderAndIsrRequest is received..." but I suppose it applies to both
> types.
>
> Best,
> David
>
> On Thu, Sep 24, 2020 at 1:40 AM Justine Olshan <jo...@confluent.io>
> wrote:
>
> > Hi Jun,
> >
> > Thanks for the comments. I apologize for some of the typos and confusion.
> > I’ve updated the KIP to fix some of the issues you mentioned.
> >
> > 20.2 I’ve changed the type to String.
> > 20.1/3 I’ve updated the TopicZNode to fix formatting and reflect the
> latest
> > version before this change.
> >
> > 21. You are correct and I’ve removed this line. I’ve also added a line
> > mentioning an IBP bump is necessary for migration
> >
> > 22. I think the wording was unclear but your summary is what was intended
> > by this line. I’ve updated to make this point more clear. “Remove deleted
> > topics from replicas by sending StopReplicaRequest V3 before the IBP bump
> > using the old logic, and using V4 and the new logic with topic IDs after
> > the IBP bump.”
> >
> > 23. I’ve removed the unspecified type from the KIP and mention that IBP
> > will be used to handle this request. “IBP will be used to determine
> whether
> > this new form of the request will be used. For older requests, we will
> > ignore this field and default to previous behavior.”
> >
> > 24. I’ve fixed this typo.
> >
> > 25. I've added a topics at a higher level for LeaderAndIsrResponse v5,
> > StopReplicaResponse v4. I've also changed StopReplicaRequest v4 to have
> > topics at a higher level.
> >
> > 26. I’ve updated forgotten_topics_data--added the topic ID and removed
> the
> > topic name
> >
> > 27. I’ve decided on plain text, and I’ve added an example.
> >
> > 28. I’ve added this idea to future work.
> >
> > Thanks again for taking a look,
> >
> > Justine
> >
> > On Wed, Sep 23, 2020 at 10:28 AM Jun Rao <ju...@confluent.io> wrote:
> >
> > > Hi, Justine,
> > >
> > > Thanks for the response. Made another pass. A few more comments below.
> > >
> > > 20. znode schema:
> > > 20.1 It seems that {"name": "version", "type": "int", "id": "UUID",
> > "doc":
> > > "version id"} should be {"name": "version", "type": "int"}, {"name":
> > "id",
> > > "type": "UUID", "doc": "version id"}.
> > > 20.2 The znode format is JSON which doesn't have UUID type. So the type
> > > probably should be string?
> > > 20.3 Also, the existing format used seems outdated. It should have the
> > > following format.
> > >     Json.encodeAsBytes(Map(
> > >       "version" -> 2,
> > >       "partitions" -> replicaAssignmentJson.asJava,
> > >       "adding_replicas" -> addingReplicasAssignmentJson.asJava,
> > >       "removing_replicas" -> removingReplicasAssignmentJson.asJava
> > >     ).asJava)
> > >   }
> > >
> > > 21. Migration: The KIP says "The migration process can take place
> without
> > > an inter-broker protocol bump, as the format stored in
> > > /brokers/topics/[topic] will be compatible with older broker versions."
> > > However, since we are bumping up the version of inter-broker requests,
> it
> > > seems that we need to use IBP for migration.
> > >
> > > 22. The KIP says "Remove deleted topics from replicas by sending
> > > StopReplicaRequest V3 for any topics which do not contain a topic ID,
> and
> > > V4 for any topics which do contain a topic ID.". However, if we use
> IBP,
> > it
> > > seems that the controller will either send StopReplicaRequest V3
> > > or StopReplicaRequest V4, but never mixed V3 and V4 for different
> topics.
> > > Basically, before the IBP bump, V3 will be used. After the IBP bump,
> > > topicId will be created and V4 will be used.
> > >
> > > 23. Given that we depend on IBP, do we still need "0 UNSPECIFIED"
> > > in LeaderAndIsr?
> > >
> > > 24. LeaderAndIsrResponse v5 : It still has the topic field.
> > >
> > > 25. LeaderAndIsrResponse v5, StopReplicaResponse v4: Could we use this
> > > opportunity to organize the response in 2 levels, first by topic, then
> by
> > > partition, as most other requests/responses?
> > >
> > > 26. FetchRequest v13 : Should forgotten_topics_data use topicId too?
> > >
> > > 27. "This file can either be plain text (key/value pairs) or JSON."
> Have
> > we
> > > decided which one to use? Also, it would be helpful to provide an
> > example.
> > >
> > > 28. Future improvement: Another future improvement opportunity is to
> use
> > > topicId in GroupMetadataManager.offsetCommitKey in the offset_commit
> > topic.
> > > This may save some space.
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Wed, Sep 23, 2020 at 8:50 AM Justine Olshan <jo...@confluent.io>
> > > wrote:
> > >
> > > > Hi Tom,
> > > >
> > > > Thanks for the comment. I think this is a really good idea and it has
> > > been
> > > > added to the KIP under the newly added tooling section.
> > > >
> > > > Thanks again,
> > > > Justine
> > > >
> > > > On Wed, Sep 23, 2020 at 3:17 AM Tom Bentley <tb...@redhat.com>
> > wrote:
> > > >
> > > > > Hi Justine,
> > > > >
> > > > > I know you started the vote thread, but on re-reading the KIP I
> > noticed
> > > > > that although the topic id is included in the MetadataResponse it's
> > not
> > > > > surfaced in the output from `kafka-topics.sh --describe`. Maybe
> that
> > > was
> > > > > intentional because ids are intentionally not really something the
> > user
> > > > > should care deeply about, but it would also make life harder for
> > anyone
> > > > > debugging Kafka and this would likely get worse the more topic ids
> > got
> > > > > rolled out across the protocols, clients etc. It seems likely that
> > > > > `kafka-topics.sh` will eventually need the ability to show the id
> of
> > a
> > > > > topic and perhaps find a topic name given an id. Is there any
> reason
> > > not
> > > > to
> > > > > implement that in this KIP?
> > > > >
> > > > > Many thanks,
> > > > >
> > > > > Tom
> > > > >
> > > > > On Mon, Sep 21, 2020 at 9:54 PM Justine Olshan <
> jolshan@confluent.io
> > >
> > > > > wrote:
> > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > After thinking about it, I've decided to remove the topic name
> from
> > > the
> > > > > > Fetch Request and Response after all. Since there are so many of
> > > these
> > > > > > requests per second, it is worth removing the extra information.
> > I've
> > > > > > updated the KIP to reflect this change.
> > > > > >
> > > > > > Please let me know if there is anything else we should discuss
> > before
> > > > > > voting.
> > > > > >
> > > > > > Thank you,
> > > > > > Justine
> > > > > >
> > > > > > On Fri, Sep 18, 2020 at 9:46 AM Justine Olshan <
> > jolshan@confluent.io
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Jun,
> > > > > > >
> > > > > > > I see what you are saying. For now we can remove the extra
> > > > information.
> > > > > > > I'll leave the option to add more fields to the file in the
> > future.
> > > > The
> > > > > > KIP
> > > > > > > has been updated to reflect this change.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Justine
> > > > > > >
> > > > > > > On Fri, Sep 18, 2020 at 8:46 AM Jun Rao <ju...@confluent.io>
> > wrote:
> > > > > > >
> > > > > > >> Hi, Justine,
> > > > > > >>
> > > > > > >> Thanks for the reply.
> > > > > > >>
> > > > > > >> 13. If the log directory is the source of truth, it means that
> > the
> > > > > > >> redundant info in the metadata file will be ignored. Then the
> > > > question
> > > > > > is
> > > > > > >> why do we need to put the redundant info in the metadata file
> > now?
> > > > > > >>
> > > > > > >> Thanks,
> > > > > > >>
> > > > > > >> Jun
> > > > > > >>
> > > > > > >> On Thu, Sep 17, 2020 at 5:07 PM Justine Olshan <
> > > > jolshan@confluent.io>
> > > > > > >> wrote:
> > > > > > >>
> > > > > > >> > Hi Jun,
> > > > > > >> > Thanks for the quick response!
> > > > > > >> >
> > > > > > >> > 12. I've decided to bump up the versions on the requests and
> > > > updated
> > > > > > the
> > > > > > >> > KIP. I think it's good we thoroughly discussed the options
> > here,
> > > > so
> > > > > we
> > > > > > >> know
> > > > > > >> > we made a good choice. :)
> > > > > > >> >
> > > > > > >> > 13. This is an interesting situation. I think if this does
> > occur
> > > > we
> > > > > > >> should
> > > > > > >> > give a warning. I agree that it's hard to know the source of
> > > truth
> > > > > for
> > > > > > >> sure
> > > > > > >> > since the directory or the file could be manually modified.
> I
> > > > guess
> > > > > > the
> > > > > > >> > directory could be used as the source of truth. To be
> honest,
> > > I'm
> > > > > not
> > > > > > >> > really sure what happens in kafka when the log directory is
> > > > renamed
> > > > > > >> > manually in such a way. I'm also wondering if the situation
> is
> > > > > > >> recoverable
> > > > > > >> > in this scenario.
> > > > > > >> >
> > > > > > >> > Thanks,
> > > > > > >> > Justine
> > > > > > >> >
> > > > > > >> > On Thu, Sep 17, 2020 at 4:28 PM Jun Rao <ju...@confluent.io>
> > > wrote:
> > > > > > >> >
> > > > > > >> > > Hi, Justine,
> > > > > > >> > >
> > > > > > >> > > Thanks for the reply.
> > > > > > >> > >
> > > > > > >> > > 12. I don't have a strong preference either. However, if
> we
> > > need
> > > > > IBP
> > > > > > >> > > anyway, maybe it's easier to just bump up the version for
> > all
> > > > > inter
> > > > > > >> > broker
> > > > > > >> > > requests and add the topic id field as a regular field. A
> > > > regular
> > > > > > >> field
> > > > > > >> > is
> > > > > > >> > > a bit more concise in wire transfer than a flexible field.
> > > > > > >> > >
> > > > > > >> > > 13. The confusion that I was referring to is between the
> > topic
> > > > > name
> > > > > > >> and
> > > > > > >> > > partition number between the log dir and the metadata
> file.
> > > For
> > > > > > >> example,
> > > > > > >> > if
> > > > > > >> > > the log dir is topicA-1 and the metadata file in it has
> > topicB
> > > > and
> > > > > > >> > > partition 0 (say due to a bug or manual modification),
> which
> > > one
> > > > > do
> > > > > > we
> > > > > > >> > use
> > > > > > >> > > as the source of truth?
> > > > > > >> > >
> > > > > > >> > > Jun
> > > > > > >> > >
> > > > > > >> > > On Thu, Sep 17, 2020 at 3:43 PM Justine Olshan <
> > > > > > jolshan@confluent.io>
> > > > > > >> > > wrote:
> > > > > > >> > >
> > > > > > >> > > > Hi Jun,
> > > > > > >> > > > Thanks for the comments.
> > > > > > >> > > >
> > > > > > >> > > > 12. I bumped the LeaderAndIsrRequest because I removed
> the
> > > > topic
> > > > > > >> name
> > > > > > >> > > field
> > > > > > >> > > > in the response. It may be possible to avoid bumping the
> > > > version
> > > > > > >> > without
> > > > > > >> > > > that change, but I may be missing something.
> > > > > > >> > > > I believe StopReplica is actually on version 3 now, but
> > > > because
> > > > > > >> > version 2
> > > > > > >> > > > is flexible, I kept that listed as version 2 on the KIP
> > > page.
> > > > > > >> However,
> > > > > > >> > > you
> > > > > > >> > > > may be right in that we may need to bump the version on
> > > > > > StopReplica
> > > > > > >> to
> > > > > > >> > > deal
> > > > > > >> > > > with deletion differently as mentioned above. I don't
> know
> > > if
> > > > I
> > > > > > >> have a
> > > > > > >> > > big
> > > > > > >> > > > preference over used tagged fields or not.
> > > > > > >> > > >
> > > > > > >> > > > 13. I was thinking that in the case where the file and
> the
> > > > > request
> > > > > > >> > topic
> > > > > > >> > > > ids don't match, it means that the broker's topic/the
> one
> > in
> > > > the
> > > > > > >> file
> > > > > > >> > has
> > > > > > >> > > > been deleted. In that case, we would need to delete the
> > old
> > > > > topic
> > > > > > >> and
> > > > > > >> > > start
> > > > > > >> > > > receiving the new version. If the topic name were to
> > change,
> > > > but
> > > > > > the
> > > > > > >> > ids
> > > > > > >> > > > still matched, the file would also need to update. Am I
> > > > missing
> > > > > a
> > > > > > >> case
> > > > > > >> > > > where the file would be correct and not the request?
> > > > > > >> > > >
> > > > > > >> > > > Thanks,
> > > > > > >> > > > Justine
> > > > > > >> > > >
> > > > > > >> > > > On Thu, Sep 17, 2020 at 3:18 PM Jun Rao <
> jun@confluent.io
> > >
> > > > > wrote:
> > > > > > >> > > >
> > > > > > >> > > > > Hi, Justine,
> > > > > > >> > > > >
> > > > > > >> > > > > Thanks for the reply. A couple of more comments below.
> > > > > > >> > > > >
> > > > > > >> > > > > 12. ListOffset and OffsetForLeader currently don't
> > support
> > > > > > >> flexible
> > > > > > >> > > > fields.
> > > > > > >> > > > > So, we have to bump up the version number and use IBP
> at
> > > > least
> > > > > > for
> > > > > > >> > > these
> > > > > > >> > > > > two requests. Note that it seems 2.7.0 will require
> IBP
> > > > anyway
> > > > > > >> > because
> > > > > > >> > > of
> > > > > > >> > > > > changes in KAFKA-10435. Also, it seems that the
> version
> > > for
> > > > > > >> > > > > LeaderAndIsrRequest and StopReplica are bumped even
> > though
> > > > we
> > > > > > only
> > > > > > >> > > added
> > > > > > >> > > > a
> > > > > > >> > > > > tagged field. But since IBP is needed anyway, we may
> > want
> > > to
> > > > > > >> revisit
> > > > > > >> > > the
> > > > > > >> > > > > overall tagged field choice.
> > > > > > >> > > > >
> > > > > > >> > > > > 13. The only downside is the potential confusion on
> > which
> > > > one
> > > > > is
> > > > > > >> the
> > > > > > >> > > > source
> > > > > > >> > > > > of truth if they don't match. Another option is to
> > include
> > > > > those
> > > > > > >> > fields
> > > > > > >> > > > in
> > > > > > >> > > > > the metadata file when we actually change the
> directory
> > > > > > structure.
> > > > > > >> > > > >
> > > > > > >> > > > > Thanks,
> > > > > > >> > > > >
> > > > > > >> > > > > Jun
> > > > > > >> > > > >
> > > > > > >> > > > > On Thu, Sep 17, 2020 at 2:01 PM Justine Olshan <
> > > > > > >> jolshan@confluent.io
> > > > > > >> > >
> > > > > > >> > > > > wrote:
> > > > > > >> > > > >
> > > > > > >> > > > > > Hello all,
> > > > > > >> > > > > >
> > > > > > >> > > > > > I've thought some more about removing the topic name
> > > field
> > > > > > from
> > > > > > >> > some
> > > > > > >> > > of
> > > > > > >> > > > > the
> > > > > > >> > > > > > requests. On closer inspection of the
> > > requests/responses,
> > > > it
> > > > > > >> seems
> > > > > > >> > > that
> > > > > > >> > > > > the
> > > > > > >> > > > > > internal changes would be much larger than I
> expected.
> > > > Some
> > > > > > >> > protocols
> > > > > > >> > > > > > involve clients, so they would require changes too.
> > I'm
> > > > > > thinking
> > > > > > >> > that
> > > > > > >> > > > for
> > > > > > >> > > > > > now, removing the topic name from these requests and
> > > > > responses
> > > > > > >> are
> > > > > > >> > > out
> > > > > > >> > > > of
> > > > > > >> > > > > > scope.
> > > > > > >> > > > > >
> > > > > > >> > > > > > I have decided to just keep the change
> > > > LeaderAndIsrResponse
> > > > > to
> > > > > > >> > remove
> > > > > > >> > > > the
> > > > > > >> > > > > > topic name, and have updated the KIP to reflect this
> > > > > change. I
> > > > > > >> have
> > > > > > >> > > > also
> > > > > > >> > > > > > mentioned the other requests and responses in future
> > > work.
> > > > > > >> > > > > >
> > > > > > >> > > > > > I'm hoping to start the voting process soon, so let
> me
> > > > know
> > > > > if
> > > > > > >> > there
> > > > > > >> > > is
> > > > > > >> > > > > > anything else we should discuss.
> > > > > > >> > > > > >
> > > > > > >> > > > > > Thank you,
> > > > > > >> > > > > > Justine
> > > > > > >> > > > > >
> > > > > > >> > > > > > On Tue, Sep 15, 2020 at 3:57 PM Justine Olshan <
> > > > > > >> > jolshan@confluent.io
> > > > > > >> > > >
> > > > > > >> > > > > > wrote:
> > > > > > >> > > > > >
> > > > > > >> > > > > > > Hello again,
> > > > > > >> > > > > > > To follow up on some of the other comments:
> > > > > > >> > > > > > >
> > > > > > >> > > > > > > 10/11) We can remove the topic name from these
> > > > > > >> > requests/responses,
> > > > > > >> > > > and
> > > > > > >> > > > > > > that means we will just have to make a few
> internal
> > > > > changes
> > > > > > to
> > > > > > >> > make
> > > > > > >> > > > > > > partitions accessible by topic id and partition. I
> > can
> > > > > > update
> > > > > > >> the
> > > > > > >> > > KIP
> > > > > > >> > > > > to
> > > > > > >> > > > > > > remove them unless anyone thinks they should stay.
> > > > > > >> > > > > > >
> > > > > > >> > > > > > > 12) Addressed in the previous email. I've updated
> > the
> > > > KIP
> > > > > to
> > > > > > >> > > include
> > > > > > >> > > > > > > tagged fields for the requests and responses.
> (More
> > on
> > > > > that
> > > > > > >> > below)
> > > > > > >> > > > > > >
> > > > > > >> > > > > > > 13) I think part of the idea for including this
> > > > > information
> > > > > > >> is to
> > > > > > >> > > > > prepare
> > > > > > >> > > > > > > for future changes. Perhaps the directory
> structure
> > > > might
> > > > > > >> change
> > > > > > >> > > from
> > > > > > >> > > > > > > topicName_partitionNumber to something like
> > > > > > >> > > topicID_partitionNumber.
> > > > > > >> > > > > Then
> > > > > > >> > > > > > > it would be useful to have the topic name in the
> > file
> > > > > since
> > > > > > it
> > > > > > >> > > would
> > > > > > >> > > > > not
> > > > > > >> > > > > > be
> > > > > > >> > > > > > > in the directory structure. Supporting topic
> renames
> > > > might
> > > > > > be
> > > > > > >> > > easier
> > > > > > >> > > > if
> > > > > > >> > > > > > the
> > > > > > >> > > > > > > other fields are included. Would there be any
> > > downsides
> > > > to
> > > > > > >> > > including
> > > > > > >> > > > > this
> > > > > > >> > > > > > > information?
> > > > > > >> > > > > > >
> > > > > > >> > > > > > > 14)  Yes, we would need to copy the partition
> > metadata
> > > > > file
> > > > > > in
> > > > > > >> > this
> > > > > > >> > > > > > > process. I've updated the KIP to include this.
> > > > > > >> > > > > > >
> > > > > > >> > > > > > > 15) I believe Lucas meant v1 and v2 here. He was
> > > > referring
> > > > > > to
> > > > > > >> how
> > > > > > >> > > the
> > > > > > >> > > > > > > requests would fall under different IBP and meant
> > that
> > > > > older
> > > > > > >> > > brokers
> > > > > > >> > > > > > would
> > > > > > >> > > > > > > have to use the older version of the request and
> the
> > > > > > existing
> > > > > > >> > topic
> > > > > > >> > > > > > > deletion process. At first, it seemed like tagged
> > > fields
> > > > > > would
> > > > > > >> > > > resolve
> > > > > > >> > > > > > > the IBP issue. However, we may need IBP for this
> > > request
> > > > > > after
> > > > > > >> > all
> > > > > > >> > > > > since
> > > > > > >> > > > > > > the controller handles the topic deletion
> > differently
> > > > > > >> depending
> > > > > > >> > on
> > > > > > >> > > > the
> > > > > > >> > > > > > IBP
> > > > > > >> > > > > > > version. In an older version, we can't just send a
> > > > > > StopReplica
> > > > > > >> > > delete
> > > > > > >> > > > > the
> > > > > > >> > > > > > > topic immediately like we'd want to for this KIP.
> > > > > > >> > > > > > >
> > > > > > >> > > > > > > This makes me wonder if we want tagged fields on
> all
> > > the
> > > > > > >> requests
> > > > > > >> > > > after
> > > > > > >> > > > > > > all. Let me know your thoughts!
> > > > > > >> > > > > > >
> > > > > > >> > > > > > > Justine
> > > > > > >> > > > > > >
> > > > > > >> > > > > > > On Tue, Sep 15, 2020 at 1:03 PM Justine Olshan <
> > > > > > >> > > jolshan@confluent.io
> > > > > > >> > > > >
> > > > > > >> > > > > > > wrote:
> > > > > > >> > > > > > >
> > > > > > >> > > > > > >> Hi all,
> > > > > > >> > > > > > >> Jun brought up a good point in his last email
> about
> > > > > tagged
> > > > > > >> > fields,
> > > > > > >> > > > and
> > > > > > >> > > > > > >> I've updated the KIP to reflect that the changes
> to
> > > > > > requests
> > > > > > >> and
> > > > > > >> > > > > > responses
> > > > > > >> > > > > > >> will be in the form of tagged fields to avoid
> > > changing
> > > > > IBP.
> > > > > > >> > > > > > >>
> > > > > > >> > > > > > >> Jun: I plan on sending a followup email to
> address
> > > some
> > > > > of
> > > > > > >> the
> > > > > > >> > > other
> > > > > > >> > > > > > >> points.
> > > > > > >> > > > > > >>
> > > > > > >> > > > > > >> Thanks,
> > > > > > >> > > > > > >> Justine
> > > > > > >> > > > > > >>
> > > > > > >> > > > > > >> On Mon, Sep 14, 2020 at 4:25 PM Jun Rao <
> > > > > jun@confluent.io>
> > > > > > >> > wrote:
> > > > > > >> > > > > > >>
> > > > > > >> > > > > > >>> Hi, Justine,
> > > > > > >> > > > > > >>>
> > > > > > >> > > > > > >>> Thanks for the updated KIP. A few comments
> below.
> > > > > > >> > > > > > >>>
> > > > > > >> > > > > > >>> 10. LeaderAndIsr Response: Do we need the topic
> > > name?
> > > > > > >> > > > > > >>>
> > > > > > >> > > > > > >>> 11. For the changed request/response, other than
> > > > > > >> LeaderAndIsr,
> > > > > > >> > > > > > >>> UpdateMetadata, Metadata, do we need to include
> > the
> > > > > topic
> > > > > > >> name?
> > > > > > >> > > > > > >>>
> > > > > > >> > > > > > >>> 12. It seems that upgrades don't require IBP.
> Does
> > > > that
> > > > > > mean
> > > > > > >> > the
> > > > > > >> > > > new
> > > > > > >> > > > > > >>> fields
> > > > > > >> > > > > > >>> in all the request/response are added as tagged
> > > fields
> > > > > > >> without
> > > > > > >> > > > > bumping
> > > > > > >> > > > > > up
> > > > > > >> > > > > > >>> the request version? It would be useful to make
> > that
> > > > > > clear.
> > > > > > >> > > > > > >>>
> > > > > > >> > > > > > >>> 13. Partition Metadata file: Do we need to
> include
> > > the
> > > > > > topic
> > > > > > >> > name
> > > > > > >> > > > and
> > > > > > >> > > > > > the
> > > > > > >> > > > > > >>> partition id since they are implied in the
> > directory
> > > > > name?
> > > > > > >> > > > > > >>>
> > > > > > >> > > > > > >>> 14. In the JBOD mode, we support moving a
> > > partition's
> > > > > data
> > > > > > >> from
> > > > > > >> > > one
> > > > > > >> > > > > > disk
> > > > > > >> > > > > > >>> to
> > > > > > >> > > > > > >>> another. Will the new partition metadata file be
> > > > copied
> > > > > > >> during
> > > > > > >> > > that
> > > > > > >> > > > > > >>> process?
> > > > > > >> > > > > > >>>
> > > > > > >> > > > > > >>> 15. The KIP says "Remove deleted topics from
> > > replicas
> > > > by
> > > > > > >> > sending
> > > > > > >> > > > > > >>> StopReplicaRequest V2 for any topics which do
> not
> > > > > contain
> > > > > > a
> > > > > > >> > topic
> > > > > > >> > > > ID,
> > > > > > >> > > > > > and
> > > > > > >> > > > > > >>> V3 for any topics which do contain a topic ID.".
> > > > > However,
> > > > > > it
> > > > > > >> > > seems
> > > > > > >> > > > > the
> > > > > > >> > > > > > >>> updated controller will create all missing topic
> > IDs
> > > > > first
> > > > > > >> > before
> > > > > > >> > > > > doing
> > > > > > >> > > > > > >>> other actions. So, is StopReplicaRequest V2
> > needed?
> > > > > > >> > > > > > >>>
> > > > > > >> > > > > > >>> Jun
> > > > > > >> > > > > > >>>
> > > > > > >> > > > > > >>> On Fri, Sep 11, 2020 at 10:31 AM John Roesler <
> > > > > > >> > > vvcephei@apache.org
> > > > > > >> > > > >
> > > > > > >> > > > > > >>> wrote:
> > > > > > >> > > > > > >>>
> > > > > > >> > > > > > >>> > Thanks, Justine!
> > > > > > >> > > > > > >>> >
> > > > > > >> > > > > > >>> > Your response seems compelling to me.
> > > > > > >> > > > > > >>> >
> > > > > > >> > > > > > >>> > -John
> > > > > > >> > > > > > >>> >
> > > > > > >> > > > > > >>> > On Fri, 2020-09-11 at 10:17 -0700, Justine
> > Olshan
> > > > > wrote:
> > > > > > >> > > > > > >>> > > Hello all,
> > > > > > >> > > > > > >>> > > Thanks for continuing the discussion! I
> have a
> > > few
> > > > > > >> > responses
> > > > > > >> > > to
> > > > > > >> > > > > > your
> > > > > > >> > > > > > >>> > points.
> > > > > > >> > > > > > >>> > >
> > > > > > >> > > > > > >>> > > Tom: You are correct in that this KIP has
> not
> > > > > > mentioned
> > > > > > >> the
> > > > > > >> > > > > > >>> > > DeleteTopicsRequest. I think that this would
> > be
> > > > out
> > > > > of
> > > > > > >> > scope
> > > > > > >> > > > for
> > > > > > >> > > > > > >>> now, but
> > > > > > >> > > > > > >>> > > may be something worth adding in the future.
> > > > > > >> > > > > > >>> > >
> > > > > > >> > > > > > >>> > > John: We did consider sequence ids, but
> there
> > > are
> > > > a
> > > > > > few
> > > > > > >> > > reasons
> > > > > > >> > > > > to
> > > > > > >> > > > > > >>> favor
> > > > > > >> > > > > > >>> > > UUIDs. There are several cases where topics
> > from
> > > > > > >> different
> > > > > > >> > > > > clusters
> > > > > > >> > > > > > >>> may
> > > > > > >> > > > > > >>> > > interact now and in the future. For example,
> > > > Mirror
> > > > > > >> Maker 2
> > > > > > >> > > may
> > > > > > >> > > > > > >>> benefit
> > > > > > >> > > > > > >>> > > from being able to detect when a cluster
> being
> > > > > > mirrored
> > > > > > >> is
> > > > > > >> > > > > deleted
> > > > > > >> > > > > > >>> and
> > > > > > >> > > > > > >>> > > recreated and globally unique identifiers
> > would
> > > > make
> > > > > > >> > > resolving
> > > > > > >> > > > > > issues
> > > > > > >> > > > > > >>> > > easier than sequence IDs which may collide
> > > between
> > > > > > >> > clusters.
> > > > > > >> > > > > > KIP-405
> > > > > > >> > > > > > >>> > > (tiered storage) will also benefit from
> > globally
> > > > > > unique
> > > > > > >> IDs
> > > > > > >> > > as
> > > > > > >> > > > > > shared
> > > > > > >> > > > > > >>> > > buckets may be used between clusters.
> > > > > > >> > > > > > >>> > >
> > > > > > >> > > > > > >>> > > Globally unique IDs would also make
> > > functionality
> > > > > like
> > > > > > >> > moving
> > > > > > >> > > > > > topics
> > > > > > >> > > > > > >>> > > between disparate clusters easier in the
> > future,
> > > > > > >> simplify
> > > > > > >> > any
> > > > > > >> > > > > > future
> > > > > > >> > > > > > >>> > > implementations of backups and restores, and
> > > more.
> > > > > In
> > > > > > >> > > general,
> > > > > > >> > > > > > >>> unique IDs
> > > > > > >> > > > > > >>> > > would ensure that the source cluster topics
> do
> > > not
> > > > > > >> conflict
> > > > > > >> > > > with
> > > > > > >> > > > > > the
> > > > > > >> > > > > > >>> > > destination cluster topics.
> > > > > > >> > > > > > >>> > >
> > > > > > >> > > > > > >>> > > If we were to use sequence ids, we would
> need
> > > > > > >> sufficiently
> > > > > > >> > > > large
> > > > > > >> > > > > > >>> cluster
> > > > > > >> > > > > > >>> > > ids to be stored with the topic identifiers
> or
> > > we
> > > > > run
> > > > > > >> the
> > > > > > >> > > risk
> > > > > > >> > > > of
> > > > > > >> > > > > > >>> > > collisions. This will give up any advantage
> in
> > > > > > >> compactness
> > > > > > >> > > that
> > > > > > >> > > > > > >>> sequence
> > > > > > >> > > > > > >>> > > numbers may bring. Given these advantages I
> > > think
> > > > it
> > > > > > >> makes
> > > > > > >> > > > sense
> > > > > > >> > > > > to
> > > > > > >> > > > > > >>> use
> > > > > > >> > > > > > >>> > > UUIDs.
> > > > > > >> > > > > > >>> > >
> > > > > > >> > > > > > >>> > > Gokul: This is an interesting idea, but this
> > is
> > > a
> > > > > > >> breaking
> > > > > > >> > > > > change.
> > > > > > >> > > > > > >>> Out of
> > > > > > >> > > > > > >>> > > scope for now, but maybe worth discussing in
> > the
> > > > > > future.
> > > > > > >> > > > > > >>> > >
> > > > > > >> > > > > > >>> > > Hope this explains some of the decisions,
> > > > > > >> > > > > > >>> > >
> > > > > > >> > > > > > >>> > > Justine
> > > > > > >> > > > > > >>> > >
> > > > > > >> > > > > > >>> > >
> > > > > > >> > > > > > >>> > >
> > > > > > >> > > > > > >>> > > On Fri, Sep 11, 2020 at 8:27 AM Gokul
> Ramanan
> > > > > > >> Subramanian <
> > > > > > >> > > > > > >>> > > gokul2411s@gmail.com> wrote:
> > > > > > >> > > > > > >>> > >
> > > > > > >> > > > > > >>> > > > Hi.
> > > > > > >> > > > > > >>> > > >
> > > > > > >> > > > > > >>> > > > Thanks for the KIP.
> > > > > > >> > > > > > >>> > > >
> > > > > > >> > > > > > >>> > > > Have you thought about whether it makes
> > sense
> > > to
> > > > > > >> support
> > > > > > >> > > > > > >>> authorizing a
> > > > > > >> > > > > > >>> > > > principal for a topic ID rather than a
> topic
> > > > name
> > > > > to
> > > > > > >> > > achieve
> > > > > > >> > > > > > >>> tighter
> > > > > > >> > > > > > >>> > > > security?
> > > > > > >> > > > > > >>> > > >
> > > > > > >> > > > > > >>> > > > Or is the topic ID fundamentally an
> internal
> > > > > detail
> > > > > > >> > similar
> > > > > > >> > > > to
> > > > > > >> > > > > > >>> epochs
> > > > > > >> > > > > > >>> > used
> > > > > > >> > > > > > >>> > > > in a bunch of other places in Kafka?
> > > > > > >> > > > > > >>> > > >
> > > > > > >> > > > > > >>> > > > Thanks.
> > > > > > >> > > > > > >>> > > >
> > > > > > >> > > > > > >>> > > > On Fri, Sep 11, 2020 at 4:06 PM John
> > Roesler <
> > > > > > >> > > > > > vvcephei@apache.org>
> > > > > > >> > > > > > >>> > wrote:
> > > > > > >> > > > > > >>> > > >
> > > > > > >> > > > > > >>> > > > > Hello Justine,
> > > > > > >> > > > > > >>> > > > >
> > > > > > >> > > > > > >>> > > > > Thanks for the KIP!
> > > > > > >> > > > > > >>> > > > >
> > > > > > >> > > > > > >>> > > > > I happen to have been confronted
> recently
> > > with
> > > > > the
> > > > > > >> need
> > > > > > >> > > to
> > > > > > >> > > > > keep
> > > > > > >> > > > > > >>> > track of
> > > > > > >> > > > > > >>> > > > a
> > > > > > >> > > > > > >>> > > > > large number of topics as compactly as
> > > > > possible. I
> > > > > > >> was
> > > > > > >> > > > going
> > > > > > >> > > > > to
> > > > > > >> > > > > > >>> come
> > > > > > >> > > > > > >>> > up
> > > > > > >> > > > > > >>> > > > > with some way to dictionary encode the
> > topic
> > > > > names
> > > > > > >> as
> > > > > > >> > > > > integers,
> > > > > > >> > > > > > >>> but
> > > > > > >> > > > > > >>> > this
> > > > > > >> > > > > > >>> > > > > seems much better!
> > > > > > >> > > > > > >>> > > > >
> > > > > > >> > > > > > >>> > > > > Apologies if this has been raised
> before,
> > > but
> > > > > I’m
> > > > > > >> > > wondering
> > > > > > >> > > > > > >>> about the
> > > > > > >> > > > > > >>> > > > > choice of UUID vs sequence number for
> the
> > > ids.
> > > > > > >> > Typically,
> > > > > > >> > > > > I’ve
> > > > > > >> > > > > > >>> seen
> > > > > > >> > > > > > >>> > UUIDs
> > > > > > >> > > > > > >>> > > > > in two situations:
> > > > > > >> > > > > > >>> > > > > 1. When processes need to generate
> > > > non-colliding
> > > > > > >> > > > identifiers
> > > > > > >> > > > > > >>> without
> > > > > > >> > > > > > >>> > > > > coordination.
> > > > > > >> > > > > > >>> > > > > 2. When the identifier needs to be
> > > > “universally
> > > > > > >> > unique”;
> > > > > > >> > > > > I.e.,
> > > > > > >> > > > > > >>> the
> > > > > > >> > > > > > >>> > > > > identifier needs to distinguish the
> entity
> > > > from
> > > > > > all
> > > > > > >> > other
> > > > > > >> > > > > > >>> entities
> > > > > > >> > > > > > >>> > that
> > > > > > >> > > > > > >>> > > > > could ever exist. This is useful in
> cases
> > > > where
> > > > > > >> > entities
> > > > > > >> > > > from
> > > > > > >> > > > > > all
> > > > > > >> > > > > > >>> > kinds
> > > > > > >> > > > > > >>> > > > of
> > > > > > >> > > > > > >>> > > > > systems get mixed together, such as when
> > > > dumping
> > > > > > >> logs
> > > > > > >> > > from
> > > > > > >> > > > > all
> > > > > > >> > > > > > >>> > processes
> > > > > > >> > > > > > >>> > > > in
> > > > > > >> > > > > > >>> > > > > a company into a common system.
> > > > > > >> > > > > > >>> > > > >
> > > > > > >> > > > > > >>> > > > > Maybe I’m being short-sighted, but it
> > > doesn’t
> > > > > seem
> > > > > > >> like
> > > > > > >> > > > > either
> > > > > > >> > > > > > >>> really
> > > > > > >> > > > > > >>> > > > > applies here. It seems like the brokers
> > > could
> > > > > and
> > > > > > >> would
> > > > > > >> > > > > achieve
> > > > > > >> > > > > > >>> > consensus
> > > > > > >> > > > > > >>> > > > > when creating a topic anyway, which is
> all
> > > > > that’s
> > > > > > >> > > required
> > > > > > >> > > > to
> > > > > > >> > > > > > >>> > generate
> > > > > > >> > > > > > >>> > > > > non-colliding sequence ids. For the
> > second,
> > > as
> > > > > you
> > > > > > >> > > mention,
> > > > > > >> > > > > we
> > > > > > >> > > > > > >>> could
> > > > > > >> > > > > > >>> > > > assign
> > > > > > >> > > > > > >>> > > > > a UUID to the cluster as a whole, which
> > > would
> > > > > > render
> > > > > > >> > any
> > > > > > >> > > > > > resource
> > > > > > >> > > > > > >>> > scoped
> > > > > > >> > > > > > >>> > > > to
> > > > > > >> > > > > > >>> > > > > the broker universally unique as well.
> > > > > > >> > > > > > >>> > > > >
> > > > > > >> > > > > > >>> > > > > The reason I mention this is that,
> > although
> > > a
> > > > > UUID
> > > > > > >> is
> > > > > > >> > way
> > > > > > >> > > > > more
> > > > > > >> > > > > > >>> > compact
> > > > > > >> > > > > > >>> > > > > than topic names, it’s still 16 bytes.
> In
> > > > > > contrast,
> > > > > > >> a
> > > > > > >> > > > 4-byte
> > > > > > >> > > > > > >>> integer
> > > > > > >> > > > > > >>> > > > > sequence id would give us 4 billion
> unique
> > > > > topics
> > > > > > >> per
> > > > > > >> > > > > cluster,
> > > > > > >> > > > > > >>> which
> > > > > > >> > > > > > >>> > > > seems
> > > > > > >> > > > > > >>> > > > > like enough ;)
> > > > > > >> > > > > > >>> > > > >
> > > > > > >> > > > > > >>> > > > > Considering the number of different
> times
> > > > these
> > > > > > >> topic
> > > > > > >> > > > > > >>> identifiers are
> > > > > > >> > > > > > >>> > > > sent
> > > > > > >> > > > > > >>> > > > > over the wire or stored in memory, it
> > seems
> > > > like
> > > > > > it
> > > > > > >> > might
> > > > > > >> > > > be
> > > > > > >> > > > > > >>> worth
> > > > > > >> > > > > > >>> > the
> > > > > > >> > > > > > >>> > > > > additional 4x space savings.
> > > > > > >> > > > > > >>> > > > >
> > > > > > >> > > > > > >>> > > > > What do you think about this?
> > > > > > >> > > > > > >>> > > > >
> > > > > > >> > > > > > >>> > > > > Thanks,
> > > > > > >> > > > > > >>> > > > > John
> > > > > > >> > > > > > >>> > > > >
> > > > > > >> > > > > > >>> > > > > On Fri, Sep 11, 2020, at 03:20, Tom
> > Bentley
> > > > > wrote:
> > > > > > >> > > > > > >>> > > > > > Hi Justine,
> > > > > > >> > > > > > >>> > > > > >
> > > > > > >> > > > > > >>> > > > > > This looks like a very welcome
> > > improvement.
> > > > > > >> Thanks!
> > > > > > >> > > > > > >>> > > > > >
> > > > > > >> > > > > > >>> > > > > > Maybe I missed it, but the KIP doesn't
> > > seem
> > > > to
> > > > > > >> > mention
> > > > > > >> > > > > > changing
> > > > > > >> > > > > > >>> > > > > > DeleteTopicsRequest to identify the
> > topic
> > > > > using
> > > > > > an
> > > > > > >> > id.
> > > > > > >> > > > > Maybe
> > > > > > >> > > > > > >>> > that's out
> > > > > > >> > > > > > >>> > > > > of
> > > > > > >> > > > > > >>> > > > > > scope, but DeleteTopicsRequest is not
> > > listed
> > > > > > among
> > > > > > >> > the
> > > > > > >> > > > > Future
> > > > > > >> > > > > > >>> Work
> > > > > > >> > > > > > >>> > APIs
> > > > > > >> > > > > > >>> > > > > > either.
> > > > > > >> > > > > > >>> > > > > >
> > > > > > >> > > > > > >>> > > > > > Kind regards,
> > > > > > >> > > > > > >>> > > > > >
> > > > > > >> > > > > > >>> > > > > > Tom
> > > > > > >> > > > > > >>> > > > > >
> > > > > > >> > > > > > >>> > > > > > On Thu, Sep 10, 2020 at 3:59 PM Satish
> > > > > Duggana <
> > > > > > >> > > > > > >>> > > > satish.duggana@gmail.com
> > > > > > >> > > > > > >>> > > > > > wrote:
> > > > > > >> > > > > > >>> > > > > >
> > > > > > >> > > > > > >>> > > > > > > Thanks Lucas/Justine for the nice
> KIP.
> > > > > > >> > > > > > >>> > > > > > >
> > > > > > >> > > > > > >>> > > > > > > It has several benefits which also
> > > include
> > > > > > >> > > simplifying
> > > > > > >> > > > > the
> > > > > > >> > > > > > >>> topic
> > > > > > >> > > > > > >>> > > > > > > deletion process by controller and
> > logs
> > > > > > cleanup
> > > > > > >> by
> > > > > > >> > > > > brokers
> > > > > > >> > > > > > in
> > > > > > >> > > > > > >>> > corner
> > > > > > >> > > > > > >>> > > > > > > cases.
> > > > > > >> > > > > > >>> > > > > > >
> > > > > > >> > > > > > >>> > > > > > > Best,
> > > > > > >> > > > > > >>> > > > > > > Satish.
> > > > > > >> > > > > > >>> > > > > > >
> > > > > > >> > > > > > >>> > > > > > > On Wed, Sep 9, 2020 at 10:07 PM
> > Justine
> > > > > > Olshan <
> > > > > > >> > > > > > >>> > jolshan@confluent.io
> > > > > > >> > > > > > >>> > > > > > > wrote:
> > > > > > >> > > > > > >>> > > > > > > > Hello all, it's been almost a
> year!
> > > I've
> > > > > > made
> > > > > > >> > some
> > > > > > >> > > > > > changes
> > > > > > >> > > > > > >>> to
> > > > > > >> > > > > > >>> > this
> > > > > > >> > > > > > >>> > > > > KIP
> > > > > > >> > > > > > >>> > > > > > > and hope to continue the discussion.
> > > > > > >> > > > > > >>> > > > > > > > One of the main changes I've added
> > is
> > > > now
> > > > > > the
> > > > > > >> > > > metadata
> > > > > > >> > > > > > >>> response
> > > > > > >> > > > > > >>> > > > will
> > > > > > >> > > > > > >>> > > > > > > include the topic ID (as Colin
> > > suggested).
> > > > > > >> Clients
> > > > > > >> > > can
> > > > > > >> > > > > > >>> obtain the
> > > > > > >> > > > > > >>> > > > > topicID
> > > > > > >> > > > > > >>> > > > > > > of a given topic through a
> > > > TopicDescription.
> > > > > > The
> > > > > > >> > > > topicId
> > > > > > >> > > > > > will
> > > > > > >> > > > > > >>> > also be
> > > > > > >> > > > > > >>> > > > > > > included with the UpdateMetadata
> > > request.
> > > > > > >> > > > > > >>> > > > > > > > Let me know what you all think.
> > > > > > >> > > > > > >>> > > > > > > > Thank you,
> > > > > > >> > > > > > >>> > > > > > > > Justine
> > > > > > >> > > > > > >>> > > > > > > >
> > > > > > >> > > > > > >>> > > > > > > > On 2019/09/13 16:38:26, "Colin
> > > McCabe" <
> > > > > > >> > > > > > cmccabe@apache.org
> > > > > > >> > > > > > >>> >
> > > > > > >> > > > > > >>> > wrote:
> > > > > > >> > > > > > >>> > > > > > > > > Hi Lucas,
> > > > > > >> > > > > > >>> > > > > > > > >
> > > > > > >> > > > > > >>> > > > > > > > > Thanks for tackling this.  Topic
> > IDs
> > > > > are a
> > > > > > >> > great
> > > > > > >> > > > > idea,
> > > > > > >> > > > > > >>> and
> > > > > > >> > > > > > >>> > this
> > > > > > >> > > > > > >>> > > > is
> > > > > > >> > > > > > >>> > > > > a
> > > > > > >> > > > > > >>> > > > > > > really good writeup.
> > > > > > >> > > > > > >>> > > > > > > > > For /brokers/topics/[topic], the
> > > > schema
> > > > > > >> version
> > > > > > >> > > > > should
> > > > > > >> > > > > > be
> > > > > > >> > > > > > >>> > bumped
> > > > > > >> > > > > > >>> > > > to
> > > > > > >> > > > > > >>> > > > > > > version 3, rather than 2.  KIP-455
> > > bumped
> > > > > the
> > > > > > >> > version
> > > > > > >> > > > of
> > > > > > >> > > > > > this
> > > > > > >> > > > > > >>> > znode
> > > > > > >> > > > > > >>> > > > to
> > > > > > >> > > > > > >>> > > > > 2
> > > > > > >> > > > > > >>> > > > > > > already :)
> > > > > > >> > > > > > >>> > > > > > > > > Given that we're going to be
> > seeing
> > > > > these
> > > > > > >> > things
> > > > > > >> > > as
> > > > > > >> > > > > > >>> strings
> > > > > > >> > > > > > >>> > as
> > > > > > >> > > > > > >>> > > > lot
> > > > > > >> > > > > > >>> > > > > (in
> > > > > > >> > > > > > >>> > > > > > > logs, in ZooKeeper, on the
> > command-line,
> > > > > > etc.),
> > > > > > >> > does
> > > > > > >> > > it
> > > > > > >> > > > > > make
> > > > > > >> > > > > > >>> > sense to
> > > > > > >> > > > > > >>> > > > > use
> > > > > > >> > > > > > >>> > > > > > > base64 when converting them to
> > strings?
> > > > > > >> > > > > > >>> > > > > > > > > Here is an example of the hex
> > > > > > >> representation:
> > > > > > >> > > > > > >>> > > > > > > > >
> > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> > > > > > >> > > > > > >>> > > > > > > > >
> > > > > > >> > > > > > >>> > > > > > > > > And here is an example in
> base64.
> > > > > > >> > > > > > >>> > > > > > > > > b8tRS7h4TJ2Vt43Dp85v2A
> > > > > > >> > > > > > >>> > > > > > > > >
> > > > > > >> > > > > > >>> > > > > > > > > The base64 version saves 15
> > letters
> > > > (to
> > > > > be
> > > > > > >> > fair,
> > > > > > >> > > 4
> > > > > > >> > > > of
> > > > > > >> > > > > > >>> those
> > > > > > >> > > > > > >>> > were
> > > > > > >> > > > > > >>> > > > > > > dashes that we could have elided in
> > the
> > > > hex
> > > > > > >> > > > > > representation.)
> > > > > > >> > > > > > >>> > > > > > > > > Another thing to consider is
> that
> > we
> > > > > > should
> > > > > > >> > > specify
> > > > > > >> > > > > > that
> > > > > > >> > > > > > >>> the
> > > > > > >> > > > > > >>> > > > > > > all-zeroes UUID is not a valid topic
> > > UUID.
> > > > > >  We
> > > > > > >> > can't
> > > > > > >> > > > use
> > > > > > >> > > > > > >>> null
> > > > > > >> > > > > > >>> > for
> > > > > > >> > > > > > >>> > > > this
> > > > > > >> > > > > > >>> > > > > > > because we can't pass a null UUID
> over
> > > the
> > > > > RPC
> > > > > > >> > > protocol
> > > > > > >> > > > > > >>> (there
> > > > > > >> > > > > > >>> > is no
> > > > > > >> > > > > > >>> > > > > > > special pattern for null, nor do we
> > want
> > > > to
> > > > > > >> waste
> > > > > > >> > > space
> > > > > > >> > > > > > >>> reserving
> > > > > > >> > > > > > >>> > > > such
> > > > > > >> > > > > > >>> > > > > a
> > > > > > >> > > > > > >>> > > > > > > pattern.)
> > > > > > >> > > > > > >>> > > > > > > > > Maybe I missed it, but did you
> > > > describe
> > > > > > >> > > "migration
> > > > > > >> > > > > > of...
> > > > > > >> > > > > > >>> > existing
> > > > > > >> > > > > > >>> > > > > > > topic[s] without topic IDs" in
> detail
> > in
> > > > any
> > > > > > >> > section?
> > > > > > >> > > > It
> > > > > > >> > > > > > >>> seems
> > > > > > >> > > > > > >>> > like
> > > > > > >> > > > > > >>> > > > > when
> > > > > > >> > > > > > >>> > > > > > > the new controller becomes active,
> it
> > > > should
> > > > > > >> just
> > > > > > >> > > > > generate
> > > > > > >> > > > > > >>> random
> > > > > > >> > > > > > >>> > > > > UUIDs for
> > > > > > >> > > > > > >>> > > > > > > these, and write the random UUIDs
> back
> > > to
> > > > > > >> > ZooKeeper.
> > > > > > >> > > > It
> > > > > > >> > > > > > >>> would be
> > > > > > >> > > > > > >>> > > > good
> > > > > > >> > > > > > >>> > > > > to
> > > > > > >> > > > > > >>> > > > > > > spell that out.  We should make it
> > clear
> > > > > that
> > > > > > >> this
> > > > > > >> > > > > happens
> > > > > > >> > > > > > >>> > regardless
> > > > > > >> > > > > > >>> > > > > of
> > > > > > >> > > > > > >>> > > > > > > the inter-broker protocol version
> > (it's
> > > a
> > > > > > >> > compatible
> > > > > > >> > > > > > change).
> > > > > > >> > > > > > >>> > > > > > > > > "LeaderAndIsrRequests including
> an
> > > > > > >> > > > is_every_partition
> > > > > > >> > > > > > >>> flag"
> > > > > > >> > > > > > >>> > > > seems a
> > > > > > >> > > > > > >>> > > > > > > bit wordy.  Can we just call these
> > "full
> > > > > > >> > > > > > >>> LeaderAndIsrRequests"?
> > > > > > >> > > > > > >>> > Then
> > > > > > >> > > > > > >>> > > > > the
> > > > > > >> > > > > > >>> > > > > > > RPC field could be named "full".
> > Also,
> > > it
> > > > > > would
> > > > > > >> > > > probably
> > > > > > >> > > > > > be
> > > > > > >> > > > > > >>> > better
> > > > > > >> > > > > > >>> > > > > for the
> > > > > > >> > > > > > >>> > > > > > > RPC field to be an enum of {
> > > UNSPECIFIED,
> > > > > > >> > > INCREMENTAL,
> > > > > > >> > > > > FULL
> > > > > > >> > > > > > >>> }, so
> > > > > > >> > > > > > >>> > > > that
> > > > > > >> > > > > > >>> > > > > we
> > > > > > >> > > > > > >>> > > > > > > can cleanly handle old versions (by
> > > > treating
> > > > > > >> them
> > > > > > >> > as
> > > > > > >> > > > > > >>> UNSPECIFIED)
> > > > > > >> > > > > > >>> > > > > > > > > In the LeaderAndIsrRequest
> > section,
> > > > you
> > > > > > >> write
> > > > > > >> > "A
> > > > > > >> > > > > final
> > > > > > >> > > > > > >>> > deletion
> > > > > > >> > > > > > >>> > > > > event
> > > > > > >> > > > > > >>> > > > > > > will be secheduled for X ms after
> the
> > > > > > >> > > > LeaderAndIsrRequest
> > > > > > >> > > > > > was
> > > > > > >> > > > > > >>> > first
> > > > > > >> > > > > > >>> > > > > > > received..."  I guess the X was a
> > > > > placeholder
> > > > > > >> that
> > > > > > >> > > you
> > > > > > >> > > > > > >>> intended
> > > > > > >> > > > > > >>> > to
> > > > > > >> > > > > > >>> > > > > replace
> > > > > > >> > > > > > >>> > > > > > > before posting? :)  In any case,
> this
> > > > seems
> > > > > > like
> > > > > > >> > the
> > > > > > >> > > > kind
> > > > > > >> > > > > > of
> > > > > > >> > > > > > >>> > thing
> > > > > > >> > > > > > >>> > > > we'd
> > > > > > >> > > > > > >>> > > > > > > want a configuration for.  Let's
> > > describe
> > > > > that
> > > > > > >> > > > > > configuration
> > > > > > >> > > > > > >>> key
> > > > > > >> > > > > > >>> > > > > somewhere
> > > > > > >> > > > > > >>> > > > > > > in this KIP, including what its
> > default
> > > > > value
> > > > > > >> is.
> > > > > > >> > > > > > >>> > > > > > > > > We should probably also log a
> > bunch
> > > of
> > > > > > >> messages
> > > > > > >> > > at
> > > > > > >> > > > > WARN
> > > > > > >> > > > > > >>> level
> > > > > > >> > > > > > >>> > > > when
> > > > > > >> > > > > > >>> > > > > > > something is scheduled for deletion,
> > as
> > > > > well.
> > > > > > >> > (Maybe
> > > > > > >> > > > > this
> > > > > > >> > > > > > >>> was
> > > > > > >> > > > > > >>> > > > > assumed, but
> > > > > > >> > > > > > >>> > > > > > > it would be good to mention it).
> > > > > > >> > > > > > >>> > > > > > > > > I feel like there are a few
> > sections
> > > > > that
> > > > > > >> > should
> > > > > > >> > > be
> > > > > > >> > > > > > >>> moved to
> > > > > > >> > > > > > >>> > > > > "rejected
> > > > > > >> > > > > > >>> > > > > > > alternatives."  For example, in the
> > > > > > DeleteTopics
> > > > > > >> > > > section,
> > > > > > >> > > > > > >>> since
> > > > > > >> > > > > > >>> > we're
> > > > > > >> > > > > > >>> > > > > not
> > > > > > >> > > > > > >>> > > > > > > going to do option 1 or 2, these
> > should
> > > be
> > > > > > moved
> > > > > > >> > into
> > > > > > >> > > > > > >>> "rejected
> > > > > > >> > > > > > >>> > > > > > > alternatives,"  rather than
> appearing
> > > > > inline.
> > > > > > >> > > Another
> > > > > > >> > > > > case
> > > > > > >> > > > > > >>> is
> > > > > > >> > > > > > >>> > the
> > > > > > >> > > > > > >>> > > > > "Should
> > > > > > >> > > > > > >>> > > > > > > we remove topic name from the
> protocol
> > > > where
> > > > > > >> > > possible"
> > > > > > >> > > > > > >>> section.
> > > > > > >> > > > > > >>> > This
> > > > > > >> > > > > > >>> > > > > is
> > > > > > >> > > > > > >>> > > > > > > clearly discussing a design
> > alternative
> > > > that
> > > > > > >> we're
> > > > > > >> > > not
> > > > > > >> > > > > > >>> proposing
> > > > > > >> > > > > > >>> > to
> > > > > > >> > > > > > >>> > > > > > > implement: removing the topic name
> > from
> > > > > those
> > > > > > >> > > > protocols.
> > > > > > >> > > > > > >>> > > > > > > > > Is it really necessary to have a
> > new
> > > > > > >> > > > > > >>> > /admin/delete_topics_by_id
> > > > > > >> > > > > > >>> > > > > path
> > > > > > >> > > > > > >>> > > > > > > in ZooKeeper?  It seems like we
> don't
> > > > really
> > > > > > >> need
> > > > > > >> > > this.
> > > > > > >> > > > > > >>> Whenever
> > > > > > >> > > > > > >>> > > > > there is
> > > > > > >> > > > > > >>> > > > > > > a new controller, we'll send out
> full
> > > > > > >> > > > > LeaderAndIsrRequests
> > > > > > >> > > > > > >>> which
> > > > > > >> > > > > > >>> > will
> > > > > > >> > > > > > >>> > > > > > > trigger the stale topics to be
> cleaned
> > > up.
> > > > > >  The
> > > > > > >> > > active
> > > > > > >> > > > > > >>> > controller
> > > > > > >> > > > > > >>> > > > will
> > > > > > >> > > > > > >>> > > > > > > also send the full
> LeaderAndIsrRequest
> > > to
> > > > > > >> brokers
> > > > > > >> > > that
> > > > > > >> > > > > are
> > > > > > >> > > > > > >>> just
> > > > > > >> > > > > > >>> > > > > starting
> > > > > > >> > > > > > >>> > > > > > > up.    So we don't really need this
> > kind
> > > > of
> > > > > > >> > two-phase
> > > > > > >> > > > > > commit
> > > > > > >> > > > > > >>> > (send
> > > > > > >> > > > > > >>> > > > out
> > > > > > >> > > > > > >>> > > > > > > StopReplicasRequest, get ACKs from
> all
> > > > > nodes,
> > > > > > >> > commit
> > > > > > >> > > by
> > > > > > >> > > > > > >>> removing
> > > > > > >> > > > > > >>> > > > > > > /admin/delete_topics node) any more.
> > > > > > >> > > > > > >>> > > > > > > > > You mention that FetchRequest
> will
> > > now
> > > > > > >> include
> > > > > > >> > > UUID
> > > > > > >> > > > > to
> > > > > > >> > > > > > >>> avoid
> > > > > > >> > > > > > >>> > > > issues
> > > > > > >> > > > > > >>> > > > > > > where requests are made to stale
> > > > partitions.
> > > > > > >> > > However,
> > > > > > >> > > > > > >>> adding a
> > > > > > >> > > > > > >>> > UUID
> > > > > > >> > > > > > >>> > > > to
> > > > > > >> > > > > > >>> > > > > > > MetadataRequest is listed as future
> > > work,
> > > > > out
> > > > > > of
> > > > > > >> > > scope
> > > > > > >> > > > > for
> > > > > > >> > > > > > >>> this
> > > > > > >> > > > > > >>> > KIP.
> > > > > > >> > > > > > >>> > > > > How
> > > > > > >> > > > > > >>> > > > > > > will the client learn what the topic
> > > UUID
> > > > > is,
> > > > > > if
> > > > > > >> > the
> > > > > > >> > > > > > metadata
> > > > > > >> > > > > > >>> > > > response
> > > > > > >> > > > > > >>> > > > > > > doesn't include that information?
> It
> > > > seems
> > > > > > like
> > > > > > >> > > adding
> > > > > > >> > > > > the
> > > > > > >> > > > > > >>> UUID
> > > > > > >> > > > > > >>> > to
> > > > > > >> > > > > > >>> > > > > > > MetadataResponse would be an
> > improvement
> > > > > here
> > > > > > >> that
> > > > > > >> > > > might
> > > > > > >> > > > > > not
> > > > > > >> > > > > > >>> be
> > > > > > >> > > > > > >>> > too
> > > > > > >> > > > > > >>> > > > > hard to
> > > > > > >> > > > > > >>> > > > > > > make.
> > > > > > >> > > > > > >>> > > > > > > > > best,
> > > > > > >> > > > > > >>> > > > > > > > > Colin
> > > > > > >> > > > > > >>> > > > > > > > >
> > > > > > >> > > > > > >>> > > > > > > > >
> > > > > > >> > > > > > >>> > > > > > > > > On Mon, Sep 9, 2019, at 17:48,
> > > Ryanne
> > > > > > Dolan
> > > > > > >> > > wrote:
> > > > > > >> > > > > > >>> > > > > > > > > > Lucas, this would be great.
> I've
> > > run
> > > > > > into
> > > > > > >> > > issues
> > > > > > >> > > > > with
> > > > > > >> > > > > > >>> > topics
> > > > > > >> > > > > > >>> > > > > being
> > > > > > >> > > > > > >>> > > > > > > > > > resurrected accidentally,
> since
> > a
> > > > > client
> > > > > > >> > cannot
> > > > > > >> > > > > > easily
> > > > > > >> > > > > > >>> > > > > distinguish
> > > > > > >> > > > > > >>> > > > > > > between
> > > > > > >> > > > > > >>> > > > > > > > > > a deleted topic and a new
> topic
> > > with
> > > > > the
> > > > > > >> same
> > > > > > >> > > > name.
> > > > > > >> > > > > > I'd
> > > > > > >> > > > > > >>> > need
> > > > > > >> > > > > > >>> > > > the
> > > > > > >> > > > > > >>> > > > > ID
> > > > > > >> > > > > > >>> > > > > > > > > > accessible from the client to
> > > solve
> > > > > that
> > > > > > >> > issue,
> > > > > > >> > > > but
> > > > > > >> > > > > > >>> this
> > > > > > >> > > > > > >>> > is a
> > > > > > >> > > > > > >>> > > > > good
> > > > > > >> > > > > > >>> > > > > > > first
> > > > > > >> > > > > > >>> > > > > > > > > > step.
> > > > > > >> > > > > > >>> > > > > > > > > >
> > > > > > >> > > > > > >>> > > > > > > > > > Ryanne
> > > > > > >> > > > > > >>> > > > > > > > > >
> > > > > > >> > > > > > >>> > > > > > > > > > On Wed, Sep 4, 2019 at 1:41 PM
> > > Lucas
> > > > > > >> > > Bradstreet <
> > > > > > >> > > > > > >>> > > > > lucas@confluent.io>
> > > > > > >> > > > > > >>> > > > > > > wrote:
> > > > > > >> > > > > > >>> > > > > > > > > > > Hi all,
> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > > > >> > > > > > >>> > > > > > > > > > > I would like to kick off
> > > > discussion
> > > > > of
> > > > > > >> > > KIP-516,
> > > > > > >> > > > > an
> > > > > > >> > > > > > >>> > > > > implementation
> > > > > > >> > > > > > >>> > > > > > > of topic
> > > > > > >> > > > > > >>> > > > > > > > > > > IDs for Kafka. Topic IDs aim
> > to
> > > > > solve
> > > > > > >> topic
> > > > > > >> > > > > > >>> uniqueness
> > > > > > >> > > > > > >>> > > > > problems in
> > > > > > >> > > > > > >>> > > > > > > Kafka,
> > > > > > >> > > > > > >>> > > > > > > > > > > where referring to a topic
> by
> > > name
> > > > > > >> alone is
> > > > > > >> > > > > > >>> insufficient.
> > > > > > >> > > > > > >>> > > > Such
> > > > > > >> > > > > > >>> > > > > > > cases
> > > > > > >> > > > > > >>> > > > > > > > > > > include when a topic has
> been
> > > > > deleted
> > > > > > >> and
> > > > > > >> > > > > recreated
> > > > > > >> > > > > > >>> with
> > > > > > >> > > > > > >>> > the
> > > > > > >> > > > > > >>> > > > > same
> > > > > > >> > > > > > >>> > > > > > > name.
> > > > > > >> > > > > > >>> > > > > > > > > > > Unique identifiers will help
> > > > > simplify
> > > > > > >> and
> > > > > > >> > > > improve
> > > > > > >> > > > > > >>> Kafka's
> > > > > > >> > > > > > >>> > > > topic
> > > > > > >> > > > > > >>> > > > > > > deletion
> > > > > > >> > > > > > >>> > > > > > > > > > > process, as well as prevent
> > > cases
> > > > > > where
> > > > > > >> > > brokers
> > > > > > >> > > > > may
> > > > > > >> > > > > > >>> > > > incorrectly
> > > > > > >> > > > > > >>> > > > > > > interact
> > > > > > >> > > > > > >>> > > > > > > > > > > with stale versions of
> topics.
> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > > > >> > > > > > >>> > > >
> > > > > > >> > > > > > >>> >
> > > > > > >> > > > > > >>>
> > > > > > >> > > > > >
> > > > > > >> > > > >
> > > > > > >> > > >
> > > > > > >> > >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > > > > > >> > > > > > >>> > > > > > > > > > > Looking forward to your
> > > thoughts.
> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > > > >> > > > > > >>> > > > > > > > > > > Lucas
> > > > > > >> > > > > > >>> > > > > > > > > > >
> > > > > > >> > > > > > >>> >
> > > > > > >> > > > > > >>> >
> > > > > > >> > > > > > >>>
> > > > > > >> > > > > > >>
> > > > > > >> > > > > >
> > > > > > >> > > > >
> > > > > > >> > > >
> > > > > > >> > >
> > > > > > >> >
> > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[David Jacot <dj...@confluent.io>]

Hi Justine,

Thanks for the KIP. I finally had time to read it :). It is a great
improvement.

I have a few comments/questions:

1. It seems that the schema of the StopReplicaRequest is slightly outdated.
We
did some changes as part of KIP-570. V3 is already organized by topics.

2. I just want to make sure that I understand the reconciliation
logic correctly. When
an "INCREMENTAL" LeaderAndIsr Request is received, the broker will also
reconcile
when the local uuid does not match the uuid in the request, right? In this
case, the
local log is staged for deletion.

3. In the documentation of the `delete.stale.topic.delay.ms` config, it
says "When a
FULL LeaderAndIsrRequest is received..." but I suppose it applies to both
types.

Best,
David

On Thu, Sep 24, 2020 at 1:40 AM Justine Olshan <jo...@confluent.io> wrote:

> Hi Jun,
>
> Thanks for the comments. I apologize for some of the typos and confusion.
> I’ve updated the KIP to fix some of the issues you mentioned.
>
> 20.2 I’ve changed the type to String.
> 20.1/3 I’ve updated the TopicZNode to fix formatting and reflect the latest
> version before this change.
>
> 21. You are correct and I’ve removed this line. I’ve also added a line
> mentioning an IBP bump is necessary for migration
>
> 22. I think the wording was unclear but your summary is what was intended
> by this line. I’ve updated to make this point more clear. “Remove deleted
> topics from replicas by sending StopReplicaRequest V3 before the IBP bump
> using the old logic, and using V4 and the new logic with topic IDs after
> the IBP bump.”
>
> 23. I’ve removed the unspecified type from the KIP and mention that IBP
> will be used to handle this request. “IBP will be used to determine whether
> this new form of the request will be used. For older requests, we will
> ignore this field and default to previous behavior.”
>
> 24. I’ve fixed this typo.
>
> 25. I've added a topics at a higher level for LeaderAndIsrResponse v5,
> StopReplicaResponse v4. I've also changed StopReplicaRequest v4 to have
> topics at a higher level.
>
> 26. I’ve updated forgotten_topics_data--added the topic ID and removed the
> topic name
>
> 27. I’ve decided on plain text, and I’ve added an example.
>
> 28. I’ve added this idea to future work.
>
> Thanks again for taking a look,
>
> Justine
>
> On Wed, Sep 23, 2020 at 10:28 AM Jun Rao <ju...@confluent.io> wrote:
>
> > Hi, Justine,
> >
> > Thanks for the response. Made another pass. A few more comments below.
> >
> > 20. znode schema:
> > 20.1 It seems that {"name": "version", "type": "int", "id": "UUID",
> "doc":
> > "version id"} should be {"name": "version", "type": "int"}, {"name":
> "id",
> > "type": "UUID", "doc": "version id"}.
> > 20.2 The znode format is JSON which doesn't have UUID type. So the type
> > probably should be string?
> > 20.3 Also, the existing format used seems outdated. It should have the
> > following format.
> >     Json.encodeAsBytes(Map(
> >       "version" -> 2,
> >       "partitions" -> replicaAssignmentJson.asJava,
> >       "adding_replicas" -> addingReplicasAssignmentJson.asJava,
> >       "removing_replicas" -> removingReplicasAssignmentJson.asJava
> >     ).asJava)
> >   }
> >
> > 21. Migration: The KIP says "The migration process can take place without
> > an inter-broker protocol bump, as the format stored in
> > /brokers/topics/[topic] will be compatible with older broker versions."
> > However, since we are bumping up the version of inter-broker requests, it
> > seems that we need to use IBP for migration.
> >
> > 22. The KIP says "Remove deleted topics from replicas by sending
> > StopReplicaRequest V3 for any topics which do not contain a topic ID, and
> > V4 for any topics which do contain a topic ID.". However, if we use IBP,
> it
> > seems that the controller will either send StopReplicaRequest V3
> > or StopReplicaRequest V4, but never mixed V3 and V4 for different topics.
> > Basically, before the IBP bump, V3 will be used. After the IBP bump,
> > topicId will be created and V4 will be used.
> >
> > 23. Given that we depend on IBP, do we still need "0 UNSPECIFIED"
> > in LeaderAndIsr?
> >
> > 24. LeaderAndIsrResponse v5 : It still has the topic field.
> >
> > 25. LeaderAndIsrResponse v5, StopReplicaResponse v4: Could we use this
> > opportunity to organize the response in 2 levels, first by topic, then by
> > partition, as most other requests/responses?
> >
> > 26. FetchRequest v13 : Should forgotten_topics_data use topicId too?
> >
> > 27. "This file can either be plain text (key/value pairs) or JSON." Have
> we
> > decided which one to use? Also, it would be helpful to provide an
> example.
> >
> > 28. Future improvement: Another future improvement opportunity is to use
> > topicId in GroupMetadataManager.offsetCommitKey in the offset_commit
> topic.
> > This may save some space.
> >
> > Thanks,
> >
> > Jun
> >
> > On Wed, Sep 23, 2020 at 8:50 AM Justine Olshan <jo...@confluent.io>
> > wrote:
> >
> > > Hi Tom,
> > >
> > > Thanks for the comment. I think this is a really good idea and it has
> > been
> > > added to the KIP under the newly added tooling section.
> > >
> > > Thanks again,
> > > Justine
> > >
> > > On Wed, Sep 23, 2020 at 3:17 AM Tom Bentley <tb...@redhat.com>
> wrote:
> > >
> > > > Hi Justine,
> > > >
> > > > I know you started the vote thread, but on re-reading the KIP I
> noticed
> > > > that although the topic id is included in the MetadataResponse it's
> not
> > > > surfaced in the output from `kafka-topics.sh --describe`. Maybe that
> > was
> > > > intentional because ids are intentionally not really something the
> user
> > > > should care deeply about, but it would also make life harder for
> anyone
> > > > debugging Kafka and this would likely get worse the more topic ids
> got
> > > > rolled out across the protocols, clients etc. It seems likely that
> > > > `kafka-topics.sh` will eventually need the ability to show the id of
> a
> > > > topic and perhaps find a topic name given an id. Is there any reason
> > not
> > > to
> > > > implement that in this KIP?
> > > >
> > > > Many thanks,
> > > >
> > > > Tom
> > > >
> > > > On Mon, Sep 21, 2020 at 9:54 PM Justine Olshan <jolshan@confluent.io
> >
> > > > wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > After thinking about it, I've decided to remove the topic name from
> > the
> > > > > Fetch Request and Response after all. Since there are so many of
> > these
> > > > > requests per second, it is worth removing the extra information.
> I've
> > > > > updated the KIP to reflect this change.
> > > > >
> > > > > Please let me know if there is anything else we should discuss
> before
> > > > > voting.
> > > > >
> > > > > Thank you,
> > > > > Justine
> > > > >
> > > > > On Fri, Sep 18, 2020 at 9:46 AM Justine Olshan <
> jolshan@confluent.io
> > >
> > > > > wrote:
> > > > >
> > > > > > Hi Jun,
> > > > > >
> > > > > > I see what you are saying. For now we can remove the extra
> > > information.
> > > > > > I'll leave the option to add more fields to the file in the
> future.
> > > The
> > > > > KIP
> > > > > > has been updated to reflect this change.
> > > > > >
> > > > > > Thanks,
> > > > > > Justine
> > > > > >
> > > > > > On Fri, Sep 18, 2020 at 8:46 AM Jun Rao <ju...@confluent.io>
> wrote:
> > > > > >
> > > > > >> Hi, Justine,
> > > > > >>
> > > > > >> Thanks for the reply.
> > > > > >>
> > > > > >> 13. If the log directory is the source of truth, it means that
> the
> > > > > >> redundant info in the metadata file will be ignored. Then the
> > > question
> > > > > is
> > > > > >> why do we need to put the redundant info in the metadata file
> now?
> > > > > >>
> > > > > >> Thanks,
> > > > > >>
> > > > > >> Jun
> > > > > >>
> > > > > >> On Thu, Sep 17, 2020 at 5:07 PM Justine Olshan <
> > > jolshan@confluent.io>
> > > > > >> wrote:
> > > > > >>
> > > > > >> > Hi Jun,
> > > > > >> > Thanks for the quick response!
> > > > > >> >
> > > > > >> > 12. I've decided to bump up the versions on the requests and
> > > updated
> > > > > the
> > > > > >> > KIP. I think it's good we thoroughly discussed the options
> here,
> > > so
> > > > we
> > > > > >> know
> > > > > >> > we made a good choice. :)
> > > > > >> >
> > > > > >> > 13. This is an interesting situation. I think if this does
> occur
> > > we
> > > > > >> should
> > > > > >> > give a warning. I agree that it's hard to know the source of
> > truth
> > > > for
> > > > > >> sure
> > > > > >> > since the directory or the file could be manually modified. I
> > > guess
> > > > > the
> > > > > >> > directory could be used as the source of truth. To be honest,
> > I'm
> > > > not
> > > > > >> > really sure what happens in kafka when the log directory is
> > > renamed
> > > > > >> > manually in such a way. I'm also wondering if the situation is
> > > > > >> recoverable
> > > > > >> > in this scenario.
> > > > > >> >
> > > > > >> > Thanks,
> > > > > >> > Justine
> > > > > >> >
> > > > > >> > On Thu, Sep 17, 2020 at 4:28 PM Jun Rao <ju...@confluent.io>
> > wrote:
> > > > > >> >
> > > > > >> > > Hi, Justine,
> > > > > >> > >
> > > > > >> > > Thanks for the reply.
> > > > > >> > >
> > > > > >> > > 12. I don't have a strong preference either. However, if we
> > need
> > > > IBP
> > > > > >> > > anyway, maybe it's easier to just bump up the version for
> all
> > > > inter
> > > > > >> > broker
> > > > > >> > > requests and add the topic id field as a regular field. A
> > > regular
> > > > > >> field
> > > > > >> > is
> > > > > >> > > a bit more concise in wire transfer than a flexible field.
> > > > > >> > >
> > > > > >> > > 13. The confusion that I was referring to is between the
> topic
> > > > name
> > > > > >> and
> > > > > >> > > partition number between the log dir and the metadata file.
> > For
> > > > > >> example,
> > > > > >> > if
> > > > > >> > > the log dir is topicA-1 and the metadata file in it has
> topicB
> > > and
> > > > > >> > > partition 0 (say due to a bug or manual modification), which
> > one
> > > > do
> > > > > we
> > > > > >> > use
> > > > > >> > > as the source of truth?
> > > > > >> > >
> > > > > >> > > Jun
> > > > > >> > >
> > > > > >> > > On Thu, Sep 17, 2020 at 3:43 PM Justine Olshan <
> > > > > jolshan@confluent.io>
> > > > > >> > > wrote:
> > > > > >> > >
> > > > > >> > > > Hi Jun,
> > > > > >> > > > Thanks for the comments.
> > > > > >> > > >
> > > > > >> > > > 12. I bumped the LeaderAndIsrRequest because I removed the
> > > topic
> > > > > >> name
> > > > > >> > > field
> > > > > >> > > > in the response. It may be possible to avoid bumping the
> > > version
> > > > > >> > without
> > > > > >> > > > that change, but I may be missing something.
> > > > > >> > > > I believe StopReplica is actually on version 3 now, but
> > > because
> > > > > >> > version 2
> > > > > >> > > > is flexible, I kept that listed as version 2 on the KIP
> > page.
> > > > > >> However,
> > > > > >> > > you
> > > > > >> > > > may be right in that we may need to bump the version on
> > > > > StopReplica
> > > > > >> to
> > > > > >> > > deal
> > > > > >> > > > with deletion differently as mentioned above. I don't know
> > if
> > > I
> > > > > >> have a
> > > > > >> > > big
> > > > > >> > > > preference over used tagged fields or not.
> > > > > >> > > >
> > > > > >> > > > 13. I was thinking that in the case where the file and the
> > > > request
> > > > > >> > topic
> > > > > >> > > > ids don't match, it means that the broker's topic/the one
> in
> > > the
> > > > > >> file
> > > > > >> > has
> > > > > >> > > > been deleted. In that case, we would need to delete the
> old
> > > > topic
> > > > > >> and
> > > > > >> > > start
> > > > > >> > > > receiving the new version. If the topic name were to
> change,
> > > but
> > > > > the
> > > > > >> > ids
> > > > > >> > > > still matched, the file would also need to update. Am I
> > > missing
> > > > a
> > > > > >> case
> > > > > >> > > > where the file would be correct and not the request?
> > > > > >> > > >
> > > > > >> > > > Thanks,
> > > > > >> > > > Justine
> > > > > >> > > >
> > > > > >> > > > On Thu, Sep 17, 2020 at 3:18 PM Jun Rao <jun@confluent.io
> >
> > > > wrote:
> > > > > >> > > >
> > > > > >> > > > > Hi, Justine,
> > > > > >> > > > >
> > > > > >> > > > > Thanks for the reply. A couple of more comments below.
> > > > > >> > > > >
> > > > > >> > > > > 12. ListOffset and OffsetForLeader currently don't
> support
> > > > > >> flexible
> > > > > >> > > > fields.
> > > > > >> > > > > So, we have to bump up the version number and use IBP at
> > > least
> > > > > for
> > > > > >> > > these
> > > > > >> > > > > two requests. Note that it seems 2.7.0 will require IBP
> > > anyway
> > > > > >> > because
> > > > > >> > > of
> > > > > >> > > > > changes in KAFKA-10435. Also, it seems that the version
> > for
> > > > > >> > > > > LeaderAndIsrRequest and StopReplica are bumped even
> though
> > > we
> > > > > only
> > > > > >> > > added
> > > > > >> > > > a
> > > > > >> > > > > tagged field. But since IBP is needed anyway, we may
> want
> > to
> > > > > >> revisit
> > > > > >> > > the
> > > > > >> > > > > overall tagged field choice.
> > > > > >> > > > >
> > > > > >> > > > > 13. The only downside is the potential confusion on
> which
> > > one
> > > > is
> > > > > >> the
> > > > > >> > > > source
> > > > > >> > > > > of truth if they don't match. Another option is to
> include
> > > > those
> > > > > >> > fields
> > > > > >> > > > in
> > > > > >> > > > > the metadata file when we actually change the directory
> > > > > structure.
> > > > > >> > > > >
> > > > > >> > > > > Thanks,
> > > > > >> > > > >
> > > > > >> > > > > Jun
> > > > > >> > > > >
> > > > > >> > > > > On Thu, Sep 17, 2020 at 2:01 PM Justine Olshan <
> > > > > >> jolshan@confluent.io
> > > > > >> > >
> > > > > >> > > > > wrote:
> > > > > >> > > > >
> > > > > >> > > > > > Hello all,
> > > > > >> > > > > >
> > > > > >> > > > > > I've thought some more about removing the topic name
> > field
> > > > > from
> > > > > >> > some
> > > > > >> > > of
> > > > > >> > > > > the
> > > > > >> > > > > > requests. On closer inspection of the
> > requests/responses,
> > > it
> > > > > >> seems
> > > > > >> > > that
> > > > > >> > > > > the
> > > > > >> > > > > > internal changes would be much larger than I expected.
> > > Some
> > > > > >> > protocols
> > > > > >> > > > > > involve clients, so they would require changes too.
> I'm
> > > > > thinking
> > > > > >> > that
> > > > > >> > > > for
> > > > > >> > > > > > now, removing the topic name from these requests and
> > > > responses
> > > > > >> are
> > > > > >> > > out
> > > > > >> > > > of
> > > > > >> > > > > > scope.
> > > > > >> > > > > >
> > > > > >> > > > > > I have decided to just keep the change
> > > LeaderAndIsrResponse
> > > > to
> > > > > >> > remove
> > > > > >> > > > the
> > > > > >> > > > > > topic name, and have updated the KIP to reflect this
> > > > change. I
> > > > > >> have
> > > > > >> > > > also
> > > > > >> > > > > > mentioned the other requests and responses in future
> > work.
> > > > > >> > > > > >
> > > > > >> > > > > > I'm hoping to start the voting process soon, so let me
> > > know
> > > > if
> > > > > >> > there
> > > > > >> > > is
> > > > > >> > > > > > anything else we should discuss.
> > > > > >> > > > > >
> > > > > >> > > > > > Thank you,
> > > > > >> > > > > > Justine
> > > > > >> > > > > >
> > > > > >> > > > > > On Tue, Sep 15, 2020 at 3:57 PM Justine Olshan <
> > > > > >> > jolshan@confluent.io
> > > > > >> > > >
> > > > > >> > > > > > wrote:
> > > > > >> > > > > >
> > > > > >> > > > > > > Hello again,
> > > > > >> > > > > > > To follow up on some of the other comments:
> > > > > >> > > > > > >
> > > > > >> > > > > > > 10/11) We can remove the topic name from these
> > > > > >> > requests/responses,
> > > > > >> > > > and
> > > > > >> > > > > > > that means we will just have to make a few internal
> > > > changes
> > > > > to
> > > > > >> > make
> > > > > >> > > > > > > partitions accessible by topic id and partition. I
> can
> > > > > update
> > > > > >> the
> > > > > >> > > KIP
> > > > > >> > > > > to
> > > > > >> > > > > > > remove them unless anyone thinks they should stay.
> > > > > >> > > > > > >
> > > > > >> > > > > > > 12) Addressed in the previous email. I've updated
> the
> > > KIP
> > > > to
> > > > > >> > > include
> > > > > >> > > > > > > tagged fields for the requests and responses. (More
> on
> > > > that
> > > > > >> > below)
> > > > > >> > > > > > >
> > > > > >> > > > > > > 13) I think part of the idea for including this
> > > > information
> > > > > >> is to
> > > > > >> > > > > prepare
> > > > > >> > > > > > > for future changes. Perhaps the directory structure
> > > might
> > > > > >> change
> > > > > >> > > from
> > > > > >> > > > > > > topicName_partitionNumber to something like
> > > > > >> > > topicID_partitionNumber.
> > > > > >> > > > > Then
> > > > > >> > > > > > > it would be useful to have the topic name in the
> file
> > > > since
> > > > > it
> > > > > >> > > would
> > > > > >> > > > > not
> > > > > >> > > > > > be
> > > > > >> > > > > > > in the directory structure. Supporting topic renames
> > > might
> > > > > be
> > > > > >> > > easier
> > > > > >> > > > if
> > > > > >> > > > > > the
> > > > > >> > > > > > > other fields are included. Would there be any
> > downsides
> > > to
> > > > > >> > > including
> > > > > >> > > > > this
> > > > > >> > > > > > > information?
> > > > > >> > > > > > >
> > > > > >> > > > > > > 14)  Yes, we would need to copy the partition
> metadata
> > > > file
> > > > > in
> > > > > >> > this
> > > > > >> > > > > > > process. I've updated the KIP to include this.
> > > > > >> > > > > > >
> > > > > >> > > > > > > 15) I believe Lucas meant v1 and v2 here. He was
> > > referring
> > > > > to
> > > > > >> how
> > > > > >> > > the
> > > > > >> > > > > > > requests would fall under different IBP and meant
> that
> > > > older
> > > > > >> > > brokers
> > > > > >> > > > > > would
> > > > > >> > > > > > > have to use the older version of the request and the
> > > > > existing
> > > > > >> > topic
> > > > > >> > > > > > > deletion process. At first, it seemed like tagged
> > fields
> > > > > would
> > > > > >> > > > resolve
> > > > > >> > > > > > > the IBP issue. However, we may need IBP for this
> > request
> > > > > after
> > > > > >> > all
> > > > > >> > > > > since
> > > > > >> > > > > > > the controller handles the topic deletion
> differently
> > > > > >> depending
> > > > > >> > on
> > > > > >> > > > the
> > > > > >> > > > > > IBP
> > > > > >> > > > > > > version. In an older version, we can't just send a
> > > > > StopReplica
> > > > > >> > > delete
> > > > > >> > > > > the
> > > > > >> > > > > > > topic immediately like we'd want to for this KIP.
> > > > > >> > > > > > >
> > > > > >> > > > > > > This makes me wonder if we want tagged fields on all
> > the
> > > > > >> requests
> > > > > >> > > > after
> > > > > >> > > > > > > all. Let me know your thoughts!
> > > > > >> > > > > > >
> > > > > >> > > > > > > Justine
> > > > > >> > > > > > >
> > > > > >> > > > > > > On Tue, Sep 15, 2020 at 1:03 PM Justine Olshan <
> > > > > >> > > jolshan@confluent.io
> > > > > >> > > > >
> > > > > >> > > > > > > wrote:
> > > > > >> > > > > > >
> > > > > >> > > > > > >> Hi all,
> > > > > >> > > > > > >> Jun brought up a good point in his last email about
> > > > tagged
> > > > > >> > fields,
> > > > > >> > > > and
> > > > > >> > > > > > >> I've updated the KIP to reflect that the changes to
> > > > > requests
> > > > > >> and
> > > > > >> > > > > > responses
> > > > > >> > > > > > >> will be in the form of tagged fields to avoid
> > changing
> > > > IBP.
> > > > > >> > > > > > >>
> > > > > >> > > > > > >> Jun: I plan on sending a followup email to address
> > some
> > > > of
> > > > > >> the
> > > > > >> > > other
> > > > > >> > > > > > >> points.
> > > > > >> > > > > > >>
> > > > > >> > > > > > >> Thanks,
> > > > > >> > > > > > >> Justine
> > > > > >> > > > > > >>
> > > > > >> > > > > > >> On Mon, Sep 14, 2020 at 4:25 PM Jun Rao <
> > > > jun@confluent.io>
> > > > > >> > wrote:
> > > > > >> > > > > > >>
> > > > > >> > > > > > >>> Hi, Justine,
> > > > > >> > > > > > >>>
> > > > > >> > > > > > >>> Thanks for the updated KIP. A few comments below.
> > > > > >> > > > > > >>>
> > > > > >> > > > > > >>> 10. LeaderAndIsr Response: Do we need the topic
> > name?
> > > > > >> > > > > > >>>
> > > > > >> > > > > > >>> 11. For the changed request/response, other than
> > > > > >> LeaderAndIsr,
> > > > > >> > > > > > >>> UpdateMetadata, Metadata, do we need to include
> the
> > > > topic
> > > > > >> name?
> > > > > >> > > > > > >>>
> > > > > >> > > > > > >>> 12. It seems that upgrades don't require IBP. Does
> > > that
> > > > > mean
> > > > > >> > the
> > > > > >> > > > new
> > > > > >> > > > > > >>> fields
> > > > > >> > > > > > >>> in all the request/response are added as tagged
> > fields
> > > > > >> without
> > > > > >> > > > > bumping
> > > > > >> > > > > > up
> > > > > >> > > > > > >>> the request version? It would be useful to make
> that
> > > > > clear.
> > > > > >> > > > > > >>>
> > > > > >> > > > > > >>> 13. Partition Metadata file: Do we need to include
> > the
> > > > > topic
> > > > > >> > name
> > > > > >> > > > and
> > > > > >> > > > > > the
> > > > > >> > > > > > >>> partition id since they are implied in the
> directory
> > > > name?
> > > > > >> > > > > > >>>
> > > > > >> > > > > > >>> 14. In the JBOD mode, we support moving a
> > partition's
> > > > data
> > > > > >> from
> > > > > >> > > one
> > > > > >> > > > > > disk
> > > > > >> > > > > > >>> to
> > > > > >> > > > > > >>> another. Will the new partition metadata file be
> > > copied
> > > > > >> during
> > > > > >> > > that
> > > > > >> > > > > > >>> process?
> > > > > >> > > > > > >>>
> > > > > >> > > > > > >>> 15. The KIP says "Remove deleted topics from
> > replicas
> > > by
> > > > > >> > sending
> > > > > >> > > > > > >>> StopReplicaRequest V2 for any topics which do not
> > > > contain
> > > > > a
> > > > > >> > topic
> > > > > >> > > > ID,
> > > > > >> > > > > > and
> > > > > >> > > > > > >>> V3 for any topics which do contain a topic ID.".
> > > > However,
> > > > > it
> > > > > >> > > seems
> > > > > >> > > > > the
> > > > > >> > > > > > >>> updated controller will create all missing topic
> IDs
> > > > first
> > > > > >> > before
> > > > > >> > > > > doing
> > > > > >> > > > > > >>> other actions. So, is StopReplicaRequest V2
> needed?
> > > > > >> > > > > > >>>
> > > > > >> > > > > > >>> Jun
> > > > > >> > > > > > >>>
> > > > > >> > > > > > >>> On Fri, Sep 11, 2020 at 10:31 AM John Roesler <
> > > > > >> > > vvcephei@apache.org
> > > > > >> > > > >
> > > > > >> > > > > > >>> wrote:
> > > > > >> > > > > > >>>
> > > > > >> > > > > > >>> > Thanks, Justine!
> > > > > >> > > > > > >>> >
> > > > > >> > > > > > >>> > Your response seems compelling to me.
> > > > > >> > > > > > >>> >
> > > > > >> > > > > > >>> > -John
> > > > > >> > > > > > >>> >
> > > > > >> > > > > > >>> > On Fri, 2020-09-11 at 10:17 -0700, Justine
> Olshan
> > > > wrote:
> > > > > >> > > > > > >>> > > Hello all,
> > > > > >> > > > > > >>> > > Thanks for continuing the discussion! I have a
> > few
> > > > > >> > responses
> > > > > >> > > to
> > > > > >> > > > > > your
> > > > > >> > > > > > >>> > points.
> > > > > >> > > > > > >>> > >
> > > > > >> > > > > > >>> > > Tom: You are correct in that this KIP has not
> > > > > mentioned
> > > > > >> the
> > > > > >> > > > > > >>> > > DeleteTopicsRequest. I think that this would
> be
> > > out
> > > > of
> > > > > >> > scope
> > > > > >> > > > for
> > > > > >> > > > > > >>> now, but
> > > > > >> > > > > > >>> > > may be something worth adding in the future.
> > > > > >> > > > > > >>> > >
> > > > > >> > > > > > >>> > > John: We did consider sequence ids, but there
> > are
> > > a
> > > > > few
> > > > > >> > > reasons
> > > > > >> > > > > to
> > > > > >> > > > > > >>> favor
> > > > > >> > > > > > >>> > > UUIDs. There are several cases where topics
> from
> > > > > >> different
> > > > > >> > > > > clusters
> > > > > >> > > > > > >>> may
> > > > > >> > > > > > >>> > > interact now and in the future. For example,
> > > Mirror
> > > > > >> Maker 2
> > > > > >> > > may
> > > > > >> > > > > > >>> benefit
> > > > > >> > > > > > >>> > > from being able to detect when a cluster being
> > > > > mirrored
> > > > > >> is
> > > > > >> > > > > deleted
> > > > > >> > > > > > >>> and
> > > > > >> > > > > > >>> > > recreated and globally unique identifiers
> would
> > > make
> > > > > >> > > resolving
> > > > > >> > > > > > issues
> > > > > >> > > > > > >>> > > easier than sequence IDs which may collide
> > between
> > > > > >> > clusters.
> > > > > >> > > > > > KIP-405
> > > > > >> > > > > > >>> > > (tiered storage) will also benefit from
> globally
> > > > > unique
> > > > > >> IDs
> > > > > >> > > as
> > > > > >> > > > > > shared
> > > > > >> > > > > > >>> > > buckets may be used between clusters.
> > > > > >> > > > > > >>> > >
> > > > > >> > > > > > >>> > > Globally unique IDs would also make
> > functionality
> > > > like
> > > > > >> > moving
> > > > > >> > > > > > topics
> > > > > >> > > > > > >>> > > between disparate clusters easier in the
> future,
> > > > > >> simplify
> > > > > >> > any
> > > > > >> > > > > > future
> > > > > >> > > > > > >>> > > implementations of backups and restores, and
> > more.
> > > > In
> > > > > >> > > general,
> > > > > >> > > > > > >>> unique IDs
> > > > > >> > > > > > >>> > > would ensure that the source cluster topics do
> > not
> > > > > >> conflict
> > > > > >> > > > with
> > > > > >> > > > > > the
> > > > > >> > > > > > >>> > > destination cluster topics.
> > > > > >> > > > > > >>> > >
> > > > > >> > > > > > >>> > > If we were to use sequence ids, we would need
> > > > > >> sufficiently
> > > > > >> > > > large
> > > > > >> > > > > > >>> cluster
> > > > > >> > > > > > >>> > > ids to be stored with the topic identifiers or
> > we
> > > > run
> > > > > >> the
> > > > > >> > > risk
> > > > > >> > > > of
> > > > > >> > > > > > >>> > > collisions. This will give up any advantage in
> > > > > >> compactness
> > > > > >> > > that
> > > > > >> > > > > > >>> sequence
> > > > > >> > > > > > >>> > > numbers may bring. Given these advantages I
> > think
> > > it
> > > > > >> makes
> > > > > >> > > > sense
> > > > > >> > > > > to
> > > > > >> > > > > > >>> use
> > > > > >> > > > > > >>> > > UUIDs.
> > > > > >> > > > > > >>> > >
> > > > > >> > > > > > >>> > > Gokul: This is an interesting idea, but this
> is
> > a
> > > > > >> breaking
> > > > > >> > > > > change.
> > > > > >> > > > > > >>> Out of
> > > > > >> > > > > > >>> > > scope for now, but maybe worth discussing in
> the
> > > > > future.
> > > > > >> > > > > > >>> > >
> > > > > >> > > > > > >>> > > Hope this explains some of the decisions,
> > > > > >> > > > > > >>> > >
> > > > > >> > > > > > >>> > > Justine
> > > > > >> > > > > > >>> > >
> > > > > >> > > > > > >>> > >
> > > > > >> > > > > > >>> > >
> > > > > >> > > > > > >>> > > On Fri, Sep 11, 2020 at 8:27 AM Gokul Ramanan
> > > > > >> Subramanian <
> > > > > >> > > > > > >>> > > gokul2411s@gmail.com> wrote:
> > > > > >> > > > > > >>> > >
> > > > > >> > > > > > >>> > > > Hi.
> > > > > >> > > > > > >>> > > >
> > > > > >> > > > > > >>> > > > Thanks for the KIP.
> > > > > >> > > > > > >>> > > >
> > > > > >> > > > > > >>> > > > Have you thought about whether it makes
> sense
> > to
> > > > > >> support
> > > > > >> > > > > > >>> authorizing a
> > > > > >> > > > > > >>> > > > principal for a topic ID rather than a topic
> > > name
> > > > to
> > > > > >> > > achieve
> > > > > >> > > > > > >>> tighter
> > > > > >> > > > > > >>> > > > security?
> > > > > >> > > > > > >>> > > >
> > > > > >> > > > > > >>> > > > Or is the topic ID fundamentally an internal
> > > > detail
> > > > > >> > similar
> > > > > >> > > > to
> > > > > >> > > > > > >>> epochs
> > > > > >> > > > > > >>> > used
> > > > > >> > > > > > >>> > > > in a bunch of other places in Kafka?
> > > > > >> > > > > > >>> > > >
> > > > > >> > > > > > >>> > > > Thanks.
> > > > > >> > > > > > >>> > > >
> > > > > >> > > > > > >>> > > > On Fri, Sep 11, 2020 at 4:06 PM John
> Roesler <
> > > > > >> > > > > > vvcephei@apache.org>
> > > > > >> > > > > > >>> > wrote:
> > > > > >> > > > > > >>> > > >
> > > > > >> > > > > > >>> > > > > Hello Justine,
> > > > > >> > > > > > >>> > > > >
> > > > > >> > > > > > >>> > > > > Thanks for the KIP!
> > > > > >> > > > > > >>> > > > >
> > > > > >> > > > > > >>> > > > > I happen to have been confronted recently
> > with
> > > > the
> > > > > >> need
> > > > > >> > > to
> > > > > >> > > > > keep
> > > > > >> > > > > > >>> > track of
> > > > > >> > > > > > >>> > > > a
> > > > > >> > > > > > >>> > > > > large number of topics as compactly as
> > > > possible. I
> > > > > >> was
> > > > > >> > > > going
> > > > > >> > > > > to
> > > > > >> > > > > > >>> come
> > > > > >> > > > > > >>> > up
> > > > > >> > > > > > >>> > > > > with some way to dictionary encode the
> topic
> > > > names
> > > > > >> as
> > > > > >> > > > > integers,
> > > > > >> > > > > > >>> but
> > > > > >> > > > > > >>> > this
> > > > > >> > > > > > >>> > > > > seems much better!
> > > > > >> > > > > > >>> > > > >
> > > > > >> > > > > > >>> > > > > Apologies if this has been raised before,
> > but
> > > > I’m
> > > > > >> > > wondering
> > > > > >> > > > > > >>> about the
> > > > > >> > > > > > >>> > > > > choice of UUID vs sequence number for the
> > ids.
> > > > > >> > Typically,
> > > > > >> > > > > I’ve
> > > > > >> > > > > > >>> seen
> > > > > >> > > > > > >>> > UUIDs
> > > > > >> > > > > > >>> > > > > in two situations:
> > > > > >> > > > > > >>> > > > > 1. When processes need to generate
> > > non-colliding
> > > > > >> > > > identifiers
> > > > > >> > > > > > >>> without
> > > > > >> > > > > > >>> > > > > coordination.
> > > > > >> > > > > > >>> > > > > 2. When the identifier needs to be
> > > “universally
> > > > > >> > unique”;
> > > > > >> > > > > I.e.,
> > > > > >> > > > > > >>> the
> > > > > >> > > > > > >>> > > > > identifier needs to distinguish the entity
> > > from
> > > > > all
> > > > > >> > other
> > > > > >> > > > > > >>> entities
> > > > > >> > > > > > >>> > that
> > > > > >> > > > > > >>> > > > > could ever exist. This is useful in cases
> > > where
> > > > > >> > entities
> > > > > >> > > > from
> > > > > >> > > > > > all
> > > > > >> > > > > > >>> > kinds
> > > > > >> > > > > > >>> > > > of
> > > > > >> > > > > > >>> > > > > systems get mixed together, such as when
> > > dumping
> > > > > >> logs
> > > > > >> > > from
> > > > > >> > > > > all
> > > > > >> > > > > > >>> > processes
> > > > > >> > > > > > >>> > > > in
> > > > > >> > > > > > >>> > > > > a company into a common system.
> > > > > >> > > > > > >>> > > > >
> > > > > >> > > > > > >>> > > > > Maybe I’m being short-sighted, but it
> > doesn’t
> > > > seem
> > > > > >> like
> > > > > >> > > > > either
> > > > > >> > > > > > >>> really
> > > > > >> > > > > > >>> > > > > applies here. It seems like the brokers
> > could
> > > > and
> > > > > >> would
> > > > > >> > > > > achieve
> > > > > >> > > > > > >>> > consensus
> > > > > >> > > > > > >>> > > > > when creating a topic anyway, which is all
> > > > that’s
> > > > > >> > > required
> > > > > >> > > > to
> > > > > >> > > > > > >>> > generate
> > > > > >> > > > > > >>> > > > > non-colliding sequence ids. For the
> second,
> > as
> > > > you
> > > > > >> > > mention,
> > > > > >> > > > > we
> > > > > >> > > > > > >>> could
> > > > > >> > > > > > >>> > > > assign
> > > > > >> > > > > > >>> > > > > a UUID to the cluster as a whole, which
> > would
> > > > > render
> > > > > >> > any
> > > > > >> > > > > > resource
> > > > > >> > > > > > >>> > scoped
> > > > > >> > > > > > >>> > > > to
> > > > > >> > > > > > >>> > > > > the broker universally unique as well.
> > > > > >> > > > > > >>> > > > >
> > > > > >> > > > > > >>> > > > > The reason I mention this is that,
> although
> > a
> > > > UUID
> > > > > >> is
> > > > > >> > way
> > > > > >> > > > > more
> > > > > >> > > > > > >>> > compact
> > > > > >> > > > > > >>> > > > > than topic names, it’s still 16 bytes. In
> > > > > contrast,
> > > > > >> a
> > > > > >> > > > 4-byte
> > > > > >> > > > > > >>> integer
> > > > > >> > > > > > >>> > > > > sequence id would give us 4 billion unique
> > > > topics
> > > > > >> per
> > > > > >> > > > > cluster,
> > > > > >> > > > > > >>> which
> > > > > >> > > > > > >>> > > > seems
> > > > > >> > > > > > >>> > > > > like enough ;)
> > > > > >> > > > > > >>> > > > >
> > > > > >> > > > > > >>> > > > > Considering the number of different times
> > > these
> > > > > >> topic
> > > > > >> > > > > > >>> identifiers are
> > > > > >> > > > > > >>> > > > sent
> > > > > >> > > > > > >>> > > > > over the wire or stored in memory, it
> seems
> > > like
> > > > > it
> > > > > >> > might
> > > > > >> > > > be
> > > > > >> > > > > > >>> worth
> > > > > >> > > > > > >>> > the
> > > > > >> > > > > > >>> > > > > additional 4x space savings.
> > > > > >> > > > > > >>> > > > >
> > > > > >> > > > > > >>> > > > > What do you think about this?
> > > > > >> > > > > > >>> > > > >
> > > > > >> > > > > > >>> > > > > Thanks,
> > > > > >> > > > > > >>> > > > > John
> > > > > >> > > > > > >>> > > > >
> > > > > >> > > > > > >>> > > > > On Fri, Sep 11, 2020, at 03:20, Tom
> Bentley
> > > > wrote:
> > > > > >> > > > > > >>> > > > > > Hi Justine,
> > > > > >> > > > > > >>> > > > > >
> > > > > >> > > > > > >>> > > > > > This looks like a very welcome
> > improvement.
> > > > > >> Thanks!
> > > > > >> > > > > > >>> > > > > >
> > > > > >> > > > > > >>> > > > > > Maybe I missed it, but the KIP doesn't
> > seem
> > > to
> > > > > >> > mention
> > > > > >> > > > > > changing
> > > > > >> > > > > > >>> > > > > > DeleteTopicsRequest to identify the
> topic
> > > > using
> > > > > an
> > > > > >> > id.
> > > > > >> > > > > Maybe
> > > > > >> > > > > > >>> > that's out
> > > > > >> > > > > > >>> > > > > of
> > > > > >> > > > > > >>> > > > > > scope, but DeleteTopicsRequest is not
> > listed
> > > > > among
> > > > > >> > the
> > > > > >> > > > > Future
> > > > > >> > > > > > >>> Work
> > > > > >> > > > > > >>> > APIs
> > > > > >> > > > > > >>> > > > > > either.
> > > > > >> > > > > > >>> > > > > >
> > > > > >> > > > > > >>> > > > > > Kind regards,
> > > > > >> > > > > > >>> > > > > >
> > > > > >> > > > > > >>> > > > > > Tom
> > > > > >> > > > > > >>> > > > > >
> > > > > >> > > > > > >>> > > > > > On Thu, Sep 10, 2020 at 3:59 PM Satish
> > > > Duggana <
> > > > > >> > > > > > >>> > > > satish.duggana@gmail.com
> > > > > >> > > > > > >>> > > > > > wrote:
> > > > > >> > > > > > >>> > > > > >
> > > > > >> > > > > > >>> > > > > > > Thanks Lucas/Justine for the nice KIP.
> > > > > >> > > > > > >>> > > > > > >
> > > > > >> > > > > > >>> > > > > > > It has several benefits which also
> > include
> > > > > >> > > simplifying
> > > > > >> > > > > the
> > > > > >> > > > > > >>> topic
> > > > > >> > > > > > >>> > > > > > > deletion process by controller and
> logs
> > > > > cleanup
> > > > > >> by
> > > > > >> > > > > brokers
> > > > > >> > > > > > in
> > > > > >> > > > > > >>> > corner
> > > > > >> > > > > > >>> > > > > > > cases.
> > > > > >> > > > > > >>> > > > > > >
> > > > > >> > > > > > >>> > > > > > > Best,
> > > > > >> > > > > > >>> > > > > > > Satish.
> > > > > >> > > > > > >>> > > > > > >
> > > > > >> > > > > > >>> > > > > > > On Wed, Sep 9, 2020 at 10:07 PM
> Justine
> > > > > Olshan <
> > > > > >> > > > > > >>> > jolshan@confluent.io
> > > > > >> > > > > > >>> > > > > > > wrote:
> > > > > >> > > > > > >>> > > > > > > > Hello all, it's been almost a year!
> > I've
> > > > > made
> > > > > >> > some
> > > > > >> > > > > > changes
> > > > > >> > > > > > >>> to
> > > > > >> > > > > > >>> > this
> > > > > >> > > > > > >>> > > > > KIP
> > > > > >> > > > > > >>> > > > > > > and hope to continue the discussion.
> > > > > >> > > > > > >>> > > > > > > > One of the main changes I've added
> is
> > > now
> > > > > the
> > > > > >> > > > metadata
> > > > > >> > > > > > >>> response
> > > > > >> > > > > > >>> > > > will
> > > > > >> > > > > > >>> > > > > > > include the topic ID (as Colin
> > suggested).
> > > > > >> Clients
> > > > > >> > > can
> > > > > >> > > > > > >>> obtain the
> > > > > >> > > > > > >>> > > > > topicID
> > > > > >> > > > > > >>> > > > > > > of a given topic through a
> > > TopicDescription.
> > > > > The
> > > > > >> > > > topicId
> > > > > >> > > > > > will
> > > > > >> > > > > > >>> > also be
> > > > > >> > > > > > >>> > > > > > > included with the UpdateMetadata
> > request.
> > > > > >> > > > > > >>> > > > > > > > Let me know what you all think.
> > > > > >> > > > > > >>> > > > > > > > Thank you,
> > > > > >> > > > > > >>> > > > > > > > Justine
> > > > > >> > > > > > >>> > > > > > > >
> > > > > >> > > > > > >>> > > > > > > > On 2019/09/13 16:38:26, "Colin
> > McCabe" <
> > > > > >> > > > > > cmccabe@apache.org
> > > > > >> > > > > > >>> >
> > > > > >> > > > > > >>> > wrote:
> > > > > >> > > > > > >>> > > > > > > > > Hi Lucas,
> > > > > >> > > > > > >>> > > > > > > > >
> > > > > >> > > > > > >>> > > > > > > > > Thanks for tackling this.  Topic
> IDs
> > > > are a
> > > > > >> > great
> > > > > >> > > > > idea,
> > > > > >> > > > > > >>> and
> > > > > >> > > > > > >>> > this
> > > > > >> > > > > > >>> > > > is
> > > > > >> > > > > > >>> > > > > a
> > > > > >> > > > > > >>> > > > > > > really good writeup.
> > > > > >> > > > > > >>> > > > > > > > > For /brokers/topics/[topic], the
> > > schema
> > > > > >> version
> > > > > >> > > > > should
> > > > > >> > > > > > be
> > > > > >> > > > > > >>> > bumped
> > > > > >> > > > > > >>> > > > to
> > > > > >> > > > > > >>> > > > > > > version 3, rather than 2.  KIP-455
> > bumped
> > > > the
> > > > > >> > version
> > > > > >> > > > of
> > > > > >> > > > > > this
> > > > > >> > > > > > >>> > znode
> > > > > >> > > > > > >>> > > > to
> > > > > >> > > > > > >>> > > > > 2
> > > > > >> > > > > > >>> > > > > > > already :)
> > > > > >> > > > > > >>> > > > > > > > > Given that we're going to be
> seeing
> > > > these
> > > > > >> > things
> > > > > >> > > as
> > > > > >> > > > > > >>> strings
> > > > > >> > > > > > >>> > as
> > > > > >> > > > > > >>> > > > lot
> > > > > >> > > > > > >>> > > > > (in
> > > > > >> > > > > > >>> > > > > > > logs, in ZooKeeper, on the
> command-line,
> > > > > etc.),
> > > > > >> > does
> > > > > >> > > it
> > > > > >> > > > > > make
> > > > > >> > > > > > >>> > sense to
> > > > > >> > > > > > >>> > > > > use
> > > > > >> > > > > > >>> > > > > > > base64 when converting them to
> strings?
> > > > > >> > > > > > >>> > > > > > > > > Here is an example of the hex
> > > > > >> representation:
> > > > > >> > > > > > >>> > > > > > > > >
> 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> > > > > >> > > > > > >>> > > > > > > > >
> > > > > >> > > > > > >>> > > > > > > > > And here is an example in base64.
> > > > > >> > > > > > >>> > > > > > > > > b8tRS7h4TJ2Vt43Dp85v2A
> > > > > >> > > > > > >>> > > > > > > > >
> > > > > >> > > > > > >>> > > > > > > > > The base64 version saves 15
> letters
> > > (to
> > > > be
> > > > > >> > fair,
> > > > > >> > > 4
> > > > > >> > > > of
> > > > > >> > > > > > >>> those
> > > > > >> > > > > > >>> > were
> > > > > >> > > > > > >>> > > > > > > dashes that we could have elided in
> the
> > > hex
> > > > > >> > > > > > representation.)
> > > > > >> > > > > > >>> > > > > > > > > Another thing to consider is that
> we
> > > > > should
> > > > > >> > > specify
> > > > > >> > > > > > that
> > > > > >> > > > > > >>> the
> > > > > >> > > > > > >>> > > > > > > all-zeroes UUID is not a valid topic
> > UUID.
> > > > >  We
> > > > > >> > can't
> > > > > >> > > > use
> > > > > >> > > > > > >>> null
> > > > > >> > > > > > >>> > for
> > > > > >> > > > > > >>> > > > this
> > > > > >> > > > > > >>> > > > > > > because we can't pass a null UUID over
> > the
> > > > RPC
> > > > > >> > > protocol
> > > > > >> > > > > > >>> (there
> > > > > >> > > > > > >>> > is no
> > > > > >> > > > > > >>> > > > > > > special pattern for null, nor do we
> want
> > > to
> > > > > >> waste
> > > > > >> > > space
> > > > > >> > > > > > >>> reserving
> > > > > >> > > > > > >>> > > > such
> > > > > >> > > > > > >>> > > > > a
> > > > > >> > > > > > >>> > > > > > > pattern.)
> > > > > >> > > > > > >>> > > > > > > > > Maybe I missed it, but did you
> > > describe
> > > > > >> > > "migration
> > > > > >> > > > > > of...
> > > > > >> > > > > > >>> > existing
> > > > > >> > > > > > >>> > > > > > > topic[s] without topic IDs" in detail
> in
> > > any
> > > > > >> > section?
> > > > > >> > > > It
> > > > > >> > > > > > >>> seems
> > > > > >> > > > > > >>> > like
> > > > > >> > > > > > >>> > > > > when
> > > > > >> > > > > > >>> > > > > > > the new controller becomes active, it
> > > should
> > > > > >> just
> > > > > >> > > > > generate
> > > > > >> > > > > > >>> random
> > > > > >> > > > > > >>> > > > > UUIDs for
> > > > > >> > > > > > >>> > > > > > > these, and write the random UUIDs back
> > to
> > > > > >> > ZooKeeper.
> > > > > >> > > > It
> > > > > >> > > > > > >>> would be
> > > > > >> > > > > > >>> > > > good
> > > > > >> > > > > > >>> > > > > to
> > > > > >> > > > > > >>> > > > > > > spell that out.  We should make it
> clear
> > > > that
> > > > > >> this
> > > > > >> > > > > happens
> > > > > >> > > > > > >>> > regardless
> > > > > >> > > > > > >>> > > > > of
> > > > > >> > > > > > >>> > > > > > > the inter-broker protocol version
> (it's
> > a
> > > > > >> > compatible
> > > > > >> > > > > > change).
> > > > > >> > > > > > >>> > > > > > > > > "LeaderAndIsrRequests including an
> > > > > >> > > > is_every_partition
> > > > > >> > > > > > >>> flag"
> > > > > >> > > > > > >>> > > > seems a
> > > > > >> > > > > > >>> > > > > > > bit wordy.  Can we just call these
> "full
> > > > > >> > > > > > >>> LeaderAndIsrRequests"?
> > > > > >> > > > > > >>> > Then
> > > > > >> > > > > > >>> > > > > the
> > > > > >> > > > > > >>> > > > > > > RPC field could be named "full".
> Also,
> > it
> > > > > would
> > > > > >> > > > probably
> > > > > >> > > > > > be
> > > > > >> > > > > > >>> > better
> > > > > >> > > > > > >>> > > > > for the
> > > > > >> > > > > > >>> > > > > > > RPC field to be an enum of {
> > UNSPECIFIED,
> > > > > >> > > INCREMENTAL,
> > > > > >> > > > > FULL
> > > > > >> > > > > > >>> }, so
> > > > > >> > > > > > >>> > > > that
> > > > > >> > > > > > >>> > > > > we
> > > > > >> > > > > > >>> > > > > > > can cleanly handle old versions (by
> > > treating
> > > > > >> them
> > > > > >> > as
> > > > > >> > > > > > >>> UNSPECIFIED)
> > > > > >> > > > > > >>> > > > > > > > > In the LeaderAndIsrRequest
> section,
> > > you
> > > > > >> write
> > > > > >> > "A
> > > > > >> > > > > final
> > > > > >> > > > > > >>> > deletion
> > > > > >> > > > > > >>> > > > > event
> > > > > >> > > > > > >>> > > > > > > will be secheduled for X ms after the
> > > > > >> > > > LeaderAndIsrRequest
> > > > > >> > > > > > was
> > > > > >> > > > > > >>> > first
> > > > > >> > > > > > >>> > > > > > > received..."  I guess the X was a
> > > > placeholder
> > > > > >> that
> > > > > >> > > you
> > > > > >> > > > > > >>> intended
> > > > > >> > > > > > >>> > to
> > > > > >> > > > > > >>> > > > > replace
> > > > > >> > > > > > >>> > > > > > > before posting? :)  In any case, this
> > > seems
> > > > > like
> > > > > >> > the
> > > > > >> > > > kind
> > > > > >> > > > > > of
> > > > > >> > > > > > >>> > thing
> > > > > >> > > > > > >>> > > > we'd
> > > > > >> > > > > > >>> > > > > > > want a configuration for.  Let's
> > describe
> > > > that
> > > > > >> > > > > > configuration
> > > > > >> > > > > > >>> key
> > > > > >> > > > > > >>> > > > > somewhere
> > > > > >> > > > > > >>> > > > > > > in this KIP, including what its
> default
> > > > value
> > > > > >> is.
> > > > > >> > > > > > >>> > > > > > > > > We should probably also log a
> bunch
> > of
> > > > > >> messages
> > > > > >> > > at
> > > > > >> > > > > WARN
> > > > > >> > > > > > >>> level
> > > > > >> > > > > > >>> > > > when
> > > > > >> > > > > > >>> > > > > > > something is scheduled for deletion,
> as
> > > > well.
> > > > > >> > (Maybe
> > > > > >> > > > > this
> > > > > >> > > > > > >>> was
> > > > > >> > > > > > >>> > > > > assumed, but
> > > > > >> > > > > > >>> > > > > > > it would be good to mention it).
> > > > > >> > > > > > >>> > > > > > > > > I feel like there are a few
> sections
> > > > that
> > > > > >> > should
> > > > > >> > > be
> > > > > >> > > > > > >>> moved to
> > > > > >> > > > > > >>> > > > > "rejected
> > > > > >> > > > > > >>> > > > > > > alternatives."  For example, in the
> > > > > DeleteTopics
> > > > > >> > > > section,
> > > > > >> > > > > > >>> since
> > > > > >> > > > > > >>> > we're
> > > > > >> > > > > > >>> > > > > not
> > > > > >> > > > > > >>> > > > > > > going to do option 1 or 2, these
> should
> > be
> > > > > moved
> > > > > >> > into
> > > > > >> > > > > > >>> "rejected
> > > > > >> > > > > > >>> > > > > > > alternatives,"  rather than appearing
> > > > inline.
> > > > > >> > > Another
> > > > > >> > > > > case
> > > > > >> > > > > > >>> is
> > > > > >> > > > > > >>> > the
> > > > > >> > > > > > >>> > > > > "Should
> > > > > >> > > > > > >>> > > > > > > we remove topic name from the protocol
> > > where
> > > > > >> > > possible"
> > > > > >> > > > > > >>> section.
> > > > > >> > > > > > >>> > This
> > > > > >> > > > > > >>> > > > > is
> > > > > >> > > > > > >>> > > > > > > clearly discussing a design
> alternative
> > > that
> > > > > >> we're
> > > > > >> > > not
> > > > > >> > > > > > >>> proposing
> > > > > >> > > > > > >>> > to
> > > > > >> > > > > > >>> > > > > > > implement: removing the topic name
> from
> > > > those
> > > > > >> > > > protocols.
> > > > > >> > > > > > >>> > > > > > > > > Is it really necessary to have a
> new
> > > > > >> > > > > > >>> > /admin/delete_topics_by_id
> > > > > >> > > > > > >>> > > > > path
> > > > > >> > > > > > >>> > > > > > > in ZooKeeper?  It seems like we don't
> > > really
> > > > > >> need
> > > > > >> > > this.
> > > > > >> > > > > > >>> Whenever
> > > > > >> > > > > > >>> > > > > there is
> > > > > >> > > > > > >>> > > > > > > a new controller, we'll send out full
> > > > > >> > > > > LeaderAndIsrRequests
> > > > > >> > > > > > >>> which
> > > > > >> > > > > > >>> > will
> > > > > >> > > > > > >>> > > > > > > trigger the stale topics to be cleaned
> > up.
> > > > >  The
> > > > > >> > > active
> > > > > >> > > > > > >>> > controller
> > > > > >> > > > > > >>> > > > will
> > > > > >> > > > > > >>> > > > > > > also send the full LeaderAndIsrRequest
> > to
> > > > > >> brokers
> > > > > >> > > that
> > > > > >> > > > > are
> > > > > >> > > > > > >>> just
> > > > > >> > > > > > >>> > > > > starting
> > > > > >> > > > > > >>> > > > > > > up.    So we don't really need this
> kind
> > > of
> > > > > >> > two-phase
> > > > > >> > > > > > commit
> > > > > >> > > > > > >>> > (send
> > > > > >> > > > > > >>> > > > out
> > > > > >> > > > > > >>> > > > > > > StopReplicasRequest, get ACKs from all
> > > > nodes,
> > > > > >> > commit
> > > > > >> > > by
> > > > > >> > > > > > >>> removing
> > > > > >> > > > > > >>> > > > > > > /admin/delete_topics node) any more.
> > > > > >> > > > > > >>> > > > > > > > > You mention that FetchRequest will
> > now
> > > > > >> include
> > > > > >> > > UUID
> > > > > >> > > > > to
> > > > > >> > > > > > >>> avoid
> > > > > >> > > > > > >>> > > > issues
> > > > > >> > > > > > >>> > > > > > > where requests are made to stale
> > > partitions.
> > > > > >> > > However,
> > > > > >> > > > > > >>> adding a
> > > > > >> > > > > > >>> > UUID
> > > > > >> > > > > > >>> > > > to
> > > > > >> > > > > > >>> > > > > > > MetadataRequest is listed as future
> > work,
> > > > out
> > > > > of
> > > > > >> > > scope
> > > > > >> > > > > for
> > > > > >> > > > > > >>> this
> > > > > >> > > > > > >>> > KIP.
> > > > > >> > > > > > >>> > > > > How
> > > > > >> > > > > > >>> > > > > > > will the client learn what the topic
> > UUID
> > > > is,
> > > > > if
> > > > > >> > the
> > > > > >> > > > > > metadata
> > > > > >> > > > > > >>> > > > response
> > > > > >> > > > > > >>> > > > > > > doesn't include that information?  It
> > > seems
> > > > > like
> > > > > >> > > adding
> > > > > >> > > > > the
> > > > > >> > > > > > >>> UUID
> > > > > >> > > > > > >>> > to
> > > > > >> > > > > > >>> > > > > > > MetadataResponse would be an
> improvement
> > > > here
> > > > > >> that
> > > > > >> > > > might
> > > > > >> > > > > > not
> > > > > >> > > > > > >>> be
> > > > > >> > > > > > >>> > too
> > > > > >> > > > > > >>> > > > > hard to
> > > > > >> > > > > > >>> > > > > > > make.
> > > > > >> > > > > > >>> > > > > > > > > best,
> > > > > >> > > > > > >>> > > > > > > > > Colin
> > > > > >> > > > > > >>> > > > > > > > >
> > > > > >> > > > > > >>> > > > > > > > >
> > > > > >> > > > > > >>> > > > > > > > > On Mon, Sep 9, 2019, at 17:48,
> > Ryanne
> > > > > Dolan
> > > > > >> > > wrote:
> > > > > >> > > > > > >>> > > > > > > > > > Lucas, this would be great. I've
> > run
> > > > > into
> > > > > >> > > issues
> > > > > >> > > > > with
> > > > > >> > > > > > >>> > topics
> > > > > >> > > > > > >>> > > > > being
> > > > > >> > > > > > >>> > > > > > > > > > resurrected accidentally, since
> a
> > > > client
> > > > > >> > cannot
> > > > > >> > > > > > easily
> > > > > >> > > > > > >>> > > > > distinguish
> > > > > >> > > > > > >>> > > > > > > between
> > > > > >> > > > > > >>> > > > > > > > > > a deleted topic and a new topic
> > with
> > > > the
> > > > > >> same
> > > > > >> > > > name.
> > > > > >> > > > > > I'd
> > > > > >> > > > > > >>> > need
> > > > > >> > > > > > >>> > > > the
> > > > > >> > > > > > >>> > > > > ID
> > > > > >> > > > > > >>> > > > > > > > > > accessible from the client to
> > solve
> > > > that
> > > > > >> > issue,
> > > > > >> > > > but
> > > > > >> > > > > > >>> this
> > > > > >> > > > > > >>> > is a
> > > > > >> > > > > > >>> > > > > good
> > > > > >> > > > > > >>> > > > > > > first
> > > > > >> > > > > > >>> > > > > > > > > > step.
> > > > > >> > > > > > >>> > > > > > > > > >
> > > > > >> > > > > > >>> > > > > > > > > > Ryanne
> > > > > >> > > > > > >>> > > > > > > > > >
> > > > > >> > > > > > >>> > > > > > > > > > On Wed, Sep 4, 2019 at 1:41 PM
> > Lucas
> > > > > >> > > Bradstreet <
> > > > > >> > > > > > >>> > > > > lucas@confluent.io>
> > > > > >> > > > > > >>> > > > > > > wrote:
> > > > > >> > > > > > >>> > > > > > > > > > > Hi all,
> > > > > >> > > > > > >>> > > > > > > > > > >
> > > > > >> > > > > > >>> > > > > > > > > > > I would like to kick off
> > > discussion
> > > > of
> > > > > >> > > KIP-516,
> > > > > >> > > > > an
> > > > > >> > > > > > >>> > > > > implementation
> > > > > >> > > > > > >>> > > > > > > of topic
> > > > > >> > > > > > >>> > > > > > > > > > > IDs for Kafka. Topic IDs aim
> to
> > > > solve
> > > > > >> topic
> > > > > >> > > > > > >>> uniqueness
> > > > > >> > > > > > >>> > > > > problems in
> > > > > >> > > > > > >>> > > > > > > Kafka,
> > > > > >> > > > > > >>> > > > > > > > > > > where referring to a topic by
> > name
> > > > > >> alone is
> > > > > >> > > > > > >>> insufficient.
> > > > > >> > > > > > >>> > > > Such
> > > > > >> > > > > > >>> > > > > > > cases
> > > > > >> > > > > > >>> > > > > > > > > > > include when a topic has been
> > > > deleted
> > > > > >> and
> > > > > >> > > > > recreated
> > > > > >> > > > > > >>> with
> > > > > >> > > > > > >>> > the
> > > > > >> > > > > > >>> > > > > same
> > > > > >> > > > > > >>> > > > > > > name.
> > > > > >> > > > > > >>> > > > > > > > > > > Unique identifiers will help
> > > > simplify
> > > > > >> and
> > > > > >> > > > improve
> > > > > >> > > > > > >>> Kafka's
> > > > > >> > > > > > >>> > > > topic
> > > > > >> > > > > > >>> > > > > > > deletion
> > > > > >> > > > > > >>> > > > > > > > > > > process, as well as prevent
> > cases
> > > > > where
> > > > > >> > > brokers
> > > > > >> > > > > may
> > > > > >> > > > > > >>> > > > incorrectly
> > > > > >> > > > > > >>> > > > > > > interact
> > > > > >> > > > > > >>> > > > > > > > > > > with stale versions of topics.
> > > > > >> > > > > > >>> > > > > > > > > > >
> > > > > >> > > > > > >>> > > > > > > > > > >
> > > > > >> > > > > > >>> > > > > > > > > > >
> > > > > >> > > > > > >>> > > >
> > > > > >> > > > > > >>> >
> > > > > >> > > > > > >>>
> > > > > >> > > > > >
> > > > > >> > > > >
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > > > > >> > > > > > >>> > > > > > > > > > > Looking forward to your
> > thoughts.
> > > > > >> > > > > > >>> > > > > > > > > > >
> > > > > >> > > > > > >>> > > > > > > > > > > Lucas
> > > > > >> > > > > > >>> > > > > > > > > > >
> > > > > >> > > > > > >>> >
> > > > > >> > > > > > >>> >
> > > > > >> > > > > > >>>
> > > > > >> > > > > > >>
> > > > > >> > > > > >
> > > > > >> > > > >
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Justine Olshan <jo...@confluent.io>]

Hi Jun,

Thanks for the comments. I apologize for some of the typos and confusion.
I’ve updated the KIP to fix some of the issues you mentioned.

20.2 I’ve changed the type to String.
20.1/3 I’ve updated the TopicZNode to fix formatting and reflect the latest
version before this change.

21. You are correct and I’ve removed this line. I’ve also added a line
mentioning an IBP bump is necessary for migration

22. I think the wording was unclear but your summary is what was intended
by this line. I’ve updated to make this point more clear. “Remove deleted
topics from replicas by sending StopReplicaRequest V3 before the IBP bump
using the old logic, and using V4 and the new logic with topic IDs after
the IBP bump.”

23. I’ve removed the unspecified type from the KIP and mention that IBP
will be used to handle this request. “IBP will be used to determine whether
this new form of the request will be used. For older requests, we will
ignore this field and default to previous behavior.”

24. I’ve fixed this typo.

25. I've added a topics at a higher level for LeaderAndIsrResponse v5,
StopReplicaResponse v4. I've also changed StopReplicaRequest v4 to have
topics at a higher level.

26. I’ve updated forgotten_topics_data--added the topic ID and removed the
topic name

27. I’ve decided on plain text, and I’ve added an example.

28. I’ve added this idea to future work.

Thanks again for taking a look,

Justine

On Wed, Sep 23, 2020 at 10:28 AM Jun Rao <ju...@confluent.io> wrote:

> Hi, Justine,
>
> Thanks for the response. Made another pass. A few more comments below.
>
> 20. znode schema:
> 20.1 It seems that {"name": "version", "type": "int", "id": "UUID", "doc":
> "version id"} should be {"name": "version", "type": "int"}, {"name": "id",
> "type": "UUID", "doc": "version id"}.
> 20.2 The znode format is JSON which doesn't have UUID type. So the type
> probably should be string?
> 20.3 Also, the existing format used seems outdated. It should have the
> following format.
>     Json.encodeAsBytes(Map(
>       "version" -> 2,
>       "partitions" -> replicaAssignmentJson.asJava,
>       "adding_replicas" -> addingReplicasAssignmentJson.asJava,
>       "removing_replicas" -> removingReplicasAssignmentJson.asJava
>     ).asJava)
>   }
>
> 21. Migration: The KIP says "The migration process can take place without
> an inter-broker protocol bump, as the format stored in
> /brokers/topics/[topic] will be compatible with older broker versions."
> However, since we are bumping up the version of inter-broker requests, it
> seems that we need to use IBP for migration.
>
> 22. The KIP says "Remove deleted topics from replicas by sending
> StopReplicaRequest V3 for any topics which do not contain a topic ID, and
> V4 for any topics which do contain a topic ID.". However, if we use IBP, it
> seems that the controller will either send StopReplicaRequest V3
> or StopReplicaRequest V4, but never mixed V3 and V4 for different topics.
> Basically, before the IBP bump, V3 will be used. After the IBP bump,
> topicId will be created and V4 will be used.
>
> 23. Given that we depend on IBP, do we still need "0 UNSPECIFIED"
> in LeaderAndIsr?
>
> 24. LeaderAndIsrResponse v5 : It still has the topic field.
>
> 25. LeaderAndIsrResponse v5, StopReplicaResponse v4: Could we use this
> opportunity to organize the response in 2 levels, first by topic, then by
> partition, as most other requests/responses?
>
> 26. FetchRequest v13 : Should forgotten_topics_data use topicId too?
>
> 27. "This file can either be plain text (key/value pairs) or JSON." Have we
> decided which one to use? Also, it would be helpful to provide an example.
>
> 28. Future improvement: Another future improvement opportunity is to use
> topicId in GroupMetadataManager.offsetCommitKey in the offset_commit topic.
> This may save some space.
>
> Thanks,
>
> Jun
>
> On Wed, Sep 23, 2020 at 8:50 AM Justine Olshan <jo...@confluent.io>
> wrote:
>
> > Hi Tom,
> >
> > Thanks for the comment. I think this is a really good idea and it has
> been
> > added to the KIP under the newly added tooling section.
> >
> > Thanks again,
> > Justine
> >
> > On Wed, Sep 23, 2020 at 3:17 AM Tom Bentley <tb...@redhat.com> wrote:
> >
> > > Hi Justine,
> > >
> > > I know you started the vote thread, but on re-reading the KIP I noticed
> > > that although the topic id is included in the MetadataResponse it's not
> > > surfaced in the output from `kafka-topics.sh --describe`. Maybe that
> was
> > > intentional because ids are intentionally not really something the user
> > > should care deeply about, but it would also make life harder for anyone
> > > debugging Kafka and this would likely get worse the more topic ids got
> > > rolled out across the protocols, clients etc. It seems likely that
> > > `kafka-topics.sh` will eventually need the ability to show the id of a
> > > topic and perhaps find a topic name given an id. Is there any reason
> not
> > to
> > > implement that in this KIP?
> > >
> > > Many thanks,
> > >
> > > Tom
> > >
> > > On Mon, Sep 21, 2020 at 9:54 PM Justine Olshan <jo...@confluent.io>
> > > wrote:
> > >
> > > > Hi all,
> > > >
> > > > After thinking about it, I've decided to remove the topic name from
> the
> > > > Fetch Request and Response after all. Since there are so many of
> these
> > > > requests per second, it is worth removing the extra information. I've
> > > > updated the KIP to reflect this change.
> > > >
> > > > Please let me know if there is anything else we should discuss before
> > > > voting.
> > > >
> > > > Thank you,
> > > > Justine
> > > >
> > > > On Fri, Sep 18, 2020 at 9:46 AM Justine Olshan <jolshan@confluent.io
> >
> > > > wrote:
> > > >
> > > > > Hi Jun,
> > > > >
> > > > > I see what you are saying. For now we can remove the extra
> > information.
> > > > > I'll leave the option to add more fields to the file in the future.
> > The
> > > > KIP
> > > > > has been updated to reflect this change.
> > > > >
> > > > > Thanks,
> > > > > Justine
> > > > >
> > > > > On Fri, Sep 18, 2020 at 8:46 AM Jun Rao <ju...@confluent.io> wrote:
> > > > >
> > > > >> Hi, Justine,
> > > > >>
> > > > >> Thanks for the reply.
> > > > >>
> > > > >> 13. If the log directory is the source of truth, it means that the
> > > > >> redundant info in the metadata file will be ignored. Then the
> > question
> > > > is
> > > > >> why do we need to put the redundant info in the metadata file now?
> > > > >>
> > > > >> Thanks,
> > > > >>
> > > > >> Jun
> > > > >>
> > > > >> On Thu, Sep 17, 2020 at 5:07 PM Justine Olshan <
> > jolshan@confluent.io>
> > > > >> wrote:
> > > > >>
> > > > >> > Hi Jun,
> > > > >> > Thanks for the quick response!
> > > > >> >
> > > > >> > 12. I've decided to bump up the versions on the requests and
> > updated
> > > > the
> > > > >> > KIP. I think it's good we thoroughly discussed the options here,
> > so
> > > we
> > > > >> know
> > > > >> > we made a good choice. :)
> > > > >> >
> > > > >> > 13. This is an interesting situation. I think if this does occur
> > we
> > > > >> should
> > > > >> > give a warning. I agree that it's hard to know the source of
> truth
> > > for
> > > > >> sure
> > > > >> > since the directory or the file could be manually modified. I
> > guess
> > > > the
> > > > >> > directory could be used as the source of truth. To be honest,
> I'm
> > > not
> > > > >> > really sure what happens in kafka when the log directory is
> > renamed
> > > > >> > manually in such a way. I'm also wondering if the situation is
> > > > >> recoverable
> > > > >> > in this scenario.
> > > > >> >
> > > > >> > Thanks,
> > > > >> > Justine
> > > > >> >
> > > > >> > On Thu, Sep 17, 2020 at 4:28 PM Jun Rao <ju...@confluent.io>
> wrote:
> > > > >> >
> > > > >> > > Hi, Justine,
> > > > >> > >
> > > > >> > > Thanks for the reply.
> > > > >> > >
> > > > >> > > 12. I don't have a strong preference either. However, if we
> need
> > > IBP
> > > > >> > > anyway, maybe it's easier to just bump up the version for all
> > > inter
> > > > >> > broker
> > > > >> > > requests and add the topic id field as a regular field. A
> > regular
> > > > >> field
> > > > >> > is
> > > > >> > > a bit more concise in wire transfer than a flexible field.
> > > > >> > >
> > > > >> > > 13. The confusion that I was referring to is between the topic
> > > name
> > > > >> and
> > > > >> > > partition number between the log dir and the metadata file.
> For
> > > > >> example,
> > > > >> > if
> > > > >> > > the log dir is topicA-1 and the metadata file in it has topicB
> > and
> > > > >> > > partition 0 (say due to a bug or manual modification), which
> one
> > > do
> > > > we
> > > > >> > use
> > > > >> > > as the source of truth?
> > > > >> > >
> > > > >> > > Jun
> > > > >> > >
> > > > >> > > On Thu, Sep 17, 2020 at 3:43 PM Justine Olshan <
> > > > jolshan@confluent.io>
> > > > >> > > wrote:
> > > > >> > >
> > > > >> > > > Hi Jun,
> > > > >> > > > Thanks for the comments.
> > > > >> > > >
> > > > >> > > > 12. I bumped the LeaderAndIsrRequest because I removed the
> > topic
> > > > >> name
> > > > >> > > field
> > > > >> > > > in the response. It may be possible to avoid bumping the
> > version
> > > > >> > without
> > > > >> > > > that change, but I may be missing something.
> > > > >> > > > I believe StopReplica is actually on version 3 now, but
> > because
> > > > >> > version 2
> > > > >> > > > is flexible, I kept that listed as version 2 on the KIP
> page.
> > > > >> However,
> > > > >> > > you
> > > > >> > > > may be right in that we may need to bump the version on
> > > > StopReplica
> > > > >> to
> > > > >> > > deal
> > > > >> > > > with deletion differently as mentioned above. I don't know
> if
> > I
> > > > >> have a
> > > > >> > > big
> > > > >> > > > preference over used tagged fields or not.
> > > > >> > > >
> > > > >> > > > 13. I was thinking that in the case where the file and the
> > > request
> > > > >> > topic
> > > > >> > > > ids don't match, it means that the broker's topic/the one in
> > the
> > > > >> file
> > > > >> > has
> > > > >> > > > been deleted. In that case, we would need to delete the old
> > > topic
> > > > >> and
> > > > >> > > start
> > > > >> > > > receiving the new version. If the topic name were to change,
> > but
> > > > the
> > > > >> > ids
> > > > >> > > > still matched, the file would also need to update. Am I
> > missing
> > > a
> > > > >> case
> > > > >> > > > where the file would be correct and not the request?
> > > > >> > > >
> > > > >> > > > Thanks,
> > > > >> > > > Justine
> > > > >> > > >
> > > > >> > > > On Thu, Sep 17, 2020 at 3:18 PM Jun Rao <ju...@confluent.io>
> > > wrote:
> > > > >> > > >
> > > > >> > > > > Hi, Justine,
> > > > >> > > > >
> > > > >> > > > > Thanks for the reply. A couple of more comments below.
> > > > >> > > > >
> > > > >> > > > > 12. ListOffset and OffsetForLeader currently don't support
> > > > >> flexible
> > > > >> > > > fields.
> > > > >> > > > > So, we have to bump up the version number and use IBP at
> > least
> > > > for
> > > > >> > > these
> > > > >> > > > > two requests. Note that it seems 2.7.0 will require IBP
> > anyway
> > > > >> > because
> > > > >> > > of
> > > > >> > > > > changes in KAFKA-10435. Also, it seems that the version
> for
> > > > >> > > > > LeaderAndIsrRequest and StopReplica are bumped even though
> > we
> > > > only
> > > > >> > > added
> > > > >> > > > a
> > > > >> > > > > tagged field. But since IBP is needed anyway, we may want
> to
> > > > >> revisit
> > > > >> > > the
> > > > >> > > > > overall tagged field choice.
> > > > >> > > > >
> > > > >> > > > > 13. The only downside is the potential confusion on which
> > one
> > > is
> > > > >> the
> > > > >> > > > source
> > > > >> > > > > of truth if they don't match. Another option is to include
> > > those
> > > > >> > fields
> > > > >> > > > in
> > > > >> > > > > the metadata file when we actually change the directory
> > > > structure.
> > > > >> > > > >
> > > > >> > > > > Thanks,
> > > > >> > > > >
> > > > >> > > > > Jun
> > > > >> > > > >
> > > > >> > > > > On Thu, Sep 17, 2020 at 2:01 PM Justine Olshan <
> > > > >> jolshan@confluent.io
> > > > >> > >
> > > > >> > > > > wrote:
> > > > >> > > > >
> > > > >> > > > > > Hello all,
> > > > >> > > > > >
> > > > >> > > > > > I've thought some more about removing the topic name
> field
> > > > from
> > > > >> > some
> > > > >> > > of
> > > > >> > > > > the
> > > > >> > > > > > requests. On closer inspection of the
> requests/responses,
> > it
> > > > >> seems
> > > > >> > > that
> > > > >> > > > > the
> > > > >> > > > > > internal changes would be much larger than I expected.
> > Some
> > > > >> > protocols
> > > > >> > > > > > involve clients, so they would require changes too. I'm
> > > > thinking
> > > > >> > that
> > > > >> > > > for
> > > > >> > > > > > now, removing the topic name from these requests and
> > > responses
> > > > >> are
> > > > >> > > out
> > > > >> > > > of
> > > > >> > > > > > scope.
> > > > >> > > > > >
> > > > >> > > > > > I have decided to just keep the change
> > LeaderAndIsrResponse
> > > to
> > > > >> > remove
> > > > >> > > > the
> > > > >> > > > > > topic name, and have updated the KIP to reflect this
> > > change. I
> > > > >> have
> > > > >> > > > also
> > > > >> > > > > > mentioned the other requests and responses in future
> work.
> > > > >> > > > > >
> > > > >> > > > > > I'm hoping to start the voting process soon, so let me
> > know
> > > if
> > > > >> > there
> > > > >> > > is
> > > > >> > > > > > anything else we should discuss.
> > > > >> > > > > >
> > > > >> > > > > > Thank you,
> > > > >> > > > > > Justine
> > > > >> > > > > >
> > > > >> > > > > > On Tue, Sep 15, 2020 at 3:57 PM Justine Olshan <
> > > > >> > jolshan@confluent.io
> > > > >> > > >
> > > > >> > > > > > wrote:
> > > > >> > > > > >
> > > > >> > > > > > > Hello again,
> > > > >> > > > > > > To follow up on some of the other comments:
> > > > >> > > > > > >
> > > > >> > > > > > > 10/11) We can remove the topic name from these
> > > > >> > requests/responses,
> > > > >> > > > and
> > > > >> > > > > > > that means we will just have to make a few internal
> > > changes
> > > > to
> > > > >> > make
> > > > >> > > > > > > partitions accessible by topic id and partition. I can
> > > > update
> > > > >> the
> > > > >> > > KIP
> > > > >> > > > > to
> > > > >> > > > > > > remove them unless anyone thinks they should stay.
> > > > >> > > > > > >
> > > > >> > > > > > > 12) Addressed in the previous email. I've updated the
> > KIP
> > > to
> > > > >> > > include
> > > > >> > > > > > > tagged fields for the requests and responses. (More on
> > > that
> > > > >> > below)
> > > > >> > > > > > >
> > > > >> > > > > > > 13) I think part of the idea for including this
> > > information
> > > > >> is to
> > > > >> > > > > prepare
> > > > >> > > > > > > for future changes. Perhaps the directory structure
> > might
> > > > >> change
> > > > >> > > from
> > > > >> > > > > > > topicName_partitionNumber to something like
> > > > >> > > topicID_partitionNumber.
> > > > >> > > > > Then
> > > > >> > > > > > > it would be useful to have the topic name in the file
> > > since
> > > > it
> > > > >> > > would
> > > > >> > > > > not
> > > > >> > > > > > be
> > > > >> > > > > > > in the directory structure. Supporting topic renames
> > might
> > > > be
> > > > >> > > easier
> > > > >> > > > if
> > > > >> > > > > > the
> > > > >> > > > > > > other fields are included. Would there be any
> downsides
> > to
> > > > >> > > including
> > > > >> > > > > this
> > > > >> > > > > > > information?
> > > > >> > > > > > >
> > > > >> > > > > > > 14)  Yes, we would need to copy the partition metadata
> > > file
> > > > in
> > > > >> > this
> > > > >> > > > > > > process. I've updated the KIP to include this.
> > > > >> > > > > > >
> > > > >> > > > > > > 15) I believe Lucas meant v1 and v2 here. He was
> > referring
> > > > to
> > > > >> how
> > > > >> > > the
> > > > >> > > > > > > requests would fall under different IBP and meant that
> > > older
> > > > >> > > brokers
> > > > >> > > > > > would
> > > > >> > > > > > > have to use the older version of the request and the
> > > > existing
> > > > >> > topic
> > > > >> > > > > > > deletion process. At first, it seemed like tagged
> fields
> > > > would
> > > > >> > > > resolve
> > > > >> > > > > > > the IBP issue. However, we may need IBP for this
> request
> > > > after
> > > > >> > all
> > > > >> > > > > since
> > > > >> > > > > > > the controller handles the topic deletion differently
> > > > >> depending
> > > > >> > on
> > > > >> > > > the
> > > > >> > > > > > IBP
> > > > >> > > > > > > version. In an older version, we can't just send a
> > > > StopReplica
> > > > >> > > delete
> > > > >> > > > > the
> > > > >> > > > > > > topic immediately like we'd want to for this KIP.
> > > > >> > > > > > >
> > > > >> > > > > > > This makes me wonder if we want tagged fields on all
> the
> > > > >> requests
> > > > >> > > > after
> > > > >> > > > > > > all. Let me know your thoughts!
> > > > >> > > > > > >
> > > > >> > > > > > > Justine
> > > > >> > > > > > >
> > > > >> > > > > > > On Tue, Sep 15, 2020 at 1:03 PM Justine Olshan <
> > > > >> > > jolshan@confluent.io
> > > > >> > > > >
> > > > >> > > > > > > wrote:
> > > > >> > > > > > >
> > > > >> > > > > > >> Hi all,
> > > > >> > > > > > >> Jun brought up a good point in his last email about
> > > tagged
> > > > >> > fields,
> > > > >> > > > and
> > > > >> > > > > > >> I've updated the KIP to reflect that the changes to
> > > > requests
> > > > >> and
> > > > >> > > > > > responses
> > > > >> > > > > > >> will be in the form of tagged fields to avoid
> changing
> > > IBP.
> > > > >> > > > > > >>
> > > > >> > > > > > >> Jun: I plan on sending a followup email to address
> some
> > > of
> > > > >> the
> > > > >> > > other
> > > > >> > > > > > >> points.
> > > > >> > > > > > >>
> > > > >> > > > > > >> Thanks,
> > > > >> > > > > > >> Justine
> > > > >> > > > > > >>
> > > > >> > > > > > >> On Mon, Sep 14, 2020 at 4:25 PM Jun Rao <
> > > jun@confluent.io>
> > > > >> > wrote:
> > > > >> > > > > > >>
> > > > >> > > > > > >>> Hi, Justine,
> > > > >> > > > > > >>>
> > > > >> > > > > > >>> Thanks for the updated KIP. A few comments below.
> > > > >> > > > > > >>>
> > > > >> > > > > > >>> 10. LeaderAndIsr Response: Do we need the topic
> name?
> > > > >> > > > > > >>>
> > > > >> > > > > > >>> 11. For the changed request/response, other than
> > > > >> LeaderAndIsr,
> > > > >> > > > > > >>> UpdateMetadata, Metadata, do we need to include the
> > > topic
> > > > >> name?
> > > > >> > > > > > >>>
> > > > >> > > > > > >>> 12. It seems that upgrades don't require IBP. Does
> > that
> > > > mean
> > > > >> > the
> > > > >> > > > new
> > > > >> > > > > > >>> fields
> > > > >> > > > > > >>> in all the request/response are added as tagged
> fields
> > > > >> without
> > > > >> > > > > bumping
> > > > >> > > > > > up
> > > > >> > > > > > >>> the request version? It would be useful to make that
> > > > clear.
> > > > >> > > > > > >>>
> > > > >> > > > > > >>> 13. Partition Metadata file: Do we need to include
> the
> > > > topic
> > > > >> > name
> > > > >> > > > and
> > > > >> > > > > > the
> > > > >> > > > > > >>> partition id since they are implied in the directory
> > > name?
> > > > >> > > > > > >>>
> > > > >> > > > > > >>> 14. In the JBOD mode, we support moving a
> partition's
> > > data
> > > > >> from
> > > > >> > > one
> > > > >> > > > > > disk
> > > > >> > > > > > >>> to
> > > > >> > > > > > >>> another. Will the new partition metadata file be
> > copied
> > > > >> during
> > > > >> > > that
> > > > >> > > > > > >>> process?
> > > > >> > > > > > >>>
> > > > >> > > > > > >>> 15. The KIP says "Remove deleted topics from
> replicas
> > by
> > > > >> > sending
> > > > >> > > > > > >>> StopReplicaRequest V2 for any topics which do not
> > > contain
> > > > a
> > > > >> > topic
> > > > >> > > > ID,
> > > > >> > > > > > and
> > > > >> > > > > > >>> V3 for any topics which do contain a topic ID.".
> > > However,
> > > > it
> > > > >> > > seems
> > > > >> > > > > the
> > > > >> > > > > > >>> updated controller will create all missing topic IDs
> > > first
> > > > >> > before
> > > > >> > > > > doing
> > > > >> > > > > > >>> other actions. So, is StopReplicaRequest V2 needed?
> > > > >> > > > > > >>>
> > > > >> > > > > > >>> Jun
> > > > >> > > > > > >>>
> > > > >> > > > > > >>> On Fri, Sep 11, 2020 at 10:31 AM John Roesler <
> > > > >> > > vvcephei@apache.org
> > > > >> > > > >
> > > > >> > > > > > >>> wrote:
> > > > >> > > > > > >>>
> > > > >> > > > > > >>> > Thanks, Justine!
> > > > >> > > > > > >>> >
> > > > >> > > > > > >>> > Your response seems compelling to me.
> > > > >> > > > > > >>> >
> > > > >> > > > > > >>> > -John
> > > > >> > > > > > >>> >
> > > > >> > > > > > >>> > On Fri, 2020-09-11 at 10:17 -0700, Justine Olshan
> > > wrote:
> > > > >> > > > > > >>> > > Hello all,
> > > > >> > > > > > >>> > > Thanks for continuing the discussion! I have a
> few
> > > > >> > responses
> > > > >> > > to
> > > > >> > > > > > your
> > > > >> > > > > > >>> > points.
> > > > >> > > > > > >>> > >
> > > > >> > > > > > >>> > > Tom: You are correct in that this KIP has not
> > > > mentioned
> > > > >> the
> > > > >> > > > > > >>> > > DeleteTopicsRequest. I think that this would be
> > out
> > > of
> > > > >> > scope
> > > > >> > > > for
> > > > >> > > > > > >>> now, but
> > > > >> > > > > > >>> > > may be something worth adding in the future.
> > > > >> > > > > > >>> > >
> > > > >> > > > > > >>> > > John: We did consider sequence ids, but there
> are
> > a
> > > > few
> > > > >> > > reasons
> > > > >> > > > > to
> > > > >> > > > > > >>> favor
> > > > >> > > > > > >>> > > UUIDs. There are several cases where topics from
> > > > >> different
> > > > >> > > > > clusters
> > > > >> > > > > > >>> may
> > > > >> > > > > > >>> > > interact now and in the future. For example,
> > Mirror
> > > > >> Maker 2
> > > > >> > > may
> > > > >> > > > > > >>> benefit
> > > > >> > > > > > >>> > > from being able to detect when a cluster being
> > > > mirrored
> > > > >> is
> > > > >> > > > > deleted
> > > > >> > > > > > >>> and
> > > > >> > > > > > >>> > > recreated and globally unique identifiers would
> > make
> > > > >> > > resolving
> > > > >> > > > > > issues
> > > > >> > > > > > >>> > > easier than sequence IDs which may collide
> between
> > > > >> > clusters.
> > > > >> > > > > > KIP-405
> > > > >> > > > > > >>> > > (tiered storage) will also benefit from globally
> > > > unique
> > > > >> IDs
> > > > >> > > as
> > > > >> > > > > > shared
> > > > >> > > > > > >>> > > buckets may be used between clusters.
> > > > >> > > > > > >>> > >
> > > > >> > > > > > >>> > > Globally unique IDs would also make
> functionality
> > > like
> > > > >> > moving
> > > > >> > > > > > topics
> > > > >> > > > > > >>> > > between disparate clusters easier in the future,
> > > > >> simplify
> > > > >> > any
> > > > >> > > > > > future
> > > > >> > > > > > >>> > > implementations of backups and restores, and
> more.
> > > In
> > > > >> > > general,
> > > > >> > > > > > >>> unique IDs
> > > > >> > > > > > >>> > > would ensure that the source cluster topics do
> not
> > > > >> conflict
> > > > >> > > > with
> > > > >> > > > > > the
> > > > >> > > > > > >>> > > destination cluster topics.
> > > > >> > > > > > >>> > >
> > > > >> > > > > > >>> > > If we were to use sequence ids, we would need
> > > > >> sufficiently
> > > > >> > > > large
> > > > >> > > > > > >>> cluster
> > > > >> > > > > > >>> > > ids to be stored with the topic identifiers or
> we
> > > run
> > > > >> the
> > > > >> > > risk
> > > > >> > > > of
> > > > >> > > > > > >>> > > collisions. This will give up any advantage in
> > > > >> compactness
> > > > >> > > that
> > > > >> > > > > > >>> sequence
> > > > >> > > > > > >>> > > numbers may bring. Given these advantages I
> think
> > it
> > > > >> makes
> > > > >> > > > sense
> > > > >> > > > > to
> > > > >> > > > > > >>> use
> > > > >> > > > > > >>> > > UUIDs.
> > > > >> > > > > > >>> > >
> > > > >> > > > > > >>> > > Gokul: This is an interesting idea, but this is
> a
> > > > >> breaking
> > > > >> > > > > change.
> > > > >> > > > > > >>> Out of
> > > > >> > > > > > >>> > > scope for now, but maybe worth discussing in the
> > > > future.
> > > > >> > > > > > >>> > >
> > > > >> > > > > > >>> > > Hope this explains some of the decisions,
> > > > >> > > > > > >>> > >
> > > > >> > > > > > >>> > > Justine
> > > > >> > > > > > >>> > >
> > > > >> > > > > > >>> > >
> > > > >> > > > > > >>> > >
> > > > >> > > > > > >>> > > On Fri, Sep 11, 2020 at 8:27 AM Gokul Ramanan
> > > > >> Subramanian <
> > > > >> > > > > > >>> > > gokul2411s@gmail.com> wrote:
> > > > >> > > > > > >>> > >
> > > > >> > > > > > >>> > > > Hi.
> > > > >> > > > > > >>> > > >
> > > > >> > > > > > >>> > > > Thanks for the KIP.
> > > > >> > > > > > >>> > > >
> > > > >> > > > > > >>> > > > Have you thought about whether it makes sense
> to
> > > > >> support
> > > > >> > > > > > >>> authorizing a
> > > > >> > > > > > >>> > > > principal for a topic ID rather than a topic
> > name
> > > to
> > > > >> > > achieve
> > > > >> > > > > > >>> tighter
> > > > >> > > > > > >>> > > > security?
> > > > >> > > > > > >>> > > >
> > > > >> > > > > > >>> > > > Or is the topic ID fundamentally an internal
> > > detail
> > > > >> > similar
> > > > >> > > > to
> > > > >> > > > > > >>> epochs
> > > > >> > > > > > >>> > used
> > > > >> > > > > > >>> > > > in a bunch of other places in Kafka?
> > > > >> > > > > > >>> > > >
> > > > >> > > > > > >>> > > > Thanks.
> > > > >> > > > > > >>> > > >
> > > > >> > > > > > >>> > > > On Fri, Sep 11, 2020 at 4:06 PM John Roesler <
> > > > >> > > > > > vvcephei@apache.org>
> > > > >> > > > > > >>> > wrote:
> > > > >> > > > > > >>> > > >
> > > > >> > > > > > >>> > > > > Hello Justine,
> > > > >> > > > > > >>> > > > >
> > > > >> > > > > > >>> > > > > Thanks for the KIP!
> > > > >> > > > > > >>> > > > >
> > > > >> > > > > > >>> > > > > I happen to have been confronted recently
> with
> > > the
> > > > >> need
> > > > >> > > to
> > > > >> > > > > keep
> > > > >> > > > > > >>> > track of
> > > > >> > > > > > >>> > > > a
> > > > >> > > > > > >>> > > > > large number of topics as compactly as
> > > possible. I
> > > > >> was
> > > > >> > > > going
> > > > >> > > > > to
> > > > >> > > > > > >>> come
> > > > >> > > > > > >>> > up
> > > > >> > > > > > >>> > > > > with some way to dictionary encode the topic
> > > names
> > > > >> as
> > > > >> > > > > integers,
> > > > >> > > > > > >>> but
> > > > >> > > > > > >>> > this
> > > > >> > > > > > >>> > > > > seems much better!
> > > > >> > > > > > >>> > > > >
> > > > >> > > > > > >>> > > > > Apologies if this has been raised before,
> but
> > > I’m
> > > > >> > > wondering
> > > > >> > > > > > >>> about the
> > > > >> > > > > > >>> > > > > choice of UUID vs sequence number for the
> ids.
> > > > >> > Typically,
> > > > >> > > > > I’ve
> > > > >> > > > > > >>> seen
> > > > >> > > > > > >>> > UUIDs
> > > > >> > > > > > >>> > > > > in two situations:
> > > > >> > > > > > >>> > > > > 1. When processes need to generate
> > non-colliding
> > > > >> > > > identifiers
> > > > >> > > > > > >>> without
> > > > >> > > > > > >>> > > > > coordination.
> > > > >> > > > > > >>> > > > > 2. When the identifier needs to be
> > “universally
> > > > >> > unique”;
> > > > >> > > > > I.e.,
> > > > >> > > > > > >>> the
> > > > >> > > > > > >>> > > > > identifier needs to distinguish the entity
> > from
> > > > all
> > > > >> > other
> > > > >> > > > > > >>> entities
> > > > >> > > > > > >>> > that
> > > > >> > > > > > >>> > > > > could ever exist. This is useful in cases
> > where
> > > > >> > entities
> > > > >> > > > from
> > > > >> > > > > > all
> > > > >> > > > > > >>> > kinds
> > > > >> > > > > > >>> > > > of
> > > > >> > > > > > >>> > > > > systems get mixed together, such as when
> > dumping
> > > > >> logs
> > > > >> > > from
> > > > >> > > > > all
> > > > >> > > > > > >>> > processes
> > > > >> > > > > > >>> > > > in
> > > > >> > > > > > >>> > > > > a company into a common system.
> > > > >> > > > > > >>> > > > >
> > > > >> > > > > > >>> > > > > Maybe I’m being short-sighted, but it
> doesn’t
> > > seem
> > > > >> like
> > > > >> > > > > either
> > > > >> > > > > > >>> really
> > > > >> > > > > > >>> > > > > applies here. It seems like the brokers
> could
> > > and
> > > > >> would
> > > > >> > > > > achieve
> > > > >> > > > > > >>> > consensus
> > > > >> > > > > > >>> > > > > when creating a topic anyway, which is all
> > > that’s
> > > > >> > > required
> > > > >> > > > to
> > > > >> > > > > > >>> > generate
> > > > >> > > > > > >>> > > > > non-colliding sequence ids. For the second,
> as
> > > you
> > > > >> > > mention,
> > > > >> > > > > we
> > > > >> > > > > > >>> could
> > > > >> > > > > > >>> > > > assign
> > > > >> > > > > > >>> > > > > a UUID to the cluster as a whole, which
> would
> > > > render
> > > > >> > any
> > > > >> > > > > > resource
> > > > >> > > > > > >>> > scoped
> > > > >> > > > > > >>> > > > to
> > > > >> > > > > > >>> > > > > the broker universally unique as well.
> > > > >> > > > > > >>> > > > >
> > > > >> > > > > > >>> > > > > The reason I mention this is that, although
> a
> > > UUID
> > > > >> is
> > > > >> > way
> > > > >> > > > > more
> > > > >> > > > > > >>> > compact
> > > > >> > > > > > >>> > > > > than topic names, it’s still 16 bytes. In
> > > > contrast,
> > > > >> a
> > > > >> > > > 4-byte
> > > > >> > > > > > >>> integer
> > > > >> > > > > > >>> > > > > sequence id would give us 4 billion unique
> > > topics
> > > > >> per
> > > > >> > > > > cluster,
> > > > >> > > > > > >>> which
> > > > >> > > > > > >>> > > > seems
> > > > >> > > > > > >>> > > > > like enough ;)
> > > > >> > > > > > >>> > > > >
> > > > >> > > > > > >>> > > > > Considering the number of different times
> > these
> > > > >> topic
> > > > >> > > > > > >>> identifiers are
> > > > >> > > > > > >>> > > > sent
> > > > >> > > > > > >>> > > > > over the wire or stored in memory, it seems
> > like
> > > > it
> > > > >> > might
> > > > >> > > > be
> > > > >> > > > > > >>> worth
> > > > >> > > > > > >>> > the
> > > > >> > > > > > >>> > > > > additional 4x space savings.
> > > > >> > > > > > >>> > > > >
> > > > >> > > > > > >>> > > > > What do you think about this?
> > > > >> > > > > > >>> > > > >
> > > > >> > > > > > >>> > > > > Thanks,
> > > > >> > > > > > >>> > > > > John
> > > > >> > > > > > >>> > > > >
> > > > >> > > > > > >>> > > > > On Fri, Sep 11, 2020, at 03:20, Tom Bentley
> > > wrote:
> > > > >> > > > > > >>> > > > > > Hi Justine,
> > > > >> > > > > > >>> > > > > >
> > > > >> > > > > > >>> > > > > > This looks like a very welcome
> improvement.
> > > > >> Thanks!
> > > > >> > > > > > >>> > > > > >
> > > > >> > > > > > >>> > > > > > Maybe I missed it, but the KIP doesn't
> seem
> > to
> > > > >> > mention
> > > > >> > > > > > changing
> > > > >> > > > > > >>> > > > > > DeleteTopicsRequest to identify the topic
> > > using
> > > > an
> > > > >> > id.
> > > > >> > > > > Maybe
> > > > >> > > > > > >>> > that's out
> > > > >> > > > > > >>> > > > > of
> > > > >> > > > > > >>> > > > > > scope, but DeleteTopicsRequest is not
> listed
> > > > among
> > > > >> > the
> > > > >> > > > > Future
> > > > >> > > > > > >>> Work
> > > > >> > > > > > >>> > APIs
> > > > >> > > > > > >>> > > > > > either.
> > > > >> > > > > > >>> > > > > >
> > > > >> > > > > > >>> > > > > > Kind regards,
> > > > >> > > > > > >>> > > > > >
> > > > >> > > > > > >>> > > > > > Tom
> > > > >> > > > > > >>> > > > > >
> > > > >> > > > > > >>> > > > > > On Thu, Sep 10, 2020 at 3:59 PM Satish
> > > Duggana <
> > > > >> > > > > > >>> > > > satish.duggana@gmail.com
> > > > >> > > > > > >>> > > > > > wrote:
> > > > >> > > > > > >>> > > > > >
> > > > >> > > > > > >>> > > > > > > Thanks Lucas/Justine for the nice KIP.
> > > > >> > > > > > >>> > > > > > >
> > > > >> > > > > > >>> > > > > > > It has several benefits which also
> include
> > > > >> > > simplifying
> > > > >> > > > > the
> > > > >> > > > > > >>> topic
> > > > >> > > > > > >>> > > > > > > deletion process by controller and logs
> > > > cleanup
> > > > >> by
> > > > >> > > > > brokers
> > > > >> > > > > > in
> > > > >> > > > > > >>> > corner
> > > > >> > > > > > >>> > > > > > > cases.
> > > > >> > > > > > >>> > > > > > >
> > > > >> > > > > > >>> > > > > > > Best,
> > > > >> > > > > > >>> > > > > > > Satish.
> > > > >> > > > > > >>> > > > > > >
> > > > >> > > > > > >>> > > > > > > On Wed, Sep 9, 2020 at 10:07 PM Justine
> > > > Olshan <
> > > > >> > > > > > >>> > jolshan@confluent.io
> > > > >> > > > > > >>> > > > > > > wrote:
> > > > >> > > > > > >>> > > > > > > > Hello all, it's been almost a year!
> I've
> > > > made
> > > > >> > some
> > > > >> > > > > > changes
> > > > >> > > > > > >>> to
> > > > >> > > > > > >>> > this
> > > > >> > > > > > >>> > > > > KIP
> > > > >> > > > > > >>> > > > > > > and hope to continue the discussion.
> > > > >> > > > > > >>> > > > > > > > One of the main changes I've added is
> > now
> > > > the
> > > > >> > > > metadata
> > > > >> > > > > > >>> response
> > > > >> > > > > > >>> > > > will
> > > > >> > > > > > >>> > > > > > > include the topic ID (as Colin
> suggested).
> > > > >> Clients
> > > > >> > > can
> > > > >> > > > > > >>> obtain the
> > > > >> > > > > > >>> > > > > topicID
> > > > >> > > > > > >>> > > > > > > of a given topic through a
> > TopicDescription.
> > > > The
> > > > >> > > > topicId
> > > > >> > > > > > will
> > > > >> > > > > > >>> > also be
> > > > >> > > > > > >>> > > > > > > included with the UpdateMetadata
> request.
> > > > >> > > > > > >>> > > > > > > > Let me know what you all think.
> > > > >> > > > > > >>> > > > > > > > Thank you,
> > > > >> > > > > > >>> > > > > > > > Justine
> > > > >> > > > > > >>> > > > > > > >
> > > > >> > > > > > >>> > > > > > > > On 2019/09/13 16:38:26, "Colin
> McCabe" <
> > > > >> > > > > > cmccabe@apache.org
> > > > >> > > > > > >>> >
> > > > >> > > > > > >>> > wrote:
> > > > >> > > > > > >>> > > > > > > > > Hi Lucas,
> > > > >> > > > > > >>> > > > > > > > >
> > > > >> > > > > > >>> > > > > > > > > Thanks for tackling this.  Topic IDs
> > > are a
> > > > >> > great
> > > > >> > > > > idea,
> > > > >> > > > > > >>> and
> > > > >> > > > > > >>> > this
> > > > >> > > > > > >>> > > > is
> > > > >> > > > > > >>> > > > > a
> > > > >> > > > > > >>> > > > > > > really good writeup.
> > > > >> > > > > > >>> > > > > > > > > For /brokers/topics/[topic], the
> > schema
> > > > >> version
> > > > >> > > > > should
> > > > >> > > > > > be
> > > > >> > > > > > >>> > bumped
> > > > >> > > > > > >>> > > > to
> > > > >> > > > > > >>> > > > > > > version 3, rather than 2.  KIP-455
> bumped
> > > the
> > > > >> > version
> > > > >> > > > of
> > > > >> > > > > > this
> > > > >> > > > > > >>> > znode
> > > > >> > > > > > >>> > > > to
> > > > >> > > > > > >>> > > > > 2
> > > > >> > > > > > >>> > > > > > > already :)
> > > > >> > > > > > >>> > > > > > > > > Given that we're going to be seeing
> > > these
> > > > >> > things
> > > > >> > > as
> > > > >> > > > > > >>> strings
> > > > >> > > > > > >>> > as
> > > > >> > > > > > >>> > > > lot
> > > > >> > > > > > >>> > > > > (in
> > > > >> > > > > > >>> > > > > > > logs, in ZooKeeper, on the command-line,
> > > > etc.),
> > > > >> > does
> > > > >> > > it
> > > > >> > > > > > make
> > > > >> > > > > > >>> > sense to
> > > > >> > > > > > >>> > > > > use
> > > > >> > > > > > >>> > > > > > > base64 when converting them to strings?
> > > > >> > > > > > >>> > > > > > > > > Here is an example of the hex
> > > > >> representation:
> > > > >> > > > > > >>> > > > > > > > > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> > > > >> > > > > > >>> > > > > > > > >
> > > > >> > > > > > >>> > > > > > > > > And here is an example in base64.
> > > > >> > > > > > >>> > > > > > > > > b8tRS7h4TJ2Vt43Dp85v2A
> > > > >> > > > > > >>> > > > > > > > >
> > > > >> > > > > > >>> > > > > > > > > The base64 version saves 15 letters
> > (to
> > > be
> > > > >> > fair,
> > > > >> > > 4
> > > > >> > > > of
> > > > >> > > > > > >>> those
> > > > >> > > > > > >>> > were
> > > > >> > > > > > >>> > > > > > > dashes that we could have elided in the
> > hex
> > > > >> > > > > > representation.)
> > > > >> > > > > > >>> > > > > > > > > Another thing to consider is that we
> > > > should
> > > > >> > > specify
> > > > >> > > > > > that
> > > > >> > > > > > >>> the
> > > > >> > > > > > >>> > > > > > > all-zeroes UUID is not a valid topic
> UUID.
> > > >  We
> > > > >> > can't
> > > > >> > > > use
> > > > >> > > > > > >>> null
> > > > >> > > > > > >>> > for
> > > > >> > > > > > >>> > > > this
> > > > >> > > > > > >>> > > > > > > because we can't pass a null UUID over
> the
> > > RPC
> > > > >> > > protocol
> > > > >> > > > > > >>> (there
> > > > >> > > > > > >>> > is no
> > > > >> > > > > > >>> > > > > > > special pattern for null, nor do we want
> > to
> > > > >> waste
> > > > >> > > space
> > > > >> > > > > > >>> reserving
> > > > >> > > > > > >>> > > > such
> > > > >> > > > > > >>> > > > > a
> > > > >> > > > > > >>> > > > > > > pattern.)
> > > > >> > > > > > >>> > > > > > > > > Maybe I missed it, but did you
> > describe
> > > > >> > > "migration
> > > > >> > > > > > of...
> > > > >> > > > > > >>> > existing
> > > > >> > > > > > >>> > > > > > > topic[s] without topic IDs" in detail in
> > any
> > > > >> > section?
> > > > >> > > > It
> > > > >> > > > > > >>> seems
> > > > >> > > > > > >>> > like
> > > > >> > > > > > >>> > > > > when
> > > > >> > > > > > >>> > > > > > > the new controller becomes active, it
> > should
> > > > >> just
> > > > >> > > > > generate
> > > > >> > > > > > >>> random
> > > > >> > > > > > >>> > > > > UUIDs for
> > > > >> > > > > > >>> > > > > > > these, and write the random UUIDs back
> to
> > > > >> > ZooKeeper.
> > > > >> > > > It
> > > > >> > > > > > >>> would be
> > > > >> > > > > > >>> > > > good
> > > > >> > > > > > >>> > > > > to
> > > > >> > > > > > >>> > > > > > > spell that out.  We should make it clear
> > > that
> > > > >> this
> > > > >> > > > > happens
> > > > >> > > > > > >>> > regardless
> > > > >> > > > > > >>> > > > > of
> > > > >> > > > > > >>> > > > > > > the inter-broker protocol version (it's
> a
> > > > >> > compatible
> > > > >> > > > > > change).
> > > > >> > > > > > >>> > > > > > > > > "LeaderAndIsrRequests including an
> > > > >> > > > is_every_partition
> > > > >> > > > > > >>> flag"
> > > > >> > > > > > >>> > > > seems a
> > > > >> > > > > > >>> > > > > > > bit wordy.  Can we just call these "full
> > > > >> > > > > > >>> LeaderAndIsrRequests"?
> > > > >> > > > > > >>> > Then
> > > > >> > > > > > >>> > > > > the
> > > > >> > > > > > >>> > > > > > > RPC field could be named "full".  Also,
> it
> > > > would
> > > > >> > > > probably
> > > > >> > > > > > be
> > > > >> > > > > > >>> > better
> > > > >> > > > > > >>> > > > > for the
> > > > >> > > > > > >>> > > > > > > RPC field to be an enum of {
> UNSPECIFIED,
> > > > >> > > INCREMENTAL,
> > > > >> > > > > FULL
> > > > >> > > > > > >>> }, so
> > > > >> > > > > > >>> > > > that
> > > > >> > > > > > >>> > > > > we
> > > > >> > > > > > >>> > > > > > > can cleanly handle old versions (by
> > treating
> > > > >> them
> > > > >> > as
> > > > >> > > > > > >>> UNSPECIFIED)
> > > > >> > > > > > >>> > > > > > > > > In the LeaderAndIsrRequest section,
> > you
> > > > >> write
> > > > >> > "A
> > > > >> > > > > final
> > > > >> > > > > > >>> > deletion
> > > > >> > > > > > >>> > > > > event
> > > > >> > > > > > >>> > > > > > > will be secheduled for X ms after the
> > > > >> > > > LeaderAndIsrRequest
> > > > >> > > > > > was
> > > > >> > > > > > >>> > first
> > > > >> > > > > > >>> > > > > > > received..."  I guess the X was a
> > > placeholder
> > > > >> that
> > > > >> > > you
> > > > >> > > > > > >>> intended
> > > > >> > > > > > >>> > to
> > > > >> > > > > > >>> > > > > replace
> > > > >> > > > > > >>> > > > > > > before posting? :)  In any case, this
> > seems
> > > > like
> > > > >> > the
> > > > >> > > > kind
> > > > >> > > > > > of
> > > > >> > > > > > >>> > thing
> > > > >> > > > > > >>> > > > we'd
> > > > >> > > > > > >>> > > > > > > want a configuration for.  Let's
> describe
> > > that
> > > > >> > > > > > configuration
> > > > >> > > > > > >>> key
> > > > >> > > > > > >>> > > > > somewhere
> > > > >> > > > > > >>> > > > > > > in this KIP, including what its default
> > > value
> > > > >> is.
> > > > >> > > > > > >>> > > > > > > > > We should probably also log a bunch
> of
> > > > >> messages
> > > > >> > > at
> > > > >> > > > > WARN
> > > > >> > > > > > >>> level
> > > > >> > > > > > >>> > > > when
> > > > >> > > > > > >>> > > > > > > something is scheduled for deletion, as
> > > well.
> > > > >> > (Maybe
> > > > >> > > > > this
> > > > >> > > > > > >>> was
> > > > >> > > > > > >>> > > > > assumed, but
> > > > >> > > > > > >>> > > > > > > it would be good to mention it).
> > > > >> > > > > > >>> > > > > > > > > I feel like there are a few sections
> > > that
> > > > >> > should
> > > > >> > > be
> > > > >> > > > > > >>> moved to
> > > > >> > > > > > >>> > > > > "rejected
> > > > >> > > > > > >>> > > > > > > alternatives."  For example, in the
> > > > DeleteTopics
> > > > >> > > > section,
> > > > >> > > > > > >>> since
> > > > >> > > > > > >>> > we're
> > > > >> > > > > > >>> > > > > not
> > > > >> > > > > > >>> > > > > > > going to do option 1 or 2, these should
> be
> > > > moved
> > > > >> > into
> > > > >> > > > > > >>> "rejected
> > > > >> > > > > > >>> > > > > > > alternatives,"  rather than appearing
> > > inline.
> > > > >> > > Another
> > > > >> > > > > case
> > > > >> > > > > > >>> is
> > > > >> > > > > > >>> > the
> > > > >> > > > > > >>> > > > > "Should
> > > > >> > > > > > >>> > > > > > > we remove topic name from the protocol
> > where
> > > > >> > > possible"
> > > > >> > > > > > >>> section.
> > > > >> > > > > > >>> > This
> > > > >> > > > > > >>> > > > > is
> > > > >> > > > > > >>> > > > > > > clearly discussing a design alternative
> > that
> > > > >> we're
> > > > >> > > not
> > > > >> > > > > > >>> proposing
> > > > >> > > > > > >>> > to
> > > > >> > > > > > >>> > > > > > > implement: removing the topic name from
> > > those
> > > > >> > > > protocols.
> > > > >> > > > > > >>> > > > > > > > > Is it really necessary to have a new
> > > > >> > > > > > >>> > /admin/delete_topics_by_id
> > > > >> > > > > > >>> > > > > path
> > > > >> > > > > > >>> > > > > > > in ZooKeeper?  It seems like we don't
> > really
> > > > >> need
> > > > >> > > this.
> > > > >> > > > > > >>> Whenever
> > > > >> > > > > > >>> > > > > there is
> > > > >> > > > > > >>> > > > > > > a new controller, we'll send out full
> > > > >> > > > > LeaderAndIsrRequests
> > > > >> > > > > > >>> which
> > > > >> > > > > > >>> > will
> > > > >> > > > > > >>> > > > > > > trigger the stale topics to be cleaned
> up.
> > > >  The
> > > > >> > > active
> > > > >> > > > > > >>> > controller
> > > > >> > > > > > >>> > > > will
> > > > >> > > > > > >>> > > > > > > also send the full LeaderAndIsrRequest
> to
> > > > >> brokers
> > > > >> > > that
> > > > >> > > > > are
> > > > >> > > > > > >>> just
> > > > >> > > > > > >>> > > > > starting
> > > > >> > > > > > >>> > > > > > > up.    So we don't really need this kind
> > of
> > > > >> > two-phase
> > > > >> > > > > > commit
> > > > >> > > > > > >>> > (send
> > > > >> > > > > > >>> > > > out
> > > > >> > > > > > >>> > > > > > > StopReplicasRequest, get ACKs from all
> > > nodes,
> > > > >> > commit
> > > > >> > > by
> > > > >> > > > > > >>> removing
> > > > >> > > > > > >>> > > > > > > /admin/delete_topics node) any more.
> > > > >> > > > > > >>> > > > > > > > > You mention that FetchRequest will
> now
> > > > >> include
> > > > >> > > UUID
> > > > >> > > > > to
> > > > >> > > > > > >>> avoid
> > > > >> > > > > > >>> > > > issues
> > > > >> > > > > > >>> > > > > > > where requests are made to stale
> > partitions.
> > > > >> > > However,
> > > > >> > > > > > >>> adding a
> > > > >> > > > > > >>> > UUID
> > > > >> > > > > > >>> > > > to
> > > > >> > > > > > >>> > > > > > > MetadataRequest is listed as future
> work,
> > > out
> > > > of
> > > > >> > > scope
> > > > >> > > > > for
> > > > >> > > > > > >>> this
> > > > >> > > > > > >>> > KIP.
> > > > >> > > > > > >>> > > > > How
> > > > >> > > > > > >>> > > > > > > will the client learn what the topic
> UUID
> > > is,
> > > > if
> > > > >> > the
> > > > >> > > > > > metadata
> > > > >> > > > > > >>> > > > response
> > > > >> > > > > > >>> > > > > > > doesn't include that information?  It
> > seems
> > > > like
> > > > >> > > adding
> > > > >> > > > > the
> > > > >> > > > > > >>> UUID
> > > > >> > > > > > >>> > to
> > > > >> > > > > > >>> > > > > > > MetadataResponse would be an improvement
> > > here
> > > > >> that
> > > > >> > > > might
> > > > >> > > > > > not
> > > > >> > > > > > >>> be
> > > > >> > > > > > >>> > too
> > > > >> > > > > > >>> > > > > hard to
> > > > >> > > > > > >>> > > > > > > make.
> > > > >> > > > > > >>> > > > > > > > > best,
> > > > >> > > > > > >>> > > > > > > > > Colin
> > > > >> > > > > > >>> > > > > > > > >
> > > > >> > > > > > >>> > > > > > > > >
> > > > >> > > > > > >>> > > > > > > > > On Mon, Sep 9, 2019, at 17:48,
> Ryanne
> > > > Dolan
> > > > >> > > wrote:
> > > > >> > > > > > >>> > > > > > > > > > Lucas, this would be great. I've
> run
> > > > into
> > > > >> > > issues
> > > > >> > > > > with
> > > > >> > > > > > >>> > topics
> > > > >> > > > > > >>> > > > > being
> > > > >> > > > > > >>> > > > > > > > > > resurrected accidentally, since a
> > > client
> > > > >> > cannot
> > > > >> > > > > > easily
> > > > >> > > > > > >>> > > > > distinguish
> > > > >> > > > > > >>> > > > > > > between
> > > > >> > > > > > >>> > > > > > > > > > a deleted topic and a new topic
> with
> > > the
> > > > >> same
> > > > >> > > > name.
> > > > >> > > > > > I'd
> > > > >> > > > > > >>> > need
> > > > >> > > > > > >>> > > > the
> > > > >> > > > > > >>> > > > > ID
> > > > >> > > > > > >>> > > > > > > > > > accessible from the client to
> solve
> > > that
> > > > >> > issue,
> > > > >> > > > but
> > > > >> > > > > > >>> this
> > > > >> > > > > > >>> > is a
> > > > >> > > > > > >>> > > > > good
> > > > >> > > > > > >>> > > > > > > first
> > > > >> > > > > > >>> > > > > > > > > > step.
> > > > >> > > > > > >>> > > > > > > > > >
> > > > >> > > > > > >>> > > > > > > > > > Ryanne
> > > > >> > > > > > >>> > > > > > > > > >
> > > > >> > > > > > >>> > > > > > > > > > On Wed, Sep 4, 2019 at 1:41 PM
> Lucas
> > > > >> > > Bradstreet <
> > > > >> > > > > > >>> > > > > lucas@confluent.io>
> > > > >> > > > > > >>> > > > > > > wrote:
> > > > >> > > > > > >>> > > > > > > > > > > Hi all,
> > > > >> > > > > > >>> > > > > > > > > > >
> > > > >> > > > > > >>> > > > > > > > > > > I would like to kick off
> > discussion
> > > of
> > > > >> > > KIP-516,
> > > > >> > > > > an
> > > > >> > > > > > >>> > > > > implementation
> > > > >> > > > > > >>> > > > > > > of topic
> > > > >> > > > > > >>> > > > > > > > > > > IDs for Kafka. Topic IDs aim to
> > > solve
> > > > >> topic
> > > > >> > > > > > >>> uniqueness
> > > > >> > > > > > >>> > > > > problems in
> > > > >> > > > > > >>> > > > > > > Kafka,
> > > > >> > > > > > >>> > > > > > > > > > > where referring to a topic by
> name
> > > > >> alone is
> > > > >> > > > > > >>> insufficient.
> > > > >> > > > > > >>> > > > Such
> > > > >> > > > > > >>> > > > > > > cases
> > > > >> > > > > > >>> > > > > > > > > > > include when a topic has been
> > > deleted
> > > > >> and
> > > > >> > > > > recreated
> > > > >> > > > > > >>> with
> > > > >> > > > > > >>> > the
> > > > >> > > > > > >>> > > > > same
> > > > >> > > > > > >>> > > > > > > name.
> > > > >> > > > > > >>> > > > > > > > > > > Unique identifiers will help
> > > simplify
> > > > >> and
> > > > >> > > > improve
> > > > >> > > > > > >>> Kafka's
> > > > >> > > > > > >>> > > > topic
> > > > >> > > > > > >>> > > > > > > deletion
> > > > >> > > > > > >>> > > > > > > > > > > process, as well as prevent
> cases
> > > > where
> > > > >> > > brokers
> > > > >> > > > > may
> > > > >> > > > > > >>> > > > incorrectly
> > > > >> > > > > > >>> > > > > > > interact
> > > > >> > > > > > >>> > > > > > > > > > > with stale versions of topics.
> > > > >> > > > > > >>> > > > > > > > > > >
> > > > >> > > > > > >>> > > > > > > > > > >
> > > > >> > > > > > >>> > > > > > > > > > >
> > > > >> > > > > > >>> > > >
> > > > >> > > > > > >>> >
> > > > >> > > > > > >>>
> > > > >> > > > > >
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > > > >> > > > > > >>> > > > > > > > > > > Looking forward to your
> thoughts.
> > > > >> > > > > > >>> > > > > > > > > > >
> > > > >> > > > > > >>> > > > > > > > > > > Lucas
> > > > >> > > > > > >>> > > > > > > > > > >
> > > > >> > > > > > >>> >
> > > > >> > > > > > >>> >
> > > > >> > > > > > >>>
> > > > >> > > > > > >>
> > > > >> > > > > >
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Jun Rao <ju...@confluent.io>]

Hi, Justine,

Thanks for the response. Made another pass. A few more comments below.

20. znode schema:
20.1 It seems that {"name": "version", "type": "int", "id": "UUID", "doc":
"version id"} should be {"name": "version", "type": "int"}, {"name": "id",
"type": "UUID", "doc": "version id"}.
20.2 The znode format is JSON which doesn't have UUID type. So the type
probably should be string?
20.3 Also, the existing format used seems outdated. It should have the
following format.
    Json.encodeAsBytes(Map(
      "version" -> 2,
      "partitions" -> replicaAssignmentJson.asJava,
      "adding_replicas" -> addingReplicasAssignmentJson.asJava,
      "removing_replicas" -> removingReplicasAssignmentJson.asJava
    ).asJava)
  }

21. Migration: The KIP says "The migration process can take place without
an inter-broker protocol bump, as the format stored in
/brokers/topics/[topic] will be compatible with older broker versions."
However, since we are bumping up the version of inter-broker requests, it
seems that we need to use IBP for migration.

22. The KIP says "Remove deleted topics from replicas by sending
StopReplicaRequest V3 for any topics which do not contain a topic ID, and
V4 for any topics which do contain a topic ID.". However, if we use IBP, it
seems that the controller will either send StopReplicaRequest V3
or StopReplicaRequest V4, but never mixed V3 and V4 for different topics.
Basically, before the IBP bump, V3 will be used. After the IBP bump,
topicId will be created and V4 will be used.

23. Given that we depend on IBP, do we still need "0 UNSPECIFIED"
in LeaderAndIsr?

24. LeaderAndIsrResponse v5 : It still has the topic field.

25. LeaderAndIsrResponse v5, StopReplicaResponse v4: Could we use this
opportunity to organize the response in 2 levels, first by topic, then by
partition, as most other requests/responses?

26. FetchRequest v13 : Should forgotten_topics_data use topicId too?

27. "This file can either be plain text (key/value pairs) or JSON." Have we
decided which one to use? Also, it would be helpful to provide an example.

28. Future improvement: Another future improvement opportunity is to use
topicId in GroupMetadataManager.offsetCommitKey in the offset_commit topic.
This may save some space.

Thanks,

Jun

On Wed, Sep 23, 2020 at 8:50 AM Justine Olshan <jo...@confluent.io> wrote:

> Hi Tom,
>
> Thanks for the comment. I think this is a really good idea and it has been
> added to the KIP under the newly added tooling section.
>
> Thanks again,
> Justine
>
> On Wed, Sep 23, 2020 at 3:17 AM Tom Bentley <tb...@redhat.com> wrote:
>
> > Hi Justine,
> >
> > I know you started the vote thread, but on re-reading the KIP I noticed
> > that although the topic id is included in the MetadataResponse it's not
> > surfaced in the output from `kafka-topics.sh --describe`. Maybe that was
> > intentional because ids are intentionally not really something the user
> > should care deeply about, but it would also make life harder for anyone
> > debugging Kafka and this would likely get worse the more topic ids got
> > rolled out across the protocols, clients etc. It seems likely that
> > `kafka-topics.sh` will eventually need the ability to show the id of a
> > topic and perhaps find a topic name given an id. Is there any reason not
> to
> > implement that in this KIP?
> >
> > Many thanks,
> >
> > Tom
> >
> > On Mon, Sep 21, 2020 at 9:54 PM Justine Olshan <jo...@confluent.io>
> > wrote:
> >
> > > Hi all,
> > >
> > > After thinking about it, I've decided to remove the topic name from the
> > > Fetch Request and Response after all. Since there are so many of these
> > > requests per second, it is worth removing the extra information. I've
> > > updated the KIP to reflect this change.
> > >
> > > Please let me know if there is anything else we should discuss before
> > > voting.
> > >
> > > Thank you,
> > > Justine
> > >
> > > On Fri, Sep 18, 2020 at 9:46 AM Justine Olshan <jo...@confluent.io>
> > > wrote:
> > >
> > > > Hi Jun,
> > > >
> > > > I see what you are saying. For now we can remove the extra
> information.
> > > > I'll leave the option to add more fields to the file in the future.
> The
> > > KIP
> > > > has been updated to reflect this change.
> > > >
> > > > Thanks,
> > > > Justine
> > > >
> > > > On Fri, Sep 18, 2020 at 8:46 AM Jun Rao <ju...@confluent.io> wrote:
> > > >
> > > >> Hi, Justine,
> > > >>
> > > >> Thanks for the reply.
> > > >>
> > > >> 13. If the log directory is the source of truth, it means that the
> > > >> redundant info in the metadata file will be ignored. Then the
> question
> > > is
> > > >> why do we need to put the redundant info in the metadata file now?
> > > >>
> > > >> Thanks,
> > > >>
> > > >> Jun
> > > >>
> > > >> On Thu, Sep 17, 2020 at 5:07 PM Justine Olshan <
> jolshan@confluent.io>
> > > >> wrote:
> > > >>
> > > >> > Hi Jun,
> > > >> > Thanks for the quick response!
> > > >> >
> > > >> > 12. I've decided to bump up the versions on the requests and
> updated
> > > the
> > > >> > KIP. I think it's good we thoroughly discussed the options here,
> so
> > we
> > > >> know
> > > >> > we made a good choice. :)
> > > >> >
> > > >> > 13. This is an interesting situation. I think if this does occur
> we
> > > >> should
> > > >> > give a warning. I agree that it's hard to know the source of truth
> > for
> > > >> sure
> > > >> > since the directory or the file could be manually modified. I
> guess
> > > the
> > > >> > directory could be used as the source of truth. To be honest, I'm
> > not
> > > >> > really sure what happens in kafka when the log directory is
> renamed
> > > >> > manually in such a way. I'm also wondering if the situation is
> > > >> recoverable
> > > >> > in this scenario.
> > > >> >
> > > >> > Thanks,
> > > >> > Justine
> > > >> >
> > > >> > On Thu, Sep 17, 2020 at 4:28 PM Jun Rao <ju...@confluent.io> wrote:
> > > >> >
> > > >> > > Hi, Justine,
> > > >> > >
> > > >> > > Thanks for the reply.
> > > >> > >
> > > >> > > 12. I don't have a strong preference either. However, if we need
> > IBP
> > > >> > > anyway, maybe it's easier to just bump up the version for all
> > inter
> > > >> > broker
> > > >> > > requests and add the topic id field as a regular field. A
> regular
> > > >> field
> > > >> > is
> > > >> > > a bit more concise in wire transfer than a flexible field.
> > > >> > >
> > > >> > > 13. The confusion that I was referring to is between the topic
> > name
> > > >> and
> > > >> > > partition number between the log dir and the metadata file. For
> > > >> example,
> > > >> > if
> > > >> > > the log dir is topicA-1 and the metadata file in it has topicB
> and
> > > >> > > partition 0 (say due to a bug or manual modification), which one
> > do
> > > we
> > > >> > use
> > > >> > > as the source of truth?
> > > >> > >
> > > >> > > Jun
> > > >> > >
> > > >> > > On Thu, Sep 17, 2020 at 3:43 PM Justine Olshan <
> > > jolshan@confluent.io>
> > > >> > > wrote:
> > > >> > >
> > > >> > > > Hi Jun,
> > > >> > > > Thanks for the comments.
> > > >> > > >
> > > >> > > > 12. I bumped the LeaderAndIsrRequest because I removed the
> topic
> > > >> name
> > > >> > > field
> > > >> > > > in the response. It may be possible to avoid bumping the
> version
> > > >> > without
> > > >> > > > that change, but I may be missing something.
> > > >> > > > I believe StopReplica is actually on version 3 now, but
> because
> > > >> > version 2
> > > >> > > > is flexible, I kept that listed as version 2 on the KIP page.
> > > >> However,
> > > >> > > you
> > > >> > > > may be right in that we may need to bump the version on
> > > StopReplica
> > > >> to
> > > >> > > deal
> > > >> > > > with deletion differently as mentioned above. I don't know if
> I
> > > >> have a
> > > >> > > big
> > > >> > > > preference over used tagged fields or not.
> > > >> > > >
> > > >> > > > 13. I was thinking that in the case where the file and the
> > request
> > > >> > topic
> > > >> > > > ids don't match, it means that the broker's topic/the one in
> the
> > > >> file
> > > >> > has
> > > >> > > > been deleted. In that case, we would need to delete the old
> > topic
> > > >> and
> > > >> > > start
> > > >> > > > receiving the new version. If the topic name were to change,
> but
> > > the
> > > >> > ids
> > > >> > > > still matched, the file would also need to update. Am I
> missing
> > a
> > > >> case
> > > >> > > > where the file would be correct and not the request?
> > > >> > > >
> > > >> > > > Thanks,
> > > >> > > > Justine
> > > >> > > >
> > > >> > > > On Thu, Sep 17, 2020 at 3:18 PM Jun Rao <ju...@confluent.io>
> > wrote:
> > > >> > > >
> > > >> > > > > Hi, Justine,
> > > >> > > > >
> > > >> > > > > Thanks for the reply. A couple of more comments below.
> > > >> > > > >
> > > >> > > > > 12. ListOffset and OffsetForLeader currently don't support
> > > >> flexible
> > > >> > > > fields.
> > > >> > > > > So, we have to bump up the version number and use IBP at
> least
> > > for
> > > >> > > these
> > > >> > > > > two requests. Note that it seems 2.7.0 will require IBP
> anyway
> > > >> > because
> > > >> > > of
> > > >> > > > > changes in KAFKA-10435. Also, it seems that the version for
> > > >> > > > > LeaderAndIsrRequest and StopReplica are bumped even though
> we
> > > only
> > > >> > > added
> > > >> > > > a
> > > >> > > > > tagged field. But since IBP is needed anyway, we may want to
> > > >> revisit
> > > >> > > the
> > > >> > > > > overall tagged field choice.
> > > >> > > > >
> > > >> > > > > 13. The only downside is the potential confusion on which
> one
> > is
> > > >> the
> > > >> > > > source
> > > >> > > > > of truth if they don't match. Another option is to include
> > those
> > > >> > fields
> > > >> > > > in
> > > >> > > > > the metadata file when we actually change the directory
> > > structure.
> > > >> > > > >
> > > >> > > > > Thanks,
> > > >> > > > >
> > > >> > > > > Jun
> > > >> > > > >
> > > >> > > > > On Thu, Sep 17, 2020 at 2:01 PM Justine Olshan <
> > > >> jolshan@confluent.io
> > > >> > >
> > > >> > > > > wrote:
> > > >> > > > >
> > > >> > > > > > Hello all,
> > > >> > > > > >
> > > >> > > > > > I've thought some more about removing the topic name field
> > > from
> > > >> > some
> > > >> > > of
> > > >> > > > > the
> > > >> > > > > > requests. On closer inspection of the requests/responses,
> it
> > > >> seems
> > > >> > > that
> > > >> > > > > the
> > > >> > > > > > internal changes would be much larger than I expected.
> Some
> > > >> > protocols
> > > >> > > > > > involve clients, so they would require changes too. I'm
> > > thinking
> > > >> > that
> > > >> > > > for
> > > >> > > > > > now, removing the topic name from these requests and
> > responses
> > > >> are
> > > >> > > out
> > > >> > > > of
> > > >> > > > > > scope.
> > > >> > > > > >
> > > >> > > > > > I have decided to just keep the change
> LeaderAndIsrResponse
> > to
> > > >> > remove
> > > >> > > > the
> > > >> > > > > > topic name, and have updated the KIP to reflect this
> > change. I
> > > >> have
> > > >> > > > also
> > > >> > > > > > mentioned the other requests and responses in future work.
> > > >> > > > > >
> > > >> > > > > > I'm hoping to start the voting process soon, so let me
> know
> > if
> > > >> > there
> > > >> > > is
> > > >> > > > > > anything else we should discuss.
> > > >> > > > > >
> > > >> > > > > > Thank you,
> > > >> > > > > > Justine
> > > >> > > > > >
> > > >> > > > > > On Tue, Sep 15, 2020 at 3:57 PM Justine Olshan <
> > > >> > jolshan@confluent.io
> > > >> > > >
> > > >> > > > > > wrote:
> > > >> > > > > >
> > > >> > > > > > > Hello again,
> > > >> > > > > > > To follow up on some of the other comments:
> > > >> > > > > > >
> > > >> > > > > > > 10/11) We can remove the topic name from these
> > > >> > requests/responses,
> > > >> > > > and
> > > >> > > > > > > that means we will just have to make a few internal
> > changes
> > > to
> > > >> > make
> > > >> > > > > > > partitions accessible by topic id and partition. I can
> > > update
> > > >> the
> > > >> > > KIP
> > > >> > > > > to
> > > >> > > > > > > remove them unless anyone thinks they should stay.
> > > >> > > > > > >
> > > >> > > > > > > 12) Addressed in the previous email. I've updated the
> KIP
> > to
> > > >> > > include
> > > >> > > > > > > tagged fields for the requests and responses. (More on
> > that
> > > >> > below)
> > > >> > > > > > >
> > > >> > > > > > > 13) I think part of the idea for including this
> > information
> > > >> is to
> > > >> > > > > prepare
> > > >> > > > > > > for future changes. Perhaps the directory structure
> might
> > > >> change
> > > >> > > from
> > > >> > > > > > > topicName_partitionNumber to something like
> > > >> > > topicID_partitionNumber.
> > > >> > > > > Then
> > > >> > > > > > > it would be useful to have the topic name in the file
> > since
> > > it
> > > >> > > would
> > > >> > > > > not
> > > >> > > > > > be
> > > >> > > > > > > in the directory structure. Supporting topic renames
> might
> > > be
> > > >> > > easier
> > > >> > > > if
> > > >> > > > > > the
> > > >> > > > > > > other fields are included. Would there be any downsides
> to
> > > >> > > including
> > > >> > > > > this
> > > >> > > > > > > information?
> > > >> > > > > > >
> > > >> > > > > > > 14)  Yes, we would need to copy the partition metadata
> > file
> > > in
> > > >> > this
> > > >> > > > > > > process. I've updated the KIP to include this.
> > > >> > > > > > >
> > > >> > > > > > > 15) I believe Lucas meant v1 and v2 here. He was
> referring
> > > to
> > > >> how
> > > >> > > the
> > > >> > > > > > > requests would fall under different IBP and meant that
> > older
> > > >> > > brokers
> > > >> > > > > > would
> > > >> > > > > > > have to use the older version of the request and the
> > > existing
> > > >> > topic
> > > >> > > > > > > deletion process. At first, it seemed like tagged fields
> > > would
> > > >> > > > resolve
> > > >> > > > > > > the IBP issue. However, we may need IBP for this request
> > > after
> > > >> > all
> > > >> > > > > since
> > > >> > > > > > > the controller handles the topic deletion differently
> > > >> depending
> > > >> > on
> > > >> > > > the
> > > >> > > > > > IBP
> > > >> > > > > > > version. In an older version, we can't just send a
> > > StopReplica
> > > >> > > delete
> > > >> > > > > the
> > > >> > > > > > > topic immediately like we'd want to for this KIP.
> > > >> > > > > > >
> > > >> > > > > > > This makes me wonder if we want tagged fields on all the
> > > >> requests
> > > >> > > > after
> > > >> > > > > > > all. Let me know your thoughts!
> > > >> > > > > > >
> > > >> > > > > > > Justine
> > > >> > > > > > >
> > > >> > > > > > > On Tue, Sep 15, 2020 at 1:03 PM Justine Olshan <
> > > >> > > jolshan@confluent.io
> > > >> > > > >
> > > >> > > > > > > wrote:
> > > >> > > > > > >
> > > >> > > > > > >> Hi all,
> > > >> > > > > > >> Jun brought up a good point in his last email about
> > tagged
> > > >> > fields,
> > > >> > > > and
> > > >> > > > > > >> I've updated the KIP to reflect that the changes to
> > > requests
> > > >> and
> > > >> > > > > > responses
> > > >> > > > > > >> will be in the form of tagged fields to avoid changing
> > IBP.
> > > >> > > > > > >>
> > > >> > > > > > >> Jun: I plan on sending a followup email to address some
> > of
> > > >> the
> > > >> > > other
> > > >> > > > > > >> points.
> > > >> > > > > > >>
> > > >> > > > > > >> Thanks,
> > > >> > > > > > >> Justine
> > > >> > > > > > >>
> > > >> > > > > > >> On Mon, Sep 14, 2020 at 4:25 PM Jun Rao <
> > jun@confluent.io>
> > > >> > wrote:
> > > >> > > > > > >>
> > > >> > > > > > >>> Hi, Justine,
> > > >> > > > > > >>>
> > > >> > > > > > >>> Thanks for the updated KIP. A few comments below.
> > > >> > > > > > >>>
> > > >> > > > > > >>> 10. LeaderAndIsr Response: Do we need the topic name?
> > > >> > > > > > >>>
> > > >> > > > > > >>> 11. For the changed request/response, other than
> > > >> LeaderAndIsr,
> > > >> > > > > > >>> UpdateMetadata, Metadata, do we need to include the
> > topic
> > > >> name?
> > > >> > > > > > >>>
> > > >> > > > > > >>> 12. It seems that upgrades don't require IBP. Does
> that
> > > mean
> > > >> > the
> > > >> > > > new
> > > >> > > > > > >>> fields
> > > >> > > > > > >>> in all the request/response are added as tagged fields
> > > >> without
> > > >> > > > > bumping
> > > >> > > > > > up
> > > >> > > > > > >>> the request version? It would be useful to make that
> > > clear.
> > > >> > > > > > >>>
> > > >> > > > > > >>> 13. Partition Metadata file: Do we need to include the
> > > topic
> > > >> > name
> > > >> > > > and
> > > >> > > > > > the
> > > >> > > > > > >>> partition id since they are implied in the directory
> > name?
> > > >> > > > > > >>>
> > > >> > > > > > >>> 14. In the JBOD mode, we support moving a partition's
> > data
> > > >> from
> > > >> > > one
> > > >> > > > > > disk
> > > >> > > > > > >>> to
> > > >> > > > > > >>> another. Will the new partition metadata file be
> copied
> > > >> during
> > > >> > > that
> > > >> > > > > > >>> process?
> > > >> > > > > > >>>
> > > >> > > > > > >>> 15. The KIP says "Remove deleted topics from replicas
> by
> > > >> > sending
> > > >> > > > > > >>> StopReplicaRequest V2 for any topics which do not
> > contain
> > > a
> > > >> > topic
> > > >> > > > ID,
> > > >> > > > > > and
> > > >> > > > > > >>> V3 for any topics which do contain a topic ID.".
> > However,
> > > it
> > > >> > > seems
> > > >> > > > > the
> > > >> > > > > > >>> updated controller will create all missing topic IDs
> > first
> > > >> > before
> > > >> > > > > doing
> > > >> > > > > > >>> other actions. So, is StopReplicaRequest V2 needed?
> > > >> > > > > > >>>
> > > >> > > > > > >>> Jun
> > > >> > > > > > >>>
> > > >> > > > > > >>> On Fri, Sep 11, 2020 at 10:31 AM John Roesler <
> > > >> > > vvcephei@apache.org
> > > >> > > > >
> > > >> > > > > > >>> wrote:
> > > >> > > > > > >>>
> > > >> > > > > > >>> > Thanks, Justine!
> > > >> > > > > > >>> >
> > > >> > > > > > >>> > Your response seems compelling to me.
> > > >> > > > > > >>> >
> > > >> > > > > > >>> > -John
> > > >> > > > > > >>> >
> > > >> > > > > > >>> > On Fri, 2020-09-11 at 10:17 -0700, Justine Olshan
> > wrote:
> > > >> > > > > > >>> > > Hello all,
> > > >> > > > > > >>> > > Thanks for continuing the discussion! I have a few
> > > >> > responses
> > > >> > > to
> > > >> > > > > > your
> > > >> > > > > > >>> > points.
> > > >> > > > > > >>> > >
> > > >> > > > > > >>> > > Tom: You are correct in that this KIP has not
> > > mentioned
> > > >> the
> > > >> > > > > > >>> > > DeleteTopicsRequest. I think that this would be
> out
> > of
> > > >> > scope
> > > >> > > > for
> > > >> > > > > > >>> now, but
> > > >> > > > > > >>> > > may be something worth adding in the future.
> > > >> > > > > > >>> > >
> > > >> > > > > > >>> > > John: We did consider sequence ids, but there are
> a
> > > few
> > > >> > > reasons
> > > >> > > > > to
> > > >> > > > > > >>> favor
> > > >> > > > > > >>> > > UUIDs. There are several cases where topics from
> > > >> different
> > > >> > > > > clusters
> > > >> > > > > > >>> may
> > > >> > > > > > >>> > > interact now and in the future. For example,
> Mirror
> > > >> Maker 2
> > > >> > > may
> > > >> > > > > > >>> benefit
> > > >> > > > > > >>> > > from being able to detect when a cluster being
> > > mirrored
> > > >> is
> > > >> > > > > deleted
> > > >> > > > > > >>> and
> > > >> > > > > > >>> > > recreated and globally unique identifiers would
> make
> > > >> > > resolving
> > > >> > > > > > issues
> > > >> > > > > > >>> > > easier than sequence IDs which may collide between
> > > >> > clusters.
> > > >> > > > > > KIP-405
> > > >> > > > > > >>> > > (tiered storage) will also benefit from globally
> > > unique
> > > >> IDs
> > > >> > > as
> > > >> > > > > > shared
> > > >> > > > > > >>> > > buckets may be used between clusters.
> > > >> > > > > > >>> > >
> > > >> > > > > > >>> > > Globally unique IDs would also make functionality
> > like
> > > >> > moving
> > > >> > > > > > topics
> > > >> > > > > > >>> > > between disparate clusters easier in the future,
> > > >> simplify
> > > >> > any
> > > >> > > > > > future
> > > >> > > > > > >>> > > implementations of backups and restores, and more.
> > In
> > > >> > > general,
> > > >> > > > > > >>> unique IDs
> > > >> > > > > > >>> > > would ensure that the source cluster topics do not
> > > >> conflict
> > > >> > > > with
> > > >> > > > > > the
> > > >> > > > > > >>> > > destination cluster topics.
> > > >> > > > > > >>> > >
> > > >> > > > > > >>> > > If we were to use sequence ids, we would need
> > > >> sufficiently
> > > >> > > > large
> > > >> > > > > > >>> cluster
> > > >> > > > > > >>> > > ids to be stored with the topic identifiers or we
> > run
> > > >> the
> > > >> > > risk
> > > >> > > > of
> > > >> > > > > > >>> > > collisions. This will give up any advantage in
> > > >> compactness
> > > >> > > that
> > > >> > > > > > >>> sequence
> > > >> > > > > > >>> > > numbers may bring. Given these advantages I think
> it
> > > >> makes
> > > >> > > > sense
> > > >> > > > > to
> > > >> > > > > > >>> use
> > > >> > > > > > >>> > > UUIDs.
> > > >> > > > > > >>> > >
> > > >> > > > > > >>> > > Gokul: This is an interesting idea, but this is a
> > > >> breaking
> > > >> > > > > change.
> > > >> > > > > > >>> Out of
> > > >> > > > > > >>> > > scope for now, but maybe worth discussing in the
> > > future.
> > > >> > > > > > >>> > >
> > > >> > > > > > >>> > > Hope this explains some of the decisions,
> > > >> > > > > > >>> > >
> > > >> > > > > > >>> > > Justine
> > > >> > > > > > >>> > >
> > > >> > > > > > >>> > >
> > > >> > > > > > >>> > >
> > > >> > > > > > >>> > > On Fri, Sep 11, 2020 at 8:27 AM Gokul Ramanan
> > > >> Subramanian <
> > > >> > > > > > >>> > > gokul2411s@gmail.com> wrote:
> > > >> > > > > > >>> > >
> > > >> > > > > > >>> > > > Hi.
> > > >> > > > > > >>> > > >
> > > >> > > > > > >>> > > > Thanks for the KIP.
> > > >> > > > > > >>> > > >
> > > >> > > > > > >>> > > > Have you thought about whether it makes sense to
> > > >> support
> > > >> > > > > > >>> authorizing a
> > > >> > > > > > >>> > > > principal for a topic ID rather than a topic
> name
> > to
> > > >> > > achieve
> > > >> > > > > > >>> tighter
> > > >> > > > > > >>> > > > security?
> > > >> > > > > > >>> > > >
> > > >> > > > > > >>> > > > Or is the topic ID fundamentally an internal
> > detail
> > > >> > similar
> > > >> > > > to
> > > >> > > > > > >>> epochs
> > > >> > > > > > >>> > used
> > > >> > > > > > >>> > > > in a bunch of other places in Kafka?
> > > >> > > > > > >>> > > >
> > > >> > > > > > >>> > > > Thanks.
> > > >> > > > > > >>> > > >
> > > >> > > > > > >>> > > > On Fri, Sep 11, 2020 at 4:06 PM John Roesler <
> > > >> > > > > > vvcephei@apache.org>
> > > >> > > > > > >>> > wrote:
> > > >> > > > > > >>> > > >
> > > >> > > > > > >>> > > > > Hello Justine,
> > > >> > > > > > >>> > > > >
> > > >> > > > > > >>> > > > > Thanks for the KIP!
> > > >> > > > > > >>> > > > >
> > > >> > > > > > >>> > > > > I happen to have been confronted recently with
> > the
> > > >> need
> > > >> > > to
> > > >> > > > > keep
> > > >> > > > > > >>> > track of
> > > >> > > > > > >>> > > > a
> > > >> > > > > > >>> > > > > large number of topics as compactly as
> > possible. I
> > > >> was
> > > >> > > > going
> > > >> > > > > to
> > > >> > > > > > >>> come
> > > >> > > > > > >>> > up
> > > >> > > > > > >>> > > > > with some way to dictionary encode the topic
> > names
> > > >> as
> > > >> > > > > integers,
> > > >> > > > > > >>> but
> > > >> > > > > > >>> > this
> > > >> > > > > > >>> > > > > seems much better!
> > > >> > > > > > >>> > > > >
> > > >> > > > > > >>> > > > > Apologies if this has been raised before, but
> > I’m
> > > >> > > wondering
> > > >> > > > > > >>> about the
> > > >> > > > > > >>> > > > > choice of UUID vs sequence number for the ids.
> > > >> > Typically,
> > > >> > > > > I’ve
> > > >> > > > > > >>> seen
> > > >> > > > > > >>> > UUIDs
> > > >> > > > > > >>> > > > > in two situations:
> > > >> > > > > > >>> > > > > 1. When processes need to generate
> non-colliding
> > > >> > > > identifiers
> > > >> > > > > > >>> without
> > > >> > > > > > >>> > > > > coordination.
> > > >> > > > > > >>> > > > > 2. When the identifier needs to be
> “universally
> > > >> > unique”;
> > > >> > > > > I.e.,
> > > >> > > > > > >>> the
> > > >> > > > > > >>> > > > > identifier needs to distinguish the entity
> from
> > > all
> > > >> > other
> > > >> > > > > > >>> entities
> > > >> > > > > > >>> > that
> > > >> > > > > > >>> > > > > could ever exist. This is useful in cases
> where
> > > >> > entities
> > > >> > > > from
> > > >> > > > > > all
> > > >> > > > > > >>> > kinds
> > > >> > > > > > >>> > > > of
> > > >> > > > > > >>> > > > > systems get mixed together, such as when
> dumping
> > > >> logs
> > > >> > > from
> > > >> > > > > all
> > > >> > > > > > >>> > processes
> > > >> > > > > > >>> > > > in
> > > >> > > > > > >>> > > > > a company into a common system.
> > > >> > > > > > >>> > > > >
> > > >> > > > > > >>> > > > > Maybe I’m being short-sighted, but it doesn’t
> > seem
> > > >> like
> > > >> > > > > either
> > > >> > > > > > >>> really
> > > >> > > > > > >>> > > > > applies here. It seems like the brokers could
> > and
> > > >> would
> > > >> > > > > achieve
> > > >> > > > > > >>> > consensus
> > > >> > > > > > >>> > > > > when creating a topic anyway, which is all
> > that’s
> > > >> > > required
> > > >> > > > to
> > > >> > > > > > >>> > generate
> > > >> > > > > > >>> > > > > non-colliding sequence ids. For the second, as
> > you
> > > >> > > mention,
> > > >> > > > > we
> > > >> > > > > > >>> could
> > > >> > > > > > >>> > > > assign
> > > >> > > > > > >>> > > > > a UUID to the cluster as a whole, which would
> > > render
> > > >> > any
> > > >> > > > > > resource
> > > >> > > > > > >>> > scoped
> > > >> > > > > > >>> > > > to
> > > >> > > > > > >>> > > > > the broker universally unique as well.
> > > >> > > > > > >>> > > > >
> > > >> > > > > > >>> > > > > The reason I mention this is that, although a
> > UUID
> > > >> is
> > > >> > way
> > > >> > > > > more
> > > >> > > > > > >>> > compact
> > > >> > > > > > >>> > > > > than topic names, it’s still 16 bytes. In
> > > contrast,
> > > >> a
> > > >> > > > 4-byte
> > > >> > > > > > >>> integer
> > > >> > > > > > >>> > > > > sequence id would give us 4 billion unique
> > topics
> > > >> per
> > > >> > > > > cluster,
> > > >> > > > > > >>> which
> > > >> > > > > > >>> > > > seems
> > > >> > > > > > >>> > > > > like enough ;)
> > > >> > > > > > >>> > > > >
> > > >> > > > > > >>> > > > > Considering the number of different times
> these
> > > >> topic
> > > >> > > > > > >>> identifiers are
> > > >> > > > > > >>> > > > sent
> > > >> > > > > > >>> > > > > over the wire or stored in memory, it seems
> like
> > > it
> > > >> > might
> > > >> > > > be
> > > >> > > > > > >>> worth
> > > >> > > > > > >>> > the
> > > >> > > > > > >>> > > > > additional 4x space savings.
> > > >> > > > > > >>> > > > >
> > > >> > > > > > >>> > > > > What do you think about this?
> > > >> > > > > > >>> > > > >
> > > >> > > > > > >>> > > > > Thanks,
> > > >> > > > > > >>> > > > > John
> > > >> > > > > > >>> > > > >
> > > >> > > > > > >>> > > > > On Fri, Sep 11, 2020, at 03:20, Tom Bentley
> > wrote:
> > > >> > > > > > >>> > > > > > Hi Justine,
> > > >> > > > > > >>> > > > > >
> > > >> > > > > > >>> > > > > > This looks like a very welcome improvement.
> > > >> Thanks!
> > > >> > > > > > >>> > > > > >
> > > >> > > > > > >>> > > > > > Maybe I missed it, but the KIP doesn't seem
> to
> > > >> > mention
> > > >> > > > > > changing
> > > >> > > > > > >>> > > > > > DeleteTopicsRequest to identify the topic
> > using
> > > an
> > > >> > id.
> > > >> > > > > Maybe
> > > >> > > > > > >>> > that's out
> > > >> > > > > > >>> > > > > of
> > > >> > > > > > >>> > > > > > scope, but DeleteTopicsRequest is not listed
> > > among
> > > >> > the
> > > >> > > > > Future
> > > >> > > > > > >>> Work
> > > >> > > > > > >>> > APIs
> > > >> > > > > > >>> > > > > > either.
> > > >> > > > > > >>> > > > > >
> > > >> > > > > > >>> > > > > > Kind regards,
> > > >> > > > > > >>> > > > > >
> > > >> > > > > > >>> > > > > > Tom
> > > >> > > > > > >>> > > > > >
> > > >> > > > > > >>> > > > > > On Thu, Sep 10, 2020 at 3:59 PM Satish
> > Duggana <
> > > >> > > > > > >>> > > > satish.duggana@gmail.com
> > > >> > > > > > >>> > > > > > wrote:
> > > >> > > > > > >>> > > > > >
> > > >> > > > > > >>> > > > > > > Thanks Lucas/Justine for the nice KIP.
> > > >> > > > > > >>> > > > > > >
> > > >> > > > > > >>> > > > > > > It has several benefits which also include
> > > >> > > simplifying
> > > >> > > > > the
> > > >> > > > > > >>> topic
> > > >> > > > > > >>> > > > > > > deletion process by controller and logs
> > > cleanup
> > > >> by
> > > >> > > > > brokers
> > > >> > > > > > in
> > > >> > > > > > >>> > corner
> > > >> > > > > > >>> > > > > > > cases.
> > > >> > > > > > >>> > > > > > >
> > > >> > > > > > >>> > > > > > > Best,
> > > >> > > > > > >>> > > > > > > Satish.
> > > >> > > > > > >>> > > > > > >
> > > >> > > > > > >>> > > > > > > On Wed, Sep 9, 2020 at 10:07 PM Justine
> > > Olshan <
> > > >> > > > > > >>> > jolshan@confluent.io
> > > >> > > > > > >>> > > > > > > wrote:
> > > >> > > > > > >>> > > > > > > > Hello all, it's been almost a year! I've
> > > made
> > > >> > some
> > > >> > > > > > changes
> > > >> > > > > > >>> to
> > > >> > > > > > >>> > this
> > > >> > > > > > >>> > > > > KIP
> > > >> > > > > > >>> > > > > > > and hope to continue the discussion.
> > > >> > > > > > >>> > > > > > > > One of the main changes I've added is
> now
> > > the
> > > >> > > > metadata
> > > >> > > > > > >>> response
> > > >> > > > > > >>> > > > will
> > > >> > > > > > >>> > > > > > > include the topic ID (as Colin suggested).
> > > >> Clients
> > > >> > > can
> > > >> > > > > > >>> obtain the
> > > >> > > > > > >>> > > > > topicID
> > > >> > > > > > >>> > > > > > > of a given topic through a
> TopicDescription.
> > > The
> > > >> > > > topicId
> > > >> > > > > > will
> > > >> > > > > > >>> > also be
> > > >> > > > > > >>> > > > > > > included with the UpdateMetadata request.
> > > >> > > > > > >>> > > > > > > > Let me know what you all think.
> > > >> > > > > > >>> > > > > > > > Thank you,
> > > >> > > > > > >>> > > > > > > > Justine
> > > >> > > > > > >>> > > > > > > >
> > > >> > > > > > >>> > > > > > > > On 2019/09/13 16:38:26, "Colin McCabe" <
> > > >> > > > > > cmccabe@apache.org
> > > >> > > > > > >>> >
> > > >> > > > > > >>> > wrote:
> > > >> > > > > > >>> > > > > > > > > Hi Lucas,
> > > >> > > > > > >>> > > > > > > > >
> > > >> > > > > > >>> > > > > > > > > Thanks for tackling this.  Topic IDs
> > are a
> > > >> > great
> > > >> > > > > idea,
> > > >> > > > > > >>> and
> > > >> > > > > > >>> > this
> > > >> > > > > > >>> > > > is
> > > >> > > > > > >>> > > > > a
> > > >> > > > > > >>> > > > > > > really good writeup.
> > > >> > > > > > >>> > > > > > > > > For /brokers/topics/[topic], the
> schema
> > > >> version
> > > >> > > > > should
> > > >> > > > > > be
> > > >> > > > > > >>> > bumped
> > > >> > > > > > >>> > > > to
> > > >> > > > > > >>> > > > > > > version 3, rather than 2.  KIP-455 bumped
> > the
> > > >> > version
> > > >> > > > of
> > > >> > > > > > this
> > > >> > > > > > >>> > znode
> > > >> > > > > > >>> > > > to
> > > >> > > > > > >>> > > > > 2
> > > >> > > > > > >>> > > > > > > already :)
> > > >> > > > > > >>> > > > > > > > > Given that we're going to be seeing
> > these
> > > >> > things
> > > >> > > as
> > > >> > > > > > >>> strings
> > > >> > > > > > >>> > as
> > > >> > > > > > >>> > > > lot
> > > >> > > > > > >>> > > > > (in
> > > >> > > > > > >>> > > > > > > logs, in ZooKeeper, on the command-line,
> > > etc.),
> > > >> > does
> > > >> > > it
> > > >> > > > > > make
> > > >> > > > > > >>> > sense to
> > > >> > > > > > >>> > > > > use
> > > >> > > > > > >>> > > > > > > base64 when converting them to strings?
> > > >> > > > > > >>> > > > > > > > > Here is an example of the hex
> > > >> representation:
> > > >> > > > > > >>> > > > > > > > > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> > > >> > > > > > >>> > > > > > > > >
> > > >> > > > > > >>> > > > > > > > > And here is an example in base64.
> > > >> > > > > > >>> > > > > > > > > b8tRS7h4TJ2Vt43Dp85v2A
> > > >> > > > > > >>> > > > > > > > >
> > > >> > > > > > >>> > > > > > > > > The base64 version saves 15 letters
> (to
> > be
> > > >> > fair,
> > > >> > > 4
> > > >> > > > of
> > > >> > > > > > >>> those
> > > >> > > > > > >>> > were
> > > >> > > > > > >>> > > > > > > dashes that we could have elided in the
> hex
> > > >> > > > > > representation.)
> > > >> > > > > > >>> > > > > > > > > Another thing to consider is that we
> > > should
> > > >> > > specify
> > > >> > > > > > that
> > > >> > > > > > >>> the
> > > >> > > > > > >>> > > > > > > all-zeroes UUID is not a valid topic UUID.
> > >  We
> > > >> > can't
> > > >> > > > use
> > > >> > > > > > >>> null
> > > >> > > > > > >>> > for
> > > >> > > > > > >>> > > > this
> > > >> > > > > > >>> > > > > > > because we can't pass a null UUID over the
> > RPC
> > > >> > > protocol
> > > >> > > > > > >>> (there
> > > >> > > > > > >>> > is no
> > > >> > > > > > >>> > > > > > > special pattern for null, nor do we want
> to
> > > >> waste
> > > >> > > space
> > > >> > > > > > >>> reserving
> > > >> > > > > > >>> > > > such
> > > >> > > > > > >>> > > > > a
> > > >> > > > > > >>> > > > > > > pattern.)
> > > >> > > > > > >>> > > > > > > > > Maybe I missed it, but did you
> describe
> > > >> > > "migration
> > > >> > > > > > of...
> > > >> > > > > > >>> > existing
> > > >> > > > > > >>> > > > > > > topic[s] without topic IDs" in detail in
> any
> > > >> > section?
> > > >> > > > It
> > > >> > > > > > >>> seems
> > > >> > > > > > >>> > like
> > > >> > > > > > >>> > > > > when
> > > >> > > > > > >>> > > > > > > the new controller becomes active, it
> should
> > > >> just
> > > >> > > > > generate
> > > >> > > > > > >>> random
> > > >> > > > > > >>> > > > > UUIDs for
> > > >> > > > > > >>> > > > > > > these, and write the random UUIDs back to
> > > >> > ZooKeeper.
> > > >> > > > It
> > > >> > > > > > >>> would be
> > > >> > > > > > >>> > > > good
> > > >> > > > > > >>> > > > > to
> > > >> > > > > > >>> > > > > > > spell that out.  We should make it clear
> > that
> > > >> this
> > > >> > > > > happens
> > > >> > > > > > >>> > regardless
> > > >> > > > > > >>> > > > > of
> > > >> > > > > > >>> > > > > > > the inter-broker protocol version (it's a
> > > >> > compatible
> > > >> > > > > > change).
> > > >> > > > > > >>> > > > > > > > > "LeaderAndIsrRequests including an
> > > >> > > > is_every_partition
> > > >> > > > > > >>> flag"
> > > >> > > > > > >>> > > > seems a
> > > >> > > > > > >>> > > > > > > bit wordy.  Can we just call these "full
> > > >> > > > > > >>> LeaderAndIsrRequests"?
> > > >> > > > > > >>> > Then
> > > >> > > > > > >>> > > > > the
> > > >> > > > > > >>> > > > > > > RPC field could be named "full".  Also, it
> > > would
> > > >> > > > probably
> > > >> > > > > > be
> > > >> > > > > > >>> > better
> > > >> > > > > > >>> > > > > for the
> > > >> > > > > > >>> > > > > > > RPC field to be an enum of { UNSPECIFIED,
> > > >> > > INCREMENTAL,
> > > >> > > > > FULL
> > > >> > > > > > >>> }, so
> > > >> > > > > > >>> > > > that
> > > >> > > > > > >>> > > > > we
> > > >> > > > > > >>> > > > > > > can cleanly handle old versions (by
> treating
> > > >> them
> > > >> > as
> > > >> > > > > > >>> UNSPECIFIED)
> > > >> > > > > > >>> > > > > > > > > In the LeaderAndIsrRequest section,
> you
> > > >> write
> > > >> > "A
> > > >> > > > > final
> > > >> > > > > > >>> > deletion
> > > >> > > > > > >>> > > > > event
> > > >> > > > > > >>> > > > > > > will be secheduled for X ms after the
> > > >> > > > LeaderAndIsrRequest
> > > >> > > > > > was
> > > >> > > > > > >>> > first
> > > >> > > > > > >>> > > > > > > received..."  I guess the X was a
> > placeholder
> > > >> that
> > > >> > > you
> > > >> > > > > > >>> intended
> > > >> > > > > > >>> > to
> > > >> > > > > > >>> > > > > replace
> > > >> > > > > > >>> > > > > > > before posting? :)  In any case, this
> seems
> > > like
> > > >> > the
> > > >> > > > kind
> > > >> > > > > > of
> > > >> > > > > > >>> > thing
> > > >> > > > > > >>> > > > we'd
> > > >> > > > > > >>> > > > > > > want a configuration for.  Let's describe
> > that
> > > >> > > > > > configuration
> > > >> > > > > > >>> key
> > > >> > > > > > >>> > > > > somewhere
> > > >> > > > > > >>> > > > > > > in this KIP, including what its default
> > value
> > > >> is.
> > > >> > > > > > >>> > > > > > > > > We should probably also log a bunch of
> > > >> messages
> > > >> > > at
> > > >> > > > > WARN
> > > >> > > > > > >>> level
> > > >> > > > > > >>> > > > when
> > > >> > > > > > >>> > > > > > > something is scheduled for deletion, as
> > well.
> > > >> > (Maybe
> > > >> > > > > this
> > > >> > > > > > >>> was
> > > >> > > > > > >>> > > > > assumed, but
> > > >> > > > > > >>> > > > > > > it would be good to mention it).
> > > >> > > > > > >>> > > > > > > > > I feel like there are a few sections
> > that
> > > >> > should
> > > >> > > be
> > > >> > > > > > >>> moved to
> > > >> > > > > > >>> > > > > "rejected
> > > >> > > > > > >>> > > > > > > alternatives."  For example, in the
> > > DeleteTopics
> > > >> > > > section,
> > > >> > > > > > >>> since
> > > >> > > > > > >>> > we're
> > > >> > > > > > >>> > > > > not
> > > >> > > > > > >>> > > > > > > going to do option 1 or 2, these should be
> > > moved
> > > >> > into
> > > >> > > > > > >>> "rejected
> > > >> > > > > > >>> > > > > > > alternatives,"  rather than appearing
> > inline.
> > > >> > > Another
> > > >> > > > > case
> > > >> > > > > > >>> is
> > > >> > > > > > >>> > the
> > > >> > > > > > >>> > > > > "Should
> > > >> > > > > > >>> > > > > > > we remove topic name from the protocol
> where
> > > >> > > possible"
> > > >> > > > > > >>> section.
> > > >> > > > > > >>> > This
> > > >> > > > > > >>> > > > > is
> > > >> > > > > > >>> > > > > > > clearly discussing a design alternative
> that
> > > >> we're
> > > >> > > not
> > > >> > > > > > >>> proposing
> > > >> > > > > > >>> > to
> > > >> > > > > > >>> > > > > > > implement: removing the topic name from
> > those
> > > >> > > > protocols.
> > > >> > > > > > >>> > > > > > > > > Is it really necessary to have a new
> > > >> > > > > > >>> > /admin/delete_topics_by_id
> > > >> > > > > > >>> > > > > path
> > > >> > > > > > >>> > > > > > > in ZooKeeper?  It seems like we don't
> really
> > > >> need
> > > >> > > this.
> > > >> > > > > > >>> Whenever
> > > >> > > > > > >>> > > > > there is
> > > >> > > > > > >>> > > > > > > a new controller, we'll send out full
> > > >> > > > > LeaderAndIsrRequests
> > > >> > > > > > >>> which
> > > >> > > > > > >>> > will
> > > >> > > > > > >>> > > > > > > trigger the stale topics to be cleaned up.
> > >  The
> > > >> > > active
> > > >> > > > > > >>> > controller
> > > >> > > > > > >>> > > > will
> > > >> > > > > > >>> > > > > > > also send the full LeaderAndIsrRequest to
> > > >> brokers
> > > >> > > that
> > > >> > > > > are
> > > >> > > > > > >>> just
> > > >> > > > > > >>> > > > > starting
> > > >> > > > > > >>> > > > > > > up.    So we don't really need this kind
> of
> > > >> > two-phase
> > > >> > > > > > commit
> > > >> > > > > > >>> > (send
> > > >> > > > > > >>> > > > out
> > > >> > > > > > >>> > > > > > > StopReplicasRequest, get ACKs from all
> > nodes,
> > > >> > commit
> > > >> > > by
> > > >> > > > > > >>> removing
> > > >> > > > > > >>> > > > > > > /admin/delete_topics node) any more.
> > > >> > > > > > >>> > > > > > > > > You mention that FetchRequest will now
> > > >> include
> > > >> > > UUID
> > > >> > > > > to
> > > >> > > > > > >>> avoid
> > > >> > > > > > >>> > > > issues
> > > >> > > > > > >>> > > > > > > where requests are made to stale
> partitions.
> > > >> > > However,
> > > >> > > > > > >>> adding a
> > > >> > > > > > >>> > UUID
> > > >> > > > > > >>> > > > to
> > > >> > > > > > >>> > > > > > > MetadataRequest is listed as future work,
> > out
> > > of
> > > >> > > scope
> > > >> > > > > for
> > > >> > > > > > >>> this
> > > >> > > > > > >>> > KIP.
> > > >> > > > > > >>> > > > > How
> > > >> > > > > > >>> > > > > > > will the client learn what the topic UUID
> > is,
> > > if
> > > >> > the
> > > >> > > > > > metadata
> > > >> > > > > > >>> > > > response
> > > >> > > > > > >>> > > > > > > doesn't include that information?  It
> seems
> > > like
> > > >> > > adding
> > > >> > > > > the
> > > >> > > > > > >>> UUID
> > > >> > > > > > >>> > to
> > > >> > > > > > >>> > > > > > > MetadataResponse would be an improvement
> > here
> > > >> that
> > > >> > > > might
> > > >> > > > > > not
> > > >> > > > > > >>> be
> > > >> > > > > > >>> > too
> > > >> > > > > > >>> > > > > hard to
> > > >> > > > > > >>> > > > > > > make.
> > > >> > > > > > >>> > > > > > > > > best,
> > > >> > > > > > >>> > > > > > > > > Colin
> > > >> > > > > > >>> > > > > > > > >
> > > >> > > > > > >>> > > > > > > > >
> > > >> > > > > > >>> > > > > > > > > On Mon, Sep 9, 2019, at 17:48, Ryanne
> > > Dolan
> > > >> > > wrote:
> > > >> > > > > > >>> > > > > > > > > > Lucas, this would be great. I've run
> > > into
> > > >> > > issues
> > > >> > > > > with
> > > >> > > > > > >>> > topics
> > > >> > > > > > >>> > > > > being
> > > >> > > > > > >>> > > > > > > > > > resurrected accidentally, since a
> > client
> > > >> > cannot
> > > >> > > > > > easily
> > > >> > > > > > >>> > > > > distinguish
> > > >> > > > > > >>> > > > > > > between
> > > >> > > > > > >>> > > > > > > > > > a deleted topic and a new topic with
> > the
> > > >> same
> > > >> > > > name.
> > > >> > > > > > I'd
> > > >> > > > > > >>> > need
> > > >> > > > > > >>> > > > the
> > > >> > > > > > >>> > > > > ID
> > > >> > > > > > >>> > > > > > > > > > accessible from the client to solve
> > that
> > > >> > issue,
> > > >> > > > but
> > > >> > > > > > >>> this
> > > >> > > > > > >>> > is a
> > > >> > > > > > >>> > > > > good
> > > >> > > > > > >>> > > > > > > first
> > > >> > > > > > >>> > > > > > > > > > step.
> > > >> > > > > > >>> > > > > > > > > >
> > > >> > > > > > >>> > > > > > > > > > Ryanne
> > > >> > > > > > >>> > > > > > > > > >
> > > >> > > > > > >>> > > > > > > > > > On Wed, Sep 4, 2019 at 1:41 PM Lucas
> > > >> > > Bradstreet <
> > > >> > > > > > >>> > > > > lucas@confluent.io>
> > > >> > > > > > >>> > > > > > > wrote:
> > > >> > > > > > >>> > > > > > > > > > > Hi all,
> > > >> > > > > > >>> > > > > > > > > > >
> > > >> > > > > > >>> > > > > > > > > > > I would like to kick off
> discussion
> > of
> > > >> > > KIP-516,
> > > >> > > > > an
> > > >> > > > > > >>> > > > > implementation
> > > >> > > > > > >>> > > > > > > of topic
> > > >> > > > > > >>> > > > > > > > > > > IDs for Kafka. Topic IDs aim to
> > solve
> > > >> topic
> > > >> > > > > > >>> uniqueness
> > > >> > > > > > >>> > > > > problems in
> > > >> > > > > > >>> > > > > > > Kafka,
> > > >> > > > > > >>> > > > > > > > > > > where referring to a topic by name
> > > >> alone is
> > > >> > > > > > >>> insufficient.
> > > >> > > > > > >>> > > > Such
> > > >> > > > > > >>> > > > > > > cases
> > > >> > > > > > >>> > > > > > > > > > > include when a topic has been
> > deleted
> > > >> and
> > > >> > > > > recreated
> > > >> > > > > > >>> with
> > > >> > > > > > >>> > the
> > > >> > > > > > >>> > > > > same
> > > >> > > > > > >>> > > > > > > name.
> > > >> > > > > > >>> > > > > > > > > > > Unique identifiers will help
> > simplify
> > > >> and
> > > >> > > > improve
> > > >> > > > > > >>> Kafka's
> > > >> > > > > > >>> > > > topic
> > > >> > > > > > >>> > > > > > > deletion
> > > >> > > > > > >>> > > > > > > > > > > process, as well as prevent cases
> > > where
> > > >> > > brokers
> > > >> > > > > may
> > > >> > > > > > >>> > > > incorrectly
> > > >> > > > > > >>> > > > > > > interact
> > > >> > > > > > >>> > > > > > > > > > > with stale versions of topics.
> > > >> > > > > > >>> > > > > > > > > > >
> > > >> > > > > > >>> > > > > > > > > > >
> > > >> > > > > > >>> > > > > > > > > > >
> > > >> > > > > > >>> > > >
> > > >> > > > > > >>> >
> > > >> > > > > > >>>
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > > >> > > > > > >>> > > > > > > > > > > Looking forward to your thoughts.
> > > >> > > > > > >>> > > > > > > > > > >
> > > >> > > > > > >>> > > > > > > > > > > Lucas
> > > >> > > > > > >>> > > > > > > > > > >
> > > >> > > > > > >>> >
> > > >> > > > > > >>> >
> > > >> > > > > > >>>
> > > >> > > > > > >>
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > > >
> > >
> >
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Justine Olshan <jo...@confluent.io>]

Hi Tom,

Thanks for the comment. I think this is a really good idea and it has been
added to the KIP under the newly added tooling section.

Thanks again,
Justine

On Wed, Sep 23, 2020 at 3:17 AM Tom Bentley <tb...@redhat.com> wrote:

> Hi Justine,
>
> I know you started the vote thread, but on re-reading the KIP I noticed
> that although the topic id is included in the MetadataResponse it's not
> surfaced in the output from `kafka-topics.sh --describe`. Maybe that was
> intentional because ids are intentionally not really something the user
> should care deeply about, but it would also make life harder for anyone
> debugging Kafka and this would likely get worse the more topic ids got
> rolled out across the protocols, clients etc. It seems likely that
> `kafka-topics.sh` will eventually need the ability to show the id of a
> topic and perhaps find a topic name given an id. Is there any reason not to
> implement that in this KIP?
>
> Many thanks,
>
> Tom
>
> On Mon, Sep 21, 2020 at 9:54 PM Justine Olshan <jo...@confluent.io>
> wrote:
>
> > Hi all,
> >
> > After thinking about it, I've decided to remove the topic name from the
> > Fetch Request and Response after all. Since there are so many of these
> > requests per second, it is worth removing the extra information. I've
> > updated the KIP to reflect this change.
> >
> > Please let me know if there is anything else we should discuss before
> > voting.
> >
> > Thank you,
> > Justine
> >
> > On Fri, Sep 18, 2020 at 9:46 AM Justine Olshan <jo...@confluent.io>
> > wrote:
> >
> > > Hi Jun,
> > >
> > > I see what you are saying. For now we can remove the extra information.
> > > I'll leave the option to add more fields to the file in the future. The
> > KIP
> > > has been updated to reflect this change.
> > >
> > > Thanks,
> > > Justine
> > >
> > > On Fri, Sep 18, 2020 at 8:46 AM Jun Rao <ju...@confluent.io> wrote:
> > >
> > >> Hi, Justine,
> > >>
> > >> Thanks for the reply.
> > >>
> > >> 13. If the log directory is the source of truth, it means that the
> > >> redundant info in the metadata file will be ignored. Then the question
> > is
> > >> why do we need to put the redundant info in the metadata file now?
> > >>
> > >> Thanks,
> > >>
> > >> Jun
> > >>
> > >> On Thu, Sep 17, 2020 at 5:07 PM Justine Olshan <jo...@confluent.io>
> > >> wrote:
> > >>
> > >> > Hi Jun,
> > >> > Thanks for the quick response!
> > >> >
> > >> > 12. I've decided to bump up the versions on the requests and updated
> > the
> > >> > KIP. I think it's good we thoroughly discussed the options here, so
> we
> > >> know
> > >> > we made a good choice. :)
> > >> >
> > >> > 13. This is an interesting situation. I think if this does occur we
> > >> should
> > >> > give a warning. I agree that it's hard to know the source of truth
> for
> > >> sure
> > >> > since the directory or the file could be manually modified. I guess
> > the
> > >> > directory could be used as the source of truth. To be honest, I'm
> not
> > >> > really sure what happens in kafka when the log directory is renamed
> > >> > manually in such a way. I'm also wondering if the situation is
> > >> recoverable
> > >> > in this scenario.
> > >> >
> > >> > Thanks,
> > >> > Justine
> > >> >
> > >> > On Thu, Sep 17, 2020 at 4:28 PM Jun Rao <ju...@confluent.io> wrote:
> > >> >
> > >> > > Hi, Justine,
> > >> > >
> > >> > > Thanks for the reply.
> > >> > >
> > >> > > 12. I don't have a strong preference either. However, if we need
> IBP
> > >> > > anyway, maybe it's easier to just bump up the version for all
> inter
> > >> > broker
> > >> > > requests and add the topic id field as a regular field. A regular
> > >> field
> > >> > is
> > >> > > a bit more concise in wire transfer than a flexible field.
> > >> > >
> > >> > > 13. The confusion that I was referring to is between the topic
> name
> > >> and
> > >> > > partition number between the log dir and the metadata file. For
> > >> example,
> > >> > if
> > >> > > the log dir is topicA-1 and the metadata file in it has topicB and
> > >> > > partition 0 (say due to a bug or manual modification), which one
> do
> > we
> > >> > use
> > >> > > as the source of truth?
> > >> > >
> > >> > > Jun
> > >> > >
> > >> > > On Thu, Sep 17, 2020 at 3:43 PM Justine Olshan <
> > jolshan@confluent.io>
> > >> > > wrote:
> > >> > >
> > >> > > > Hi Jun,
> > >> > > > Thanks for the comments.
> > >> > > >
> > >> > > > 12. I bumped the LeaderAndIsrRequest because I removed the topic
> > >> name
> > >> > > field
> > >> > > > in the response. It may be possible to avoid bumping the version
> > >> > without
> > >> > > > that change, but I may be missing something.
> > >> > > > I believe StopReplica is actually on version 3 now, but because
> > >> > version 2
> > >> > > > is flexible, I kept that listed as version 2 on the KIP page.
> > >> However,
> > >> > > you
> > >> > > > may be right in that we may need to bump the version on
> > StopReplica
> > >> to
> > >> > > deal
> > >> > > > with deletion differently as mentioned above. I don't know if I
> > >> have a
> > >> > > big
> > >> > > > preference over used tagged fields or not.
> > >> > > >
> > >> > > > 13. I was thinking that in the case where the file and the
> request
> > >> > topic
> > >> > > > ids don't match, it means that the broker's topic/the one in the
> > >> file
> > >> > has
> > >> > > > been deleted. In that case, we would need to delete the old
> topic
> > >> and
> > >> > > start
> > >> > > > receiving the new version. If the topic name were to change, but
> > the
> > >> > ids
> > >> > > > still matched, the file would also need to update. Am I missing
> a
> > >> case
> > >> > > > where the file would be correct and not the request?
> > >> > > >
> > >> > > > Thanks,
> > >> > > > Justine
> > >> > > >
> > >> > > > On Thu, Sep 17, 2020 at 3:18 PM Jun Rao <ju...@confluent.io>
> wrote:
> > >> > > >
> > >> > > > > Hi, Justine,
> > >> > > > >
> > >> > > > > Thanks for the reply. A couple of more comments below.
> > >> > > > >
> > >> > > > > 12. ListOffset and OffsetForLeader currently don't support
> > >> flexible
> > >> > > > fields.
> > >> > > > > So, we have to bump up the version number and use IBP at least
> > for
> > >> > > these
> > >> > > > > two requests. Note that it seems 2.7.0 will require IBP anyway
> > >> > because
> > >> > > of
> > >> > > > > changes in KAFKA-10435. Also, it seems that the version for
> > >> > > > > LeaderAndIsrRequest and StopReplica are bumped even though we
> > only
> > >> > > added
> > >> > > > a
> > >> > > > > tagged field. But since IBP is needed anyway, we may want to
> > >> revisit
> > >> > > the
> > >> > > > > overall tagged field choice.
> > >> > > > >
> > >> > > > > 13. The only downside is the potential confusion on which one
> is
> > >> the
> > >> > > > source
> > >> > > > > of truth if they don't match. Another option is to include
> those
> > >> > fields
> > >> > > > in
> > >> > > > > the metadata file when we actually change the directory
> > structure.
> > >> > > > >
> > >> > > > > Thanks,
> > >> > > > >
> > >> > > > > Jun
> > >> > > > >
> > >> > > > > On Thu, Sep 17, 2020 at 2:01 PM Justine Olshan <
> > >> jolshan@confluent.io
> > >> > >
> > >> > > > > wrote:
> > >> > > > >
> > >> > > > > > Hello all,
> > >> > > > > >
> > >> > > > > > I've thought some more about removing the topic name field
> > from
> > >> > some
> > >> > > of
> > >> > > > > the
> > >> > > > > > requests. On closer inspection of the requests/responses, it
> > >> seems
> > >> > > that
> > >> > > > > the
> > >> > > > > > internal changes would be much larger than I expected. Some
> > >> > protocols
> > >> > > > > > involve clients, so they would require changes too. I'm
> > thinking
> > >> > that
> > >> > > > for
> > >> > > > > > now, removing the topic name from these requests and
> responses
> > >> are
> > >> > > out
> > >> > > > of
> > >> > > > > > scope.
> > >> > > > > >
> > >> > > > > > I have decided to just keep the change LeaderAndIsrResponse
> to
> > >> > remove
> > >> > > > the
> > >> > > > > > topic name, and have updated the KIP to reflect this
> change. I
> > >> have
> > >> > > > also
> > >> > > > > > mentioned the other requests and responses in future work.
> > >> > > > > >
> > >> > > > > > I'm hoping to start the voting process soon, so let me know
> if
> > >> > there
> > >> > > is
> > >> > > > > > anything else we should discuss.
> > >> > > > > >
> > >> > > > > > Thank you,
> > >> > > > > > Justine
> > >> > > > > >
> > >> > > > > > On Tue, Sep 15, 2020 at 3:57 PM Justine Olshan <
> > >> > jolshan@confluent.io
> > >> > > >
> > >> > > > > > wrote:
> > >> > > > > >
> > >> > > > > > > Hello again,
> > >> > > > > > > To follow up on some of the other comments:
> > >> > > > > > >
> > >> > > > > > > 10/11) We can remove the topic name from these
> > >> > requests/responses,
> > >> > > > and
> > >> > > > > > > that means we will just have to make a few internal
> changes
> > to
> > >> > make
> > >> > > > > > > partitions accessible by topic id and partition. I can
> > update
> > >> the
> > >> > > KIP
> > >> > > > > to
> > >> > > > > > > remove them unless anyone thinks they should stay.
> > >> > > > > > >
> > >> > > > > > > 12) Addressed in the previous email. I've updated the KIP
> to
> > >> > > include
> > >> > > > > > > tagged fields for the requests and responses. (More on
> that
> > >> > below)
> > >> > > > > > >
> > >> > > > > > > 13) I think part of the idea for including this
> information
> > >> is to
> > >> > > > > prepare
> > >> > > > > > > for future changes. Perhaps the directory structure might
> > >> change
> > >> > > from
> > >> > > > > > > topicName_partitionNumber to something like
> > >> > > topicID_partitionNumber.
> > >> > > > > Then
> > >> > > > > > > it would be useful to have the topic name in the file
> since
> > it
> > >> > > would
> > >> > > > > not
> > >> > > > > > be
> > >> > > > > > > in the directory structure. Supporting topic renames might
> > be
> > >> > > easier
> > >> > > > if
> > >> > > > > > the
> > >> > > > > > > other fields are included. Would there be any downsides to
> > >> > > including
> > >> > > > > this
> > >> > > > > > > information?
> > >> > > > > > >
> > >> > > > > > > 14)  Yes, we would need to copy the partition metadata
> file
> > in
> > >> > this
> > >> > > > > > > process. I've updated the KIP to include this.
> > >> > > > > > >
> > >> > > > > > > 15) I believe Lucas meant v1 and v2 here. He was referring
> > to
> > >> how
> > >> > > the
> > >> > > > > > > requests would fall under different IBP and meant that
> older
> > >> > > brokers
> > >> > > > > > would
> > >> > > > > > > have to use the older version of the request and the
> > existing
> > >> > topic
> > >> > > > > > > deletion process. At first, it seemed like tagged fields
> > would
> > >> > > > resolve
> > >> > > > > > > the IBP issue. However, we may need IBP for this request
> > after
> > >> > all
> > >> > > > > since
> > >> > > > > > > the controller handles the topic deletion differently
> > >> depending
> > >> > on
> > >> > > > the
> > >> > > > > > IBP
> > >> > > > > > > version. In an older version, we can't just send a
> > StopReplica
> > >> > > delete
> > >> > > > > the
> > >> > > > > > > topic immediately like we'd want to for this KIP.
> > >> > > > > > >
> > >> > > > > > > This makes me wonder if we want tagged fields on all the
> > >> requests
> > >> > > > after
> > >> > > > > > > all. Let me know your thoughts!
> > >> > > > > > >
> > >> > > > > > > Justine
> > >> > > > > > >
> > >> > > > > > > On Tue, Sep 15, 2020 at 1:03 PM Justine Olshan <
> > >> > > jolshan@confluent.io
> > >> > > > >
> > >> > > > > > > wrote:
> > >> > > > > > >
> > >> > > > > > >> Hi all,
> > >> > > > > > >> Jun brought up a good point in his last email about
> tagged
> > >> > fields,
> > >> > > > and
> > >> > > > > > >> I've updated the KIP to reflect that the changes to
> > requests
> > >> and
> > >> > > > > > responses
> > >> > > > > > >> will be in the form of tagged fields to avoid changing
> IBP.
> > >> > > > > > >>
> > >> > > > > > >> Jun: I plan on sending a followup email to address some
> of
> > >> the
> > >> > > other
> > >> > > > > > >> points.
> > >> > > > > > >>
> > >> > > > > > >> Thanks,
> > >> > > > > > >> Justine
> > >> > > > > > >>
> > >> > > > > > >> On Mon, Sep 14, 2020 at 4:25 PM Jun Rao <
> jun@confluent.io>
> > >> > wrote:
> > >> > > > > > >>
> > >> > > > > > >>> Hi, Justine,
> > >> > > > > > >>>
> > >> > > > > > >>> Thanks for the updated KIP. A few comments below.
> > >> > > > > > >>>
> > >> > > > > > >>> 10. LeaderAndIsr Response: Do we need the topic name?
> > >> > > > > > >>>
> > >> > > > > > >>> 11. For the changed request/response, other than
> > >> LeaderAndIsr,
> > >> > > > > > >>> UpdateMetadata, Metadata, do we need to include the
> topic
> > >> name?
> > >> > > > > > >>>
> > >> > > > > > >>> 12. It seems that upgrades don't require IBP. Does that
> > mean
> > >> > the
> > >> > > > new
> > >> > > > > > >>> fields
> > >> > > > > > >>> in all the request/response are added as tagged fields
> > >> without
> > >> > > > > bumping
> > >> > > > > > up
> > >> > > > > > >>> the request version? It would be useful to make that
> > clear.
> > >> > > > > > >>>
> > >> > > > > > >>> 13. Partition Metadata file: Do we need to include the
> > topic
> > >> > name
> > >> > > > and
> > >> > > > > > the
> > >> > > > > > >>> partition id since they are implied in the directory
> name?
> > >> > > > > > >>>
> > >> > > > > > >>> 14. In the JBOD mode, we support moving a partition's
> data
> > >> from
> > >> > > one
> > >> > > > > > disk
> > >> > > > > > >>> to
> > >> > > > > > >>> another. Will the new partition metadata file be copied
> > >> during
> > >> > > that
> > >> > > > > > >>> process?
> > >> > > > > > >>>
> > >> > > > > > >>> 15. The KIP says "Remove deleted topics from replicas by
> > >> > sending
> > >> > > > > > >>> StopReplicaRequest V2 for any topics which do not
> contain
> > a
> > >> > topic
> > >> > > > ID,
> > >> > > > > > and
> > >> > > > > > >>> V3 for any topics which do contain a topic ID.".
> However,
> > it
> > >> > > seems
> > >> > > > > the
> > >> > > > > > >>> updated controller will create all missing topic IDs
> first
> > >> > before
> > >> > > > > doing
> > >> > > > > > >>> other actions. So, is StopReplicaRequest V2 needed?
> > >> > > > > > >>>
> > >> > > > > > >>> Jun
> > >> > > > > > >>>
> > >> > > > > > >>> On Fri, Sep 11, 2020 at 10:31 AM John Roesler <
> > >> > > vvcephei@apache.org
> > >> > > > >
> > >> > > > > > >>> wrote:
> > >> > > > > > >>>
> > >> > > > > > >>> > Thanks, Justine!
> > >> > > > > > >>> >
> > >> > > > > > >>> > Your response seems compelling to me.
> > >> > > > > > >>> >
> > >> > > > > > >>> > -John
> > >> > > > > > >>> >
> > >> > > > > > >>> > On Fri, 2020-09-11 at 10:17 -0700, Justine Olshan
> wrote:
> > >> > > > > > >>> > > Hello all,
> > >> > > > > > >>> > > Thanks for continuing the discussion! I have a few
> > >> > responses
> > >> > > to
> > >> > > > > > your
> > >> > > > > > >>> > points.
> > >> > > > > > >>> > >
> > >> > > > > > >>> > > Tom: You are correct in that this KIP has not
> > mentioned
> > >> the
> > >> > > > > > >>> > > DeleteTopicsRequest. I think that this would be out
> of
> > >> > scope
> > >> > > > for
> > >> > > > > > >>> now, but
> > >> > > > > > >>> > > may be something worth adding in the future.
> > >> > > > > > >>> > >
> > >> > > > > > >>> > > John: We did consider sequence ids, but there are a
> > few
> > >> > > reasons
> > >> > > > > to
> > >> > > > > > >>> favor
> > >> > > > > > >>> > > UUIDs. There are several cases where topics from
> > >> different
> > >> > > > > clusters
> > >> > > > > > >>> may
> > >> > > > > > >>> > > interact now and in the future. For example, Mirror
> > >> Maker 2
> > >> > > may
> > >> > > > > > >>> benefit
> > >> > > > > > >>> > > from being able to detect when a cluster being
> > mirrored
> > >> is
> > >> > > > > deleted
> > >> > > > > > >>> and
> > >> > > > > > >>> > > recreated and globally unique identifiers would make
> > >> > > resolving
> > >> > > > > > issues
> > >> > > > > > >>> > > easier than sequence IDs which may collide between
> > >> > clusters.
> > >> > > > > > KIP-405
> > >> > > > > > >>> > > (tiered storage) will also benefit from globally
> > unique
> > >> IDs
> > >> > > as
> > >> > > > > > shared
> > >> > > > > > >>> > > buckets may be used between clusters.
> > >> > > > > > >>> > >
> > >> > > > > > >>> > > Globally unique IDs would also make functionality
> like
> > >> > moving
> > >> > > > > > topics
> > >> > > > > > >>> > > between disparate clusters easier in the future,
> > >> simplify
> > >> > any
> > >> > > > > > future
> > >> > > > > > >>> > > implementations of backups and restores, and more.
> In
> > >> > > general,
> > >> > > > > > >>> unique IDs
> > >> > > > > > >>> > > would ensure that the source cluster topics do not
> > >> conflict
> > >> > > > with
> > >> > > > > > the
> > >> > > > > > >>> > > destination cluster topics.
> > >> > > > > > >>> > >
> > >> > > > > > >>> > > If we were to use sequence ids, we would need
> > >> sufficiently
> > >> > > > large
> > >> > > > > > >>> cluster
> > >> > > > > > >>> > > ids to be stored with the topic identifiers or we
> run
> > >> the
> > >> > > risk
> > >> > > > of
> > >> > > > > > >>> > > collisions. This will give up any advantage in
> > >> compactness
> > >> > > that
> > >> > > > > > >>> sequence
> > >> > > > > > >>> > > numbers may bring. Given these advantages I think it
> > >> makes
> > >> > > > sense
> > >> > > > > to
> > >> > > > > > >>> use
> > >> > > > > > >>> > > UUIDs.
> > >> > > > > > >>> > >
> > >> > > > > > >>> > > Gokul: This is an interesting idea, but this is a
> > >> breaking
> > >> > > > > change.
> > >> > > > > > >>> Out of
> > >> > > > > > >>> > > scope for now, but maybe worth discussing in the
> > future.
> > >> > > > > > >>> > >
> > >> > > > > > >>> > > Hope this explains some of the decisions,
> > >> > > > > > >>> > >
> > >> > > > > > >>> > > Justine
> > >> > > > > > >>> > >
> > >> > > > > > >>> > >
> > >> > > > > > >>> > >
> > >> > > > > > >>> > > On Fri, Sep 11, 2020 at 8:27 AM Gokul Ramanan
> > >> Subramanian <
> > >> > > > > > >>> > > gokul2411s@gmail.com> wrote:
> > >> > > > > > >>> > >
> > >> > > > > > >>> > > > Hi.
> > >> > > > > > >>> > > >
> > >> > > > > > >>> > > > Thanks for the KIP.
> > >> > > > > > >>> > > >
> > >> > > > > > >>> > > > Have you thought about whether it makes sense to
> > >> support
> > >> > > > > > >>> authorizing a
> > >> > > > > > >>> > > > principal for a topic ID rather than a topic name
> to
> > >> > > achieve
> > >> > > > > > >>> tighter
> > >> > > > > > >>> > > > security?
> > >> > > > > > >>> > > >
> > >> > > > > > >>> > > > Or is the topic ID fundamentally an internal
> detail
> > >> > similar
> > >> > > > to
> > >> > > > > > >>> epochs
> > >> > > > > > >>> > used
> > >> > > > > > >>> > > > in a bunch of other places in Kafka?
> > >> > > > > > >>> > > >
> > >> > > > > > >>> > > > Thanks.
> > >> > > > > > >>> > > >
> > >> > > > > > >>> > > > On Fri, Sep 11, 2020 at 4:06 PM John Roesler <
> > >> > > > > > vvcephei@apache.org>
> > >> > > > > > >>> > wrote:
> > >> > > > > > >>> > > >
> > >> > > > > > >>> > > > > Hello Justine,
> > >> > > > > > >>> > > > >
> > >> > > > > > >>> > > > > Thanks for the KIP!
> > >> > > > > > >>> > > > >
> > >> > > > > > >>> > > > > I happen to have been confronted recently with
> the
> > >> need
> > >> > > to
> > >> > > > > keep
> > >> > > > > > >>> > track of
> > >> > > > > > >>> > > > a
> > >> > > > > > >>> > > > > large number of topics as compactly as
> possible. I
> > >> was
> > >> > > > going
> > >> > > > > to
> > >> > > > > > >>> come
> > >> > > > > > >>> > up
> > >> > > > > > >>> > > > > with some way to dictionary encode the topic
> names
> > >> as
> > >> > > > > integers,
> > >> > > > > > >>> but
> > >> > > > > > >>> > this
> > >> > > > > > >>> > > > > seems much better!
> > >> > > > > > >>> > > > >
> > >> > > > > > >>> > > > > Apologies if this has been raised before, but
> I’m
> > >> > > wondering
> > >> > > > > > >>> about the
> > >> > > > > > >>> > > > > choice of UUID vs sequence number for the ids.
> > >> > Typically,
> > >> > > > > I’ve
> > >> > > > > > >>> seen
> > >> > > > > > >>> > UUIDs
> > >> > > > > > >>> > > > > in two situations:
> > >> > > > > > >>> > > > > 1. When processes need to generate non-colliding
> > >> > > > identifiers
> > >> > > > > > >>> without
> > >> > > > > > >>> > > > > coordination.
> > >> > > > > > >>> > > > > 2. When the identifier needs to be “universally
> > >> > unique”;
> > >> > > > > I.e.,
> > >> > > > > > >>> the
> > >> > > > > > >>> > > > > identifier needs to distinguish the entity from
> > all
> > >> > other
> > >> > > > > > >>> entities
> > >> > > > > > >>> > that
> > >> > > > > > >>> > > > > could ever exist. This is useful in cases where
> > >> > entities
> > >> > > > from
> > >> > > > > > all
> > >> > > > > > >>> > kinds
> > >> > > > > > >>> > > > of
> > >> > > > > > >>> > > > > systems get mixed together, such as when dumping
> > >> logs
> > >> > > from
> > >> > > > > all
> > >> > > > > > >>> > processes
> > >> > > > > > >>> > > > in
> > >> > > > > > >>> > > > > a company into a common system.
> > >> > > > > > >>> > > > >
> > >> > > > > > >>> > > > > Maybe I’m being short-sighted, but it doesn’t
> seem
> > >> like
> > >> > > > > either
> > >> > > > > > >>> really
> > >> > > > > > >>> > > > > applies here. It seems like the brokers could
> and
> > >> would
> > >> > > > > achieve
> > >> > > > > > >>> > consensus
> > >> > > > > > >>> > > > > when creating a topic anyway, which is all
> that’s
> > >> > > required
> > >> > > > to
> > >> > > > > > >>> > generate
> > >> > > > > > >>> > > > > non-colliding sequence ids. For the second, as
> you
> > >> > > mention,
> > >> > > > > we
> > >> > > > > > >>> could
> > >> > > > > > >>> > > > assign
> > >> > > > > > >>> > > > > a UUID to the cluster as a whole, which would
> > render
> > >> > any
> > >> > > > > > resource
> > >> > > > > > >>> > scoped
> > >> > > > > > >>> > > > to
> > >> > > > > > >>> > > > > the broker universally unique as well.
> > >> > > > > > >>> > > > >
> > >> > > > > > >>> > > > > The reason I mention this is that, although a
> UUID
> > >> is
> > >> > way
> > >> > > > > more
> > >> > > > > > >>> > compact
> > >> > > > > > >>> > > > > than topic names, it’s still 16 bytes. In
> > contrast,
> > >> a
> > >> > > > 4-byte
> > >> > > > > > >>> integer
> > >> > > > > > >>> > > > > sequence id would give us 4 billion unique
> topics
> > >> per
> > >> > > > > cluster,
> > >> > > > > > >>> which
> > >> > > > > > >>> > > > seems
> > >> > > > > > >>> > > > > like enough ;)
> > >> > > > > > >>> > > > >
> > >> > > > > > >>> > > > > Considering the number of different times these
> > >> topic
> > >> > > > > > >>> identifiers are
> > >> > > > > > >>> > > > sent
> > >> > > > > > >>> > > > > over the wire or stored in memory, it seems like
> > it
> > >> > might
> > >> > > > be
> > >> > > > > > >>> worth
> > >> > > > > > >>> > the
> > >> > > > > > >>> > > > > additional 4x space savings.
> > >> > > > > > >>> > > > >
> > >> > > > > > >>> > > > > What do you think about this?
> > >> > > > > > >>> > > > >
> > >> > > > > > >>> > > > > Thanks,
> > >> > > > > > >>> > > > > John
> > >> > > > > > >>> > > > >
> > >> > > > > > >>> > > > > On Fri, Sep 11, 2020, at 03:20, Tom Bentley
> wrote:
> > >> > > > > > >>> > > > > > Hi Justine,
> > >> > > > > > >>> > > > > >
> > >> > > > > > >>> > > > > > This looks like a very welcome improvement.
> > >> Thanks!
> > >> > > > > > >>> > > > > >
> > >> > > > > > >>> > > > > > Maybe I missed it, but the KIP doesn't seem to
> > >> > mention
> > >> > > > > > changing
> > >> > > > > > >>> > > > > > DeleteTopicsRequest to identify the topic
> using
> > an
> > >> > id.
> > >> > > > > Maybe
> > >> > > > > > >>> > that's out
> > >> > > > > > >>> > > > > of
> > >> > > > > > >>> > > > > > scope, but DeleteTopicsRequest is not listed
> > among
> > >> > the
> > >> > > > > Future
> > >> > > > > > >>> Work
> > >> > > > > > >>> > APIs
> > >> > > > > > >>> > > > > > either.
> > >> > > > > > >>> > > > > >
> > >> > > > > > >>> > > > > > Kind regards,
> > >> > > > > > >>> > > > > >
> > >> > > > > > >>> > > > > > Tom
> > >> > > > > > >>> > > > > >
> > >> > > > > > >>> > > > > > On Thu, Sep 10, 2020 at 3:59 PM Satish
> Duggana <
> > >> > > > > > >>> > > > satish.duggana@gmail.com
> > >> > > > > > >>> > > > > > wrote:
> > >> > > > > > >>> > > > > >
> > >> > > > > > >>> > > > > > > Thanks Lucas/Justine for the nice KIP.
> > >> > > > > > >>> > > > > > >
> > >> > > > > > >>> > > > > > > It has several benefits which also include
> > >> > > simplifying
> > >> > > > > the
> > >> > > > > > >>> topic
> > >> > > > > > >>> > > > > > > deletion process by controller and logs
> > cleanup
> > >> by
> > >> > > > > brokers
> > >> > > > > > in
> > >> > > > > > >>> > corner
> > >> > > > > > >>> > > > > > > cases.
> > >> > > > > > >>> > > > > > >
> > >> > > > > > >>> > > > > > > Best,
> > >> > > > > > >>> > > > > > > Satish.
> > >> > > > > > >>> > > > > > >
> > >> > > > > > >>> > > > > > > On Wed, Sep 9, 2020 at 10:07 PM Justine
> > Olshan <
> > >> > > > > > >>> > jolshan@confluent.io
> > >> > > > > > >>> > > > > > > wrote:
> > >> > > > > > >>> > > > > > > > Hello all, it's been almost a year! I've
> > made
> > >> > some
> > >> > > > > > changes
> > >> > > > > > >>> to
> > >> > > > > > >>> > this
> > >> > > > > > >>> > > > > KIP
> > >> > > > > > >>> > > > > > > and hope to continue the discussion.
> > >> > > > > > >>> > > > > > > > One of the main changes I've added is now
> > the
> > >> > > > metadata
> > >> > > > > > >>> response
> > >> > > > > > >>> > > > will
> > >> > > > > > >>> > > > > > > include the topic ID (as Colin suggested).
> > >> Clients
> > >> > > can
> > >> > > > > > >>> obtain the
> > >> > > > > > >>> > > > > topicID
> > >> > > > > > >>> > > > > > > of a given topic through a TopicDescription.
> > The
> > >> > > > topicId
> > >> > > > > > will
> > >> > > > > > >>> > also be
> > >> > > > > > >>> > > > > > > included with the UpdateMetadata request.
> > >> > > > > > >>> > > > > > > > Let me know what you all think.
> > >> > > > > > >>> > > > > > > > Thank you,
> > >> > > > > > >>> > > > > > > > Justine
> > >> > > > > > >>> > > > > > > >
> > >> > > > > > >>> > > > > > > > On 2019/09/13 16:38:26, "Colin McCabe" <
> > >> > > > > > cmccabe@apache.org
> > >> > > > > > >>> >
> > >> > > > > > >>> > wrote:
> > >> > > > > > >>> > > > > > > > > Hi Lucas,
> > >> > > > > > >>> > > > > > > > >
> > >> > > > > > >>> > > > > > > > > Thanks for tackling this.  Topic IDs
> are a
> > >> > great
> > >> > > > > idea,
> > >> > > > > > >>> and
> > >> > > > > > >>> > this
> > >> > > > > > >>> > > > is
> > >> > > > > > >>> > > > > a
> > >> > > > > > >>> > > > > > > really good writeup.
> > >> > > > > > >>> > > > > > > > > For /brokers/topics/[topic], the schema
> > >> version
> > >> > > > > should
> > >> > > > > > be
> > >> > > > > > >>> > bumped
> > >> > > > > > >>> > > > to
> > >> > > > > > >>> > > > > > > version 3, rather than 2.  KIP-455 bumped
> the
> > >> > version
> > >> > > > of
> > >> > > > > > this
> > >> > > > > > >>> > znode
> > >> > > > > > >>> > > > to
> > >> > > > > > >>> > > > > 2
> > >> > > > > > >>> > > > > > > already :)
> > >> > > > > > >>> > > > > > > > > Given that we're going to be seeing
> these
> > >> > things
> > >> > > as
> > >> > > > > > >>> strings
> > >> > > > > > >>> > as
> > >> > > > > > >>> > > > lot
> > >> > > > > > >>> > > > > (in
> > >> > > > > > >>> > > > > > > logs, in ZooKeeper, on the command-line,
> > etc.),
> > >> > does
> > >> > > it
> > >> > > > > > make
> > >> > > > > > >>> > sense to
> > >> > > > > > >>> > > > > use
> > >> > > > > > >>> > > > > > > base64 when converting them to strings?
> > >> > > > > > >>> > > > > > > > > Here is an example of the hex
> > >> representation:
> > >> > > > > > >>> > > > > > > > > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> > >> > > > > > >>> > > > > > > > >
> > >> > > > > > >>> > > > > > > > > And here is an example in base64.
> > >> > > > > > >>> > > > > > > > > b8tRS7h4TJ2Vt43Dp85v2A
> > >> > > > > > >>> > > > > > > > >
> > >> > > > > > >>> > > > > > > > > The base64 version saves 15 letters (to
> be
> > >> > fair,
> > >> > > 4
> > >> > > > of
> > >> > > > > > >>> those
> > >> > > > > > >>> > were
> > >> > > > > > >>> > > > > > > dashes that we could have elided in the hex
> > >> > > > > > representation.)
> > >> > > > > > >>> > > > > > > > > Another thing to consider is that we
> > should
> > >> > > specify
> > >> > > > > > that
> > >> > > > > > >>> the
> > >> > > > > > >>> > > > > > > all-zeroes UUID is not a valid topic UUID.
> >  We
> > >> > can't
> > >> > > > use
> > >> > > > > > >>> null
> > >> > > > > > >>> > for
> > >> > > > > > >>> > > > this
> > >> > > > > > >>> > > > > > > because we can't pass a null UUID over the
> RPC
> > >> > > protocol
> > >> > > > > > >>> (there
> > >> > > > > > >>> > is no
> > >> > > > > > >>> > > > > > > special pattern for null, nor do we want to
> > >> waste
> > >> > > space
> > >> > > > > > >>> reserving
> > >> > > > > > >>> > > > such
> > >> > > > > > >>> > > > > a
> > >> > > > > > >>> > > > > > > pattern.)
> > >> > > > > > >>> > > > > > > > > Maybe I missed it, but did you describe
> > >> > > "migration
> > >> > > > > > of...
> > >> > > > > > >>> > existing
> > >> > > > > > >>> > > > > > > topic[s] without topic IDs" in detail in any
> > >> > section?
> > >> > > > It
> > >> > > > > > >>> seems
> > >> > > > > > >>> > like
> > >> > > > > > >>> > > > > when
> > >> > > > > > >>> > > > > > > the new controller becomes active, it should
> > >> just
> > >> > > > > generate
> > >> > > > > > >>> random
> > >> > > > > > >>> > > > > UUIDs for
> > >> > > > > > >>> > > > > > > these, and write the random UUIDs back to
> > >> > ZooKeeper.
> > >> > > > It
> > >> > > > > > >>> would be
> > >> > > > > > >>> > > > good
> > >> > > > > > >>> > > > > to
> > >> > > > > > >>> > > > > > > spell that out.  We should make it clear
> that
> > >> this
> > >> > > > > happens
> > >> > > > > > >>> > regardless
> > >> > > > > > >>> > > > > of
> > >> > > > > > >>> > > > > > > the inter-broker protocol version (it's a
> > >> > compatible
> > >> > > > > > change).
> > >> > > > > > >>> > > > > > > > > "LeaderAndIsrRequests including an
> > >> > > > is_every_partition
> > >> > > > > > >>> flag"
> > >> > > > > > >>> > > > seems a
> > >> > > > > > >>> > > > > > > bit wordy.  Can we just call these "full
> > >> > > > > > >>> LeaderAndIsrRequests"?
> > >> > > > > > >>> > Then
> > >> > > > > > >>> > > > > the
> > >> > > > > > >>> > > > > > > RPC field could be named "full".  Also, it
> > would
> > >> > > > probably
> > >> > > > > > be
> > >> > > > > > >>> > better
> > >> > > > > > >>> > > > > for the
> > >> > > > > > >>> > > > > > > RPC field to be an enum of { UNSPECIFIED,
> > >> > > INCREMENTAL,
> > >> > > > > FULL
> > >> > > > > > >>> }, so
> > >> > > > > > >>> > > > that
> > >> > > > > > >>> > > > > we
> > >> > > > > > >>> > > > > > > can cleanly handle old versions (by treating
> > >> them
> > >> > as
> > >> > > > > > >>> UNSPECIFIED)
> > >> > > > > > >>> > > > > > > > > In the LeaderAndIsrRequest section, you
> > >> write
> > >> > "A
> > >> > > > > final
> > >> > > > > > >>> > deletion
> > >> > > > > > >>> > > > > event
> > >> > > > > > >>> > > > > > > will be secheduled for X ms after the
> > >> > > > LeaderAndIsrRequest
> > >> > > > > > was
> > >> > > > > > >>> > first
> > >> > > > > > >>> > > > > > > received..."  I guess the X was a
> placeholder
> > >> that
> > >> > > you
> > >> > > > > > >>> intended
> > >> > > > > > >>> > to
> > >> > > > > > >>> > > > > replace
> > >> > > > > > >>> > > > > > > before posting? :)  In any case, this seems
> > like
> > >> > the
> > >> > > > kind
> > >> > > > > > of
> > >> > > > > > >>> > thing
> > >> > > > > > >>> > > > we'd
> > >> > > > > > >>> > > > > > > want a configuration for.  Let's describe
> that
> > >> > > > > > configuration
> > >> > > > > > >>> key
> > >> > > > > > >>> > > > > somewhere
> > >> > > > > > >>> > > > > > > in this KIP, including what its default
> value
> > >> is.
> > >> > > > > > >>> > > > > > > > > We should probably also log a bunch of
> > >> messages
> > >> > > at
> > >> > > > > WARN
> > >> > > > > > >>> level
> > >> > > > > > >>> > > > when
> > >> > > > > > >>> > > > > > > something is scheduled for deletion, as
> well.
> > >> > (Maybe
> > >> > > > > this
> > >> > > > > > >>> was
> > >> > > > > > >>> > > > > assumed, but
> > >> > > > > > >>> > > > > > > it would be good to mention it).
> > >> > > > > > >>> > > > > > > > > I feel like there are a few sections
> that
> > >> > should
> > >> > > be
> > >> > > > > > >>> moved to
> > >> > > > > > >>> > > > > "rejected
> > >> > > > > > >>> > > > > > > alternatives."  For example, in the
> > DeleteTopics
> > >> > > > section,
> > >> > > > > > >>> since
> > >> > > > > > >>> > we're
> > >> > > > > > >>> > > > > not
> > >> > > > > > >>> > > > > > > going to do option 1 or 2, these should be
> > moved
> > >> > into
> > >> > > > > > >>> "rejected
> > >> > > > > > >>> > > > > > > alternatives,"  rather than appearing
> inline.
> > >> > > Another
> > >> > > > > case
> > >> > > > > > >>> is
> > >> > > > > > >>> > the
> > >> > > > > > >>> > > > > "Should
> > >> > > > > > >>> > > > > > > we remove topic name from the protocol where
> > >> > > possible"
> > >> > > > > > >>> section.
> > >> > > > > > >>> > This
> > >> > > > > > >>> > > > > is
> > >> > > > > > >>> > > > > > > clearly discussing a design alternative that
> > >> we're
> > >> > > not
> > >> > > > > > >>> proposing
> > >> > > > > > >>> > to
> > >> > > > > > >>> > > > > > > implement: removing the topic name from
> those
> > >> > > > protocols.
> > >> > > > > > >>> > > > > > > > > Is it really necessary to have a new
> > >> > > > > > >>> > /admin/delete_topics_by_id
> > >> > > > > > >>> > > > > path
> > >> > > > > > >>> > > > > > > in ZooKeeper?  It seems like we don't really
> > >> need
> > >> > > this.
> > >> > > > > > >>> Whenever
> > >> > > > > > >>> > > > > there is
> > >> > > > > > >>> > > > > > > a new controller, we'll send out full
> > >> > > > > LeaderAndIsrRequests
> > >> > > > > > >>> which
> > >> > > > > > >>> > will
> > >> > > > > > >>> > > > > > > trigger the stale topics to be cleaned up.
> >  The
> > >> > > active
> > >> > > > > > >>> > controller
> > >> > > > > > >>> > > > will
> > >> > > > > > >>> > > > > > > also send the full LeaderAndIsrRequest to
> > >> brokers
> > >> > > that
> > >> > > > > are
> > >> > > > > > >>> just
> > >> > > > > > >>> > > > > starting
> > >> > > > > > >>> > > > > > > up.    So we don't really need this kind of
> > >> > two-phase
> > >> > > > > > commit
> > >> > > > > > >>> > (send
> > >> > > > > > >>> > > > out
> > >> > > > > > >>> > > > > > > StopReplicasRequest, get ACKs from all
> nodes,
> > >> > commit
> > >> > > by
> > >> > > > > > >>> removing
> > >> > > > > > >>> > > > > > > /admin/delete_topics node) any more.
> > >> > > > > > >>> > > > > > > > > You mention that FetchRequest will now
> > >> include
> > >> > > UUID
> > >> > > > > to
> > >> > > > > > >>> avoid
> > >> > > > > > >>> > > > issues
> > >> > > > > > >>> > > > > > > where requests are made to stale partitions.
> > >> > > However,
> > >> > > > > > >>> adding a
> > >> > > > > > >>> > UUID
> > >> > > > > > >>> > > > to
> > >> > > > > > >>> > > > > > > MetadataRequest is listed as future work,
> out
> > of
> > >> > > scope
> > >> > > > > for
> > >> > > > > > >>> this
> > >> > > > > > >>> > KIP.
> > >> > > > > > >>> > > > > How
> > >> > > > > > >>> > > > > > > will the client learn what the topic UUID
> is,
> > if
> > >> > the
> > >> > > > > > metadata
> > >> > > > > > >>> > > > response
> > >> > > > > > >>> > > > > > > doesn't include that information?  It seems
> > like
> > >> > > adding
> > >> > > > > the
> > >> > > > > > >>> UUID
> > >> > > > > > >>> > to
> > >> > > > > > >>> > > > > > > MetadataResponse would be an improvement
> here
> > >> that
> > >> > > > might
> > >> > > > > > not
> > >> > > > > > >>> be
> > >> > > > > > >>> > too
> > >> > > > > > >>> > > > > hard to
> > >> > > > > > >>> > > > > > > make.
> > >> > > > > > >>> > > > > > > > > best,
> > >> > > > > > >>> > > > > > > > > Colin
> > >> > > > > > >>> > > > > > > > >
> > >> > > > > > >>> > > > > > > > >
> > >> > > > > > >>> > > > > > > > > On Mon, Sep 9, 2019, at 17:48, Ryanne
> > Dolan
> > >> > > wrote:
> > >> > > > > > >>> > > > > > > > > > Lucas, this would be great. I've run
> > into
> > >> > > issues
> > >> > > > > with
> > >> > > > > > >>> > topics
> > >> > > > > > >>> > > > > being
> > >> > > > > > >>> > > > > > > > > > resurrected accidentally, since a
> client
> > >> > cannot
> > >> > > > > > easily
> > >> > > > > > >>> > > > > distinguish
> > >> > > > > > >>> > > > > > > between
> > >> > > > > > >>> > > > > > > > > > a deleted topic and a new topic with
> the
> > >> same
> > >> > > > name.
> > >> > > > > > I'd
> > >> > > > > > >>> > need
> > >> > > > > > >>> > > > the
> > >> > > > > > >>> > > > > ID
> > >> > > > > > >>> > > > > > > > > > accessible from the client to solve
> that
> > >> > issue,
> > >> > > > but
> > >> > > > > > >>> this
> > >> > > > > > >>> > is a
> > >> > > > > > >>> > > > > good
> > >> > > > > > >>> > > > > > > first
> > >> > > > > > >>> > > > > > > > > > step.
> > >> > > > > > >>> > > > > > > > > >
> > >> > > > > > >>> > > > > > > > > > Ryanne
> > >> > > > > > >>> > > > > > > > > >
> > >> > > > > > >>> > > > > > > > > > On Wed, Sep 4, 2019 at 1:41 PM Lucas
> > >> > > Bradstreet <
> > >> > > > > > >>> > > > > lucas@confluent.io>
> > >> > > > > > >>> > > > > > > wrote:
> > >> > > > > > >>> > > > > > > > > > > Hi all,
> > >> > > > > > >>> > > > > > > > > > >
> > >> > > > > > >>> > > > > > > > > > > I would like to kick off discussion
> of
> > >> > > KIP-516,
> > >> > > > > an
> > >> > > > > > >>> > > > > implementation
> > >> > > > > > >>> > > > > > > of topic
> > >> > > > > > >>> > > > > > > > > > > IDs for Kafka. Topic IDs aim to
> solve
> > >> topic
> > >> > > > > > >>> uniqueness
> > >> > > > > > >>> > > > > problems in
> > >> > > > > > >>> > > > > > > Kafka,
> > >> > > > > > >>> > > > > > > > > > > where referring to a topic by name
> > >> alone is
> > >> > > > > > >>> insufficient.
> > >> > > > > > >>> > > > Such
> > >> > > > > > >>> > > > > > > cases
> > >> > > > > > >>> > > > > > > > > > > include when a topic has been
> deleted
> > >> and
> > >> > > > > recreated
> > >> > > > > > >>> with
> > >> > > > > > >>> > the
> > >> > > > > > >>> > > > > same
> > >> > > > > > >>> > > > > > > name.
> > >> > > > > > >>> > > > > > > > > > > Unique identifiers will help
> simplify
> > >> and
> > >> > > > improve
> > >> > > > > > >>> Kafka's
> > >> > > > > > >>> > > > topic
> > >> > > > > > >>> > > > > > > deletion
> > >> > > > > > >>> > > > > > > > > > > process, as well as prevent cases
> > where
> > >> > > brokers
> > >> > > > > may
> > >> > > > > > >>> > > > incorrectly
> > >> > > > > > >>> > > > > > > interact
> > >> > > > > > >>> > > > > > > > > > > with stale versions of topics.
> > >> > > > > > >>> > > > > > > > > > >
> > >> > > > > > >>> > > > > > > > > > >
> > >> > > > > > >>> > > > > > > > > > >
> > >> > > > > > >>> > > >
> > >> > > > > > >>> >
> > >> > > > > > >>>
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > >> > > > > > >>> > > > > > > > > > > Looking forward to your thoughts.
> > >> > > > > > >>> > > > > > > > > > >
> > >> > > > > > >>> > > > > > > > > > > Lucas
> > >> > > > > > >>> > > > > > > > > > >
> > >> > > > > > >>> >
> > >> > > > > > >>> >
> > >> > > > > > >>>
> > >> > > > > > >>
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> >
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Tom Bentley <tb...@redhat.com>]

Hi Justine,

I know you started the vote thread, but on re-reading the KIP I noticed
that although the topic id is included in the MetadataResponse it's not
surfaced in the output from `kafka-topics.sh --describe`. Maybe that was
intentional because ids are intentionally not really something the user
should care deeply about, but it would also make life harder for anyone
debugging Kafka and this would likely get worse the more topic ids got
rolled out across the protocols, clients etc. It seems likely that
`kafka-topics.sh` will eventually need the ability to show the id of a
topic and perhaps find a topic name given an id. Is there any reason not to
implement that in this KIP?

Many thanks,

Tom

On Mon, Sep 21, 2020 at 9:54 PM Justine Olshan <jo...@confluent.io> wrote:

> Hi all,
>
> After thinking about it, I've decided to remove the topic name from the
> Fetch Request and Response after all. Since there are so many of these
> requests per second, it is worth removing the extra information. I've
> updated the KIP to reflect this change.
>
> Please let me know if there is anything else we should discuss before
> voting.
>
> Thank you,
> Justine
>
> On Fri, Sep 18, 2020 at 9:46 AM Justine Olshan <jo...@confluent.io>
> wrote:
>
> > Hi Jun,
> >
> > I see what you are saying. For now we can remove the extra information.
> > I'll leave the option to add more fields to the file in the future. The
> KIP
> > has been updated to reflect this change.
> >
> > Thanks,
> > Justine
> >
> > On Fri, Sep 18, 2020 at 8:46 AM Jun Rao <ju...@confluent.io> wrote:
> >
> >> Hi, Justine,
> >>
> >> Thanks for the reply.
> >>
> >> 13. If the log directory is the source of truth, it means that the
> >> redundant info in the metadata file will be ignored. Then the question
> is
> >> why do we need to put the redundant info in the metadata file now?
> >>
> >> Thanks,
> >>
> >> Jun
> >>
> >> On Thu, Sep 17, 2020 at 5:07 PM Justine Olshan <jo...@confluent.io>
> >> wrote:
> >>
> >> > Hi Jun,
> >> > Thanks for the quick response!
> >> >
> >> > 12. I've decided to bump up the versions on the requests and updated
> the
> >> > KIP. I think it's good we thoroughly discussed the options here, so we
> >> know
> >> > we made a good choice. :)
> >> >
> >> > 13. This is an interesting situation. I think if this does occur we
> >> should
> >> > give a warning. I agree that it's hard to know the source of truth for
> >> sure
> >> > since the directory or the file could be manually modified. I guess
> the
> >> > directory could be used as the source of truth. To be honest, I'm not
> >> > really sure what happens in kafka when the log directory is renamed
> >> > manually in such a way. I'm also wondering if the situation is
> >> recoverable
> >> > in this scenario.
> >> >
> >> > Thanks,
> >> > Justine
> >> >
> >> > On Thu, Sep 17, 2020 at 4:28 PM Jun Rao <ju...@confluent.io> wrote:
> >> >
> >> > > Hi, Justine,
> >> > >
> >> > > Thanks for the reply.
> >> > >
> >> > > 12. I don't have a strong preference either. However, if we need IBP
> >> > > anyway, maybe it's easier to just bump up the version for all inter
> >> > broker
> >> > > requests and add the topic id field as a regular field. A regular
> >> field
> >> > is
> >> > > a bit more concise in wire transfer than a flexible field.
> >> > >
> >> > > 13. The confusion that I was referring to is between the topic name
> >> and
> >> > > partition number between the log dir and the metadata file. For
> >> example,
> >> > if
> >> > > the log dir is topicA-1 and the metadata file in it has topicB and
> >> > > partition 0 (say due to a bug or manual modification), which one do
> we
> >> > use
> >> > > as the source of truth?
> >> > >
> >> > > Jun
> >> > >
> >> > > On Thu, Sep 17, 2020 at 3:43 PM Justine Olshan <
> jolshan@confluent.io>
> >> > > wrote:
> >> > >
> >> > > > Hi Jun,
> >> > > > Thanks for the comments.
> >> > > >
> >> > > > 12. I bumped the LeaderAndIsrRequest because I removed the topic
> >> name
> >> > > field
> >> > > > in the response. It may be possible to avoid bumping the version
> >> > without
> >> > > > that change, but I may be missing something.
> >> > > > I believe StopReplica is actually on version 3 now, but because
> >> > version 2
> >> > > > is flexible, I kept that listed as version 2 on the KIP page.
> >> However,
> >> > > you
> >> > > > may be right in that we may need to bump the version on
> StopReplica
> >> to
> >> > > deal
> >> > > > with deletion differently as mentioned above. I don't know if I
> >> have a
> >> > > big
> >> > > > preference over used tagged fields or not.
> >> > > >
> >> > > > 13. I was thinking that in the case where the file and the request
> >> > topic
> >> > > > ids don't match, it means that the broker's topic/the one in the
> >> file
> >> > has
> >> > > > been deleted. In that case, we would need to delete the old topic
> >> and
> >> > > start
> >> > > > receiving the new version. If the topic name were to change, but
> the
> >> > ids
> >> > > > still matched, the file would also need to update. Am I missing a
> >> case
> >> > > > where the file would be correct and not the request?
> >> > > >
> >> > > > Thanks,
> >> > > > Justine
> >> > > >
> >> > > > On Thu, Sep 17, 2020 at 3:18 PM Jun Rao <ju...@confluent.io> wrote:
> >> > > >
> >> > > > > Hi, Justine,
> >> > > > >
> >> > > > > Thanks for the reply. A couple of more comments below.
> >> > > > >
> >> > > > > 12. ListOffset and OffsetForLeader currently don't support
> >> flexible
> >> > > > fields.
> >> > > > > So, we have to bump up the version number and use IBP at least
> for
> >> > > these
> >> > > > > two requests. Note that it seems 2.7.0 will require IBP anyway
> >> > because
> >> > > of
> >> > > > > changes in KAFKA-10435. Also, it seems that the version for
> >> > > > > LeaderAndIsrRequest and StopReplica are bumped even though we
> only
> >> > > added
> >> > > > a
> >> > > > > tagged field. But since IBP is needed anyway, we may want to
> >> revisit
> >> > > the
> >> > > > > overall tagged field choice.
> >> > > > >
> >> > > > > 13. The only downside is the potential confusion on which one is
> >> the
> >> > > > source
> >> > > > > of truth if they don't match. Another option is to include those
> >> > fields
> >> > > > in
> >> > > > > the metadata file when we actually change the directory
> structure.
> >> > > > >
> >> > > > > Thanks,
> >> > > > >
> >> > > > > Jun
> >> > > > >
> >> > > > > On Thu, Sep 17, 2020 at 2:01 PM Justine Olshan <
> >> jolshan@confluent.io
> >> > >
> >> > > > > wrote:
> >> > > > >
> >> > > > > > Hello all,
> >> > > > > >
> >> > > > > > I've thought some more about removing the topic name field
> from
> >> > some
> >> > > of
> >> > > > > the
> >> > > > > > requests. On closer inspection of the requests/responses, it
> >> seems
> >> > > that
> >> > > > > the
> >> > > > > > internal changes would be much larger than I expected. Some
> >> > protocols
> >> > > > > > involve clients, so they would require changes too. I'm
> thinking
> >> > that
> >> > > > for
> >> > > > > > now, removing the topic name from these requests and responses
> >> are
> >> > > out
> >> > > > of
> >> > > > > > scope.
> >> > > > > >
> >> > > > > > I have decided to just keep the change LeaderAndIsrResponse to
> >> > remove
> >> > > > the
> >> > > > > > topic name, and have updated the KIP to reflect this change. I
> >> have
> >> > > > also
> >> > > > > > mentioned the other requests and responses in future work.
> >> > > > > >
> >> > > > > > I'm hoping to start the voting process soon, so let me know if
> >> > there
> >> > > is
> >> > > > > > anything else we should discuss.
> >> > > > > >
> >> > > > > > Thank you,
> >> > > > > > Justine
> >> > > > > >
> >> > > > > > On Tue, Sep 15, 2020 at 3:57 PM Justine Olshan <
> >> > jolshan@confluent.io
> >> > > >
> >> > > > > > wrote:
> >> > > > > >
> >> > > > > > > Hello again,
> >> > > > > > > To follow up on some of the other comments:
> >> > > > > > >
> >> > > > > > > 10/11) We can remove the topic name from these
> >> > requests/responses,
> >> > > > and
> >> > > > > > > that means we will just have to make a few internal changes
> to
> >> > make
> >> > > > > > > partitions accessible by topic id and partition. I can
> update
> >> the
> >> > > KIP
> >> > > > > to
> >> > > > > > > remove them unless anyone thinks they should stay.
> >> > > > > > >
> >> > > > > > > 12) Addressed in the previous email. I've updated the KIP to
> >> > > include
> >> > > > > > > tagged fields for the requests and responses. (More on that
> >> > below)
> >> > > > > > >
> >> > > > > > > 13) I think part of the idea for including this information
> >> is to
> >> > > > > prepare
> >> > > > > > > for future changes. Perhaps the directory structure might
> >> change
> >> > > from
> >> > > > > > > topicName_partitionNumber to something like
> >> > > topicID_partitionNumber.
> >> > > > > Then
> >> > > > > > > it would be useful to have the topic name in the file since
> it
> >> > > would
> >> > > > > not
> >> > > > > > be
> >> > > > > > > in the directory structure. Supporting topic renames might
> be
> >> > > easier
> >> > > > if
> >> > > > > > the
> >> > > > > > > other fields are included. Would there be any downsides to
> >> > > including
> >> > > > > this
> >> > > > > > > information?
> >> > > > > > >
> >> > > > > > > 14)  Yes, we would need to copy the partition metadata file
> in
> >> > this
> >> > > > > > > process. I've updated the KIP to include this.
> >> > > > > > >
> >> > > > > > > 15) I believe Lucas meant v1 and v2 here. He was referring
> to
> >> how
> >> > > the
> >> > > > > > > requests would fall under different IBP and meant that older
> >> > > brokers
> >> > > > > > would
> >> > > > > > > have to use the older version of the request and the
> existing
> >> > topic
> >> > > > > > > deletion process. At first, it seemed like tagged fields
> would
> >> > > > resolve
> >> > > > > > > the IBP issue. However, we may need IBP for this request
> after
> >> > all
> >> > > > > since
> >> > > > > > > the controller handles the topic deletion differently
> >> depending
> >> > on
> >> > > > the
> >> > > > > > IBP
> >> > > > > > > version. In an older version, we can't just send a
> StopReplica
> >> > > delete
> >> > > > > the
> >> > > > > > > topic immediately like we'd want to for this KIP.
> >> > > > > > >
> >> > > > > > > This makes me wonder if we want tagged fields on all the
> >> requests
> >> > > > after
> >> > > > > > > all. Let me know your thoughts!
> >> > > > > > >
> >> > > > > > > Justine
> >> > > > > > >
> >> > > > > > > On Tue, Sep 15, 2020 at 1:03 PM Justine Olshan <
> >> > > jolshan@confluent.io
> >> > > > >
> >> > > > > > > wrote:
> >> > > > > > >
> >> > > > > > >> Hi all,
> >> > > > > > >> Jun brought up a good point in his last email about tagged
> >> > fields,
> >> > > > and
> >> > > > > > >> I've updated the KIP to reflect that the changes to
> requests
> >> and
> >> > > > > > responses
> >> > > > > > >> will be in the form of tagged fields to avoid changing IBP.
> >> > > > > > >>
> >> > > > > > >> Jun: I plan on sending a followup email to address some of
> >> the
> >> > > other
> >> > > > > > >> points.
> >> > > > > > >>
> >> > > > > > >> Thanks,
> >> > > > > > >> Justine
> >> > > > > > >>
> >> > > > > > >> On Mon, Sep 14, 2020 at 4:25 PM Jun Rao <ju...@confluent.io>
> >> > wrote:
> >> > > > > > >>
> >> > > > > > >>> Hi, Justine,
> >> > > > > > >>>
> >> > > > > > >>> Thanks for the updated KIP. A few comments below.
> >> > > > > > >>>
> >> > > > > > >>> 10. LeaderAndIsr Response: Do we need the topic name?
> >> > > > > > >>>
> >> > > > > > >>> 11. For the changed request/response, other than
> >> LeaderAndIsr,
> >> > > > > > >>> UpdateMetadata, Metadata, do we need to include the topic
> >> name?
> >> > > > > > >>>
> >> > > > > > >>> 12. It seems that upgrades don't require IBP. Does that
> mean
> >> > the
> >> > > > new
> >> > > > > > >>> fields
> >> > > > > > >>> in all the request/response are added as tagged fields
> >> without
> >> > > > > bumping
> >> > > > > > up
> >> > > > > > >>> the request version? It would be useful to make that
> clear.
> >> > > > > > >>>
> >> > > > > > >>> 13. Partition Metadata file: Do we need to include the
> topic
> >> > name
> >> > > > and
> >> > > > > > the
> >> > > > > > >>> partition id since they are implied in the directory name?
> >> > > > > > >>>
> >> > > > > > >>> 14. In the JBOD mode, we support moving a partition's data
> >> from
> >> > > one
> >> > > > > > disk
> >> > > > > > >>> to
> >> > > > > > >>> another. Will the new partition metadata file be copied
> >> during
> >> > > that
> >> > > > > > >>> process?
> >> > > > > > >>>
> >> > > > > > >>> 15. The KIP says "Remove deleted topics from replicas by
> >> > sending
> >> > > > > > >>> StopReplicaRequest V2 for any topics which do not contain
> a
> >> > topic
> >> > > > ID,
> >> > > > > > and
> >> > > > > > >>> V3 for any topics which do contain a topic ID.". However,
> it
> >> > > seems
> >> > > > > the
> >> > > > > > >>> updated controller will create all missing topic IDs first
> >> > before
> >> > > > > doing
> >> > > > > > >>> other actions. So, is StopReplicaRequest V2 needed?
> >> > > > > > >>>
> >> > > > > > >>> Jun
> >> > > > > > >>>
> >> > > > > > >>> On Fri, Sep 11, 2020 at 10:31 AM John Roesler <
> >> > > vvcephei@apache.org
> >> > > > >
> >> > > > > > >>> wrote:
> >> > > > > > >>>
> >> > > > > > >>> > Thanks, Justine!
> >> > > > > > >>> >
> >> > > > > > >>> > Your response seems compelling to me.
> >> > > > > > >>> >
> >> > > > > > >>> > -John
> >> > > > > > >>> >
> >> > > > > > >>> > On Fri, 2020-09-11 at 10:17 -0700, Justine Olshan wrote:
> >> > > > > > >>> > > Hello all,
> >> > > > > > >>> > > Thanks for continuing the discussion! I have a few
> >> > responses
> >> > > to
> >> > > > > > your
> >> > > > > > >>> > points.
> >> > > > > > >>> > >
> >> > > > > > >>> > > Tom: You are correct in that this KIP has not
> mentioned
> >> the
> >> > > > > > >>> > > DeleteTopicsRequest. I think that this would be out of
> >> > scope
> >> > > > for
> >> > > > > > >>> now, but
> >> > > > > > >>> > > may be something worth adding in the future.
> >> > > > > > >>> > >
> >> > > > > > >>> > > John: We did consider sequence ids, but there are a
> few
> >> > > reasons
> >> > > > > to
> >> > > > > > >>> favor
> >> > > > > > >>> > > UUIDs. There are several cases where topics from
> >> different
> >> > > > > clusters
> >> > > > > > >>> may
> >> > > > > > >>> > > interact now and in the future. For example, Mirror
> >> Maker 2
> >> > > may
> >> > > > > > >>> benefit
> >> > > > > > >>> > > from being able to detect when a cluster being
> mirrored
> >> is
> >> > > > > deleted
> >> > > > > > >>> and
> >> > > > > > >>> > > recreated and globally unique identifiers would make
> >> > > resolving
> >> > > > > > issues
> >> > > > > > >>> > > easier than sequence IDs which may collide between
> >> > clusters.
> >> > > > > > KIP-405
> >> > > > > > >>> > > (tiered storage) will also benefit from globally
> unique
> >> IDs
> >> > > as
> >> > > > > > shared
> >> > > > > > >>> > > buckets may be used between clusters.
> >> > > > > > >>> > >
> >> > > > > > >>> > > Globally unique IDs would also make functionality like
> >> > moving
> >> > > > > > topics
> >> > > > > > >>> > > between disparate clusters easier in the future,
> >> simplify
> >> > any
> >> > > > > > future
> >> > > > > > >>> > > implementations of backups and restores, and more. In
> >> > > general,
> >> > > > > > >>> unique IDs
> >> > > > > > >>> > > would ensure that the source cluster topics do not
> >> conflict
> >> > > > with
> >> > > > > > the
> >> > > > > > >>> > > destination cluster topics.
> >> > > > > > >>> > >
> >> > > > > > >>> > > If we were to use sequence ids, we would need
> >> sufficiently
> >> > > > large
> >> > > > > > >>> cluster
> >> > > > > > >>> > > ids to be stored with the topic identifiers or we run
> >> the
> >> > > risk
> >> > > > of
> >> > > > > > >>> > > collisions. This will give up any advantage in
> >> compactness
> >> > > that
> >> > > > > > >>> sequence
> >> > > > > > >>> > > numbers may bring. Given these advantages I think it
> >> makes
> >> > > > sense
> >> > > > > to
> >> > > > > > >>> use
> >> > > > > > >>> > > UUIDs.
> >> > > > > > >>> > >
> >> > > > > > >>> > > Gokul: This is an interesting idea, but this is a
> >> breaking
> >> > > > > change.
> >> > > > > > >>> Out of
> >> > > > > > >>> > > scope for now, but maybe worth discussing in the
> future.
> >> > > > > > >>> > >
> >> > > > > > >>> > > Hope this explains some of the decisions,
> >> > > > > > >>> > >
> >> > > > > > >>> > > Justine
> >> > > > > > >>> > >
> >> > > > > > >>> > >
> >> > > > > > >>> > >
> >> > > > > > >>> > > On Fri, Sep 11, 2020 at 8:27 AM Gokul Ramanan
> >> Subramanian <
> >> > > > > > >>> > > gokul2411s@gmail.com> wrote:
> >> > > > > > >>> > >
> >> > > > > > >>> > > > Hi.
> >> > > > > > >>> > > >
> >> > > > > > >>> > > > Thanks for the KIP.
> >> > > > > > >>> > > >
> >> > > > > > >>> > > > Have you thought about whether it makes sense to
> >> support
> >> > > > > > >>> authorizing a
> >> > > > > > >>> > > > principal for a topic ID rather than a topic name to
> >> > > achieve
> >> > > > > > >>> tighter
> >> > > > > > >>> > > > security?
> >> > > > > > >>> > > >
> >> > > > > > >>> > > > Or is the topic ID fundamentally an internal detail
> >> > similar
> >> > > > to
> >> > > > > > >>> epochs
> >> > > > > > >>> > used
> >> > > > > > >>> > > > in a bunch of other places in Kafka?
> >> > > > > > >>> > > >
> >> > > > > > >>> > > > Thanks.
> >> > > > > > >>> > > >
> >> > > > > > >>> > > > On Fri, Sep 11, 2020 at 4:06 PM John Roesler <
> >> > > > > > vvcephei@apache.org>
> >> > > > > > >>> > wrote:
> >> > > > > > >>> > > >
> >> > > > > > >>> > > > > Hello Justine,
> >> > > > > > >>> > > > >
> >> > > > > > >>> > > > > Thanks for the KIP!
> >> > > > > > >>> > > > >
> >> > > > > > >>> > > > > I happen to have been confronted recently with the
> >> need
> >> > > to
> >> > > > > keep
> >> > > > > > >>> > track of
> >> > > > > > >>> > > > a
> >> > > > > > >>> > > > > large number of topics as compactly as possible. I
> >> was
> >> > > > going
> >> > > > > to
> >> > > > > > >>> come
> >> > > > > > >>> > up
> >> > > > > > >>> > > > > with some way to dictionary encode the topic names
> >> as
> >> > > > > integers,
> >> > > > > > >>> but
> >> > > > > > >>> > this
> >> > > > > > >>> > > > > seems much better!
> >> > > > > > >>> > > > >
> >> > > > > > >>> > > > > Apologies if this has been raised before, but I’m
> >> > > wondering
> >> > > > > > >>> about the
> >> > > > > > >>> > > > > choice of UUID vs sequence number for the ids.
> >> > Typically,
> >> > > > > I’ve
> >> > > > > > >>> seen
> >> > > > > > >>> > UUIDs
> >> > > > > > >>> > > > > in two situations:
> >> > > > > > >>> > > > > 1. When processes need to generate non-colliding
> >> > > > identifiers
> >> > > > > > >>> without
> >> > > > > > >>> > > > > coordination.
> >> > > > > > >>> > > > > 2. When the identifier needs to be “universally
> >> > unique”;
> >> > > > > I.e.,
> >> > > > > > >>> the
> >> > > > > > >>> > > > > identifier needs to distinguish the entity from
> all
> >> > other
> >> > > > > > >>> entities
> >> > > > > > >>> > that
> >> > > > > > >>> > > > > could ever exist. This is useful in cases where
> >> > entities
> >> > > > from
> >> > > > > > all
> >> > > > > > >>> > kinds
> >> > > > > > >>> > > > of
> >> > > > > > >>> > > > > systems get mixed together, such as when dumping
> >> logs
> >> > > from
> >> > > > > all
> >> > > > > > >>> > processes
> >> > > > > > >>> > > > in
> >> > > > > > >>> > > > > a company into a common system.
> >> > > > > > >>> > > > >
> >> > > > > > >>> > > > > Maybe I’m being short-sighted, but it doesn’t seem
> >> like
> >> > > > > either
> >> > > > > > >>> really
> >> > > > > > >>> > > > > applies here. It seems like the brokers could and
> >> would
> >> > > > > achieve
> >> > > > > > >>> > consensus
> >> > > > > > >>> > > > > when creating a topic anyway, which is all that’s
> >> > > required
> >> > > > to
> >> > > > > > >>> > generate
> >> > > > > > >>> > > > > non-colliding sequence ids. For the second, as you
> >> > > mention,
> >> > > > > we
> >> > > > > > >>> could
> >> > > > > > >>> > > > assign
> >> > > > > > >>> > > > > a UUID to the cluster as a whole, which would
> render
> >> > any
> >> > > > > > resource
> >> > > > > > >>> > scoped
> >> > > > > > >>> > > > to
> >> > > > > > >>> > > > > the broker universally unique as well.
> >> > > > > > >>> > > > >
> >> > > > > > >>> > > > > The reason I mention this is that, although a UUID
> >> is
> >> > way
> >> > > > > more
> >> > > > > > >>> > compact
> >> > > > > > >>> > > > > than topic names, it’s still 16 bytes. In
> contrast,
> >> a
> >> > > > 4-byte
> >> > > > > > >>> integer
> >> > > > > > >>> > > > > sequence id would give us 4 billion unique topics
> >> per
> >> > > > > cluster,
> >> > > > > > >>> which
> >> > > > > > >>> > > > seems
> >> > > > > > >>> > > > > like enough ;)
> >> > > > > > >>> > > > >
> >> > > > > > >>> > > > > Considering the number of different times these
> >> topic
> >> > > > > > >>> identifiers are
> >> > > > > > >>> > > > sent
> >> > > > > > >>> > > > > over the wire or stored in memory, it seems like
> it
> >> > might
> >> > > > be
> >> > > > > > >>> worth
> >> > > > > > >>> > the
> >> > > > > > >>> > > > > additional 4x space savings.
> >> > > > > > >>> > > > >
> >> > > > > > >>> > > > > What do you think about this?
> >> > > > > > >>> > > > >
> >> > > > > > >>> > > > > Thanks,
> >> > > > > > >>> > > > > John
> >> > > > > > >>> > > > >
> >> > > > > > >>> > > > > On Fri, Sep 11, 2020, at 03:20, Tom Bentley wrote:
> >> > > > > > >>> > > > > > Hi Justine,
> >> > > > > > >>> > > > > >
> >> > > > > > >>> > > > > > This looks like a very welcome improvement.
> >> Thanks!
> >> > > > > > >>> > > > > >
> >> > > > > > >>> > > > > > Maybe I missed it, but the KIP doesn't seem to
> >> > mention
> >> > > > > > changing
> >> > > > > > >>> > > > > > DeleteTopicsRequest to identify the topic using
> an
> >> > id.
> >> > > > > Maybe
> >> > > > > > >>> > that's out
> >> > > > > > >>> > > > > of
> >> > > > > > >>> > > > > > scope, but DeleteTopicsRequest is not listed
> among
> >> > the
> >> > > > > Future
> >> > > > > > >>> Work
> >> > > > > > >>> > APIs
> >> > > > > > >>> > > > > > either.
> >> > > > > > >>> > > > > >
> >> > > > > > >>> > > > > > Kind regards,
> >> > > > > > >>> > > > > >
> >> > > > > > >>> > > > > > Tom
> >> > > > > > >>> > > > > >
> >> > > > > > >>> > > > > > On Thu, Sep 10, 2020 at 3:59 PM Satish Duggana <
> >> > > > > > >>> > > > satish.duggana@gmail.com
> >> > > > > > >>> > > > > > wrote:
> >> > > > > > >>> > > > > >
> >> > > > > > >>> > > > > > > Thanks Lucas/Justine for the nice KIP.
> >> > > > > > >>> > > > > > >
> >> > > > > > >>> > > > > > > It has several benefits which also include
> >> > > simplifying
> >> > > > > the
> >> > > > > > >>> topic
> >> > > > > > >>> > > > > > > deletion process by controller and logs
> cleanup
> >> by
> >> > > > > brokers
> >> > > > > > in
> >> > > > > > >>> > corner
> >> > > > > > >>> > > > > > > cases.
> >> > > > > > >>> > > > > > >
> >> > > > > > >>> > > > > > > Best,
> >> > > > > > >>> > > > > > > Satish.
> >> > > > > > >>> > > > > > >
> >> > > > > > >>> > > > > > > On Wed, Sep 9, 2020 at 10:07 PM Justine
> Olshan <
> >> > > > > > >>> > jolshan@confluent.io
> >> > > > > > >>> > > > > > > wrote:
> >> > > > > > >>> > > > > > > > Hello all, it's been almost a year! I've
> made
> >> > some
> >> > > > > > changes
> >> > > > > > >>> to
> >> > > > > > >>> > this
> >> > > > > > >>> > > > > KIP
> >> > > > > > >>> > > > > > > and hope to continue the discussion.
> >> > > > > > >>> > > > > > > > One of the main changes I've added is now
> the
> >> > > > metadata
> >> > > > > > >>> response
> >> > > > > > >>> > > > will
> >> > > > > > >>> > > > > > > include the topic ID (as Colin suggested).
> >> Clients
> >> > > can
> >> > > > > > >>> obtain the
> >> > > > > > >>> > > > > topicID
> >> > > > > > >>> > > > > > > of a given topic through a TopicDescription.
> The
> >> > > > topicId
> >> > > > > > will
> >> > > > > > >>> > also be
> >> > > > > > >>> > > > > > > included with the UpdateMetadata request.
> >> > > > > > >>> > > > > > > > Let me know what you all think.
> >> > > > > > >>> > > > > > > > Thank you,
> >> > > > > > >>> > > > > > > > Justine
> >> > > > > > >>> > > > > > > >
> >> > > > > > >>> > > > > > > > On 2019/09/13 16:38:26, "Colin McCabe" <
> >> > > > > > cmccabe@apache.org
> >> > > > > > >>> >
> >> > > > > > >>> > wrote:
> >> > > > > > >>> > > > > > > > > Hi Lucas,
> >> > > > > > >>> > > > > > > > >
> >> > > > > > >>> > > > > > > > > Thanks for tackling this.  Topic IDs are a
> >> > great
> >> > > > > idea,
> >> > > > > > >>> and
> >> > > > > > >>> > this
> >> > > > > > >>> > > > is
> >> > > > > > >>> > > > > a
> >> > > > > > >>> > > > > > > really good writeup.
> >> > > > > > >>> > > > > > > > > For /brokers/topics/[topic], the schema
> >> version
> >> > > > > should
> >> > > > > > be
> >> > > > > > >>> > bumped
> >> > > > > > >>> > > > to
> >> > > > > > >>> > > > > > > version 3, rather than 2.  KIP-455 bumped the
> >> > version
> >> > > > of
> >> > > > > > this
> >> > > > > > >>> > znode
> >> > > > > > >>> > > > to
> >> > > > > > >>> > > > > 2
> >> > > > > > >>> > > > > > > already :)
> >> > > > > > >>> > > > > > > > > Given that we're going to be seeing these
> >> > things
> >> > > as
> >> > > > > > >>> strings
> >> > > > > > >>> > as
> >> > > > > > >>> > > > lot
> >> > > > > > >>> > > > > (in
> >> > > > > > >>> > > > > > > logs, in ZooKeeper, on the command-line,
> etc.),
> >> > does
> >> > > it
> >> > > > > > make
> >> > > > > > >>> > sense to
> >> > > > > > >>> > > > > use
> >> > > > > > >>> > > > > > > base64 when converting them to strings?
> >> > > > > > >>> > > > > > > > > Here is an example of the hex
> >> representation:
> >> > > > > > >>> > > > > > > > > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> >> > > > > > >>> > > > > > > > >
> >> > > > > > >>> > > > > > > > > And here is an example in base64.
> >> > > > > > >>> > > > > > > > > b8tRS7h4TJ2Vt43Dp85v2A
> >> > > > > > >>> > > > > > > > >
> >> > > > > > >>> > > > > > > > > The base64 version saves 15 letters (to be
> >> > fair,
> >> > > 4
> >> > > > of
> >> > > > > > >>> those
> >> > > > > > >>> > were
> >> > > > > > >>> > > > > > > dashes that we could have elided in the hex
> >> > > > > > representation.)
> >> > > > > > >>> > > > > > > > > Another thing to consider is that we
> should
> >> > > specify
> >> > > > > > that
> >> > > > > > >>> the
> >> > > > > > >>> > > > > > > all-zeroes UUID is not a valid topic UUID.
>  We
> >> > can't
> >> > > > use
> >> > > > > > >>> null
> >> > > > > > >>> > for
> >> > > > > > >>> > > > this
> >> > > > > > >>> > > > > > > because we can't pass a null UUID over the RPC
> >> > > protocol
> >> > > > > > >>> (there
> >> > > > > > >>> > is no
> >> > > > > > >>> > > > > > > special pattern for null, nor do we want to
> >> waste
> >> > > space
> >> > > > > > >>> reserving
> >> > > > > > >>> > > > such
> >> > > > > > >>> > > > > a
> >> > > > > > >>> > > > > > > pattern.)
> >> > > > > > >>> > > > > > > > > Maybe I missed it, but did you describe
> >> > > "migration
> >> > > > > > of...
> >> > > > > > >>> > existing
> >> > > > > > >>> > > > > > > topic[s] without topic IDs" in detail in any
> >> > section?
> >> > > > It
> >> > > > > > >>> seems
> >> > > > > > >>> > like
> >> > > > > > >>> > > > > when
> >> > > > > > >>> > > > > > > the new controller becomes active, it should
> >> just
> >> > > > > generate
> >> > > > > > >>> random
> >> > > > > > >>> > > > > UUIDs for
> >> > > > > > >>> > > > > > > these, and write the random UUIDs back to
> >> > ZooKeeper.
> >> > > > It
> >> > > > > > >>> would be
> >> > > > > > >>> > > > good
> >> > > > > > >>> > > > > to
> >> > > > > > >>> > > > > > > spell that out.  We should make it clear that
> >> this
> >> > > > > happens
> >> > > > > > >>> > regardless
> >> > > > > > >>> > > > > of
> >> > > > > > >>> > > > > > > the inter-broker protocol version (it's a
> >> > compatible
> >> > > > > > change).
> >> > > > > > >>> > > > > > > > > "LeaderAndIsrRequests including an
> >> > > > is_every_partition
> >> > > > > > >>> flag"
> >> > > > > > >>> > > > seems a
> >> > > > > > >>> > > > > > > bit wordy.  Can we just call these "full
> >> > > > > > >>> LeaderAndIsrRequests"?
> >> > > > > > >>> > Then
> >> > > > > > >>> > > > > the
> >> > > > > > >>> > > > > > > RPC field could be named "full".  Also, it
> would
> >> > > > probably
> >> > > > > > be
> >> > > > > > >>> > better
> >> > > > > > >>> > > > > for the
> >> > > > > > >>> > > > > > > RPC field to be an enum of { UNSPECIFIED,
> >> > > INCREMENTAL,
> >> > > > > FULL
> >> > > > > > >>> }, so
> >> > > > > > >>> > > > that
> >> > > > > > >>> > > > > we
> >> > > > > > >>> > > > > > > can cleanly handle old versions (by treating
> >> them
> >> > as
> >> > > > > > >>> UNSPECIFIED)
> >> > > > > > >>> > > > > > > > > In the LeaderAndIsrRequest section, you
> >> write
> >> > "A
> >> > > > > final
> >> > > > > > >>> > deletion
> >> > > > > > >>> > > > > event
> >> > > > > > >>> > > > > > > will be secheduled for X ms after the
> >> > > > LeaderAndIsrRequest
> >> > > > > > was
> >> > > > > > >>> > first
> >> > > > > > >>> > > > > > > received..."  I guess the X was a placeholder
> >> that
> >> > > you
> >> > > > > > >>> intended
> >> > > > > > >>> > to
> >> > > > > > >>> > > > > replace
> >> > > > > > >>> > > > > > > before posting? :)  In any case, this seems
> like
> >> > the
> >> > > > kind
> >> > > > > > of
> >> > > > > > >>> > thing
> >> > > > > > >>> > > > we'd
> >> > > > > > >>> > > > > > > want a configuration for.  Let's describe that
> >> > > > > > configuration
> >> > > > > > >>> key
> >> > > > > > >>> > > > > somewhere
> >> > > > > > >>> > > > > > > in this KIP, including what its default value
> >> is.
> >> > > > > > >>> > > > > > > > > We should probably also log a bunch of
> >> messages
> >> > > at
> >> > > > > WARN
> >> > > > > > >>> level
> >> > > > > > >>> > > > when
> >> > > > > > >>> > > > > > > something is scheduled for deletion, as well.
> >> > (Maybe
> >> > > > > this
> >> > > > > > >>> was
> >> > > > > > >>> > > > > assumed, but
> >> > > > > > >>> > > > > > > it would be good to mention it).
> >> > > > > > >>> > > > > > > > > I feel like there are a few sections that
> >> > should
> >> > > be
> >> > > > > > >>> moved to
> >> > > > > > >>> > > > > "rejected
> >> > > > > > >>> > > > > > > alternatives."  For example, in the
> DeleteTopics
> >> > > > section,
> >> > > > > > >>> since
> >> > > > > > >>> > we're
> >> > > > > > >>> > > > > not
> >> > > > > > >>> > > > > > > going to do option 1 or 2, these should be
> moved
> >> > into
> >> > > > > > >>> "rejected
> >> > > > > > >>> > > > > > > alternatives,"  rather than appearing inline.
> >> > > Another
> >> > > > > case
> >> > > > > > >>> is
> >> > > > > > >>> > the
> >> > > > > > >>> > > > > "Should
> >> > > > > > >>> > > > > > > we remove topic name from the protocol where
> >> > > possible"
> >> > > > > > >>> section.
> >> > > > > > >>> > This
> >> > > > > > >>> > > > > is
> >> > > > > > >>> > > > > > > clearly discussing a design alternative that
> >> we're
> >> > > not
> >> > > > > > >>> proposing
> >> > > > > > >>> > to
> >> > > > > > >>> > > > > > > implement: removing the topic name from those
> >> > > > protocols.
> >> > > > > > >>> > > > > > > > > Is it really necessary to have a new
> >> > > > > > >>> > /admin/delete_topics_by_id
> >> > > > > > >>> > > > > path
> >> > > > > > >>> > > > > > > in ZooKeeper?  It seems like we don't really
> >> need
> >> > > this.
> >> > > > > > >>> Whenever
> >> > > > > > >>> > > > > there is
> >> > > > > > >>> > > > > > > a new controller, we'll send out full
> >> > > > > LeaderAndIsrRequests
> >> > > > > > >>> which
> >> > > > > > >>> > will
> >> > > > > > >>> > > > > > > trigger the stale topics to be cleaned up.
>  The
> >> > > active
> >> > > > > > >>> > controller
> >> > > > > > >>> > > > will
> >> > > > > > >>> > > > > > > also send the full LeaderAndIsrRequest to
> >> brokers
> >> > > that
> >> > > > > are
> >> > > > > > >>> just
> >> > > > > > >>> > > > > starting
> >> > > > > > >>> > > > > > > up.    So we don't really need this kind of
> >> > two-phase
> >> > > > > > commit
> >> > > > > > >>> > (send
> >> > > > > > >>> > > > out
> >> > > > > > >>> > > > > > > StopReplicasRequest, get ACKs from all nodes,
> >> > commit
> >> > > by
> >> > > > > > >>> removing
> >> > > > > > >>> > > > > > > /admin/delete_topics node) any more.
> >> > > > > > >>> > > > > > > > > You mention that FetchRequest will now
> >> include
> >> > > UUID
> >> > > > > to
> >> > > > > > >>> avoid
> >> > > > > > >>> > > > issues
> >> > > > > > >>> > > > > > > where requests are made to stale partitions.
> >> > > However,
> >> > > > > > >>> adding a
> >> > > > > > >>> > UUID
> >> > > > > > >>> > > > to
> >> > > > > > >>> > > > > > > MetadataRequest is listed as future work, out
> of
> >> > > scope
> >> > > > > for
> >> > > > > > >>> this
> >> > > > > > >>> > KIP.
> >> > > > > > >>> > > > > How
> >> > > > > > >>> > > > > > > will the client learn what the topic UUID is,
> if
> >> > the
> >> > > > > > metadata
> >> > > > > > >>> > > > response
> >> > > > > > >>> > > > > > > doesn't include that information?  It seems
> like
> >> > > adding
> >> > > > > the
> >> > > > > > >>> UUID
> >> > > > > > >>> > to
> >> > > > > > >>> > > > > > > MetadataResponse would be an improvement here
> >> that
> >> > > > might
> >> > > > > > not
> >> > > > > > >>> be
> >> > > > > > >>> > too
> >> > > > > > >>> > > > > hard to
> >> > > > > > >>> > > > > > > make.
> >> > > > > > >>> > > > > > > > > best,
> >> > > > > > >>> > > > > > > > > Colin
> >> > > > > > >>> > > > > > > > >
> >> > > > > > >>> > > > > > > > >
> >> > > > > > >>> > > > > > > > > On Mon, Sep 9, 2019, at 17:48, Ryanne
> Dolan
> >> > > wrote:
> >> > > > > > >>> > > > > > > > > > Lucas, this would be great. I've run
> into
> >> > > issues
> >> > > > > with
> >> > > > > > >>> > topics
> >> > > > > > >>> > > > > being
> >> > > > > > >>> > > > > > > > > > resurrected accidentally, since a client
> >> > cannot
> >> > > > > > easily
> >> > > > > > >>> > > > > distinguish
> >> > > > > > >>> > > > > > > between
> >> > > > > > >>> > > > > > > > > > a deleted topic and a new topic with the
> >> same
> >> > > > name.
> >> > > > > > I'd
> >> > > > > > >>> > need
> >> > > > > > >>> > > > the
> >> > > > > > >>> > > > > ID
> >> > > > > > >>> > > > > > > > > > accessible from the client to solve that
> >> > issue,
> >> > > > but
> >> > > > > > >>> this
> >> > > > > > >>> > is a
> >> > > > > > >>> > > > > good
> >> > > > > > >>> > > > > > > first
> >> > > > > > >>> > > > > > > > > > step.
> >> > > > > > >>> > > > > > > > > >
> >> > > > > > >>> > > > > > > > > > Ryanne
> >> > > > > > >>> > > > > > > > > >
> >> > > > > > >>> > > > > > > > > > On Wed, Sep 4, 2019 at 1:41 PM Lucas
> >> > > Bradstreet <
> >> > > > > > >>> > > > > lucas@confluent.io>
> >> > > > > > >>> > > > > > > wrote:
> >> > > > > > >>> > > > > > > > > > > Hi all,
> >> > > > > > >>> > > > > > > > > > >
> >> > > > > > >>> > > > > > > > > > > I would like to kick off discussion of
> >> > > KIP-516,
> >> > > > > an
> >> > > > > > >>> > > > > implementation
> >> > > > > > >>> > > > > > > of topic
> >> > > > > > >>> > > > > > > > > > > IDs for Kafka. Topic IDs aim to solve
> >> topic
> >> > > > > > >>> uniqueness
> >> > > > > > >>> > > > > problems in
> >> > > > > > >>> > > > > > > Kafka,
> >> > > > > > >>> > > > > > > > > > > where referring to a topic by name
> >> alone is
> >> > > > > > >>> insufficient.
> >> > > > > > >>> > > > Such
> >> > > > > > >>> > > > > > > cases
> >> > > > > > >>> > > > > > > > > > > include when a topic has been deleted
> >> and
> >> > > > > recreated
> >> > > > > > >>> with
> >> > > > > > >>> > the
> >> > > > > > >>> > > > > same
> >> > > > > > >>> > > > > > > name.
> >> > > > > > >>> > > > > > > > > > > Unique identifiers will help simplify
> >> and
> >> > > > improve
> >> > > > > > >>> Kafka's
> >> > > > > > >>> > > > topic
> >> > > > > > >>> > > > > > > deletion
> >> > > > > > >>> > > > > > > > > > > process, as well as prevent cases
> where
> >> > > brokers
> >> > > > > may
> >> > > > > > >>> > > > incorrectly
> >> > > > > > >>> > > > > > > interact
> >> > > > > > >>> > > > > > > > > > > with stale versions of topics.
> >> > > > > > >>> > > > > > > > > > >
> >> > > > > > >>> > > > > > > > > > >
> >> > > > > > >>> > > > > > > > > > >
> >> > > > > > >>> > > >
> >> > > > > > >>> >
> >> > > > > > >>>
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> >> > > > > > >>> > > > > > > > > > > Looking forward to your thoughts.
> >> > > > > > >>> > > > > > > > > > >
> >> > > > > > >>> > > > > > > > > > > Lucas
> >> > > > > > >>> > > > > > > > > > >
> >> > > > > > >>> >
> >> > > > > > >>> >
> >> > > > > > >>>
> >> > > > > > >>
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Justine Olshan <jo...@confluent.io>]

Hi all,

After thinking about it, I've decided to remove the topic name from the
Fetch Request and Response after all. Since there are so many of these
requests per second, it is worth removing the extra information. I've
updated the KIP to reflect this change.

Please let me know if there is anything else we should discuss before
voting.

Thank you,
Justine

On Fri, Sep 18, 2020 at 9:46 AM Justine Olshan <jo...@confluent.io> wrote:

> Hi Jun,
>
> I see what you are saying. For now we can remove the extra information.
> I'll leave the option to add more fields to the file in the future. The KIP
> has been updated to reflect this change.
>
> Thanks,
> Justine
>
> On Fri, Sep 18, 2020 at 8:46 AM Jun Rao <ju...@confluent.io> wrote:
>
>> Hi, Justine,
>>
>> Thanks for the reply.
>>
>> 13. If the log directory is the source of truth, it means that the
>> redundant info in the metadata file will be ignored. Then the question is
>> why do we need to put the redundant info in the metadata file now?
>>
>> Thanks,
>>
>> Jun
>>
>> On Thu, Sep 17, 2020 at 5:07 PM Justine Olshan <jo...@confluent.io>
>> wrote:
>>
>> > Hi Jun,
>> > Thanks for the quick response!
>> >
>> > 12. I've decided to bump up the versions on the requests and updated the
>> > KIP. I think it's good we thoroughly discussed the options here, so we
>> know
>> > we made a good choice. :)
>> >
>> > 13. This is an interesting situation. I think if this does occur we
>> should
>> > give a warning. I agree that it's hard to know the source of truth for
>> sure
>> > since the directory or the file could be manually modified. I guess the
>> > directory could be used as the source of truth. To be honest, I'm not
>> > really sure what happens in kafka when the log directory is renamed
>> > manually in such a way. I'm also wondering if the situation is
>> recoverable
>> > in this scenario.
>> >
>> > Thanks,
>> > Justine
>> >
>> > On Thu, Sep 17, 2020 at 4:28 PM Jun Rao <ju...@confluent.io> wrote:
>> >
>> > > Hi, Justine,
>> > >
>> > > Thanks for the reply.
>> > >
>> > > 12. I don't have a strong preference either. However, if we need IBP
>> > > anyway, maybe it's easier to just bump up the version for all inter
>> > broker
>> > > requests and add the topic id field as a regular field. A regular
>> field
>> > is
>> > > a bit more concise in wire transfer than a flexible field.
>> > >
>> > > 13. The confusion that I was referring to is between the topic name
>> and
>> > > partition number between the log dir and the metadata file. For
>> example,
>> > if
>> > > the log dir is topicA-1 and the metadata file in it has topicB and
>> > > partition 0 (say due to a bug or manual modification), which one do we
>> > use
>> > > as the source of truth?
>> > >
>> > > Jun
>> > >
>> > > On Thu, Sep 17, 2020 at 3:43 PM Justine Olshan <jo...@confluent.io>
>> > > wrote:
>> > >
>> > > > Hi Jun,
>> > > > Thanks for the comments.
>> > > >
>> > > > 12. I bumped the LeaderAndIsrRequest because I removed the topic
>> name
>> > > field
>> > > > in the response. It may be possible to avoid bumping the version
>> > without
>> > > > that change, but I may be missing something.
>> > > > I believe StopReplica is actually on version 3 now, but because
>> > version 2
>> > > > is flexible, I kept that listed as version 2 on the KIP page.
>> However,
>> > > you
>> > > > may be right in that we may need to bump the version on StopReplica
>> to
>> > > deal
>> > > > with deletion differently as mentioned above. I don't know if I
>> have a
>> > > big
>> > > > preference over used tagged fields or not.
>> > > >
>> > > > 13. I was thinking that in the case where the file and the request
>> > topic
>> > > > ids don't match, it means that the broker's topic/the one in the
>> file
>> > has
>> > > > been deleted. In that case, we would need to delete the old topic
>> and
>> > > start
>> > > > receiving the new version. If the topic name were to change, but the
>> > ids
>> > > > still matched, the file would also need to update. Am I missing a
>> case
>> > > > where the file would be correct and not the request?
>> > > >
>> > > > Thanks,
>> > > > Justine
>> > > >
>> > > > On Thu, Sep 17, 2020 at 3:18 PM Jun Rao <ju...@confluent.io> wrote:
>> > > >
>> > > > > Hi, Justine,
>> > > > >
>> > > > > Thanks for the reply. A couple of more comments below.
>> > > > >
>> > > > > 12. ListOffset and OffsetForLeader currently don't support
>> flexible
>> > > > fields.
>> > > > > So, we have to bump up the version number and use IBP at least for
>> > > these
>> > > > > two requests. Note that it seems 2.7.0 will require IBP anyway
>> > because
>> > > of
>> > > > > changes in KAFKA-10435. Also, it seems that the version for
>> > > > > LeaderAndIsrRequest and StopReplica are bumped even though we only
>> > > added
>> > > > a
>> > > > > tagged field. But since IBP is needed anyway, we may want to
>> revisit
>> > > the
>> > > > > overall tagged field choice.
>> > > > >
>> > > > > 13. The only downside is the potential confusion on which one is
>> the
>> > > > source
>> > > > > of truth if they don't match. Another option is to include those
>> > fields
>> > > > in
>> > > > > the metadata file when we actually change the directory structure.
>> > > > >
>> > > > > Thanks,
>> > > > >
>> > > > > Jun
>> > > > >
>> > > > > On Thu, Sep 17, 2020 at 2:01 PM Justine Olshan <
>> jolshan@confluent.io
>> > >
>> > > > > wrote:
>> > > > >
>> > > > > > Hello all,
>> > > > > >
>> > > > > > I've thought some more about removing the topic name field from
>> > some
>> > > of
>> > > > > the
>> > > > > > requests. On closer inspection of the requests/responses, it
>> seems
>> > > that
>> > > > > the
>> > > > > > internal changes would be much larger than I expected. Some
>> > protocols
>> > > > > > involve clients, so they would require changes too. I'm thinking
>> > that
>> > > > for
>> > > > > > now, removing the topic name from these requests and responses
>> are
>> > > out
>> > > > of
>> > > > > > scope.
>> > > > > >
>> > > > > > I have decided to just keep the change LeaderAndIsrResponse to
>> > remove
>> > > > the
>> > > > > > topic name, and have updated the KIP to reflect this change. I
>> have
>> > > > also
>> > > > > > mentioned the other requests and responses in future work.
>> > > > > >
>> > > > > > I'm hoping to start the voting process soon, so let me know if
>> > there
>> > > is
>> > > > > > anything else we should discuss.
>> > > > > >
>> > > > > > Thank you,
>> > > > > > Justine
>> > > > > >
>> > > > > > On Tue, Sep 15, 2020 at 3:57 PM Justine Olshan <
>> > jolshan@confluent.io
>> > > >
>> > > > > > wrote:
>> > > > > >
>> > > > > > > Hello again,
>> > > > > > > To follow up on some of the other comments:
>> > > > > > >
>> > > > > > > 10/11) We can remove the topic name from these
>> > requests/responses,
>> > > > and
>> > > > > > > that means we will just have to make a few internal changes to
>> > make
>> > > > > > > partitions accessible by topic id and partition. I can update
>> the
>> > > KIP
>> > > > > to
>> > > > > > > remove them unless anyone thinks they should stay.
>> > > > > > >
>> > > > > > > 12) Addressed in the previous email. I've updated the KIP to
>> > > include
>> > > > > > > tagged fields for the requests and responses. (More on that
>> > below)
>> > > > > > >
>> > > > > > > 13) I think part of the idea for including this information
>> is to
>> > > > > prepare
>> > > > > > > for future changes. Perhaps the directory structure might
>> change
>> > > from
>> > > > > > > topicName_partitionNumber to something like
>> > > topicID_partitionNumber.
>> > > > > Then
>> > > > > > > it would be useful to have the topic name in the file since it
>> > > would
>> > > > > not
>> > > > > > be
>> > > > > > > in the directory structure. Supporting topic renames might be
>> > > easier
>> > > > if
>> > > > > > the
>> > > > > > > other fields are included. Would there be any downsides to
>> > > including
>> > > > > this
>> > > > > > > information?
>> > > > > > >
>> > > > > > > 14)  Yes, we would need to copy the partition metadata file in
>> > this
>> > > > > > > process. I've updated the KIP to include this.
>> > > > > > >
>> > > > > > > 15) I believe Lucas meant v1 and v2 here. He was referring to
>> how
>> > > the
>> > > > > > > requests would fall under different IBP and meant that older
>> > > brokers
>> > > > > > would
>> > > > > > > have to use the older version of the request and the existing
>> > topic
>> > > > > > > deletion process. At first, it seemed like tagged fields would
>> > > > resolve
>> > > > > > > the IBP issue. However, we may need IBP for this request after
>> > all
>> > > > > since
>> > > > > > > the controller handles the topic deletion differently
>> depending
>> > on
>> > > > the
>> > > > > > IBP
>> > > > > > > version. In an older version, we can't just send a StopReplica
>> > > delete
>> > > > > the
>> > > > > > > topic immediately like we'd want to for this KIP.
>> > > > > > >
>> > > > > > > This makes me wonder if we want tagged fields on all the
>> requests
>> > > > after
>> > > > > > > all. Let me know your thoughts!
>> > > > > > >
>> > > > > > > Justine
>> > > > > > >
>> > > > > > > On Tue, Sep 15, 2020 at 1:03 PM Justine Olshan <
>> > > jolshan@confluent.io
>> > > > >
>> > > > > > > wrote:
>> > > > > > >
>> > > > > > >> Hi all,
>> > > > > > >> Jun brought up a good point in his last email about tagged
>> > fields,
>> > > > and
>> > > > > > >> I've updated the KIP to reflect that the changes to requests
>> and
>> > > > > > responses
>> > > > > > >> will be in the form of tagged fields to avoid changing IBP.
>> > > > > > >>
>> > > > > > >> Jun: I plan on sending a followup email to address some of
>> the
>> > > other
>> > > > > > >> points.
>> > > > > > >>
>> > > > > > >> Thanks,
>> > > > > > >> Justine
>> > > > > > >>
>> > > > > > >> On Mon, Sep 14, 2020 at 4:25 PM Jun Rao <ju...@confluent.io>
>> > wrote:
>> > > > > > >>
>> > > > > > >>> Hi, Justine,
>> > > > > > >>>
>> > > > > > >>> Thanks for the updated KIP. A few comments below.
>> > > > > > >>>
>> > > > > > >>> 10. LeaderAndIsr Response: Do we need the topic name?
>> > > > > > >>>
>> > > > > > >>> 11. For the changed request/response, other than
>> LeaderAndIsr,
>> > > > > > >>> UpdateMetadata, Metadata, do we need to include the topic
>> name?
>> > > > > > >>>
>> > > > > > >>> 12. It seems that upgrades don't require IBP. Does that mean
>> > the
>> > > > new
>> > > > > > >>> fields
>> > > > > > >>> in all the request/response are added as tagged fields
>> without
>> > > > > bumping
>> > > > > > up
>> > > > > > >>> the request version? It would be useful to make that clear.
>> > > > > > >>>
>> > > > > > >>> 13. Partition Metadata file: Do we need to include the topic
>> > name
>> > > > and
>> > > > > > the
>> > > > > > >>> partition id since they are implied in the directory name?
>> > > > > > >>>
>> > > > > > >>> 14. In the JBOD mode, we support moving a partition's data
>> from
>> > > one
>> > > > > > disk
>> > > > > > >>> to
>> > > > > > >>> another. Will the new partition metadata file be copied
>> during
>> > > that
>> > > > > > >>> process?
>> > > > > > >>>
>> > > > > > >>> 15. The KIP says "Remove deleted topics from replicas by
>> > sending
>> > > > > > >>> StopReplicaRequest V2 for any topics which do not contain a
>> > topic
>> > > > ID,
>> > > > > > and
>> > > > > > >>> V3 for any topics which do contain a topic ID.". However, it
>> > > seems
>> > > > > the
>> > > > > > >>> updated controller will create all missing topic IDs first
>> > before
>> > > > > doing
>> > > > > > >>> other actions. So, is StopReplicaRequest V2 needed?
>> > > > > > >>>
>> > > > > > >>> Jun
>> > > > > > >>>
>> > > > > > >>> On Fri, Sep 11, 2020 at 10:31 AM John Roesler <
>> > > vvcephei@apache.org
>> > > > >
>> > > > > > >>> wrote:
>> > > > > > >>>
>> > > > > > >>> > Thanks, Justine!
>> > > > > > >>> >
>> > > > > > >>> > Your response seems compelling to me.
>> > > > > > >>> >
>> > > > > > >>> > -John
>> > > > > > >>> >
>> > > > > > >>> > On Fri, 2020-09-11 at 10:17 -0700, Justine Olshan wrote:
>> > > > > > >>> > > Hello all,
>> > > > > > >>> > > Thanks for continuing the discussion! I have a few
>> > responses
>> > > to
>> > > > > > your
>> > > > > > >>> > points.
>> > > > > > >>> > >
>> > > > > > >>> > > Tom: You are correct in that this KIP has not mentioned
>> the
>> > > > > > >>> > > DeleteTopicsRequest. I think that this would be out of
>> > scope
>> > > > for
>> > > > > > >>> now, but
>> > > > > > >>> > > may be something worth adding in the future.
>> > > > > > >>> > >
>> > > > > > >>> > > John: We did consider sequence ids, but there are a few
>> > > reasons
>> > > > > to
>> > > > > > >>> favor
>> > > > > > >>> > > UUIDs. There are several cases where topics from
>> different
>> > > > > clusters
>> > > > > > >>> may
>> > > > > > >>> > > interact now and in the future. For example, Mirror
>> Maker 2
>> > > may
>> > > > > > >>> benefit
>> > > > > > >>> > > from being able to detect when a cluster being mirrored
>> is
>> > > > > deleted
>> > > > > > >>> and
>> > > > > > >>> > > recreated and globally unique identifiers would make
>> > > resolving
>> > > > > > issues
>> > > > > > >>> > > easier than sequence IDs which may collide between
>> > clusters.
>> > > > > > KIP-405
>> > > > > > >>> > > (tiered storage) will also benefit from globally unique
>> IDs
>> > > as
>> > > > > > shared
>> > > > > > >>> > > buckets may be used between clusters.
>> > > > > > >>> > >
>> > > > > > >>> > > Globally unique IDs would also make functionality like
>> > moving
>> > > > > > topics
>> > > > > > >>> > > between disparate clusters easier in the future,
>> simplify
>> > any
>> > > > > > future
>> > > > > > >>> > > implementations of backups and restores, and more. In
>> > > general,
>> > > > > > >>> unique IDs
>> > > > > > >>> > > would ensure that the source cluster topics do not
>> conflict
>> > > > with
>> > > > > > the
>> > > > > > >>> > > destination cluster topics.
>> > > > > > >>> > >
>> > > > > > >>> > > If we were to use sequence ids, we would need
>> sufficiently
>> > > > large
>> > > > > > >>> cluster
>> > > > > > >>> > > ids to be stored with the topic identifiers or we run
>> the
>> > > risk
>> > > > of
>> > > > > > >>> > > collisions. This will give up any advantage in
>> compactness
>> > > that
>> > > > > > >>> sequence
>> > > > > > >>> > > numbers may bring. Given these advantages I think it
>> makes
>> > > > sense
>> > > > > to
>> > > > > > >>> use
>> > > > > > >>> > > UUIDs.
>> > > > > > >>> > >
>> > > > > > >>> > > Gokul: This is an interesting idea, but this is a
>> breaking
>> > > > > change.
>> > > > > > >>> Out of
>> > > > > > >>> > > scope for now, but maybe worth discussing in the future.
>> > > > > > >>> > >
>> > > > > > >>> > > Hope this explains some of the decisions,
>> > > > > > >>> > >
>> > > > > > >>> > > Justine
>> > > > > > >>> > >
>> > > > > > >>> > >
>> > > > > > >>> > >
>> > > > > > >>> > > On Fri, Sep 11, 2020 at 8:27 AM Gokul Ramanan
>> Subramanian <
>> > > > > > >>> > > gokul2411s@gmail.com> wrote:
>> > > > > > >>> > >
>> > > > > > >>> > > > Hi.
>> > > > > > >>> > > >
>> > > > > > >>> > > > Thanks for the KIP.
>> > > > > > >>> > > >
>> > > > > > >>> > > > Have you thought about whether it makes sense to
>> support
>> > > > > > >>> authorizing a
>> > > > > > >>> > > > principal for a topic ID rather than a topic name to
>> > > achieve
>> > > > > > >>> tighter
>> > > > > > >>> > > > security?
>> > > > > > >>> > > >
>> > > > > > >>> > > > Or is the topic ID fundamentally an internal detail
>> > similar
>> > > > to
>> > > > > > >>> epochs
>> > > > > > >>> > used
>> > > > > > >>> > > > in a bunch of other places in Kafka?
>> > > > > > >>> > > >
>> > > > > > >>> > > > Thanks.
>> > > > > > >>> > > >
>> > > > > > >>> > > > On Fri, Sep 11, 2020 at 4:06 PM John Roesler <
>> > > > > > vvcephei@apache.org>
>> > > > > > >>> > wrote:
>> > > > > > >>> > > >
>> > > > > > >>> > > > > Hello Justine,
>> > > > > > >>> > > > >
>> > > > > > >>> > > > > Thanks for the KIP!
>> > > > > > >>> > > > >
>> > > > > > >>> > > > > I happen to have been confronted recently with the
>> need
>> > > to
>> > > > > keep
>> > > > > > >>> > track of
>> > > > > > >>> > > > a
>> > > > > > >>> > > > > large number of topics as compactly as possible. I
>> was
>> > > > going
>> > > > > to
>> > > > > > >>> come
>> > > > > > >>> > up
>> > > > > > >>> > > > > with some way to dictionary encode the topic names
>> as
>> > > > > integers,
>> > > > > > >>> but
>> > > > > > >>> > this
>> > > > > > >>> > > > > seems much better!
>> > > > > > >>> > > > >
>> > > > > > >>> > > > > Apologies if this has been raised before, but I’m
>> > > wondering
>> > > > > > >>> about the
>> > > > > > >>> > > > > choice of UUID vs sequence number for the ids.
>> > Typically,
>> > > > > I’ve
>> > > > > > >>> seen
>> > > > > > >>> > UUIDs
>> > > > > > >>> > > > > in two situations:
>> > > > > > >>> > > > > 1. When processes need to generate non-colliding
>> > > > identifiers
>> > > > > > >>> without
>> > > > > > >>> > > > > coordination.
>> > > > > > >>> > > > > 2. When the identifier needs to be “universally
>> > unique”;
>> > > > > I.e.,
>> > > > > > >>> the
>> > > > > > >>> > > > > identifier needs to distinguish the entity from all
>> > other
>> > > > > > >>> entities
>> > > > > > >>> > that
>> > > > > > >>> > > > > could ever exist. This is useful in cases where
>> > entities
>> > > > from
>> > > > > > all
>> > > > > > >>> > kinds
>> > > > > > >>> > > > of
>> > > > > > >>> > > > > systems get mixed together, such as when dumping
>> logs
>> > > from
>> > > > > all
>> > > > > > >>> > processes
>> > > > > > >>> > > > in
>> > > > > > >>> > > > > a company into a common system.
>> > > > > > >>> > > > >
>> > > > > > >>> > > > > Maybe I’m being short-sighted, but it doesn’t seem
>> like
>> > > > > either
>> > > > > > >>> really
>> > > > > > >>> > > > > applies here. It seems like the brokers could and
>> would
>> > > > > achieve
>> > > > > > >>> > consensus
>> > > > > > >>> > > > > when creating a topic anyway, which is all that’s
>> > > required
>> > > > to
>> > > > > > >>> > generate
>> > > > > > >>> > > > > non-colliding sequence ids. For the second, as you
>> > > mention,
>> > > > > we
>> > > > > > >>> could
>> > > > > > >>> > > > assign
>> > > > > > >>> > > > > a UUID to the cluster as a whole, which would render
>> > any
>> > > > > > resource
>> > > > > > >>> > scoped
>> > > > > > >>> > > > to
>> > > > > > >>> > > > > the broker universally unique as well.
>> > > > > > >>> > > > >
>> > > > > > >>> > > > > The reason I mention this is that, although a UUID
>> is
>> > way
>> > > > > more
>> > > > > > >>> > compact
>> > > > > > >>> > > > > than topic names, it’s still 16 bytes. In contrast,
>> a
>> > > > 4-byte
>> > > > > > >>> integer
>> > > > > > >>> > > > > sequence id would give us 4 billion unique topics
>> per
>> > > > > cluster,
>> > > > > > >>> which
>> > > > > > >>> > > > seems
>> > > > > > >>> > > > > like enough ;)
>> > > > > > >>> > > > >
>> > > > > > >>> > > > > Considering the number of different times these
>> topic
>> > > > > > >>> identifiers are
>> > > > > > >>> > > > sent
>> > > > > > >>> > > > > over the wire or stored in memory, it seems like it
>> > might
>> > > > be
>> > > > > > >>> worth
>> > > > > > >>> > the
>> > > > > > >>> > > > > additional 4x space savings.
>> > > > > > >>> > > > >
>> > > > > > >>> > > > > What do you think about this?
>> > > > > > >>> > > > >
>> > > > > > >>> > > > > Thanks,
>> > > > > > >>> > > > > John
>> > > > > > >>> > > > >
>> > > > > > >>> > > > > On Fri, Sep 11, 2020, at 03:20, Tom Bentley wrote:
>> > > > > > >>> > > > > > Hi Justine,
>> > > > > > >>> > > > > >
>> > > > > > >>> > > > > > This looks like a very welcome improvement.
>> Thanks!
>> > > > > > >>> > > > > >
>> > > > > > >>> > > > > > Maybe I missed it, but the KIP doesn't seem to
>> > mention
>> > > > > > changing
>> > > > > > >>> > > > > > DeleteTopicsRequest to identify the topic using an
>> > id.
>> > > > > Maybe
>> > > > > > >>> > that's out
>> > > > > > >>> > > > > of
>> > > > > > >>> > > > > > scope, but DeleteTopicsRequest is not listed among
>> > the
>> > > > > Future
>> > > > > > >>> Work
>> > > > > > >>> > APIs
>> > > > > > >>> > > > > > either.
>> > > > > > >>> > > > > >
>> > > > > > >>> > > > > > Kind regards,
>> > > > > > >>> > > > > >
>> > > > > > >>> > > > > > Tom
>> > > > > > >>> > > > > >
>> > > > > > >>> > > > > > On Thu, Sep 10, 2020 at 3:59 PM Satish Duggana <
>> > > > > > >>> > > > satish.duggana@gmail.com
>> > > > > > >>> > > > > > wrote:
>> > > > > > >>> > > > > >
>> > > > > > >>> > > > > > > Thanks Lucas/Justine for the nice KIP.
>> > > > > > >>> > > > > > >
>> > > > > > >>> > > > > > > It has several benefits which also include
>> > > simplifying
>> > > > > the
>> > > > > > >>> topic
>> > > > > > >>> > > > > > > deletion process by controller and logs cleanup
>> by
>> > > > > brokers
>> > > > > > in
>> > > > > > >>> > corner
>> > > > > > >>> > > > > > > cases.
>> > > > > > >>> > > > > > >
>> > > > > > >>> > > > > > > Best,
>> > > > > > >>> > > > > > > Satish.
>> > > > > > >>> > > > > > >
>> > > > > > >>> > > > > > > On Wed, Sep 9, 2020 at 10:07 PM Justine Olshan <
>> > > > > > >>> > jolshan@confluent.io
>> > > > > > >>> > > > > > > wrote:
>> > > > > > >>> > > > > > > > Hello all, it's been almost a year! I've made
>> > some
>> > > > > > changes
>> > > > > > >>> to
>> > > > > > >>> > this
>> > > > > > >>> > > > > KIP
>> > > > > > >>> > > > > > > and hope to continue the discussion.
>> > > > > > >>> > > > > > > > One of the main changes I've added is now the
>> > > > metadata
>> > > > > > >>> response
>> > > > > > >>> > > > will
>> > > > > > >>> > > > > > > include the topic ID (as Colin suggested).
>> Clients
>> > > can
>> > > > > > >>> obtain the
>> > > > > > >>> > > > > topicID
>> > > > > > >>> > > > > > > of a given topic through a TopicDescription. The
>> > > > topicId
>> > > > > > will
>> > > > > > >>> > also be
>> > > > > > >>> > > > > > > included with the UpdateMetadata request.
>> > > > > > >>> > > > > > > > Let me know what you all think.
>> > > > > > >>> > > > > > > > Thank you,
>> > > > > > >>> > > > > > > > Justine
>> > > > > > >>> > > > > > > >
>> > > > > > >>> > > > > > > > On 2019/09/13 16:38:26, "Colin McCabe" <
>> > > > > > cmccabe@apache.org
>> > > > > > >>> >
>> > > > > > >>> > wrote:
>> > > > > > >>> > > > > > > > > Hi Lucas,
>> > > > > > >>> > > > > > > > >
>> > > > > > >>> > > > > > > > > Thanks for tackling this.  Topic IDs are a
>> > great
>> > > > > idea,
>> > > > > > >>> and
>> > > > > > >>> > this
>> > > > > > >>> > > > is
>> > > > > > >>> > > > > a
>> > > > > > >>> > > > > > > really good writeup.
>> > > > > > >>> > > > > > > > > For /brokers/topics/[topic], the schema
>> version
>> > > > > should
>> > > > > > be
>> > > > > > >>> > bumped
>> > > > > > >>> > > > to
>> > > > > > >>> > > > > > > version 3, rather than 2.  KIP-455 bumped the
>> > version
>> > > > of
>> > > > > > this
>> > > > > > >>> > znode
>> > > > > > >>> > > > to
>> > > > > > >>> > > > > 2
>> > > > > > >>> > > > > > > already :)
>> > > > > > >>> > > > > > > > > Given that we're going to be seeing these
>> > things
>> > > as
>> > > > > > >>> strings
>> > > > > > >>> > as
>> > > > > > >>> > > > lot
>> > > > > > >>> > > > > (in
>> > > > > > >>> > > > > > > logs, in ZooKeeper, on the command-line, etc.),
>> > does
>> > > it
>> > > > > > make
>> > > > > > >>> > sense to
>> > > > > > >>> > > > > use
>> > > > > > >>> > > > > > > base64 when converting them to strings?
>> > > > > > >>> > > > > > > > > Here is an example of the hex
>> representation:
>> > > > > > >>> > > > > > > > > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
>> > > > > > >>> > > > > > > > >
>> > > > > > >>> > > > > > > > > And here is an example in base64.
>> > > > > > >>> > > > > > > > > b8tRS7h4TJ2Vt43Dp85v2A
>> > > > > > >>> > > > > > > > >
>> > > > > > >>> > > > > > > > > The base64 version saves 15 letters (to be
>> > fair,
>> > > 4
>> > > > of
>> > > > > > >>> those
>> > > > > > >>> > were
>> > > > > > >>> > > > > > > dashes that we could have elided in the hex
>> > > > > > representation.)
>> > > > > > >>> > > > > > > > > Another thing to consider is that we should
>> > > specify
>> > > > > > that
>> > > > > > >>> the
>> > > > > > >>> > > > > > > all-zeroes UUID is not a valid topic UUID.   We
>> > can't
>> > > > use
>> > > > > > >>> null
>> > > > > > >>> > for
>> > > > > > >>> > > > this
>> > > > > > >>> > > > > > > because we can't pass a null UUID over the RPC
>> > > protocol
>> > > > > > >>> (there
>> > > > > > >>> > is no
>> > > > > > >>> > > > > > > special pattern for null, nor do we want to
>> waste
>> > > space
>> > > > > > >>> reserving
>> > > > > > >>> > > > such
>> > > > > > >>> > > > > a
>> > > > > > >>> > > > > > > pattern.)
>> > > > > > >>> > > > > > > > > Maybe I missed it, but did you describe
>> > > "migration
>> > > > > > of...
>> > > > > > >>> > existing
>> > > > > > >>> > > > > > > topic[s] without topic IDs" in detail in any
>> > section?
>> > > > It
>> > > > > > >>> seems
>> > > > > > >>> > like
>> > > > > > >>> > > > > when
>> > > > > > >>> > > > > > > the new controller becomes active, it should
>> just
>> > > > > generate
>> > > > > > >>> random
>> > > > > > >>> > > > > UUIDs for
>> > > > > > >>> > > > > > > these, and write the random UUIDs back to
>> > ZooKeeper.
>> > > > It
>> > > > > > >>> would be
>> > > > > > >>> > > > good
>> > > > > > >>> > > > > to
>> > > > > > >>> > > > > > > spell that out.  We should make it clear that
>> this
>> > > > > happens
>> > > > > > >>> > regardless
>> > > > > > >>> > > > > of
>> > > > > > >>> > > > > > > the inter-broker protocol version (it's a
>> > compatible
>> > > > > > change).
>> > > > > > >>> > > > > > > > > "LeaderAndIsrRequests including an
>> > > > is_every_partition
>> > > > > > >>> flag"
>> > > > > > >>> > > > seems a
>> > > > > > >>> > > > > > > bit wordy.  Can we just call these "full
>> > > > > > >>> LeaderAndIsrRequests"?
>> > > > > > >>> > Then
>> > > > > > >>> > > > > the
>> > > > > > >>> > > > > > > RPC field could be named "full".  Also, it would
>> > > > probably
>> > > > > > be
>> > > > > > >>> > better
>> > > > > > >>> > > > > for the
>> > > > > > >>> > > > > > > RPC field to be an enum of { UNSPECIFIED,
>> > > INCREMENTAL,
>> > > > > FULL
>> > > > > > >>> }, so
>> > > > > > >>> > > > that
>> > > > > > >>> > > > > we
>> > > > > > >>> > > > > > > can cleanly handle old versions (by treating
>> them
>> > as
>> > > > > > >>> UNSPECIFIED)
>> > > > > > >>> > > > > > > > > In the LeaderAndIsrRequest section, you
>> write
>> > "A
>> > > > > final
>> > > > > > >>> > deletion
>> > > > > > >>> > > > > event
>> > > > > > >>> > > > > > > will be secheduled for X ms after the
>> > > > LeaderAndIsrRequest
>> > > > > > was
>> > > > > > >>> > first
>> > > > > > >>> > > > > > > received..."  I guess the X was a placeholder
>> that
>> > > you
>> > > > > > >>> intended
>> > > > > > >>> > to
>> > > > > > >>> > > > > replace
>> > > > > > >>> > > > > > > before posting? :)  In any case, this seems like
>> > the
>> > > > kind
>> > > > > > of
>> > > > > > >>> > thing
>> > > > > > >>> > > > we'd
>> > > > > > >>> > > > > > > want a configuration for.  Let's describe that
>> > > > > > configuration
>> > > > > > >>> key
>> > > > > > >>> > > > > somewhere
>> > > > > > >>> > > > > > > in this KIP, including what its default value
>> is.
>> > > > > > >>> > > > > > > > > We should probably also log a bunch of
>> messages
>> > > at
>> > > > > WARN
>> > > > > > >>> level
>> > > > > > >>> > > > when
>> > > > > > >>> > > > > > > something is scheduled for deletion, as well.
>> > (Maybe
>> > > > > this
>> > > > > > >>> was
>> > > > > > >>> > > > > assumed, but
>> > > > > > >>> > > > > > > it would be good to mention it).
>> > > > > > >>> > > > > > > > > I feel like there are a few sections that
>> > should
>> > > be
>> > > > > > >>> moved to
>> > > > > > >>> > > > > "rejected
>> > > > > > >>> > > > > > > alternatives."  For example, in the DeleteTopics
>> > > > section,
>> > > > > > >>> since
>> > > > > > >>> > we're
>> > > > > > >>> > > > > not
>> > > > > > >>> > > > > > > going to do option 1 or 2, these should be moved
>> > into
>> > > > > > >>> "rejected
>> > > > > > >>> > > > > > > alternatives,"  rather than appearing inline.
>> > > Another
>> > > > > case
>> > > > > > >>> is
>> > > > > > >>> > the
>> > > > > > >>> > > > > "Should
>> > > > > > >>> > > > > > > we remove topic name from the protocol where
>> > > possible"
>> > > > > > >>> section.
>> > > > > > >>> > This
>> > > > > > >>> > > > > is
>> > > > > > >>> > > > > > > clearly discussing a design alternative that
>> we're
>> > > not
>> > > > > > >>> proposing
>> > > > > > >>> > to
>> > > > > > >>> > > > > > > implement: removing the topic name from those
>> > > > protocols.
>> > > > > > >>> > > > > > > > > Is it really necessary to have a new
>> > > > > > >>> > /admin/delete_topics_by_id
>> > > > > > >>> > > > > path
>> > > > > > >>> > > > > > > in ZooKeeper?  It seems like we don't really
>> need
>> > > this.
>> > > > > > >>> Whenever
>> > > > > > >>> > > > > there is
>> > > > > > >>> > > > > > > a new controller, we'll send out full
>> > > > > LeaderAndIsrRequests
>> > > > > > >>> which
>> > > > > > >>> > will
>> > > > > > >>> > > > > > > trigger the stale topics to be cleaned up.   The
>> > > active
>> > > > > > >>> > controller
>> > > > > > >>> > > > will
>> > > > > > >>> > > > > > > also send the full LeaderAndIsrRequest to
>> brokers
>> > > that
>> > > > > are
>> > > > > > >>> just
>> > > > > > >>> > > > > starting
>> > > > > > >>> > > > > > > up.    So we don't really need this kind of
>> > two-phase
>> > > > > > commit
>> > > > > > >>> > (send
>> > > > > > >>> > > > out
>> > > > > > >>> > > > > > > StopReplicasRequest, get ACKs from all nodes,
>> > commit
>> > > by
>> > > > > > >>> removing
>> > > > > > >>> > > > > > > /admin/delete_topics node) any more.
>> > > > > > >>> > > > > > > > > You mention that FetchRequest will now
>> include
>> > > UUID
>> > > > > to
>> > > > > > >>> avoid
>> > > > > > >>> > > > issues
>> > > > > > >>> > > > > > > where requests are made to stale partitions.
>> > > However,
>> > > > > > >>> adding a
>> > > > > > >>> > UUID
>> > > > > > >>> > > > to
>> > > > > > >>> > > > > > > MetadataRequest is listed as future work, out of
>> > > scope
>> > > > > for
>> > > > > > >>> this
>> > > > > > >>> > KIP.
>> > > > > > >>> > > > > How
>> > > > > > >>> > > > > > > will the client learn what the topic UUID is, if
>> > the
>> > > > > > metadata
>> > > > > > >>> > > > response
>> > > > > > >>> > > > > > > doesn't include that information?  It seems like
>> > > adding
>> > > > > the
>> > > > > > >>> UUID
>> > > > > > >>> > to
>> > > > > > >>> > > > > > > MetadataResponse would be an improvement here
>> that
>> > > > might
>> > > > > > not
>> > > > > > >>> be
>> > > > > > >>> > too
>> > > > > > >>> > > > > hard to
>> > > > > > >>> > > > > > > make.
>> > > > > > >>> > > > > > > > > best,
>> > > > > > >>> > > > > > > > > Colin
>> > > > > > >>> > > > > > > > >
>> > > > > > >>> > > > > > > > >
>> > > > > > >>> > > > > > > > > On Mon, Sep 9, 2019, at 17:48, Ryanne Dolan
>> > > wrote:
>> > > > > > >>> > > > > > > > > > Lucas, this would be great. I've run into
>> > > issues
>> > > > > with
>> > > > > > >>> > topics
>> > > > > > >>> > > > > being
>> > > > > > >>> > > > > > > > > > resurrected accidentally, since a client
>> > cannot
>> > > > > > easily
>> > > > > > >>> > > > > distinguish
>> > > > > > >>> > > > > > > between
>> > > > > > >>> > > > > > > > > > a deleted topic and a new topic with the
>> same
>> > > > name.
>> > > > > > I'd
>> > > > > > >>> > need
>> > > > > > >>> > > > the
>> > > > > > >>> > > > > ID
>> > > > > > >>> > > > > > > > > > accessible from the client to solve that
>> > issue,
>> > > > but
>> > > > > > >>> this
>> > > > > > >>> > is a
>> > > > > > >>> > > > > good
>> > > > > > >>> > > > > > > first
>> > > > > > >>> > > > > > > > > > step.
>> > > > > > >>> > > > > > > > > >
>> > > > > > >>> > > > > > > > > > Ryanne
>> > > > > > >>> > > > > > > > > >
>> > > > > > >>> > > > > > > > > > On Wed, Sep 4, 2019 at 1:41 PM Lucas
>> > > Bradstreet <
>> > > > > > >>> > > > > lucas@confluent.io>
>> > > > > > >>> > > > > > > wrote:
>> > > > > > >>> > > > > > > > > > > Hi all,
>> > > > > > >>> > > > > > > > > > >
>> > > > > > >>> > > > > > > > > > > I would like to kick off discussion of
>> > > KIP-516,
>> > > > > an
>> > > > > > >>> > > > > implementation
>> > > > > > >>> > > > > > > of topic
>> > > > > > >>> > > > > > > > > > > IDs for Kafka. Topic IDs aim to solve
>> topic
>> > > > > > >>> uniqueness
>> > > > > > >>> > > > > problems in
>> > > > > > >>> > > > > > > Kafka,
>> > > > > > >>> > > > > > > > > > > where referring to a topic by name
>> alone is
>> > > > > > >>> insufficient.
>> > > > > > >>> > > > Such
>> > > > > > >>> > > > > > > cases
>> > > > > > >>> > > > > > > > > > > include when a topic has been deleted
>> and
>> > > > > recreated
>> > > > > > >>> with
>> > > > > > >>> > the
>> > > > > > >>> > > > > same
>> > > > > > >>> > > > > > > name.
>> > > > > > >>> > > > > > > > > > > Unique identifiers will help simplify
>> and
>> > > > improve
>> > > > > > >>> Kafka's
>> > > > > > >>> > > > topic
>> > > > > > >>> > > > > > > deletion
>> > > > > > >>> > > > > > > > > > > process, as well as prevent cases where
>> > > brokers
>> > > > > may
>> > > > > > >>> > > > incorrectly
>> > > > > > >>> > > > > > > interact
>> > > > > > >>> > > > > > > > > > > with stale versions of topics.
>> > > > > > >>> > > > > > > > > > >
>> > > > > > >>> > > > > > > > > > >
>> > > > > > >>> > > > > > > > > > >
>> > > > > > >>> > > >
>> > > > > > >>> >
>> > > > > > >>>
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
>> > > > > > >>> > > > > > > > > > > Looking forward to your thoughts.
>> > > > > > >>> > > > > > > > > > >
>> > > > > > >>> > > > > > > > > > > Lucas
>> > > > > > >>> > > > > > > > > > >
>> > > > > > >>> >
>> > > > > > >>> >
>> > > > > > >>>
>> > > > > > >>
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Justine Olshan <jo...@confluent.io>]

Hi Jun,

I see what you are saying. For now we can remove the extra information.
I'll leave the option to add more fields to the file in the future. The KIP
has been updated to reflect this change.

Thanks,
Justine

On Fri, Sep 18, 2020 at 8:46 AM Jun Rao <ju...@confluent.io> wrote:

> Hi, Justine,
>
> Thanks for the reply.
>
> 13. If the log directory is the source of truth, it means that the
> redundant info in the metadata file will be ignored. Then the question is
> why do we need to put the redundant info in the metadata file now?
>
> Thanks,
>
> Jun
>
> On Thu, Sep 17, 2020 at 5:07 PM Justine Olshan <jo...@confluent.io>
> wrote:
>
> > Hi Jun,
> > Thanks for the quick response!
> >
> > 12. I've decided to bump up the versions on the requests and updated the
> > KIP. I think it's good we thoroughly discussed the options here, so we
> know
> > we made a good choice. :)
> >
> > 13. This is an interesting situation. I think if this does occur we
> should
> > give a warning. I agree that it's hard to know the source of truth for
> sure
> > since the directory or the file could be manually modified. I guess the
> > directory could be used as the source of truth. To be honest, I'm not
> > really sure what happens in kafka when the log directory is renamed
> > manually in such a way. I'm also wondering if the situation is
> recoverable
> > in this scenario.
> >
> > Thanks,
> > Justine
> >
> > On Thu, Sep 17, 2020 at 4:28 PM Jun Rao <ju...@confluent.io> wrote:
> >
> > > Hi, Justine,
> > >
> > > Thanks for the reply.
> > >
> > > 12. I don't have a strong preference either. However, if we need IBP
> > > anyway, maybe it's easier to just bump up the version for all inter
> > broker
> > > requests and add the topic id field as a regular field. A regular field
> > is
> > > a bit more concise in wire transfer than a flexible field.
> > >
> > > 13. The confusion that I was referring to is between the topic name and
> > > partition number between the log dir and the metadata file. For
> example,
> > if
> > > the log dir is topicA-1 and the metadata file in it has topicB and
> > > partition 0 (say due to a bug or manual modification), which one do we
> > use
> > > as the source of truth?
> > >
> > > Jun
> > >
> > > On Thu, Sep 17, 2020 at 3:43 PM Justine Olshan <jo...@confluent.io>
> > > wrote:
> > >
> > > > Hi Jun,
> > > > Thanks for the comments.
> > > >
> > > > 12. I bumped the LeaderAndIsrRequest because I removed the topic name
> > > field
> > > > in the response. It may be possible to avoid bumping the version
> > without
> > > > that change, but I may be missing something.
> > > > I believe StopReplica is actually on version 3 now, but because
> > version 2
> > > > is flexible, I kept that listed as version 2 on the KIP page.
> However,
> > > you
> > > > may be right in that we may need to bump the version on StopReplica
> to
> > > deal
> > > > with deletion differently as mentioned above. I don't know if I have
> a
> > > big
> > > > preference over used tagged fields or not.
> > > >
> > > > 13. I was thinking that in the case where the file and the request
> > topic
> > > > ids don't match, it means that the broker's topic/the one in the file
> > has
> > > > been deleted. In that case, we would need to delete the old topic and
> > > start
> > > > receiving the new version. If the topic name were to change, but the
> > ids
> > > > still matched, the file would also need to update. Am I missing a
> case
> > > > where the file would be correct and not the request?
> > > >
> > > > Thanks,
> > > > Justine
> > > >
> > > > On Thu, Sep 17, 2020 at 3:18 PM Jun Rao <ju...@confluent.io> wrote:
> > > >
> > > > > Hi, Justine,
> > > > >
> > > > > Thanks for the reply. A couple of more comments below.
> > > > >
> > > > > 12. ListOffset and OffsetForLeader currently don't support flexible
> > > > fields.
> > > > > So, we have to bump up the version number and use IBP at least for
> > > these
> > > > > two requests. Note that it seems 2.7.0 will require IBP anyway
> > because
> > > of
> > > > > changes in KAFKA-10435. Also, it seems that the version for
> > > > > LeaderAndIsrRequest and StopReplica are bumped even though we only
> > > added
> > > > a
> > > > > tagged field. But since IBP is needed anyway, we may want to
> revisit
> > > the
> > > > > overall tagged field choice.
> > > > >
> > > > > 13. The only downside is the potential confusion on which one is
> the
> > > > source
> > > > > of truth if they don't match. Another option is to include those
> > fields
> > > > in
> > > > > the metadata file when we actually change the directory structure.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Jun
> > > > >
> > > > > On Thu, Sep 17, 2020 at 2:01 PM Justine Olshan <
> jolshan@confluent.io
> > >
> > > > > wrote:
> > > > >
> > > > > > Hello all,
> > > > > >
> > > > > > I've thought some more about removing the topic name field from
> > some
> > > of
> > > > > the
> > > > > > requests. On closer inspection of the requests/responses, it
> seems
> > > that
> > > > > the
> > > > > > internal changes would be much larger than I expected. Some
> > protocols
> > > > > > involve clients, so they would require changes too. I'm thinking
> > that
> > > > for
> > > > > > now, removing the topic name from these requests and responses
> are
> > > out
> > > > of
> > > > > > scope.
> > > > > >
> > > > > > I have decided to just keep the change LeaderAndIsrResponse to
> > remove
> > > > the
> > > > > > topic name, and have updated the KIP to reflect this change. I
> have
> > > > also
> > > > > > mentioned the other requests and responses in future work.
> > > > > >
> > > > > > I'm hoping to start the voting process soon, so let me know if
> > there
> > > is
> > > > > > anything else we should discuss.
> > > > > >
> > > > > > Thank you,
> > > > > > Justine
> > > > > >
> > > > > > On Tue, Sep 15, 2020 at 3:57 PM Justine Olshan <
> > jolshan@confluent.io
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hello again,
> > > > > > > To follow up on some of the other comments:
> > > > > > >
> > > > > > > 10/11) We can remove the topic name from these
> > requests/responses,
> > > > and
> > > > > > > that means we will just have to make a few internal changes to
> > make
> > > > > > > partitions accessible by topic id and partition. I can update
> the
> > > KIP
> > > > > to
> > > > > > > remove them unless anyone thinks they should stay.
> > > > > > >
> > > > > > > 12) Addressed in the previous email. I've updated the KIP to
> > > include
> > > > > > > tagged fields for the requests and responses. (More on that
> > below)
> > > > > > >
> > > > > > > 13) I think part of the idea for including this information is
> to
> > > > > prepare
> > > > > > > for future changes. Perhaps the directory structure might
> change
> > > from
> > > > > > > topicName_partitionNumber to something like
> > > topicID_partitionNumber.
> > > > > Then
> > > > > > > it would be useful to have the topic name in the file since it
> > > would
> > > > > not
> > > > > > be
> > > > > > > in the directory structure. Supporting topic renames might be
> > > easier
> > > > if
> > > > > > the
> > > > > > > other fields are included. Would there be any downsides to
> > > including
> > > > > this
> > > > > > > information?
> > > > > > >
> > > > > > > 14)  Yes, we would need to copy the partition metadata file in
> > this
> > > > > > > process. I've updated the KIP to include this.
> > > > > > >
> > > > > > > 15) I believe Lucas meant v1 and v2 here. He was referring to
> how
> > > the
> > > > > > > requests would fall under different IBP and meant that older
> > > brokers
> > > > > > would
> > > > > > > have to use the older version of the request and the existing
> > topic
> > > > > > > deletion process. At first, it seemed like tagged fields would
> > > > resolve
> > > > > > > the IBP issue. However, we may need IBP for this request after
> > all
> > > > > since
> > > > > > > the controller handles the topic deletion differently depending
> > on
> > > > the
> > > > > > IBP
> > > > > > > version. In an older version, we can't just send a StopReplica
> > > delete
> > > > > the
> > > > > > > topic immediately like we'd want to for this KIP.
> > > > > > >
> > > > > > > This makes me wonder if we want tagged fields on all the
> requests
> > > > after
> > > > > > > all. Let me know your thoughts!
> > > > > > >
> > > > > > > Justine
> > > > > > >
> > > > > > > On Tue, Sep 15, 2020 at 1:03 PM Justine Olshan <
> > > jolshan@confluent.io
> > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > >> Hi all,
> > > > > > >> Jun brought up a good point in his last email about tagged
> > fields,
> > > > and
> > > > > > >> I've updated the KIP to reflect that the changes to requests
> and
> > > > > > responses
> > > > > > >> will be in the form of tagged fields to avoid changing IBP.
> > > > > > >>
> > > > > > >> Jun: I plan on sending a followup email to address some of the
> > > other
> > > > > > >> points.
> > > > > > >>
> > > > > > >> Thanks,
> > > > > > >> Justine
> > > > > > >>
> > > > > > >> On Mon, Sep 14, 2020 at 4:25 PM Jun Rao <ju...@confluent.io>
> > wrote:
> > > > > > >>
> > > > > > >>> Hi, Justine,
> > > > > > >>>
> > > > > > >>> Thanks for the updated KIP. A few comments below.
> > > > > > >>>
> > > > > > >>> 10. LeaderAndIsr Response: Do we need the topic name?
> > > > > > >>>
> > > > > > >>> 11. For the changed request/response, other than
> LeaderAndIsr,
> > > > > > >>> UpdateMetadata, Metadata, do we need to include the topic
> name?
> > > > > > >>>
> > > > > > >>> 12. It seems that upgrades don't require IBP. Does that mean
> > the
> > > > new
> > > > > > >>> fields
> > > > > > >>> in all the request/response are added as tagged fields
> without
> > > > > bumping
> > > > > > up
> > > > > > >>> the request version? It would be useful to make that clear.
> > > > > > >>>
> > > > > > >>> 13. Partition Metadata file: Do we need to include the topic
> > name
> > > > and
> > > > > > the
> > > > > > >>> partition id since they are implied in the directory name?
> > > > > > >>>
> > > > > > >>> 14. In the JBOD mode, we support moving a partition's data
> from
> > > one
> > > > > > disk
> > > > > > >>> to
> > > > > > >>> another. Will the new partition metadata file be copied
> during
> > > that
> > > > > > >>> process?
> > > > > > >>>
> > > > > > >>> 15. The KIP says "Remove deleted topics from replicas by
> > sending
> > > > > > >>> StopReplicaRequest V2 for any topics which do not contain a
> > topic
> > > > ID,
> > > > > > and
> > > > > > >>> V3 for any topics which do contain a topic ID.". However, it
> > > seems
> > > > > the
> > > > > > >>> updated controller will create all missing topic IDs first
> > before
> > > > > doing
> > > > > > >>> other actions. So, is StopReplicaRequest V2 needed?
> > > > > > >>>
> > > > > > >>> Jun
> > > > > > >>>
> > > > > > >>> On Fri, Sep 11, 2020 at 10:31 AM John Roesler <
> > > vvcephei@apache.org
> > > > >
> > > > > > >>> wrote:
> > > > > > >>>
> > > > > > >>> > Thanks, Justine!
> > > > > > >>> >
> > > > > > >>> > Your response seems compelling to me.
> > > > > > >>> >
> > > > > > >>> > -John
> > > > > > >>> >
> > > > > > >>> > On Fri, 2020-09-11 at 10:17 -0700, Justine Olshan wrote:
> > > > > > >>> > > Hello all,
> > > > > > >>> > > Thanks for continuing the discussion! I have a few
> > responses
> > > to
> > > > > > your
> > > > > > >>> > points.
> > > > > > >>> > >
> > > > > > >>> > > Tom: You are correct in that this KIP has not mentioned
> the
> > > > > > >>> > > DeleteTopicsRequest. I think that this would be out of
> > scope
> > > > for
> > > > > > >>> now, but
> > > > > > >>> > > may be something worth adding in the future.
> > > > > > >>> > >
> > > > > > >>> > > John: We did consider sequence ids, but there are a few
> > > reasons
> > > > > to
> > > > > > >>> favor
> > > > > > >>> > > UUIDs. There are several cases where topics from
> different
> > > > > clusters
> > > > > > >>> may
> > > > > > >>> > > interact now and in the future. For example, Mirror
> Maker 2
> > > may
> > > > > > >>> benefit
> > > > > > >>> > > from being able to detect when a cluster being mirrored
> is
> > > > > deleted
> > > > > > >>> and
> > > > > > >>> > > recreated and globally unique identifiers would make
> > > resolving
> > > > > > issues
> > > > > > >>> > > easier than sequence IDs which may collide between
> > clusters.
> > > > > > KIP-405
> > > > > > >>> > > (tiered storage) will also benefit from globally unique
> IDs
> > > as
> > > > > > shared
> > > > > > >>> > > buckets may be used between clusters.
> > > > > > >>> > >
> > > > > > >>> > > Globally unique IDs would also make functionality like
> > moving
> > > > > > topics
> > > > > > >>> > > between disparate clusters easier in the future, simplify
> > any
> > > > > > future
> > > > > > >>> > > implementations of backups and restores, and more. In
> > > general,
> > > > > > >>> unique IDs
> > > > > > >>> > > would ensure that the source cluster topics do not
> conflict
> > > > with
> > > > > > the
> > > > > > >>> > > destination cluster topics.
> > > > > > >>> > >
> > > > > > >>> > > If we were to use sequence ids, we would need
> sufficiently
> > > > large
> > > > > > >>> cluster
> > > > > > >>> > > ids to be stored with the topic identifiers or we run the
> > > risk
> > > > of
> > > > > > >>> > > collisions. This will give up any advantage in
> compactness
> > > that
> > > > > > >>> sequence
> > > > > > >>> > > numbers may bring. Given these advantages I think it
> makes
> > > > sense
> > > > > to
> > > > > > >>> use
> > > > > > >>> > > UUIDs.
> > > > > > >>> > >
> > > > > > >>> > > Gokul: This is an interesting idea, but this is a
> breaking
> > > > > change.
> > > > > > >>> Out of
> > > > > > >>> > > scope for now, but maybe worth discussing in the future.
> > > > > > >>> > >
> > > > > > >>> > > Hope this explains some of the decisions,
> > > > > > >>> > >
> > > > > > >>> > > Justine
> > > > > > >>> > >
> > > > > > >>> > >
> > > > > > >>> > >
> > > > > > >>> > > On Fri, Sep 11, 2020 at 8:27 AM Gokul Ramanan
> Subramanian <
> > > > > > >>> > > gokul2411s@gmail.com> wrote:
> > > > > > >>> > >
> > > > > > >>> > > > Hi.
> > > > > > >>> > > >
> > > > > > >>> > > > Thanks for the KIP.
> > > > > > >>> > > >
> > > > > > >>> > > > Have you thought about whether it makes sense to
> support
> > > > > > >>> authorizing a
> > > > > > >>> > > > principal for a topic ID rather than a topic name to
> > > achieve
> > > > > > >>> tighter
> > > > > > >>> > > > security?
> > > > > > >>> > > >
> > > > > > >>> > > > Or is the topic ID fundamentally an internal detail
> > similar
> > > > to
> > > > > > >>> epochs
> > > > > > >>> > used
> > > > > > >>> > > > in a bunch of other places in Kafka?
> > > > > > >>> > > >
> > > > > > >>> > > > Thanks.
> > > > > > >>> > > >
> > > > > > >>> > > > On Fri, Sep 11, 2020 at 4:06 PM John Roesler <
> > > > > > vvcephei@apache.org>
> > > > > > >>> > wrote:
> > > > > > >>> > > >
> > > > > > >>> > > > > Hello Justine,
> > > > > > >>> > > > >
> > > > > > >>> > > > > Thanks for the KIP!
> > > > > > >>> > > > >
> > > > > > >>> > > > > I happen to have been confronted recently with the
> need
> > > to
> > > > > keep
> > > > > > >>> > track of
> > > > > > >>> > > > a
> > > > > > >>> > > > > large number of topics as compactly as possible. I
> was
> > > > going
> > > > > to
> > > > > > >>> come
> > > > > > >>> > up
> > > > > > >>> > > > > with some way to dictionary encode the topic names as
> > > > > integers,
> > > > > > >>> but
> > > > > > >>> > this
> > > > > > >>> > > > > seems much better!
> > > > > > >>> > > > >
> > > > > > >>> > > > > Apologies if this has been raised before, but I’m
> > > wondering
> > > > > > >>> about the
> > > > > > >>> > > > > choice of UUID vs sequence number for the ids.
> > Typically,
> > > > > I’ve
> > > > > > >>> seen
> > > > > > >>> > UUIDs
> > > > > > >>> > > > > in two situations:
> > > > > > >>> > > > > 1. When processes need to generate non-colliding
> > > > identifiers
> > > > > > >>> without
> > > > > > >>> > > > > coordination.
> > > > > > >>> > > > > 2. When the identifier needs to be “universally
> > unique”;
> > > > > I.e.,
> > > > > > >>> the
> > > > > > >>> > > > > identifier needs to distinguish the entity from all
> > other
> > > > > > >>> entities
> > > > > > >>> > that
> > > > > > >>> > > > > could ever exist. This is useful in cases where
> > entities
> > > > from
> > > > > > all
> > > > > > >>> > kinds
> > > > > > >>> > > > of
> > > > > > >>> > > > > systems get mixed together, such as when dumping logs
> > > from
> > > > > all
> > > > > > >>> > processes
> > > > > > >>> > > > in
> > > > > > >>> > > > > a company into a common system.
> > > > > > >>> > > > >
> > > > > > >>> > > > > Maybe I’m being short-sighted, but it doesn’t seem
> like
> > > > > either
> > > > > > >>> really
> > > > > > >>> > > > > applies here. It seems like the brokers could and
> would
> > > > > achieve
> > > > > > >>> > consensus
> > > > > > >>> > > > > when creating a topic anyway, which is all that’s
> > > required
> > > > to
> > > > > > >>> > generate
> > > > > > >>> > > > > non-colliding sequence ids. For the second, as you
> > > mention,
> > > > > we
> > > > > > >>> could
> > > > > > >>> > > > assign
> > > > > > >>> > > > > a UUID to the cluster as a whole, which would render
> > any
> > > > > > resource
> > > > > > >>> > scoped
> > > > > > >>> > > > to
> > > > > > >>> > > > > the broker universally unique as well.
> > > > > > >>> > > > >
> > > > > > >>> > > > > The reason I mention this is that, although a UUID is
> > way
> > > > > more
> > > > > > >>> > compact
> > > > > > >>> > > > > than topic names, it’s still 16 bytes. In contrast, a
> > > > 4-byte
> > > > > > >>> integer
> > > > > > >>> > > > > sequence id would give us 4 billion unique topics per
> > > > > cluster,
> > > > > > >>> which
> > > > > > >>> > > > seems
> > > > > > >>> > > > > like enough ;)
> > > > > > >>> > > > >
> > > > > > >>> > > > > Considering the number of different times these topic
> > > > > > >>> identifiers are
> > > > > > >>> > > > sent
> > > > > > >>> > > > > over the wire or stored in memory, it seems like it
> > might
> > > > be
> > > > > > >>> worth
> > > > > > >>> > the
> > > > > > >>> > > > > additional 4x space savings.
> > > > > > >>> > > > >
> > > > > > >>> > > > > What do you think about this?
> > > > > > >>> > > > >
> > > > > > >>> > > > > Thanks,
> > > > > > >>> > > > > John
> > > > > > >>> > > > >
> > > > > > >>> > > > > On Fri, Sep 11, 2020, at 03:20, Tom Bentley wrote:
> > > > > > >>> > > > > > Hi Justine,
> > > > > > >>> > > > > >
> > > > > > >>> > > > > > This looks like a very welcome improvement. Thanks!
> > > > > > >>> > > > > >
> > > > > > >>> > > > > > Maybe I missed it, but the KIP doesn't seem to
> > mention
> > > > > > changing
> > > > > > >>> > > > > > DeleteTopicsRequest to identify the topic using an
> > id.
> > > > > Maybe
> > > > > > >>> > that's out
> > > > > > >>> > > > > of
> > > > > > >>> > > > > > scope, but DeleteTopicsRequest is not listed among
> > the
> > > > > Future
> > > > > > >>> Work
> > > > > > >>> > APIs
> > > > > > >>> > > > > > either.
> > > > > > >>> > > > > >
> > > > > > >>> > > > > > Kind regards,
> > > > > > >>> > > > > >
> > > > > > >>> > > > > > Tom
> > > > > > >>> > > > > >
> > > > > > >>> > > > > > On Thu, Sep 10, 2020 at 3:59 PM Satish Duggana <
> > > > > > >>> > > > satish.duggana@gmail.com
> > > > > > >>> > > > > > wrote:
> > > > > > >>> > > > > >
> > > > > > >>> > > > > > > Thanks Lucas/Justine for the nice KIP.
> > > > > > >>> > > > > > >
> > > > > > >>> > > > > > > It has several benefits which also include
> > > simplifying
> > > > > the
> > > > > > >>> topic
> > > > > > >>> > > > > > > deletion process by controller and logs cleanup
> by
> > > > > brokers
> > > > > > in
> > > > > > >>> > corner
> > > > > > >>> > > > > > > cases.
> > > > > > >>> > > > > > >
> > > > > > >>> > > > > > > Best,
> > > > > > >>> > > > > > > Satish.
> > > > > > >>> > > > > > >
> > > > > > >>> > > > > > > On Wed, Sep 9, 2020 at 10:07 PM Justine Olshan <
> > > > > > >>> > jolshan@confluent.io
> > > > > > >>> > > > > > > wrote:
> > > > > > >>> > > > > > > > Hello all, it's been almost a year! I've made
> > some
> > > > > > changes
> > > > > > >>> to
> > > > > > >>> > this
> > > > > > >>> > > > > KIP
> > > > > > >>> > > > > > > and hope to continue the discussion.
> > > > > > >>> > > > > > > > One of the main changes I've added is now the
> > > > metadata
> > > > > > >>> response
> > > > > > >>> > > > will
> > > > > > >>> > > > > > > include the topic ID (as Colin suggested).
> Clients
> > > can
> > > > > > >>> obtain the
> > > > > > >>> > > > > topicID
> > > > > > >>> > > > > > > of a given topic through a TopicDescription. The
> > > > topicId
> > > > > > will
> > > > > > >>> > also be
> > > > > > >>> > > > > > > included with the UpdateMetadata request.
> > > > > > >>> > > > > > > > Let me know what you all think.
> > > > > > >>> > > > > > > > Thank you,
> > > > > > >>> > > > > > > > Justine
> > > > > > >>> > > > > > > >
> > > > > > >>> > > > > > > > On 2019/09/13 16:38:26, "Colin McCabe" <
> > > > > > cmccabe@apache.org
> > > > > > >>> >
> > > > > > >>> > wrote:
> > > > > > >>> > > > > > > > > Hi Lucas,
> > > > > > >>> > > > > > > > >
> > > > > > >>> > > > > > > > > Thanks for tackling this.  Topic IDs are a
> > great
> > > > > idea,
> > > > > > >>> and
> > > > > > >>> > this
> > > > > > >>> > > > is
> > > > > > >>> > > > > a
> > > > > > >>> > > > > > > really good writeup.
> > > > > > >>> > > > > > > > > For /brokers/topics/[topic], the schema
> version
> > > > > should
> > > > > > be
> > > > > > >>> > bumped
> > > > > > >>> > > > to
> > > > > > >>> > > > > > > version 3, rather than 2.  KIP-455 bumped the
> > version
> > > > of
> > > > > > this
> > > > > > >>> > znode
> > > > > > >>> > > > to
> > > > > > >>> > > > > 2
> > > > > > >>> > > > > > > already :)
> > > > > > >>> > > > > > > > > Given that we're going to be seeing these
> > things
> > > as
> > > > > > >>> strings
> > > > > > >>> > as
> > > > > > >>> > > > lot
> > > > > > >>> > > > > (in
> > > > > > >>> > > > > > > logs, in ZooKeeper, on the command-line, etc.),
> > does
> > > it
> > > > > > make
> > > > > > >>> > sense to
> > > > > > >>> > > > > use
> > > > > > >>> > > > > > > base64 when converting them to strings?
> > > > > > >>> > > > > > > > > Here is an example of the hex representation:
> > > > > > >>> > > > > > > > > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> > > > > > >>> > > > > > > > >
> > > > > > >>> > > > > > > > > And here is an example in base64.
> > > > > > >>> > > > > > > > > b8tRS7h4TJ2Vt43Dp85v2A
> > > > > > >>> > > > > > > > >
> > > > > > >>> > > > > > > > > The base64 version saves 15 letters (to be
> > fair,
> > > 4
> > > > of
> > > > > > >>> those
> > > > > > >>> > were
> > > > > > >>> > > > > > > dashes that we could have elided in the hex
> > > > > > representation.)
> > > > > > >>> > > > > > > > > Another thing to consider is that we should
> > > specify
> > > > > > that
> > > > > > >>> the
> > > > > > >>> > > > > > > all-zeroes UUID is not a valid topic UUID.   We
> > can't
> > > > use
> > > > > > >>> null
> > > > > > >>> > for
> > > > > > >>> > > > this
> > > > > > >>> > > > > > > because we can't pass a null UUID over the RPC
> > > protocol
> > > > > > >>> (there
> > > > > > >>> > is no
> > > > > > >>> > > > > > > special pattern for null, nor do we want to waste
> > > space
> > > > > > >>> reserving
> > > > > > >>> > > > such
> > > > > > >>> > > > > a
> > > > > > >>> > > > > > > pattern.)
> > > > > > >>> > > > > > > > > Maybe I missed it, but did you describe
> > > "migration
> > > > > > of...
> > > > > > >>> > existing
> > > > > > >>> > > > > > > topic[s] without topic IDs" in detail in any
> > section?
> > > > It
> > > > > > >>> seems
> > > > > > >>> > like
> > > > > > >>> > > > > when
> > > > > > >>> > > > > > > the new controller becomes active, it should just
> > > > > generate
> > > > > > >>> random
> > > > > > >>> > > > > UUIDs for
> > > > > > >>> > > > > > > these, and write the random UUIDs back to
> > ZooKeeper.
> > > > It
> > > > > > >>> would be
> > > > > > >>> > > > good
> > > > > > >>> > > > > to
> > > > > > >>> > > > > > > spell that out.  We should make it clear that
> this
> > > > > happens
> > > > > > >>> > regardless
> > > > > > >>> > > > > of
> > > > > > >>> > > > > > > the inter-broker protocol version (it's a
> > compatible
> > > > > > change).
> > > > > > >>> > > > > > > > > "LeaderAndIsrRequests including an
> > > > is_every_partition
> > > > > > >>> flag"
> > > > > > >>> > > > seems a
> > > > > > >>> > > > > > > bit wordy.  Can we just call these "full
> > > > > > >>> LeaderAndIsrRequests"?
> > > > > > >>> > Then
> > > > > > >>> > > > > the
> > > > > > >>> > > > > > > RPC field could be named "full".  Also, it would
> > > > probably
> > > > > > be
> > > > > > >>> > better
> > > > > > >>> > > > > for the
> > > > > > >>> > > > > > > RPC field to be an enum of { UNSPECIFIED,
> > > INCREMENTAL,
> > > > > FULL
> > > > > > >>> }, so
> > > > > > >>> > > > that
> > > > > > >>> > > > > we
> > > > > > >>> > > > > > > can cleanly handle old versions (by treating them
> > as
> > > > > > >>> UNSPECIFIED)
> > > > > > >>> > > > > > > > > In the LeaderAndIsrRequest section, you write
> > "A
> > > > > final
> > > > > > >>> > deletion
> > > > > > >>> > > > > event
> > > > > > >>> > > > > > > will be secheduled for X ms after the
> > > > LeaderAndIsrRequest
> > > > > > was
> > > > > > >>> > first
> > > > > > >>> > > > > > > received..."  I guess the X was a placeholder
> that
> > > you
> > > > > > >>> intended
> > > > > > >>> > to
> > > > > > >>> > > > > replace
> > > > > > >>> > > > > > > before posting? :)  In any case, this seems like
> > the
> > > > kind
> > > > > > of
> > > > > > >>> > thing
> > > > > > >>> > > > we'd
> > > > > > >>> > > > > > > want a configuration for.  Let's describe that
> > > > > > configuration
> > > > > > >>> key
> > > > > > >>> > > > > somewhere
> > > > > > >>> > > > > > > in this KIP, including what its default value is.
> > > > > > >>> > > > > > > > > We should probably also log a bunch of
> messages
> > > at
> > > > > WARN
> > > > > > >>> level
> > > > > > >>> > > > when
> > > > > > >>> > > > > > > something is scheduled for deletion, as well.
> > (Maybe
> > > > > this
> > > > > > >>> was
> > > > > > >>> > > > > assumed, but
> > > > > > >>> > > > > > > it would be good to mention it).
> > > > > > >>> > > > > > > > > I feel like there are a few sections that
> > should
> > > be
> > > > > > >>> moved to
> > > > > > >>> > > > > "rejected
> > > > > > >>> > > > > > > alternatives."  For example, in the DeleteTopics
> > > > section,
> > > > > > >>> since
> > > > > > >>> > we're
> > > > > > >>> > > > > not
> > > > > > >>> > > > > > > going to do option 1 or 2, these should be moved
> > into
> > > > > > >>> "rejected
> > > > > > >>> > > > > > > alternatives,"  rather than appearing inline.
> > > Another
> > > > > case
> > > > > > >>> is
> > > > > > >>> > the
> > > > > > >>> > > > > "Should
> > > > > > >>> > > > > > > we remove topic name from the protocol where
> > > possible"
> > > > > > >>> section.
> > > > > > >>> > This
> > > > > > >>> > > > > is
> > > > > > >>> > > > > > > clearly discussing a design alternative that
> we're
> > > not
> > > > > > >>> proposing
> > > > > > >>> > to
> > > > > > >>> > > > > > > implement: removing the topic name from those
> > > > protocols.
> > > > > > >>> > > > > > > > > Is it really necessary to have a new
> > > > > > >>> > /admin/delete_topics_by_id
> > > > > > >>> > > > > path
> > > > > > >>> > > > > > > in ZooKeeper?  It seems like we don't really need
> > > this.
> > > > > > >>> Whenever
> > > > > > >>> > > > > there is
> > > > > > >>> > > > > > > a new controller, we'll send out full
> > > > > LeaderAndIsrRequests
> > > > > > >>> which
> > > > > > >>> > will
> > > > > > >>> > > > > > > trigger the stale topics to be cleaned up.   The
> > > active
> > > > > > >>> > controller
> > > > > > >>> > > > will
> > > > > > >>> > > > > > > also send the full LeaderAndIsrRequest to brokers
> > > that
> > > > > are
> > > > > > >>> just
> > > > > > >>> > > > > starting
> > > > > > >>> > > > > > > up.    So we don't really need this kind of
> > two-phase
> > > > > > commit
> > > > > > >>> > (send
> > > > > > >>> > > > out
> > > > > > >>> > > > > > > StopReplicasRequest, get ACKs from all nodes,
> > commit
> > > by
> > > > > > >>> removing
> > > > > > >>> > > > > > > /admin/delete_topics node) any more.
> > > > > > >>> > > > > > > > > You mention that FetchRequest will now
> include
> > > UUID
> > > > > to
> > > > > > >>> avoid
> > > > > > >>> > > > issues
> > > > > > >>> > > > > > > where requests are made to stale partitions.
> > > However,
> > > > > > >>> adding a
> > > > > > >>> > UUID
> > > > > > >>> > > > to
> > > > > > >>> > > > > > > MetadataRequest is listed as future work, out of
> > > scope
> > > > > for
> > > > > > >>> this
> > > > > > >>> > KIP.
> > > > > > >>> > > > > How
> > > > > > >>> > > > > > > will the client learn what the topic UUID is, if
> > the
> > > > > > metadata
> > > > > > >>> > > > response
> > > > > > >>> > > > > > > doesn't include that information?  It seems like
> > > adding
> > > > > the
> > > > > > >>> UUID
> > > > > > >>> > to
> > > > > > >>> > > > > > > MetadataResponse would be an improvement here
> that
> > > > might
> > > > > > not
> > > > > > >>> be
> > > > > > >>> > too
> > > > > > >>> > > > > hard to
> > > > > > >>> > > > > > > make.
> > > > > > >>> > > > > > > > > best,
> > > > > > >>> > > > > > > > > Colin
> > > > > > >>> > > > > > > > >
> > > > > > >>> > > > > > > > >
> > > > > > >>> > > > > > > > > On Mon, Sep 9, 2019, at 17:48, Ryanne Dolan
> > > wrote:
> > > > > > >>> > > > > > > > > > Lucas, this would be great. I've run into
> > > issues
> > > > > with
> > > > > > >>> > topics
> > > > > > >>> > > > > being
> > > > > > >>> > > > > > > > > > resurrected accidentally, since a client
> > cannot
> > > > > > easily
> > > > > > >>> > > > > distinguish
> > > > > > >>> > > > > > > between
> > > > > > >>> > > > > > > > > > a deleted topic and a new topic with the
> same
> > > > name.
> > > > > > I'd
> > > > > > >>> > need
> > > > > > >>> > > > the
> > > > > > >>> > > > > ID
> > > > > > >>> > > > > > > > > > accessible from the client to solve that
> > issue,
> > > > but
> > > > > > >>> this
> > > > > > >>> > is a
> > > > > > >>> > > > > good
> > > > > > >>> > > > > > > first
> > > > > > >>> > > > > > > > > > step.
> > > > > > >>> > > > > > > > > >
> > > > > > >>> > > > > > > > > > Ryanne
> > > > > > >>> > > > > > > > > >
> > > > > > >>> > > > > > > > > > On Wed, Sep 4, 2019 at 1:41 PM Lucas
> > > Bradstreet <
> > > > > > >>> > > > > lucas@confluent.io>
> > > > > > >>> > > > > > > wrote:
> > > > > > >>> > > > > > > > > > > Hi all,
> > > > > > >>> > > > > > > > > > >
> > > > > > >>> > > > > > > > > > > I would like to kick off discussion of
> > > KIP-516,
> > > > > an
> > > > > > >>> > > > > implementation
> > > > > > >>> > > > > > > of topic
> > > > > > >>> > > > > > > > > > > IDs for Kafka. Topic IDs aim to solve
> topic
> > > > > > >>> uniqueness
> > > > > > >>> > > > > problems in
> > > > > > >>> > > > > > > Kafka,
> > > > > > >>> > > > > > > > > > > where referring to a topic by name alone
> is
> > > > > > >>> insufficient.
> > > > > > >>> > > > Such
> > > > > > >>> > > > > > > cases
> > > > > > >>> > > > > > > > > > > include when a topic has been deleted and
> > > > > recreated
> > > > > > >>> with
> > > > > > >>> > the
> > > > > > >>> > > > > same
> > > > > > >>> > > > > > > name.
> > > > > > >>> > > > > > > > > > > Unique identifiers will help simplify and
> > > > improve
> > > > > > >>> Kafka's
> > > > > > >>> > > > topic
> > > > > > >>> > > > > > > deletion
> > > > > > >>> > > > > > > > > > > process, as well as prevent cases where
> > > brokers
> > > > > may
> > > > > > >>> > > > incorrectly
> > > > > > >>> > > > > > > interact
> > > > > > >>> > > > > > > > > > > with stale versions of topics.
> > > > > > >>> > > > > > > > > > >
> > > > > > >>> > > > > > > > > > >
> > > > > > >>> > > > > > > > > > >
> > > > > > >>> > > >
> > > > > > >>> >
> > > > > > >>>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > > > > > >>> > > > > > > > > > > Looking forward to your thoughts.
> > > > > > >>> > > > > > > > > > >
> > > > > > >>> > > > > > > > > > > Lucas
> > > > > > >>> > > > > > > > > > >
> > > > > > >>> >
> > > > > > >>> >
> > > > > > >>>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Jun Rao <ju...@confluent.io>]

Hi, Justine,

Thanks for the reply.

13. If the log directory is the source of truth, it means that the
redundant info in the metadata file will be ignored. Then the question is
why do we need to put the redundant info in the metadata file now?

Thanks,

Jun

On Thu, Sep 17, 2020 at 5:07 PM Justine Olshan <jo...@confluent.io> wrote:

> Hi Jun,
> Thanks for the quick response!
>
> 12. I've decided to bump up the versions on the requests and updated the
> KIP. I think it's good we thoroughly discussed the options here, so we know
> we made a good choice. :)
>
> 13. This is an interesting situation. I think if this does occur we should
> give a warning. I agree that it's hard to know the source of truth for sure
> since the directory or the file could be manually modified. I guess the
> directory could be used as the source of truth. To be honest, I'm not
> really sure what happens in kafka when the log directory is renamed
> manually in such a way. I'm also wondering if the situation is recoverable
> in this scenario.
>
> Thanks,
> Justine
>
> On Thu, Sep 17, 2020 at 4:28 PM Jun Rao <ju...@confluent.io> wrote:
>
> > Hi, Justine,
> >
> > Thanks for the reply.
> >
> > 12. I don't have a strong preference either. However, if we need IBP
> > anyway, maybe it's easier to just bump up the version for all inter
> broker
> > requests and add the topic id field as a regular field. A regular field
> is
> > a bit more concise in wire transfer than a flexible field.
> >
> > 13. The confusion that I was referring to is between the topic name and
> > partition number between the log dir and the metadata file. For example,
> if
> > the log dir is topicA-1 and the metadata file in it has topicB and
> > partition 0 (say due to a bug or manual modification), which one do we
> use
> > as the source of truth?
> >
> > Jun
> >
> > On Thu, Sep 17, 2020 at 3:43 PM Justine Olshan <jo...@confluent.io>
> > wrote:
> >
> > > Hi Jun,
> > > Thanks for the comments.
> > >
> > > 12. I bumped the LeaderAndIsrRequest because I removed the topic name
> > field
> > > in the response. It may be possible to avoid bumping the version
> without
> > > that change, but I may be missing something.
> > > I believe StopReplica is actually on version 3 now, but because
> version 2
> > > is flexible, I kept that listed as version 2 on the KIP page. However,
> > you
> > > may be right in that we may need to bump the version on StopReplica to
> > deal
> > > with deletion differently as mentioned above. I don't know if I have a
> > big
> > > preference over used tagged fields or not.
> > >
> > > 13. I was thinking that in the case where the file and the request
> topic
> > > ids don't match, it means that the broker's topic/the one in the file
> has
> > > been deleted. In that case, we would need to delete the old topic and
> > start
> > > receiving the new version. If the topic name were to change, but the
> ids
> > > still matched, the file would also need to update. Am I missing a case
> > > where the file would be correct and not the request?
> > >
> > > Thanks,
> > > Justine
> > >
> > > On Thu, Sep 17, 2020 at 3:18 PM Jun Rao <ju...@confluent.io> wrote:
> > >
> > > > Hi, Justine,
> > > >
> > > > Thanks for the reply. A couple of more comments below.
> > > >
> > > > 12. ListOffset and OffsetForLeader currently don't support flexible
> > > fields.
> > > > So, we have to bump up the version number and use IBP at least for
> > these
> > > > two requests. Note that it seems 2.7.0 will require IBP anyway
> because
> > of
> > > > changes in KAFKA-10435. Also, it seems that the version for
> > > > LeaderAndIsrRequest and StopReplica are bumped even though we only
> > added
> > > a
> > > > tagged field. But since IBP is needed anyway, we may want to revisit
> > the
> > > > overall tagged field choice.
> > > >
> > > > 13. The only downside is the potential confusion on which one is the
> > > source
> > > > of truth if they don't match. Another option is to include those
> fields
> > > in
> > > > the metadata file when we actually change the directory structure.
> > > >
> > > > Thanks,
> > > >
> > > > Jun
> > > >
> > > > On Thu, Sep 17, 2020 at 2:01 PM Justine Olshan <jolshan@confluent.io
> >
> > > > wrote:
> > > >
> > > > > Hello all,
> > > > >
> > > > > I've thought some more about removing the topic name field from
> some
> > of
> > > > the
> > > > > requests. On closer inspection of the requests/responses, it seems
> > that
> > > > the
> > > > > internal changes would be much larger than I expected. Some
> protocols
> > > > > involve clients, so they would require changes too. I'm thinking
> that
> > > for
> > > > > now, removing the topic name from these requests and responses are
> > out
> > > of
> > > > > scope.
> > > > >
> > > > > I have decided to just keep the change LeaderAndIsrResponse to
> remove
> > > the
> > > > > topic name, and have updated the KIP to reflect this change. I have
> > > also
> > > > > mentioned the other requests and responses in future work.
> > > > >
> > > > > I'm hoping to start the voting process soon, so let me know if
> there
> > is
> > > > > anything else we should discuss.
> > > > >
> > > > > Thank you,
> > > > > Justine
> > > > >
> > > > > On Tue, Sep 15, 2020 at 3:57 PM Justine Olshan <
> jolshan@confluent.io
> > >
> > > > > wrote:
> > > > >
> > > > > > Hello again,
> > > > > > To follow up on some of the other comments:
> > > > > >
> > > > > > 10/11) We can remove the topic name from these
> requests/responses,
> > > and
> > > > > > that means we will just have to make a few internal changes to
> make
> > > > > > partitions accessible by topic id and partition. I can update the
> > KIP
> > > > to
> > > > > > remove them unless anyone thinks they should stay.
> > > > > >
> > > > > > 12) Addressed in the previous email. I've updated the KIP to
> > include
> > > > > > tagged fields for the requests and responses. (More on that
> below)
> > > > > >
> > > > > > 13) I think part of the idea for including this information is to
> > > > prepare
> > > > > > for future changes. Perhaps the directory structure might change
> > from
> > > > > > topicName_partitionNumber to something like
> > topicID_partitionNumber.
> > > > Then
> > > > > > it would be useful to have the topic name in the file since it
> > would
> > > > not
> > > > > be
> > > > > > in the directory structure. Supporting topic renames might be
> > easier
> > > if
> > > > > the
> > > > > > other fields are included. Would there be any downsides to
> > including
> > > > this
> > > > > > information?
> > > > > >
> > > > > > 14)  Yes, we would need to copy the partition metadata file in
> this
> > > > > > process. I've updated the KIP to include this.
> > > > > >
> > > > > > 15) I believe Lucas meant v1 and v2 here. He was referring to how
> > the
> > > > > > requests would fall under different IBP and meant that older
> > brokers
> > > > > would
> > > > > > have to use the older version of the request and the existing
> topic
> > > > > > deletion process. At first, it seemed like tagged fields would
> > > resolve
> > > > > > the IBP issue. However, we may need IBP for this request after
> all
> > > > since
> > > > > > the controller handles the topic deletion differently depending
> on
> > > the
> > > > > IBP
> > > > > > version. In an older version, we can't just send a StopReplica
> > delete
> > > > the
> > > > > > topic immediately like we'd want to for this KIP.
> > > > > >
> > > > > > This makes me wonder if we want tagged fields on all the requests
> > > after
> > > > > > all. Let me know your thoughts!
> > > > > >
> > > > > > Justine
> > > > > >
> > > > > > On Tue, Sep 15, 2020 at 1:03 PM Justine Olshan <
> > jolshan@confluent.io
> > > >
> > > > > > wrote:
> > > > > >
> > > > > >> Hi all,
> > > > > >> Jun brought up a good point in his last email about tagged
> fields,
> > > and
> > > > > >> I've updated the KIP to reflect that the changes to requests and
> > > > > responses
> > > > > >> will be in the form of tagged fields to avoid changing IBP.
> > > > > >>
> > > > > >> Jun: I plan on sending a followup email to address some of the
> > other
> > > > > >> points.
> > > > > >>
> > > > > >> Thanks,
> > > > > >> Justine
> > > > > >>
> > > > > >> On Mon, Sep 14, 2020 at 4:25 PM Jun Rao <ju...@confluent.io>
> wrote:
> > > > > >>
> > > > > >>> Hi, Justine,
> > > > > >>>
> > > > > >>> Thanks for the updated KIP. A few comments below.
> > > > > >>>
> > > > > >>> 10. LeaderAndIsr Response: Do we need the topic name?
> > > > > >>>
> > > > > >>> 11. For the changed request/response, other than LeaderAndIsr,
> > > > > >>> UpdateMetadata, Metadata, do we need to include the topic name?
> > > > > >>>
> > > > > >>> 12. It seems that upgrades don't require IBP. Does that mean
> the
> > > new
> > > > > >>> fields
> > > > > >>> in all the request/response are added as tagged fields without
> > > > bumping
> > > > > up
> > > > > >>> the request version? It would be useful to make that clear.
> > > > > >>>
> > > > > >>> 13. Partition Metadata file: Do we need to include the topic
> name
> > > and
> > > > > the
> > > > > >>> partition id since they are implied in the directory name?
> > > > > >>>
> > > > > >>> 14. In the JBOD mode, we support moving a partition's data from
> > one
> > > > > disk
> > > > > >>> to
> > > > > >>> another. Will the new partition metadata file be copied during
> > that
> > > > > >>> process?
> > > > > >>>
> > > > > >>> 15. The KIP says "Remove deleted topics from replicas by
> sending
> > > > > >>> StopReplicaRequest V2 for any topics which do not contain a
> topic
> > > ID,
> > > > > and
> > > > > >>> V3 for any topics which do contain a topic ID.". However, it
> > seems
> > > > the
> > > > > >>> updated controller will create all missing topic IDs first
> before
> > > > doing
> > > > > >>> other actions. So, is StopReplicaRequest V2 needed?
> > > > > >>>
> > > > > >>> Jun
> > > > > >>>
> > > > > >>> On Fri, Sep 11, 2020 at 10:31 AM John Roesler <
> > vvcephei@apache.org
> > > >
> > > > > >>> wrote:
> > > > > >>>
> > > > > >>> > Thanks, Justine!
> > > > > >>> >
> > > > > >>> > Your response seems compelling to me.
> > > > > >>> >
> > > > > >>> > -John
> > > > > >>> >
> > > > > >>> > On Fri, 2020-09-11 at 10:17 -0700, Justine Olshan wrote:
> > > > > >>> > > Hello all,
> > > > > >>> > > Thanks for continuing the discussion! I have a few
> responses
> > to
> > > > > your
> > > > > >>> > points.
> > > > > >>> > >
> > > > > >>> > > Tom: You are correct in that this KIP has not mentioned the
> > > > > >>> > > DeleteTopicsRequest. I think that this would be out of
> scope
> > > for
> > > > > >>> now, but
> > > > > >>> > > may be something worth adding in the future.
> > > > > >>> > >
> > > > > >>> > > John: We did consider sequence ids, but there are a few
> > reasons
> > > > to
> > > > > >>> favor
> > > > > >>> > > UUIDs. There are several cases where topics from different
> > > > clusters
> > > > > >>> may
> > > > > >>> > > interact now and in the future. For example, Mirror Maker 2
> > may
> > > > > >>> benefit
> > > > > >>> > > from being able to detect when a cluster being mirrored is
> > > > deleted
> > > > > >>> and
> > > > > >>> > > recreated and globally unique identifiers would make
> > resolving
> > > > > issues
> > > > > >>> > > easier than sequence IDs which may collide between
> clusters.
> > > > > KIP-405
> > > > > >>> > > (tiered storage) will also benefit from globally unique IDs
> > as
> > > > > shared
> > > > > >>> > > buckets may be used between clusters.
> > > > > >>> > >
> > > > > >>> > > Globally unique IDs would also make functionality like
> moving
> > > > > topics
> > > > > >>> > > between disparate clusters easier in the future, simplify
> any
> > > > > future
> > > > > >>> > > implementations of backups and restores, and more. In
> > general,
> > > > > >>> unique IDs
> > > > > >>> > > would ensure that the source cluster topics do not conflict
> > > with
> > > > > the
> > > > > >>> > > destination cluster topics.
> > > > > >>> > >
> > > > > >>> > > If we were to use sequence ids, we would need sufficiently
> > > large
> > > > > >>> cluster
> > > > > >>> > > ids to be stored with the topic identifiers or we run the
> > risk
> > > of
> > > > > >>> > > collisions. This will give up any advantage in compactness
> > that
> > > > > >>> sequence
> > > > > >>> > > numbers may bring. Given these advantages I think it makes
> > > sense
> > > > to
> > > > > >>> use
> > > > > >>> > > UUIDs.
> > > > > >>> > >
> > > > > >>> > > Gokul: This is an interesting idea, but this is a breaking
> > > > change.
> > > > > >>> Out of
> > > > > >>> > > scope for now, but maybe worth discussing in the future.
> > > > > >>> > >
> > > > > >>> > > Hope this explains some of the decisions,
> > > > > >>> > >
> > > > > >>> > > Justine
> > > > > >>> > >
> > > > > >>> > >
> > > > > >>> > >
> > > > > >>> > > On Fri, Sep 11, 2020 at 8:27 AM Gokul Ramanan Subramanian <
> > > > > >>> > > gokul2411s@gmail.com> wrote:
> > > > > >>> > >
> > > > > >>> > > > Hi.
> > > > > >>> > > >
> > > > > >>> > > > Thanks for the KIP.
> > > > > >>> > > >
> > > > > >>> > > > Have you thought about whether it makes sense to support
> > > > > >>> authorizing a
> > > > > >>> > > > principal for a topic ID rather than a topic name to
> > achieve
> > > > > >>> tighter
> > > > > >>> > > > security?
> > > > > >>> > > >
> > > > > >>> > > > Or is the topic ID fundamentally an internal detail
> similar
> > > to
> > > > > >>> epochs
> > > > > >>> > used
> > > > > >>> > > > in a bunch of other places in Kafka?
> > > > > >>> > > >
> > > > > >>> > > > Thanks.
> > > > > >>> > > >
> > > > > >>> > > > On Fri, Sep 11, 2020 at 4:06 PM John Roesler <
> > > > > vvcephei@apache.org>
> > > > > >>> > wrote:
> > > > > >>> > > >
> > > > > >>> > > > > Hello Justine,
> > > > > >>> > > > >
> > > > > >>> > > > > Thanks for the KIP!
> > > > > >>> > > > >
> > > > > >>> > > > > I happen to have been confronted recently with the need
> > to
> > > > keep
> > > > > >>> > track of
> > > > > >>> > > > a
> > > > > >>> > > > > large number of topics as compactly as possible. I was
> > > going
> > > > to
> > > > > >>> come
> > > > > >>> > up
> > > > > >>> > > > > with some way to dictionary encode the topic names as
> > > > integers,
> > > > > >>> but
> > > > > >>> > this
> > > > > >>> > > > > seems much better!
> > > > > >>> > > > >
> > > > > >>> > > > > Apologies if this has been raised before, but I’m
> > wondering
> > > > > >>> about the
> > > > > >>> > > > > choice of UUID vs sequence number for the ids.
> Typically,
> > > > I’ve
> > > > > >>> seen
> > > > > >>> > UUIDs
> > > > > >>> > > > > in two situations:
> > > > > >>> > > > > 1. When processes need to generate non-colliding
> > > identifiers
> > > > > >>> without
> > > > > >>> > > > > coordination.
> > > > > >>> > > > > 2. When the identifier needs to be “universally
> unique”;
> > > > I.e.,
> > > > > >>> the
> > > > > >>> > > > > identifier needs to distinguish the entity from all
> other
> > > > > >>> entities
> > > > > >>> > that
> > > > > >>> > > > > could ever exist. This is useful in cases where
> entities
> > > from
> > > > > all
> > > > > >>> > kinds
> > > > > >>> > > > of
> > > > > >>> > > > > systems get mixed together, such as when dumping logs
> > from
> > > > all
> > > > > >>> > processes
> > > > > >>> > > > in
> > > > > >>> > > > > a company into a common system.
> > > > > >>> > > > >
> > > > > >>> > > > > Maybe I’m being short-sighted, but it doesn’t seem like
> > > > either
> > > > > >>> really
> > > > > >>> > > > > applies here. It seems like the brokers could and would
> > > > achieve
> > > > > >>> > consensus
> > > > > >>> > > > > when creating a topic anyway, which is all that’s
> > required
> > > to
> > > > > >>> > generate
> > > > > >>> > > > > non-colliding sequence ids. For the second, as you
> > mention,
> > > > we
> > > > > >>> could
> > > > > >>> > > > assign
> > > > > >>> > > > > a UUID to the cluster as a whole, which would render
> any
> > > > > resource
> > > > > >>> > scoped
> > > > > >>> > > > to
> > > > > >>> > > > > the broker universally unique as well.
> > > > > >>> > > > >
> > > > > >>> > > > > The reason I mention this is that, although a UUID is
> way
> > > > more
> > > > > >>> > compact
> > > > > >>> > > > > than topic names, it’s still 16 bytes. In contrast, a
> > > 4-byte
> > > > > >>> integer
> > > > > >>> > > > > sequence id would give us 4 billion unique topics per
> > > > cluster,
> > > > > >>> which
> > > > > >>> > > > seems
> > > > > >>> > > > > like enough ;)
> > > > > >>> > > > >
> > > > > >>> > > > > Considering the number of different times these topic
> > > > > >>> identifiers are
> > > > > >>> > > > sent
> > > > > >>> > > > > over the wire or stored in memory, it seems like it
> might
> > > be
> > > > > >>> worth
> > > > > >>> > the
> > > > > >>> > > > > additional 4x space savings.
> > > > > >>> > > > >
> > > > > >>> > > > > What do you think about this?
> > > > > >>> > > > >
> > > > > >>> > > > > Thanks,
> > > > > >>> > > > > John
> > > > > >>> > > > >
> > > > > >>> > > > > On Fri, Sep 11, 2020, at 03:20, Tom Bentley wrote:
> > > > > >>> > > > > > Hi Justine,
> > > > > >>> > > > > >
> > > > > >>> > > > > > This looks like a very welcome improvement. Thanks!
> > > > > >>> > > > > >
> > > > > >>> > > > > > Maybe I missed it, but the KIP doesn't seem to
> mention
> > > > > changing
> > > > > >>> > > > > > DeleteTopicsRequest to identify the topic using an
> id.
> > > > Maybe
> > > > > >>> > that's out
> > > > > >>> > > > > of
> > > > > >>> > > > > > scope, but DeleteTopicsRequest is not listed among
> the
> > > > Future
> > > > > >>> Work
> > > > > >>> > APIs
> > > > > >>> > > > > > either.
> > > > > >>> > > > > >
> > > > > >>> > > > > > Kind regards,
> > > > > >>> > > > > >
> > > > > >>> > > > > > Tom
> > > > > >>> > > > > >
> > > > > >>> > > > > > On Thu, Sep 10, 2020 at 3:59 PM Satish Duggana <
> > > > > >>> > > > satish.duggana@gmail.com
> > > > > >>> > > > > > wrote:
> > > > > >>> > > > > >
> > > > > >>> > > > > > > Thanks Lucas/Justine for the nice KIP.
> > > > > >>> > > > > > >
> > > > > >>> > > > > > > It has several benefits which also include
> > simplifying
> > > > the
> > > > > >>> topic
> > > > > >>> > > > > > > deletion process by controller and logs cleanup by
> > > > brokers
> > > > > in
> > > > > >>> > corner
> > > > > >>> > > > > > > cases.
> > > > > >>> > > > > > >
> > > > > >>> > > > > > > Best,
> > > > > >>> > > > > > > Satish.
> > > > > >>> > > > > > >
> > > > > >>> > > > > > > On Wed, Sep 9, 2020 at 10:07 PM Justine Olshan <
> > > > > >>> > jolshan@confluent.io
> > > > > >>> > > > > > > wrote:
> > > > > >>> > > > > > > > Hello all, it's been almost a year! I've made
> some
> > > > > changes
> > > > > >>> to
> > > > > >>> > this
> > > > > >>> > > > > KIP
> > > > > >>> > > > > > > and hope to continue the discussion.
> > > > > >>> > > > > > > > One of the main changes I've added is now the
> > > metadata
> > > > > >>> response
> > > > > >>> > > > will
> > > > > >>> > > > > > > include the topic ID (as Colin suggested). Clients
> > can
> > > > > >>> obtain the
> > > > > >>> > > > > topicID
> > > > > >>> > > > > > > of a given topic through a TopicDescription. The
> > > topicId
> > > > > will
> > > > > >>> > also be
> > > > > >>> > > > > > > included with the UpdateMetadata request.
> > > > > >>> > > > > > > > Let me know what you all think.
> > > > > >>> > > > > > > > Thank you,
> > > > > >>> > > > > > > > Justine
> > > > > >>> > > > > > > >
> > > > > >>> > > > > > > > On 2019/09/13 16:38:26, "Colin McCabe" <
> > > > > cmccabe@apache.org
> > > > > >>> >
> > > > > >>> > wrote:
> > > > > >>> > > > > > > > > Hi Lucas,
> > > > > >>> > > > > > > > >
> > > > > >>> > > > > > > > > Thanks for tackling this.  Topic IDs are a
> great
> > > > idea,
> > > > > >>> and
> > > > > >>> > this
> > > > > >>> > > > is
> > > > > >>> > > > > a
> > > > > >>> > > > > > > really good writeup.
> > > > > >>> > > > > > > > > For /brokers/topics/[topic], the schema version
> > > > should
> > > > > be
> > > > > >>> > bumped
> > > > > >>> > > > to
> > > > > >>> > > > > > > version 3, rather than 2.  KIP-455 bumped the
> version
> > > of
> > > > > this
> > > > > >>> > znode
> > > > > >>> > > > to
> > > > > >>> > > > > 2
> > > > > >>> > > > > > > already :)
> > > > > >>> > > > > > > > > Given that we're going to be seeing these
> things
> > as
> > > > > >>> strings
> > > > > >>> > as
> > > > > >>> > > > lot
> > > > > >>> > > > > (in
> > > > > >>> > > > > > > logs, in ZooKeeper, on the command-line, etc.),
> does
> > it
> > > > > make
> > > > > >>> > sense to
> > > > > >>> > > > > use
> > > > > >>> > > > > > > base64 when converting them to strings?
> > > > > >>> > > > > > > > > Here is an example of the hex representation:
> > > > > >>> > > > > > > > > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> > > > > >>> > > > > > > > >
> > > > > >>> > > > > > > > > And here is an example in base64.
> > > > > >>> > > > > > > > > b8tRS7h4TJ2Vt43Dp85v2A
> > > > > >>> > > > > > > > >
> > > > > >>> > > > > > > > > The base64 version saves 15 letters (to be
> fair,
> > 4
> > > of
> > > > > >>> those
> > > > > >>> > were
> > > > > >>> > > > > > > dashes that we could have elided in the hex
> > > > > representation.)
> > > > > >>> > > > > > > > > Another thing to consider is that we should
> > specify
> > > > > that
> > > > > >>> the
> > > > > >>> > > > > > > all-zeroes UUID is not a valid topic UUID.   We
> can't
> > > use
> > > > > >>> null
> > > > > >>> > for
> > > > > >>> > > > this
> > > > > >>> > > > > > > because we can't pass a null UUID over the RPC
> > protocol
> > > > > >>> (there
> > > > > >>> > is no
> > > > > >>> > > > > > > special pattern for null, nor do we want to waste
> > space
> > > > > >>> reserving
> > > > > >>> > > > such
> > > > > >>> > > > > a
> > > > > >>> > > > > > > pattern.)
> > > > > >>> > > > > > > > > Maybe I missed it, but did you describe
> > "migration
> > > > > of...
> > > > > >>> > existing
> > > > > >>> > > > > > > topic[s] without topic IDs" in detail in any
> section?
> > > It
> > > > > >>> seems
> > > > > >>> > like
> > > > > >>> > > > > when
> > > > > >>> > > > > > > the new controller becomes active, it should just
> > > > generate
> > > > > >>> random
> > > > > >>> > > > > UUIDs for
> > > > > >>> > > > > > > these, and write the random UUIDs back to
> ZooKeeper.
> > > It
> > > > > >>> would be
> > > > > >>> > > > good
> > > > > >>> > > > > to
> > > > > >>> > > > > > > spell that out.  We should make it clear that this
> > > > happens
> > > > > >>> > regardless
> > > > > >>> > > > > of
> > > > > >>> > > > > > > the inter-broker protocol version (it's a
> compatible
> > > > > change).
> > > > > >>> > > > > > > > > "LeaderAndIsrRequests including an
> > > is_every_partition
> > > > > >>> flag"
> > > > > >>> > > > seems a
> > > > > >>> > > > > > > bit wordy.  Can we just call these "full
> > > > > >>> LeaderAndIsrRequests"?
> > > > > >>> > Then
> > > > > >>> > > > > the
> > > > > >>> > > > > > > RPC field could be named "full".  Also, it would
> > > probably
> > > > > be
> > > > > >>> > better
> > > > > >>> > > > > for the
> > > > > >>> > > > > > > RPC field to be an enum of { UNSPECIFIED,
> > INCREMENTAL,
> > > > FULL
> > > > > >>> }, so
> > > > > >>> > > > that
> > > > > >>> > > > > we
> > > > > >>> > > > > > > can cleanly handle old versions (by treating them
> as
> > > > > >>> UNSPECIFIED)
> > > > > >>> > > > > > > > > In the LeaderAndIsrRequest section, you write
> "A
> > > > final
> > > > > >>> > deletion
> > > > > >>> > > > > event
> > > > > >>> > > > > > > will be secheduled for X ms after the
> > > LeaderAndIsrRequest
> > > > > was
> > > > > >>> > first
> > > > > >>> > > > > > > received..."  I guess the X was a placeholder that
> > you
> > > > > >>> intended
> > > > > >>> > to
> > > > > >>> > > > > replace
> > > > > >>> > > > > > > before posting? :)  In any case, this seems like
> the
> > > kind
> > > > > of
> > > > > >>> > thing
> > > > > >>> > > > we'd
> > > > > >>> > > > > > > want a configuration for.  Let's describe that
> > > > > configuration
> > > > > >>> key
> > > > > >>> > > > > somewhere
> > > > > >>> > > > > > > in this KIP, including what its default value is.
> > > > > >>> > > > > > > > > We should probably also log a bunch of messages
> > at
> > > > WARN
> > > > > >>> level
> > > > > >>> > > > when
> > > > > >>> > > > > > > something is scheduled for deletion, as well.
> (Maybe
> > > > this
> > > > > >>> was
> > > > > >>> > > > > assumed, but
> > > > > >>> > > > > > > it would be good to mention it).
> > > > > >>> > > > > > > > > I feel like there are a few sections that
> should
> > be
> > > > > >>> moved to
> > > > > >>> > > > > "rejected
> > > > > >>> > > > > > > alternatives."  For example, in the DeleteTopics
> > > section,
> > > > > >>> since
> > > > > >>> > we're
> > > > > >>> > > > > not
> > > > > >>> > > > > > > going to do option 1 or 2, these should be moved
> into
> > > > > >>> "rejected
> > > > > >>> > > > > > > alternatives,"  rather than appearing inline.
> > Another
> > > > case
> > > > > >>> is
> > > > > >>> > the
> > > > > >>> > > > > "Should
> > > > > >>> > > > > > > we remove topic name from the protocol where
> > possible"
> > > > > >>> section.
> > > > > >>> > This
> > > > > >>> > > > > is
> > > > > >>> > > > > > > clearly discussing a design alternative that we're
> > not
> > > > > >>> proposing
> > > > > >>> > to
> > > > > >>> > > > > > > implement: removing the topic name from those
> > > protocols.
> > > > > >>> > > > > > > > > Is it really necessary to have a new
> > > > > >>> > /admin/delete_topics_by_id
> > > > > >>> > > > > path
> > > > > >>> > > > > > > in ZooKeeper?  It seems like we don't really need
> > this.
> > > > > >>> Whenever
> > > > > >>> > > > > there is
> > > > > >>> > > > > > > a new controller, we'll send out full
> > > > LeaderAndIsrRequests
> > > > > >>> which
> > > > > >>> > will
> > > > > >>> > > > > > > trigger the stale topics to be cleaned up.   The
> > active
> > > > > >>> > controller
> > > > > >>> > > > will
> > > > > >>> > > > > > > also send the full LeaderAndIsrRequest to brokers
> > that
> > > > are
> > > > > >>> just
> > > > > >>> > > > > starting
> > > > > >>> > > > > > > up.    So we don't really need this kind of
> two-phase
> > > > > commit
> > > > > >>> > (send
> > > > > >>> > > > out
> > > > > >>> > > > > > > StopReplicasRequest, get ACKs from all nodes,
> commit
> > by
> > > > > >>> removing
> > > > > >>> > > > > > > /admin/delete_topics node) any more.
> > > > > >>> > > > > > > > > You mention that FetchRequest will now include
> > UUID
> > > > to
> > > > > >>> avoid
> > > > > >>> > > > issues
> > > > > >>> > > > > > > where requests are made to stale partitions.
> > However,
> > > > > >>> adding a
> > > > > >>> > UUID
> > > > > >>> > > > to
> > > > > >>> > > > > > > MetadataRequest is listed as future work, out of
> > scope
> > > > for
> > > > > >>> this
> > > > > >>> > KIP.
> > > > > >>> > > > > How
> > > > > >>> > > > > > > will the client learn what the topic UUID is, if
> the
> > > > > metadata
> > > > > >>> > > > response
> > > > > >>> > > > > > > doesn't include that information?  It seems like
> > adding
> > > > the
> > > > > >>> UUID
> > > > > >>> > to
> > > > > >>> > > > > > > MetadataResponse would be an improvement here that
> > > might
> > > > > not
> > > > > >>> be
> > > > > >>> > too
> > > > > >>> > > > > hard to
> > > > > >>> > > > > > > make.
> > > > > >>> > > > > > > > > best,
> > > > > >>> > > > > > > > > Colin
> > > > > >>> > > > > > > > >
> > > > > >>> > > > > > > > >
> > > > > >>> > > > > > > > > On Mon, Sep 9, 2019, at 17:48, Ryanne Dolan
> > wrote:
> > > > > >>> > > > > > > > > > Lucas, this would be great. I've run into
> > issues
> > > > with
> > > > > >>> > topics
> > > > > >>> > > > > being
> > > > > >>> > > > > > > > > > resurrected accidentally, since a client
> cannot
> > > > > easily
> > > > > >>> > > > > distinguish
> > > > > >>> > > > > > > between
> > > > > >>> > > > > > > > > > a deleted topic and a new topic with the same
> > > name.
> > > > > I'd
> > > > > >>> > need
> > > > > >>> > > > the
> > > > > >>> > > > > ID
> > > > > >>> > > > > > > > > > accessible from the client to solve that
> issue,
> > > but
> > > > > >>> this
> > > > > >>> > is a
> > > > > >>> > > > > good
> > > > > >>> > > > > > > first
> > > > > >>> > > > > > > > > > step.
> > > > > >>> > > > > > > > > >
> > > > > >>> > > > > > > > > > Ryanne
> > > > > >>> > > > > > > > > >
> > > > > >>> > > > > > > > > > On Wed, Sep 4, 2019 at 1:41 PM Lucas
> > Bradstreet <
> > > > > >>> > > > > lucas@confluent.io>
> > > > > >>> > > > > > > wrote:
> > > > > >>> > > > > > > > > > > Hi all,
> > > > > >>> > > > > > > > > > >
> > > > > >>> > > > > > > > > > > I would like to kick off discussion of
> > KIP-516,
> > > > an
> > > > > >>> > > > > implementation
> > > > > >>> > > > > > > of topic
> > > > > >>> > > > > > > > > > > IDs for Kafka. Topic IDs aim to solve topic
> > > > > >>> uniqueness
> > > > > >>> > > > > problems in
> > > > > >>> > > > > > > Kafka,
> > > > > >>> > > > > > > > > > > where referring to a topic by name alone is
> > > > > >>> insufficient.
> > > > > >>> > > > Such
> > > > > >>> > > > > > > cases
> > > > > >>> > > > > > > > > > > include when a topic has been deleted and
> > > > recreated
> > > > > >>> with
> > > > > >>> > the
> > > > > >>> > > > > same
> > > > > >>> > > > > > > name.
> > > > > >>> > > > > > > > > > > Unique identifiers will help simplify and
> > > improve
> > > > > >>> Kafka's
> > > > > >>> > > > topic
> > > > > >>> > > > > > > deletion
> > > > > >>> > > > > > > > > > > process, as well as prevent cases where
> > brokers
> > > > may
> > > > > >>> > > > incorrectly
> > > > > >>> > > > > > > interact
> > > > > >>> > > > > > > > > > > with stale versions of topics.
> > > > > >>> > > > > > > > > > >
> > > > > >>> > > > > > > > > > >
> > > > > >>> > > > > > > > > > >
> > > > > >>> > > >
> > > > > >>> >
> > > > > >>>
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > > > > >>> > > > > > > > > > > Looking forward to your thoughts.
> > > > > >>> > > > > > > > > > >
> > > > > >>> > > > > > > > > > > Lucas
> > > > > >>> > > > > > > > > > >
> > > > > >>> >
> > > > > >>> >
> > > > > >>>
> > > > > >>
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Justine Olshan <jo...@confluent.io>]

Hi Jun,
Thanks for the quick response!

12. I've decided to bump up the versions on the requests and updated the
KIP. I think it's good we thoroughly discussed the options here, so we know
we made a good choice. :)

13. This is an interesting situation. I think if this does occur we should
give a warning. I agree that it's hard to know the source of truth for sure
since the directory or the file could be manually modified. I guess the
directory could be used as the source of truth. To be honest, I'm not
really sure what happens in kafka when the log directory is renamed
manually in such a way. I'm also wondering if the situation is recoverable
in this scenario.

Thanks,
Justine

On Thu, Sep 17, 2020 at 4:28 PM Jun Rao <ju...@confluent.io> wrote:

> Hi, Justine,
>
> Thanks for the reply.
>
> 12. I don't have a strong preference either. However, if we need IBP
> anyway, maybe it's easier to just bump up the version for all inter broker
> requests and add the topic id field as a regular field. A regular field is
> a bit more concise in wire transfer than a flexible field.
>
> 13. The confusion that I was referring to is between the topic name and
> partition number between the log dir and the metadata file. For example, if
> the log dir is topicA-1 and the metadata file in it has topicB and
> partition 0 (say due to a bug or manual modification), which one do we use
> as the source of truth?
>
> Jun
>
> On Thu, Sep 17, 2020 at 3:43 PM Justine Olshan <jo...@confluent.io>
> wrote:
>
> > Hi Jun,
> > Thanks for the comments.
> >
> > 12. I bumped the LeaderAndIsrRequest because I removed the topic name
> field
> > in the response. It may be possible to avoid bumping the version without
> > that change, but I may be missing something.
> > I believe StopReplica is actually on version 3 now, but because version 2
> > is flexible, I kept that listed as version 2 on the KIP page. However,
> you
> > may be right in that we may need to bump the version on StopReplica to
> deal
> > with deletion differently as mentioned above. I don't know if I have a
> big
> > preference over used tagged fields or not.
> >
> > 13. I was thinking that in the case where the file and the request topic
> > ids don't match, it means that the broker's topic/the one in the file has
> > been deleted. In that case, we would need to delete the old topic and
> start
> > receiving the new version. If the topic name were to change, but the ids
> > still matched, the file would also need to update. Am I missing a case
> > where the file would be correct and not the request?
> >
> > Thanks,
> > Justine
> >
> > On Thu, Sep 17, 2020 at 3:18 PM Jun Rao <ju...@confluent.io> wrote:
> >
> > > Hi, Justine,
> > >
> > > Thanks for the reply. A couple of more comments below.
> > >
> > > 12. ListOffset and OffsetForLeader currently don't support flexible
> > fields.
> > > So, we have to bump up the version number and use IBP at least for
> these
> > > two requests. Note that it seems 2.7.0 will require IBP anyway because
> of
> > > changes in KAFKA-10435. Also, it seems that the version for
> > > LeaderAndIsrRequest and StopReplica are bumped even though we only
> added
> > a
> > > tagged field. But since IBP is needed anyway, we may want to revisit
> the
> > > overall tagged field choice.
> > >
> > > 13. The only downside is the potential confusion on which one is the
> > source
> > > of truth if they don't match. Another option is to include those fields
> > in
> > > the metadata file when we actually change the directory structure.
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Thu, Sep 17, 2020 at 2:01 PM Justine Olshan <jo...@confluent.io>
> > > wrote:
> > >
> > > > Hello all,
> > > >
> > > > I've thought some more about removing the topic name field from some
> of
> > > the
> > > > requests. On closer inspection of the requests/responses, it seems
> that
> > > the
> > > > internal changes would be much larger than I expected. Some protocols
> > > > involve clients, so they would require changes too. I'm thinking that
> > for
> > > > now, removing the topic name from these requests and responses are
> out
> > of
> > > > scope.
> > > >
> > > > I have decided to just keep the change LeaderAndIsrResponse to remove
> > the
> > > > topic name, and have updated the KIP to reflect this change. I have
> > also
> > > > mentioned the other requests and responses in future work.
> > > >
> > > > I'm hoping to start the voting process soon, so let me know if there
> is
> > > > anything else we should discuss.
> > > >
> > > > Thank you,
> > > > Justine
> > > >
> > > > On Tue, Sep 15, 2020 at 3:57 PM Justine Olshan <jolshan@confluent.io
> >
> > > > wrote:
> > > >
> > > > > Hello again,
> > > > > To follow up on some of the other comments:
> > > > >
> > > > > 10/11) We can remove the topic name from these requests/responses,
> > and
> > > > > that means we will just have to make a few internal changes to make
> > > > > partitions accessible by topic id and partition. I can update the
> KIP
> > > to
> > > > > remove them unless anyone thinks they should stay.
> > > > >
> > > > > 12) Addressed in the previous email. I've updated the KIP to
> include
> > > > > tagged fields for the requests and responses. (More on that below)
> > > > >
> > > > > 13) I think part of the idea for including this information is to
> > > prepare
> > > > > for future changes. Perhaps the directory structure might change
> from
> > > > > topicName_partitionNumber to something like
> topicID_partitionNumber.
> > > Then
> > > > > it would be useful to have the topic name in the file since it
> would
> > > not
> > > > be
> > > > > in the directory structure. Supporting topic renames might be
> easier
> > if
> > > > the
> > > > > other fields are included. Would there be any downsides to
> including
> > > this
> > > > > information?
> > > > >
> > > > > 14)  Yes, we would need to copy the partition metadata file in this
> > > > > process. I've updated the KIP to include this.
> > > > >
> > > > > 15) I believe Lucas meant v1 and v2 here. He was referring to how
> the
> > > > > requests would fall under different IBP and meant that older
> brokers
> > > > would
> > > > > have to use the older version of the request and the existing topic
> > > > > deletion process. At first, it seemed like tagged fields would
> > resolve
> > > > > the IBP issue. However, we may need IBP for this request after all
> > > since
> > > > > the controller handles the topic deletion differently depending on
> > the
> > > > IBP
> > > > > version. In an older version, we can't just send a StopReplica
> delete
> > > the
> > > > > topic immediately like we'd want to for this KIP.
> > > > >
> > > > > This makes me wonder if we want tagged fields on all the requests
> > after
> > > > > all. Let me know your thoughts!
> > > > >
> > > > > Justine
> > > > >
> > > > > On Tue, Sep 15, 2020 at 1:03 PM Justine Olshan <
> jolshan@confluent.io
> > >
> > > > > wrote:
> > > > >
> > > > >> Hi all,
> > > > >> Jun brought up a good point in his last email about tagged fields,
> > and
> > > > >> I've updated the KIP to reflect that the changes to requests and
> > > > responses
> > > > >> will be in the form of tagged fields to avoid changing IBP.
> > > > >>
> > > > >> Jun: I plan on sending a followup email to address some of the
> other
> > > > >> points.
> > > > >>
> > > > >> Thanks,
> > > > >> Justine
> > > > >>
> > > > >> On Mon, Sep 14, 2020 at 4:25 PM Jun Rao <ju...@confluent.io> wrote:
> > > > >>
> > > > >>> Hi, Justine,
> > > > >>>
> > > > >>> Thanks for the updated KIP. A few comments below.
> > > > >>>
> > > > >>> 10. LeaderAndIsr Response: Do we need the topic name?
> > > > >>>
> > > > >>> 11. For the changed request/response, other than LeaderAndIsr,
> > > > >>> UpdateMetadata, Metadata, do we need to include the topic name?
> > > > >>>
> > > > >>> 12. It seems that upgrades don't require IBP. Does that mean the
> > new
> > > > >>> fields
> > > > >>> in all the request/response are added as tagged fields without
> > > bumping
> > > > up
> > > > >>> the request version? It would be useful to make that clear.
> > > > >>>
> > > > >>> 13. Partition Metadata file: Do we need to include the topic name
> > and
> > > > the
> > > > >>> partition id since they are implied in the directory name?
> > > > >>>
> > > > >>> 14. In the JBOD mode, we support moving a partition's data from
> one
> > > > disk
> > > > >>> to
> > > > >>> another. Will the new partition metadata file be copied during
> that
> > > > >>> process?
> > > > >>>
> > > > >>> 15. The KIP says "Remove deleted topics from replicas by sending
> > > > >>> StopReplicaRequest V2 for any topics which do not contain a topic
> > ID,
> > > > and
> > > > >>> V3 for any topics which do contain a topic ID.". However, it
> seems
> > > the
> > > > >>> updated controller will create all missing topic IDs first before
> > > doing
> > > > >>> other actions. So, is StopReplicaRequest V2 needed?
> > > > >>>
> > > > >>> Jun
> > > > >>>
> > > > >>> On Fri, Sep 11, 2020 at 10:31 AM John Roesler <
> vvcephei@apache.org
> > >
> > > > >>> wrote:
> > > > >>>
> > > > >>> > Thanks, Justine!
> > > > >>> >
> > > > >>> > Your response seems compelling to me.
> > > > >>> >
> > > > >>> > -John
> > > > >>> >
> > > > >>> > On Fri, 2020-09-11 at 10:17 -0700, Justine Olshan wrote:
> > > > >>> > > Hello all,
> > > > >>> > > Thanks for continuing the discussion! I have a few responses
> to
> > > > your
> > > > >>> > points.
> > > > >>> > >
> > > > >>> > > Tom: You are correct in that this KIP has not mentioned the
> > > > >>> > > DeleteTopicsRequest. I think that this would be out of scope
> > for
> > > > >>> now, but
> > > > >>> > > may be something worth adding in the future.
> > > > >>> > >
> > > > >>> > > John: We did consider sequence ids, but there are a few
> reasons
> > > to
> > > > >>> favor
> > > > >>> > > UUIDs. There are several cases where topics from different
> > > clusters
> > > > >>> may
> > > > >>> > > interact now and in the future. For example, Mirror Maker 2
> may
> > > > >>> benefit
> > > > >>> > > from being able to detect when a cluster being mirrored is
> > > deleted
> > > > >>> and
> > > > >>> > > recreated and globally unique identifiers would make
> resolving
> > > > issues
> > > > >>> > > easier than sequence IDs which may collide between clusters.
> > > > KIP-405
> > > > >>> > > (tiered storage) will also benefit from globally unique IDs
> as
> > > > shared
> > > > >>> > > buckets may be used between clusters.
> > > > >>> > >
> > > > >>> > > Globally unique IDs would also make functionality like moving
> > > > topics
> > > > >>> > > between disparate clusters easier in the future, simplify any
> > > > future
> > > > >>> > > implementations of backups and restores, and more. In
> general,
> > > > >>> unique IDs
> > > > >>> > > would ensure that the source cluster topics do not conflict
> > with
> > > > the
> > > > >>> > > destination cluster topics.
> > > > >>> > >
> > > > >>> > > If we were to use sequence ids, we would need sufficiently
> > large
> > > > >>> cluster
> > > > >>> > > ids to be stored with the topic identifiers or we run the
> risk
> > of
> > > > >>> > > collisions. This will give up any advantage in compactness
> that
> > > > >>> sequence
> > > > >>> > > numbers may bring. Given these advantages I think it makes
> > sense
> > > to
> > > > >>> use
> > > > >>> > > UUIDs.
> > > > >>> > >
> > > > >>> > > Gokul: This is an interesting idea, but this is a breaking
> > > change.
> > > > >>> Out of
> > > > >>> > > scope for now, but maybe worth discussing in the future.
> > > > >>> > >
> > > > >>> > > Hope this explains some of the decisions,
> > > > >>> > >
> > > > >>> > > Justine
> > > > >>> > >
> > > > >>> > >
> > > > >>> > >
> > > > >>> > > On Fri, Sep 11, 2020 at 8:27 AM Gokul Ramanan Subramanian <
> > > > >>> > > gokul2411s@gmail.com> wrote:
> > > > >>> > >
> > > > >>> > > > Hi.
> > > > >>> > > >
> > > > >>> > > > Thanks for the KIP.
> > > > >>> > > >
> > > > >>> > > > Have you thought about whether it makes sense to support
> > > > >>> authorizing a
> > > > >>> > > > principal for a topic ID rather than a topic name to
> achieve
> > > > >>> tighter
> > > > >>> > > > security?
> > > > >>> > > >
> > > > >>> > > > Or is the topic ID fundamentally an internal detail similar
> > to
> > > > >>> epochs
> > > > >>> > used
> > > > >>> > > > in a bunch of other places in Kafka?
> > > > >>> > > >
> > > > >>> > > > Thanks.
> > > > >>> > > >
> > > > >>> > > > On Fri, Sep 11, 2020 at 4:06 PM John Roesler <
> > > > vvcephei@apache.org>
> > > > >>> > wrote:
> > > > >>> > > >
> > > > >>> > > > > Hello Justine,
> > > > >>> > > > >
> > > > >>> > > > > Thanks for the KIP!
> > > > >>> > > > >
> > > > >>> > > > > I happen to have been confronted recently with the need
> to
> > > keep
> > > > >>> > track of
> > > > >>> > > > a
> > > > >>> > > > > large number of topics as compactly as possible. I was
> > going
> > > to
> > > > >>> come
> > > > >>> > up
> > > > >>> > > > > with some way to dictionary encode the topic names as
> > > integers,
> > > > >>> but
> > > > >>> > this
> > > > >>> > > > > seems much better!
> > > > >>> > > > >
> > > > >>> > > > > Apologies if this has been raised before, but I’m
> wondering
> > > > >>> about the
> > > > >>> > > > > choice of UUID vs sequence number for the ids. Typically,
> > > I’ve
> > > > >>> seen
> > > > >>> > UUIDs
> > > > >>> > > > > in two situations:
> > > > >>> > > > > 1. When processes need to generate non-colliding
> > identifiers
> > > > >>> without
> > > > >>> > > > > coordination.
> > > > >>> > > > > 2. When the identifier needs to be “universally unique”;
> > > I.e.,
> > > > >>> the
> > > > >>> > > > > identifier needs to distinguish the entity from all other
> > > > >>> entities
> > > > >>> > that
> > > > >>> > > > > could ever exist. This is useful in cases where entities
> > from
> > > > all
> > > > >>> > kinds
> > > > >>> > > > of
> > > > >>> > > > > systems get mixed together, such as when dumping logs
> from
> > > all
> > > > >>> > processes
> > > > >>> > > > in
> > > > >>> > > > > a company into a common system.
> > > > >>> > > > >
> > > > >>> > > > > Maybe I’m being short-sighted, but it doesn’t seem like
> > > either
> > > > >>> really
> > > > >>> > > > > applies here. It seems like the brokers could and would
> > > achieve
> > > > >>> > consensus
> > > > >>> > > > > when creating a topic anyway, which is all that’s
> required
> > to
> > > > >>> > generate
> > > > >>> > > > > non-colliding sequence ids. For the second, as you
> mention,
> > > we
> > > > >>> could
> > > > >>> > > > assign
> > > > >>> > > > > a UUID to the cluster as a whole, which would render any
> > > > resource
> > > > >>> > scoped
> > > > >>> > > > to
> > > > >>> > > > > the broker universally unique as well.
> > > > >>> > > > >
> > > > >>> > > > > The reason I mention this is that, although a UUID is way
> > > more
> > > > >>> > compact
> > > > >>> > > > > than topic names, it’s still 16 bytes. In contrast, a
> > 4-byte
> > > > >>> integer
> > > > >>> > > > > sequence id would give us 4 billion unique topics per
> > > cluster,
> > > > >>> which
> > > > >>> > > > seems
> > > > >>> > > > > like enough ;)
> > > > >>> > > > >
> > > > >>> > > > > Considering the number of different times these topic
> > > > >>> identifiers are
> > > > >>> > > > sent
> > > > >>> > > > > over the wire or stored in memory, it seems like it might
> > be
> > > > >>> worth
> > > > >>> > the
> > > > >>> > > > > additional 4x space savings.
> > > > >>> > > > >
> > > > >>> > > > > What do you think about this?
> > > > >>> > > > >
> > > > >>> > > > > Thanks,
> > > > >>> > > > > John
> > > > >>> > > > >
> > > > >>> > > > > On Fri, Sep 11, 2020, at 03:20, Tom Bentley wrote:
> > > > >>> > > > > > Hi Justine,
> > > > >>> > > > > >
> > > > >>> > > > > > This looks like a very welcome improvement. Thanks!
> > > > >>> > > > > >
> > > > >>> > > > > > Maybe I missed it, but the KIP doesn't seem to mention
> > > > changing
> > > > >>> > > > > > DeleteTopicsRequest to identify the topic using an id.
> > > Maybe
> > > > >>> > that's out
> > > > >>> > > > > of
> > > > >>> > > > > > scope, but DeleteTopicsRequest is not listed among the
> > > Future
> > > > >>> Work
> > > > >>> > APIs
> > > > >>> > > > > > either.
> > > > >>> > > > > >
> > > > >>> > > > > > Kind regards,
> > > > >>> > > > > >
> > > > >>> > > > > > Tom
> > > > >>> > > > > >
> > > > >>> > > > > > On Thu, Sep 10, 2020 at 3:59 PM Satish Duggana <
> > > > >>> > > > satish.duggana@gmail.com
> > > > >>> > > > > > wrote:
> > > > >>> > > > > >
> > > > >>> > > > > > > Thanks Lucas/Justine for the nice KIP.
> > > > >>> > > > > > >
> > > > >>> > > > > > > It has several benefits which also include
> simplifying
> > > the
> > > > >>> topic
> > > > >>> > > > > > > deletion process by controller and logs cleanup by
> > > brokers
> > > > in
> > > > >>> > corner
> > > > >>> > > > > > > cases.
> > > > >>> > > > > > >
> > > > >>> > > > > > > Best,
> > > > >>> > > > > > > Satish.
> > > > >>> > > > > > >
> > > > >>> > > > > > > On Wed, Sep 9, 2020 at 10:07 PM Justine Olshan <
> > > > >>> > jolshan@confluent.io
> > > > >>> > > > > > > wrote:
> > > > >>> > > > > > > > Hello all, it's been almost a year! I've made some
> > > > changes
> > > > >>> to
> > > > >>> > this
> > > > >>> > > > > KIP
> > > > >>> > > > > > > and hope to continue the discussion.
> > > > >>> > > > > > > > One of the main changes I've added is now the
> > metadata
> > > > >>> response
> > > > >>> > > > will
> > > > >>> > > > > > > include the topic ID (as Colin suggested). Clients
> can
> > > > >>> obtain the
> > > > >>> > > > > topicID
> > > > >>> > > > > > > of a given topic through a TopicDescription. The
> > topicId
> > > > will
> > > > >>> > also be
> > > > >>> > > > > > > included with the UpdateMetadata request.
> > > > >>> > > > > > > > Let me know what you all think.
> > > > >>> > > > > > > > Thank you,
> > > > >>> > > > > > > > Justine
> > > > >>> > > > > > > >
> > > > >>> > > > > > > > On 2019/09/13 16:38:26, "Colin McCabe" <
> > > > cmccabe@apache.org
> > > > >>> >
> > > > >>> > wrote:
> > > > >>> > > > > > > > > Hi Lucas,
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > > Thanks for tackling this.  Topic IDs are a great
> > > idea,
> > > > >>> and
> > > > >>> > this
> > > > >>> > > > is
> > > > >>> > > > > a
> > > > >>> > > > > > > really good writeup.
> > > > >>> > > > > > > > > For /brokers/topics/[topic], the schema version
> > > should
> > > > be
> > > > >>> > bumped
> > > > >>> > > > to
> > > > >>> > > > > > > version 3, rather than 2.  KIP-455 bumped the version
> > of
> > > > this
> > > > >>> > znode
> > > > >>> > > > to
> > > > >>> > > > > 2
> > > > >>> > > > > > > already :)
> > > > >>> > > > > > > > > Given that we're going to be seeing these things
> as
> > > > >>> strings
> > > > >>> > as
> > > > >>> > > > lot
> > > > >>> > > > > (in
> > > > >>> > > > > > > logs, in ZooKeeper, on the command-line, etc.), does
> it
> > > > make
> > > > >>> > sense to
> > > > >>> > > > > use
> > > > >>> > > > > > > base64 when converting them to strings?
> > > > >>> > > > > > > > > Here is an example of the hex representation:
> > > > >>> > > > > > > > > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > > And here is an example in base64.
> > > > >>> > > > > > > > > b8tRS7h4TJ2Vt43Dp85v2A
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > > The base64 version saves 15 letters (to be fair,
> 4
> > of
> > > > >>> those
> > > > >>> > were
> > > > >>> > > > > > > dashes that we could have elided in the hex
> > > > representation.)
> > > > >>> > > > > > > > > Another thing to consider is that we should
> specify
> > > > that
> > > > >>> the
> > > > >>> > > > > > > all-zeroes UUID is not a valid topic UUID.   We can't
> > use
> > > > >>> null
> > > > >>> > for
> > > > >>> > > > this
> > > > >>> > > > > > > because we can't pass a null UUID over the RPC
> protocol
> > > > >>> (there
> > > > >>> > is no
> > > > >>> > > > > > > special pattern for null, nor do we want to waste
> space
> > > > >>> reserving
> > > > >>> > > > such
> > > > >>> > > > > a
> > > > >>> > > > > > > pattern.)
> > > > >>> > > > > > > > > Maybe I missed it, but did you describe
> "migration
> > > > of...
> > > > >>> > existing
> > > > >>> > > > > > > topic[s] without topic IDs" in detail in any section?
> > It
> > > > >>> seems
> > > > >>> > like
> > > > >>> > > > > when
> > > > >>> > > > > > > the new controller becomes active, it should just
> > > generate
> > > > >>> random
> > > > >>> > > > > UUIDs for
> > > > >>> > > > > > > these, and write the random UUIDs back to ZooKeeper.
> > It
> > > > >>> would be
> > > > >>> > > > good
> > > > >>> > > > > to
> > > > >>> > > > > > > spell that out.  We should make it clear that this
> > > happens
> > > > >>> > regardless
> > > > >>> > > > > of
> > > > >>> > > > > > > the inter-broker protocol version (it's a compatible
> > > > change).
> > > > >>> > > > > > > > > "LeaderAndIsrRequests including an
> > is_every_partition
> > > > >>> flag"
> > > > >>> > > > seems a
> > > > >>> > > > > > > bit wordy.  Can we just call these "full
> > > > >>> LeaderAndIsrRequests"?
> > > > >>> > Then
> > > > >>> > > > > the
> > > > >>> > > > > > > RPC field could be named "full".  Also, it would
> > probably
> > > > be
> > > > >>> > better
> > > > >>> > > > > for the
> > > > >>> > > > > > > RPC field to be an enum of { UNSPECIFIED,
> INCREMENTAL,
> > > FULL
> > > > >>> }, so
> > > > >>> > > > that
> > > > >>> > > > > we
> > > > >>> > > > > > > can cleanly handle old versions (by treating them as
> > > > >>> UNSPECIFIED)
> > > > >>> > > > > > > > > In the LeaderAndIsrRequest section, you write "A
> > > final
> > > > >>> > deletion
> > > > >>> > > > > event
> > > > >>> > > > > > > will be secheduled for X ms after the
> > LeaderAndIsrRequest
> > > > was
> > > > >>> > first
> > > > >>> > > > > > > received..."  I guess the X was a placeholder that
> you
> > > > >>> intended
> > > > >>> > to
> > > > >>> > > > > replace
> > > > >>> > > > > > > before posting? :)  In any case, this seems like the
> > kind
> > > > of
> > > > >>> > thing
> > > > >>> > > > we'd
> > > > >>> > > > > > > want a configuration for.  Let's describe that
> > > > configuration
> > > > >>> key
> > > > >>> > > > > somewhere
> > > > >>> > > > > > > in this KIP, including what its default value is.
> > > > >>> > > > > > > > > We should probably also log a bunch of messages
> at
> > > WARN
> > > > >>> level
> > > > >>> > > > when
> > > > >>> > > > > > > something is scheduled for deletion, as well.  (Maybe
> > > this
> > > > >>> was
> > > > >>> > > > > assumed, but
> > > > >>> > > > > > > it would be good to mention it).
> > > > >>> > > > > > > > > I feel like there are a few sections that should
> be
> > > > >>> moved to
> > > > >>> > > > > "rejected
> > > > >>> > > > > > > alternatives."  For example, in the DeleteTopics
> > section,
> > > > >>> since
> > > > >>> > we're
> > > > >>> > > > > not
> > > > >>> > > > > > > going to do option 1 or 2, these should be moved into
> > > > >>> "rejected
> > > > >>> > > > > > > alternatives,"  rather than appearing inline.
> Another
> > > case
> > > > >>> is
> > > > >>> > the
> > > > >>> > > > > "Should
> > > > >>> > > > > > > we remove topic name from the protocol where
> possible"
> > > > >>> section.
> > > > >>> > This
> > > > >>> > > > > is
> > > > >>> > > > > > > clearly discussing a design alternative that we're
> not
> > > > >>> proposing
> > > > >>> > to
> > > > >>> > > > > > > implement: removing the topic name from those
> > protocols.
> > > > >>> > > > > > > > > Is it really necessary to have a new
> > > > >>> > /admin/delete_topics_by_id
> > > > >>> > > > > path
> > > > >>> > > > > > > in ZooKeeper?  It seems like we don't really need
> this.
> > > > >>> Whenever
> > > > >>> > > > > there is
> > > > >>> > > > > > > a new controller, we'll send out full
> > > LeaderAndIsrRequests
> > > > >>> which
> > > > >>> > will
> > > > >>> > > > > > > trigger the stale topics to be cleaned up.   The
> active
> > > > >>> > controller
> > > > >>> > > > will
> > > > >>> > > > > > > also send the full LeaderAndIsrRequest to brokers
> that
> > > are
> > > > >>> just
> > > > >>> > > > > starting
> > > > >>> > > > > > > up.    So we don't really need this kind of two-phase
> > > > commit
> > > > >>> > (send
> > > > >>> > > > out
> > > > >>> > > > > > > StopReplicasRequest, get ACKs from all nodes, commit
> by
> > > > >>> removing
> > > > >>> > > > > > > /admin/delete_topics node) any more.
> > > > >>> > > > > > > > > You mention that FetchRequest will now include
> UUID
> > > to
> > > > >>> avoid
> > > > >>> > > > issues
> > > > >>> > > > > > > where requests are made to stale partitions.
> However,
> > > > >>> adding a
> > > > >>> > UUID
> > > > >>> > > > to
> > > > >>> > > > > > > MetadataRequest is listed as future work, out of
> scope
> > > for
> > > > >>> this
> > > > >>> > KIP.
> > > > >>> > > > > How
> > > > >>> > > > > > > will the client learn what the topic UUID is, if the
> > > > metadata
> > > > >>> > > > response
> > > > >>> > > > > > > doesn't include that information?  It seems like
> adding
> > > the
> > > > >>> UUID
> > > > >>> > to
> > > > >>> > > > > > > MetadataResponse would be an improvement here that
> > might
> > > > not
> > > > >>> be
> > > > >>> > too
> > > > >>> > > > > hard to
> > > > >>> > > > > > > make.
> > > > >>> > > > > > > > > best,
> > > > >>> > > > > > > > > Colin
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > > On Mon, Sep 9, 2019, at 17:48, Ryanne Dolan
> wrote:
> > > > >>> > > > > > > > > > Lucas, this would be great. I've run into
> issues
> > > with
> > > > >>> > topics
> > > > >>> > > > > being
> > > > >>> > > > > > > > > > resurrected accidentally, since a client cannot
> > > > easily
> > > > >>> > > > > distinguish
> > > > >>> > > > > > > between
> > > > >>> > > > > > > > > > a deleted topic and a new topic with the same
> > name.
> > > > I'd
> > > > >>> > need
> > > > >>> > > > the
> > > > >>> > > > > ID
> > > > >>> > > > > > > > > > accessible from the client to solve that issue,
> > but
> > > > >>> this
> > > > >>> > is a
> > > > >>> > > > > good
> > > > >>> > > > > > > first
> > > > >>> > > > > > > > > > step.
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > > > Ryanne
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > > > On Wed, Sep 4, 2019 at 1:41 PM Lucas
> Bradstreet <
> > > > >>> > > > > lucas@confluent.io>
> > > > >>> > > > > > > wrote:
> > > > >>> > > > > > > > > > > Hi all,
> > > > >>> > > > > > > > > > >
> > > > >>> > > > > > > > > > > I would like to kick off discussion of
> KIP-516,
> > > an
> > > > >>> > > > > implementation
> > > > >>> > > > > > > of topic
> > > > >>> > > > > > > > > > > IDs for Kafka. Topic IDs aim to solve topic
> > > > >>> uniqueness
> > > > >>> > > > > problems in
> > > > >>> > > > > > > Kafka,
> > > > >>> > > > > > > > > > > where referring to a topic by name alone is
> > > > >>> insufficient.
> > > > >>> > > > Such
> > > > >>> > > > > > > cases
> > > > >>> > > > > > > > > > > include when a topic has been deleted and
> > > recreated
> > > > >>> with
> > > > >>> > the
> > > > >>> > > > > same
> > > > >>> > > > > > > name.
> > > > >>> > > > > > > > > > > Unique identifiers will help simplify and
> > improve
> > > > >>> Kafka's
> > > > >>> > > > topic
> > > > >>> > > > > > > deletion
> > > > >>> > > > > > > > > > > process, as well as prevent cases where
> brokers
> > > may
> > > > >>> > > > incorrectly
> > > > >>> > > > > > > interact
> > > > >>> > > > > > > > > > > with stale versions of topics.
> > > > >>> > > > > > > > > > >
> > > > >>> > > > > > > > > > >
> > > > >>> > > > > > > > > > >
> > > > >>> > > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > > > >>> > > > > > > > > > > Looking forward to your thoughts.
> > > > >>> > > > > > > > > > >
> > > > >>> > > > > > > > > > > Lucas
> > > > >>> > > > > > > > > > >
> > > > >>> >
> > > > >>> >
> > > > >>>
> > > > >>
> > > >
> > >
> >
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Jun Rao <ju...@confluent.io>]

Hi, Justine,

Thanks for the reply.

12. I don't have a strong preference either. However, if we need IBP
anyway, maybe it's easier to just bump up the version for all inter broker
requests and add the topic id field as a regular field. A regular field is
a bit more concise in wire transfer than a flexible field.

13. The confusion that I was referring to is between the topic name and
partition number between the log dir and the metadata file. For example, if
the log dir is topicA-1 and the metadata file in it has topicB and
partition 0 (say due to a bug or manual modification), which one do we use
as the source of truth?

Jun

On Thu, Sep 17, 2020 at 3:43 PM Justine Olshan <jo...@confluent.io> wrote:

> Hi Jun,
> Thanks for the comments.
>
> 12. I bumped the LeaderAndIsrRequest because I removed the topic name field
> in the response. It may be possible to avoid bumping the version without
> that change, but I may be missing something.
> I believe StopReplica is actually on version 3 now, but because version 2
> is flexible, I kept that listed as version 2 on the KIP page. However, you
> may be right in that we may need to bump the version on StopReplica to deal
> with deletion differently as mentioned above. I don't know if I have a big
> preference over used tagged fields or not.
>
> 13. I was thinking that in the case where the file and the request topic
> ids don't match, it means that the broker's topic/the one in the file has
> been deleted. In that case, we would need to delete the old topic and start
> receiving the new version. If the topic name were to change, but the ids
> still matched, the file would also need to update. Am I missing a case
> where the file would be correct and not the request?
>
> Thanks,
> Justine
>
> On Thu, Sep 17, 2020 at 3:18 PM Jun Rao <ju...@confluent.io> wrote:
>
> > Hi, Justine,
> >
> > Thanks for the reply. A couple of more comments below.
> >
> > 12. ListOffset and OffsetForLeader currently don't support flexible
> fields.
> > So, we have to bump up the version number and use IBP at least for these
> > two requests. Note that it seems 2.7.0 will require IBP anyway because of
> > changes in KAFKA-10435. Also, it seems that the version for
> > LeaderAndIsrRequest and StopReplica are bumped even though we only added
> a
> > tagged field. But since IBP is needed anyway, we may want to revisit the
> > overall tagged field choice.
> >
> > 13. The only downside is the potential confusion on which one is the
> source
> > of truth if they don't match. Another option is to include those fields
> in
> > the metadata file when we actually change the directory structure.
> >
> > Thanks,
> >
> > Jun
> >
> > On Thu, Sep 17, 2020 at 2:01 PM Justine Olshan <jo...@confluent.io>
> > wrote:
> >
> > > Hello all,
> > >
> > > I've thought some more about removing the topic name field from some of
> > the
> > > requests. On closer inspection of the requests/responses, it seems that
> > the
> > > internal changes would be much larger than I expected. Some protocols
> > > involve clients, so they would require changes too. I'm thinking that
> for
> > > now, removing the topic name from these requests and responses are out
> of
> > > scope.
> > >
> > > I have decided to just keep the change LeaderAndIsrResponse to remove
> the
> > > topic name, and have updated the KIP to reflect this change. I have
> also
> > > mentioned the other requests and responses in future work.
> > >
> > > I'm hoping to start the voting process soon, so let me know if there is
> > > anything else we should discuss.
> > >
> > > Thank you,
> > > Justine
> > >
> > > On Tue, Sep 15, 2020 at 3:57 PM Justine Olshan <jo...@confluent.io>
> > > wrote:
> > >
> > > > Hello again,
> > > > To follow up on some of the other comments:
> > > >
> > > > 10/11) We can remove the topic name from these requests/responses,
> and
> > > > that means we will just have to make a few internal changes to make
> > > > partitions accessible by topic id and partition. I can update the KIP
> > to
> > > > remove them unless anyone thinks they should stay.
> > > >
> > > > 12) Addressed in the previous email. I've updated the KIP to include
> > > > tagged fields for the requests and responses. (More on that below)
> > > >
> > > > 13) I think part of the idea for including this information is to
> > prepare
> > > > for future changes. Perhaps the directory structure might change from
> > > > topicName_partitionNumber to something like topicID_partitionNumber.
> > Then
> > > > it would be useful to have the topic name in the file since it would
> > not
> > > be
> > > > in the directory structure. Supporting topic renames might be easier
> if
> > > the
> > > > other fields are included. Would there be any downsides to including
> > this
> > > > information?
> > > >
> > > > 14)  Yes, we would need to copy the partition metadata file in this
> > > > process. I've updated the KIP to include this.
> > > >
> > > > 15) I believe Lucas meant v1 and v2 here. He was referring to how the
> > > > requests would fall under different IBP and meant that older brokers
> > > would
> > > > have to use the older version of the request and the existing topic
> > > > deletion process. At first, it seemed like tagged fields would
> resolve
> > > > the IBP issue. However, we may need IBP for this request after all
> > since
> > > > the controller handles the topic deletion differently depending on
> the
> > > IBP
> > > > version. In an older version, we can't just send a StopReplica delete
> > the
> > > > topic immediately like we'd want to for this KIP.
> > > >
> > > > This makes me wonder if we want tagged fields on all the requests
> after
> > > > all. Let me know your thoughts!
> > > >
> > > > Justine
> > > >
> > > > On Tue, Sep 15, 2020 at 1:03 PM Justine Olshan <jolshan@confluent.io
> >
> > > > wrote:
> > > >
> > > >> Hi all,
> > > >> Jun brought up a good point in his last email about tagged fields,
> and
> > > >> I've updated the KIP to reflect that the changes to requests and
> > > responses
> > > >> will be in the form of tagged fields to avoid changing IBP.
> > > >>
> > > >> Jun: I plan on sending a followup email to address some of the other
> > > >> points.
> > > >>
> > > >> Thanks,
> > > >> Justine
> > > >>
> > > >> On Mon, Sep 14, 2020 at 4:25 PM Jun Rao <ju...@confluent.io> wrote:
> > > >>
> > > >>> Hi, Justine,
> > > >>>
> > > >>> Thanks for the updated KIP. A few comments below.
> > > >>>
> > > >>> 10. LeaderAndIsr Response: Do we need the topic name?
> > > >>>
> > > >>> 11. For the changed request/response, other than LeaderAndIsr,
> > > >>> UpdateMetadata, Metadata, do we need to include the topic name?
> > > >>>
> > > >>> 12. It seems that upgrades don't require IBP. Does that mean the
> new
> > > >>> fields
> > > >>> in all the request/response are added as tagged fields without
> > bumping
> > > up
> > > >>> the request version? It would be useful to make that clear.
> > > >>>
> > > >>> 13. Partition Metadata file: Do we need to include the topic name
> and
> > > the
> > > >>> partition id since they are implied in the directory name?
> > > >>>
> > > >>> 14. In the JBOD mode, we support moving a partition's data from one
> > > disk
> > > >>> to
> > > >>> another. Will the new partition metadata file be copied during that
> > > >>> process?
> > > >>>
> > > >>> 15. The KIP says "Remove deleted topics from replicas by sending
> > > >>> StopReplicaRequest V2 for any topics which do not contain a topic
> ID,
> > > and
> > > >>> V3 for any topics which do contain a topic ID.". However, it seems
> > the
> > > >>> updated controller will create all missing topic IDs first before
> > doing
> > > >>> other actions. So, is StopReplicaRequest V2 needed?
> > > >>>
> > > >>> Jun
> > > >>>
> > > >>> On Fri, Sep 11, 2020 at 10:31 AM John Roesler <vvcephei@apache.org
> >
> > > >>> wrote:
> > > >>>
> > > >>> > Thanks, Justine!
> > > >>> >
> > > >>> > Your response seems compelling to me.
> > > >>> >
> > > >>> > -John
> > > >>> >
> > > >>> > On Fri, 2020-09-11 at 10:17 -0700, Justine Olshan wrote:
> > > >>> > > Hello all,
> > > >>> > > Thanks for continuing the discussion! I have a few responses to
> > > your
> > > >>> > points.
> > > >>> > >
> > > >>> > > Tom: You are correct in that this KIP has not mentioned the
> > > >>> > > DeleteTopicsRequest. I think that this would be out of scope
> for
> > > >>> now, but
> > > >>> > > may be something worth adding in the future.
> > > >>> > >
> > > >>> > > John: We did consider sequence ids, but there are a few reasons
> > to
> > > >>> favor
> > > >>> > > UUIDs. There are several cases where topics from different
> > clusters
> > > >>> may
> > > >>> > > interact now and in the future. For example, Mirror Maker 2 may
> > > >>> benefit
> > > >>> > > from being able to detect when a cluster being mirrored is
> > deleted
> > > >>> and
> > > >>> > > recreated and globally unique identifiers would make resolving
> > > issues
> > > >>> > > easier than sequence IDs which may collide between clusters.
> > > KIP-405
> > > >>> > > (tiered storage) will also benefit from globally unique IDs as
> > > shared
> > > >>> > > buckets may be used between clusters.
> > > >>> > >
> > > >>> > > Globally unique IDs would also make functionality like moving
> > > topics
> > > >>> > > between disparate clusters easier in the future, simplify any
> > > future
> > > >>> > > implementations of backups and restores, and more. In general,
> > > >>> unique IDs
> > > >>> > > would ensure that the source cluster topics do not conflict
> with
> > > the
> > > >>> > > destination cluster topics.
> > > >>> > >
> > > >>> > > If we were to use sequence ids, we would need sufficiently
> large
> > > >>> cluster
> > > >>> > > ids to be stored with the topic identifiers or we run the risk
> of
> > > >>> > > collisions. This will give up any advantage in compactness that
> > > >>> sequence
> > > >>> > > numbers may bring. Given these advantages I think it makes
> sense
> > to
> > > >>> use
> > > >>> > > UUIDs.
> > > >>> > >
> > > >>> > > Gokul: This is an interesting idea, but this is a breaking
> > change.
> > > >>> Out of
> > > >>> > > scope for now, but maybe worth discussing in the future.
> > > >>> > >
> > > >>> > > Hope this explains some of the decisions,
> > > >>> > >
> > > >>> > > Justine
> > > >>> > >
> > > >>> > >
> > > >>> > >
> > > >>> > > On Fri, Sep 11, 2020 at 8:27 AM Gokul Ramanan Subramanian <
> > > >>> > > gokul2411s@gmail.com> wrote:
> > > >>> > >
> > > >>> > > > Hi.
> > > >>> > > >
> > > >>> > > > Thanks for the KIP.
> > > >>> > > >
> > > >>> > > > Have you thought about whether it makes sense to support
> > > >>> authorizing a
> > > >>> > > > principal for a topic ID rather than a topic name to achieve
> > > >>> tighter
> > > >>> > > > security?
> > > >>> > > >
> > > >>> > > > Or is the topic ID fundamentally an internal detail similar
> to
> > > >>> epochs
> > > >>> > used
> > > >>> > > > in a bunch of other places in Kafka?
> > > >>> > > >
> > > >>> > > > Thanks.
> > > >>> > > >
> > > >>> > > > On Fri, Sep 11, 2020 at 4:06 PM John Roesler <
> > > vvcephei@apache.org>
> > > >>> > wrote:
> > > >>> > > >
> > > >>> > > > > Hello Justine,
> > > >>> > > > >
> > > >>> > > > > Thanks for the KIP!
> > > >>> > > > >
> > > >>> > > > > I happen to have been confronted recently with the need to
> > keep
> > > >>> > track of
> > > >>> > > > a
> > > >>> > > > > large number of topics as compactly as possible. I was
> going
> > to
> > > >>> come
> > > >>> > up
> > > >>> > > > > with some way to dictionary encode the topic names as
> > integers,
> > > >>> but
> > > >>> > this
> > > >>> > > > > seems much better!
> > > >>> > > > >
> > > >>> > > > > Apologies if this has been raised before, but I’m wondering
> > > >>> about the
> > > >>> > > > > choice of UUID vs sequence number for the ids. Typically,
> > I’ve
> > > >>> seen
> > > >>> > UUIDs
> > > >>> > > > > in two situations:
> > > >>> > > > > 1. When processes need to generate non-colliding
> identifiers
> > > >>> without
> > > >>> > > > > coordination.
> > > >>> > > > > 2. When the identifier needs to be “universally unique”;
> > I.e.,
> > > >>> the
> > > >>> > > > > identifier needs to distinguish the entity from all other
> > > >>> entities
> > > >>> > that
> > > >>> > > > > could ever exist. This is useful in cases where entities
> from
> > > all
> > > >>> > kinds
> > > >>> > > > of
> > > >>> > > > > systems get mixed together, such as when dumping logs from
> > all
> > > >>> > processes
> > > >>> > > > in
> > > >>> > > > > a company into a common system.
> > > >>> > > > >
> > > >>> > > > > Maybe I’m being short-sighted, but it doesn’t seem like
> > either
> > > >>> really
> > > >>> > > > > applies here. It seems like the brokers could and would
> > achieve
> > > >>> > consensus
> > > >>> > > > > when creating a topic anyway, which is all that’s required
> to
> > > >>> > generate
> > > >>> > > > > non-colliding sequence ids. For the second, as you mention,
> > we
> > > >>> could
> > > >>> > > > assign
> > > >>> > > > > a UUID to the cluster as a whole, which would render any
> > > resource
> > > >>> > scoped
> > > >>> > > > to
> > > >>> > > > > the broker universally unique as well.
> > > >>> > > > >
> > > >>> > > > > The reason I mention this is that, although a UUID is way
> > more
> > > >>> > compact
> > > >>> > > > > than topic names, it’s still 16 bytes. In contrast, a
> 4-byte
> > > >>> integer
> > > >>> > > > > sequence id would give us 4 billion unique topics per
> > cluster,
> > > >>> which
> > > >>> > > > seems
> > > >>> > > > > like enough ;)
> > > >>> > > > >
> > > >>> > > > > Considering the number of different times these topic
> > > >>> identifiers are
> > > >>> > > > sent
> > > >>> > > > > over the wire or stored in memory, it seems like it might
> be
> > > >>> worth
> > > >>> > the
> > > >>> > > > > additional 4x space savings.
> > > >>> > > > >
> > > >>> > > > > What do you think about this?
> > > >>> > > > >
> > > >>> > > > > Thanks,
> > > >>> > > > > John
> > > >>> > > > >
> > > >>> > > > > On Fri, Sep 11, 2020, at 03:20, Tom Bentley wrote:
> > > >>> > > > > > Hi Justine,
> > > >>> > > > > >
> > > >>> > > > > > This looks like a very welcome improvement. Thanks!
> > > >>> > > > > >
> > > >>> > > > > > Maybe I missed it, but the KIP doesn't seem to mention
> > > changing
> > > >>> > > > > > DeleteTopicsRequest to identify the topic using an id.
> > Maybe
> > > >>> > that's out
> > > >>> > > > > of
> > > >>> > > > > > scope, but DeleteTopicsRequest is not listed among the
> > Future
> > > >>> Work
> > > >>> > APIs
> > > >>> > > > > > either.
> > > >>> > > > > >
> > > >>> > > > > > Kind regards,
> > > >>> > > > > >
> > > >>> > > > > > Tom
> > > >>> > > > > >
> > > >>> > > > > > On Thu, Sep 10, 2020 at 3:59 PM Satish Duggana <
> > > >>> > > > satish.duggana@gmail.com
> > > >>> > > > > > wrote:
> > > >>> > > > > >
> > > >>> > > > > > > Thanks Lucas/Justine for the nice KIP.
> > > >>> > > > > > >
> > > >>> > > > > > > It has several benefits which also include simplifying
> > the
> > > >>> topic
> > > >>> > > > > > > deletion process by controller and logs cleanup by
> > brokers
> > > in
> > > >>> > corner
> > > >>> > > > > > > cases.
> > > >>> > > > > > >
> > > >>> > > > > > > Best,
> > > >>> > > > > > > Satish.
> > > >>> > > > > > >
> > > >>> > > > > > > On Wed, Sep 9, 2020 at 10:07 PM Justine Olshan <
> > > >>> > jolshan@confluent.io
> > > >>> > > > > > > wrote:
> > > >>> > > > > > > > Hello all, it's been almost a year! I've made some
> > > changes
> > > >>> to
> > > >>> > this
> > > >>> > > > > KIP
> > > >>> > > > > > > and hope to continue the discussion.
> > > >>> > > > > > > > One of the main changes I've added is now the
> metadata
> > > >>> response
> > > >>> > > > will
> > > >>> > > > > > > include the topic ID (as Colin suggested). Clients can
> > > >>> obtain the
> > > >>> > > > > topicID
> > > >>> > > > > > > of a given topic through a TopicDescription. The
> topicId
> > > will
> > > >>> > also be
> > > >>> > > > > > > included with the UpdateMetadata request.
> > > >>> > > > > > > > Let me know what you all think.
> > > >>> > > > > > > > Thank you,
> > > >>> > > > > > > > Justine
> > > >>> > > > > > > >
> > > >>> > > > > > > > On 2019/09/13 16:38:26, "Colin McCabe" <
> > > cmccabe@apache.org
> > > >>> >
> > > >>> > wrote:
> > > >>> > > > > > > > > Hi Lucas,
> > > >>> > > > > > > > >
> > > >>> > > > > > > > > Thanks for tackling this.  Topic IDs are a great
> > idea,
> > > >>> and
> > > >>> > this
> > > >>> > > > is
> > > >>> > > > > a
> > > >>> > > > > > > really good writeup.
> > > >>> > > > > > > > > For /brokers/topics/[topic], the schema version
> > should
> > > be
> > > >>> > bumped
> > > >>> > > > to
> > > >>> > > > > > > version 3, rather than 2.  KIP-455 bumped the version
> of
> > > this
> > > >>> > znode
> > > >>> > > > to
> > > >>> > > > > 2
> > > >>> > > > > > > already :)
> > > >>> > > > > > > > > Given that we're going to be seeing these things as
> > > >>> strings
> > > >>> > as
> > > >>> > > > lot
> > > >>> > > > > (in
> > > >>> > > > > > > logs, in ZooKeeper, on the command-line, etc.), does it
> > > make
> > > >>> > sense to
> > > >>> > > > > use
> > > >>> > > > > > > base64 when converting them to strings?
> > > >>> > > > > > > > > Here is an example of the hex representation:
> > > >>> > > > > > > > > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> > > >>> > > > > > > > >
> > > >>> > > > > > > > > And here is an example in base64.
> > > >>> > > > > > > > > b8tRS7h4TJ2Vt43Dp85v2A
> > > >>> > > > > > > > >
> > > >>> > > > > > > > > The base64 version saves 15 letters (to be fair, 4
> of
> > > >>> those
> > > >>> > were
> > > >>> > > > > > > dashes that we could have elided in the hex
> > > representation.)
> > > >>> > > > > > > > > Another thing to consider is that we should specify
> > > that
> > > >>> the
> > > >>> > > > > > > all-zeroes UUID is not a valid topic UUID.   We can't
> use
> > > >>> null
> > > >>> > for
> > > >>> > > > this
> > > >>> > > > > > > because we can't pass a null UUID over the RPC protocol
> > > >>> (there
> > > >>> > is no
> > > >>> > > > > > > special pattern for null, nor do we want to waste space
> > > >>> reserving
> > > >>> > > > such
> > > >>> > > > > a
> > > >>> > > > > > > pattern.)
> > > >>> > > > > > > > > Maybe I missed it, but did you describe "migration
> > > of...
> > > >>> > existing
> > > >>> > > > > > > topic[s] without topic IDs" in detail in any section?
> It
> > > >>> seems
> > > >>> > like
> > > >>> > > > > when
> > > >>> > > > > > > the new controller becomes active, it should just
> > generate
> > > >>> random
> > > >>> > > > > UUIDs for
> > > >>> > > > > > > these, and write the random UUIDs back to ZooKeeper.
> It
> > > >>> would be
> > > >>> > > > good
> > > >>> > > > > to
> > > >>> > > > > > > spell that out.  We should make it clear that this
> > happens
> > > >>> > regardless
> > > >>> > > > > of
> > > >>> > > > > > > the inter-broker protocol version (it's a compatible
> > > change).
> > > >>> > > > > > > > > "LeaderAndIsrRequests including an
> is_every_partition
> > > >>> flag"
> > > >>> > > > seems a
> > > >>> > > > > > > bit wordy.  Can we just call these "full
> > > >>> LeaderAndIsrRequests"?
> > > >>> > Then
> > > >>> > > > > the
> > > >>> > > > > > > RPC field could be named "full".  Also, it would
> probably
> > > be
> > > >>> > better
> > > >>> > > > > for the
> > > >>> > > > > > > RPC field to be an enum of { UNSPECIFIED, INCREMENTAL,
> > FULL
> > > >>> }, so
> > > >>> > > > that
> > > >>> > > > > we
> > > >>> > > > > > > can cleanly handle old versions (by treating them as
> > > >>> UNSPECIFIED)
> > > >>> > > > > > > > > In the LeaderAndIsrRequest section, you write "A
> > final
> > > >>> > deletion
> > > >>> > > > > event
> > > >>> > > > > > > will be secheduled for X ms after the
> LeaderAndIsrRequest
> > > was
> > > >>> > first
> > > >>> > > > > > > received..."  I guess the X was a placeholder that you
> > > >>> intended
> > > >>> > to
> > > >>> > > > > replace
> > > >>> > > > > > > before posting? :)  In any case, this seems like the
> kind
> > > of
> > > >>> > thing
> > > >>> > > > we'd
> > > >>> > > > > > > want a configuration for.  Let's describe that
> > > configuration
> > > >>> key
> > > >>> > > > > somewhere
> > > >>> > > > > > > in this KIP, including what its default value is.
> > > >>> > > > > > > > > We should probably also log a bunch of messages at
> > WARN
> > > >>> level
> > > >>> > > > when
> > > >>> > > > > > > something is scheduled for deletion, as well.  (Maybe
> > this
> > > >>> was
> > > >>> > > > > assumed, but
> > > >>> > > > > > > it would be good to mention it).
> > > >>> > > > > > > > > I feel like there are a few sections that should be
> > > >>> moved to
> > > >>> > > > > "rejected
> > > >>> > > > > > > alternatives."  For example, in the DeleteTopics
> section,
> > > >>> since
> > > >>> > we're
> > > >>> > > > > not
> > > >>> > > > > > > going to do option 1 or 2, these should be moved into
> > > >>> "rejected
> > > >>> > > > > > > alternatives,"  rather than appearing inline.  Another
> > case
> > > >>> is
> > > >>> > the
> > > >>> > > > > "Should
> > > >>> > > > > > > we remove topic name from the protocol where possible"
> > > >>> section.
> > > >>> > This
> > > >>> > > > > is
> > > >>> > > > > > > clearly discussing a design alternative that we're not
> > > >>> proposing
> > > >>> > to
> > > >>> > > > > > > implement: removing the topic name from those
> protocols.
> > > >>> > > > > > > > > Is it really necessary to have a new
> > > >>> > /admin/delete_topics_by_id
> > > >>> > > > > path
> > > >>> > > > > > > in ZooKeeper?  It seems like we don't really need this.
> > > >>> Whenever
> > > >>> > > > > there is
> > > >>> > > > > > > a new controller, we'll send out full
> > LeaderAndIsrRequests
> > > >>> which
> > > >>> > will
> > > >>> > > > > > > trigger the stale topics to be cleaned up.   The active
> > > >>> > controller
> > > >>> > > > will
> > > >>> > > > > > > also send the full LeaderAndIsrRequest to brokers that
> > are
> > > >>> just
> > > >>> > > > > starting
> > > >>> > > > > > > up.    So we don't really need this kind of two-phase
> > > commit
> > > >>> > (send
> > > >>> > > > out
> > > >>> > > > > > > StopReplicasRequest, get ACKs from all nodes, commit by
> > > >>> removing
> > > >>> > > > > > > /admin/delete_topics node) any more.
> > > >>> > > > > > > > > You mention that FetchRequest will now include UUID
> > to
> > > >>> avoid
> > > >>> > > > issues
> > > >>> > > > > > > where requests are made to stale partitions.  However,
> > > >>> adding a
> > > >>> > UUID
> > > >>> > > > to
> > > >>> > > > > > > MetadataRequest is listed as future work, out of scope
> > for
> > > >>> this
> > > >>> > KIP.
> > > >>> > > > > How
> > > >>> > > > > > > will the client learn what the topic UUID is, if the
> > > metadata
> > > >>> > > > response
> > > >>> > > > > > > doesn't include that information?  It seems like adding
> > the
> > > >>> UUID
> > > >>> > to
> > > >>> > > > > > > MetadataResponse would be an improvement here that
> might
> > > not
> > > >>> be
> > > >>> > too
> > > >>> > > > > hard to
> > > >>> > > > > > > make.
> > > >>> > > > > > > > > best,
> > > >>> > > > > > > > > Colin
> > > >>> > > > > > > > >
> > > >>> > > > > > > > >
> > > >>> > > > > > > > > On Mon, Sep 9, 2019, at 17:48, Ryanne Dolan wrote:
> > > >>> > > > > > > > > > Lucas, this would be great. I've run into issues
> > with
> > > >>> > topics
> > > >>> > > > > being
> > > >>> > > > > > > > > > resurrected accidentally, since a client cannot
> > > easily
> > > >>> > > > > distinguish
> > > >>> > > > > > > between
> > > >>> > > > > > > > > > a deleted topic and a new topic with the same
> name.
> > > I'd
> > > >>> > need
> > > >>> > > > the
> > > >>> > > > > ID
> > > >>> > > > > > > > > > accessible from the client to solve that issue,
> but
> > > >>> this
> > > >>> > is a
> > > >>> > > > > good
> > > >>> > > > > > > first
> > > >>> > > > > > > > > > step.
> > > >>> > > > > > > > > >
> > > >>> > > > > > > > > > Ryanne
> > > >>> > > > > > > > > >
> > > >>> > > > > > > > > > On Wed, Sep 4, 2019 at 1:41 PM Lucas Bradstreet <
> > > >>> > > > > lucas@confluent.io>
> > > >>> > > > > > > wrote:
> > > >>> > > > > > > > > > > Hi all,
> > > >>> > > > > > > > > > >
> > > >>> > > > > > > > > > > I would like to kick off discussion of KIP-516,
> > an
> > > >>> > > > > implementation
> > > >>> > > > > > > of topic
> > > >>> > > > > > > > > > > IDs for Kafka. Topic IDs aim to solve topic
> > > >>> uniqueness
> > > >>> > > > > problems in
> > > >>> > > > > > > Kafka,
> > > >>> > > > > > > > > > > where referring to a topic by name alone is
> > > >>> insufficient.
> > > >>> > > > Such
> > > >>> > > > > > > cases
> > > >>> > > > > > > > > > > include when a topic has been deleted and
> > recreated
> > > >>> with
> > > >>> > the
> > > >>> > > > > same
> > > >>> > > > > > > name.
> > > >>> > > > > > > > > > > Unique identifiers will help simplify and
> improve
> > > >>> Kafka's
> > > >>> > > > topic
> > > >>> > > > > > > deletion
> > > >>> > > > > > > > > > > process, as well as prevent cases where brokers
> > may
> > > >>> > > > incorrectly
> > > >>> > > > > > > interact
> > > >>> > > > > > > > > > > with stale versions of topics.
> > > >>> > > > > > > > > > >
> > > >>> > > > > > > > > > >
> > > >>> > > > > > > > > > >
> > > >>> > > >
> > > >>> >
> > > >>>
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > > >>> > > > > > > > > > > Looking forward to your thoughts.
> > > >>> > > > > > > > > > >
> > > >>> > > > > > > > > > > Lucas
> > > >>> > > > > > > > > > >
> > > >>> >
> > > >>> >
> > > >>>
> > > >>
> > >
> >
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Justine Olshan <jo...@confluent.io>]

Hi Jun,
Thanks for the comments.

12. I bumped the LeaderAndIsrRequest because I removed the topic name field
in the response. It may be possible to avoid bumping the version without
that change, but I may be missing something.
I believe StopReplica is actually on version 3 now, but because version 2
is flexible, I kept that listed as version 2 on the KIP page. However, you
may be right in that we may need to bump the version on StopReplica to deal
with deletion differently as mentioned above. I don't know if I have a big
preference over used tagged fields or not.

13. I was thinking that in the case where the file and the request topic
ids don't match, it means that the broker's topic/the one in the file has
been deleted. In that case, we would need to delete the old topic and start
receiving the new version. If the topic name were to change, but the ids
still matched, the file would also need to update. Am I missing a case
where the file would be correct and not the request?

Thanks,
Justine

On Thu, Sep 17, 2020 at 3:18 PM Jun Rao <ju...@confluent.io> wrote:

> Hi, Justine,
>
> Thanks for the reply. A couple of more comments below.
>
> 12. ListOffset and OffsetForLeader currently don't support flexible fields.
> So, we have to bump up the version number and use IBP at least for these
> two requests. Note that it seems 2.7.0 will require IBP anyway because of
> changes in KAFKA-10435. Also, it seems that the version for
> LeaderAndIsrRequest and StopReplica are bumped even though we only added a
> tagged field. But since IBP is needed anyway, we may want to revisit the
> overall tagged field choice.
>
> 13. The only downside is the potential confusion on which one is the source
> of truth if they don't match. Another option is to include those fields in
> the metadata file when we actually change the directory structure.
>
> Thanks,
>
> Jun
>
> On Thu, Sep 17, 2020 at 2:01 PM Justine Olshan <jo...@confluent.io>
> wrote:
>
> > Hello all,
> >
> > I've thought some more about removing the topic name field from some of
> the
> > requests. On closer inspection of the requests/responses, it seems that
> the
> > internal changes would be much larger than I expected. Some protocols
> > involve clients, so they would require changes too. I'm thinking that for
> > now, removing the topic name from these requests and responses are out of
> > scope.
> >
> > I have decided to just keep the change LeaderAndIsrResponse to remove the
> > topic name, and have updated the KIP to reflect this change. I have also
> > mentioned the other requests and responses in future work.
> >
> > I'm hoping to start the voting process soon, so let me know if there is
> > anything else we should discuss.
> >
> > Thank you,
> > Justine
> >
> > On Tue, Sep 15, 2020 at 3:57 PM Justine Olshan <jo...@confluent.io>
> > wrote:
> >
> > > Hello again,
> > > To follow up on some of the other comments:
> > >
> > > 10/11) We can remove the topic name from these requests/responses, and
> > > that means we will just have to make a few internal changes to make
> > > partitions accessible by topic id and partition. I can update the KIP
> to
> > > remove them unless anyone thinks they should stay.
> > >
> > > 12) Addressed in the previous email. I've updated the KIP to include
> > > tagged fields for the requests and responses. (More on that below)
> > >
> > > 13) I think part of the idea for including this information is to
> prepare
> > > for future changes. Perhaps the directory structure might change from
> > > topicName_partitionNumber to something like topicID_partitionNumber.
> Then
> > > it would be useful to have the topic name in the file since it would
> not
> > be
> > > in the directory structure. Supporting topic renames might be easier if
> > the
> > > other fields are included. Would there be any downsides to including
> this
> > > information?
> > >
> > > 14)  Yes, we would need to copy the partition metadata file in this
> > > process. I've updated the KIP to include this.
> > >
> > > 15) I believe Lucas meant v1 and v2 here. He was referring to how the
> > > requests would fall under different IBP and meant that older brokers
> > would
> > > have to use the older version of the request and the existing topic
> > > deletion process. At first, it seemed like tagged fields would resolve
> > > the IBP issue. However, we may need IBP for this request after all
> since
> > > the controller handles the topic deletion differently depending on the
> > IBP
> > > version. In an older version, we can't just send a StopReplica delete
> the
> > > topic immediately like we'd want to for this KIP.
> > >
> > > This makes me wonder if we want tagged fields on all the requests after
> > > all. Let me know your thoughts!
> > >
> > > Justine
> > >
> > > On Tue, Sep 15, 2020 at 1:03 PM Justine Olshan <jo...@confluent.io>
> > > wrote:
> > >
> > >> Hi all,
> > >> Jun brought up a good point in his last email about tagged fields, and
> > >> I've updated the KIP to reflect that the changes to requests and
> > responses
> > >> will be in the form of tagged fields to avoid changing IBP.
> > >>
> > >> Jun: I plan on sending a followup email to address some of the other
> > >> points.
> > >>
> > >> Thanks,
> > >> Justine
> > >>
> > >> On Mon, Sep 14, 2020 at 4:25 PM Jun Rao <ju...@confluent.io> wrote:
> > >>
> > >>> Hi, Justine,
> > >>>
> > >>> Thanks for the updated KIP. A few comments below.
> > >>>
> > >>> 10. LeaderAndIsr Response: Do we need the topic name?
> > >>>
> > >>> 11. For the changed request/response, other than LeaderAndIsr,
> > >>> UpdateMetadata, Metadata, do we need to include the topic name?
> > >>>
> > >>> 12. It seems that upgrades don't require IBP. Does that mean the new
> > >>> fields
> > >>> in all the request/response are added as tagged fields without
> bumping
> > up
> > >>> the request version? It would be useful to make that clear.
> > >>>
> > >>> 13. Partition Metadata file: Do we need to include the topic name and
> > the
> > >>> partition id since they are implied in the directory name?
> > >>>
> > >>> 14. In the JBOD mode, we support moving a partition's data from one
> > disk
> > >>> to
> > >>> another. Will the new partition metadata file be copied during that
> > >>> process?
> > >>>
> > >>> 15. The KIP says "Remove deleted topics from replicas by sending
> > >>> StopReplicaRequest V2 for any topics which do not contain a topic ID,
> > and
> > >>> V3 for any topics which do contain a topic ID.". However, it seems
> the
> > >>> updated controller will create all missing topic IDs first before
> doing
> > >>> other actions. So, is StopReplicaRequest V2 needed?
> > >>>
> > >>> Jun
> > >>>
> > >>> On Fri, Sep 11, 2020 at 10:31 AM John Roesler <vv...@apache.org>
> > >>> wrote:
> > >>>
> > >>> > Thanks, Justine!
> > >>> >
> > >>> > Your response seems compelling to me.
> > >>> >
> > >>> > -John
> > >>> >
> > >>> > On Fri, 2020-09-11 at 10:17 -0700, Justine Olshan wrote:
> > >>> > > Hello all,
> > >>> > > Thanks for continuing the discussion! I have a few responses to
> > your
> > >>> > points.
> > >>> > >
> > >>> > > Tom: You are correct in that this KIP has not mentioned the
> > >>> > > DeleteTopicsRequest. I think that this would be out of scope for
> > >>> now, but
> > >>> > > may be something worth adding in the future.
> > >>> > >
> > >>> > > John: We did consider sequence ids, but there are a few reasons
> to
> > >>> favor
> > >>> > > UUIDs. There are several cases where topics from different
> clusters
> > >>> may
> > >>> > > interact now and in the future. For example, Mirror Maker 2 may
> > >>> benefit
> > >>> > > from being able to detect when a cluster being mirrored is
> deleted
> > >>> and
> > >>> > > recreated and globally unique identifiers would make resolving
> > issues
> > >>> > > easier than sequence IDs which may collide between clusters.
> > KIP-405
> > >>> > > (tiered storage) will also benefit from globally unique IDs as
> > shared
> > >>> > > buckets may be used between clusters.
> > >>> > >
> > >>> > > Globally unique IDs would also make functionality like moving
> > topics
> > >>> > > between disparate clusters easier in the future, simplify any
> > future
> > >>> > > implementations of backups and restores, and more. In general,
> > >>> unique IDs
> > >>> > > would ensure that the source cluster topics do not conflict with
> > the
> > >>> > > destination cluster topics.
> > >>> > >
> > >>> > > If we were to use sequence ids, we would need sufficiently large
> > >>> cluster
> > >>> > > ids to be stored with the topic identifiers or we run the risk of
> > >>> > > collisions. This will give up any advantage in compactness that
> > >>> sequence
> > >>> > > numbers may bring. Given these advantages I think it makes sense
> to
> > >>> use
> > >>> > > UUIDs.
> > >>> > >
> > >>> > > Gokul: This is an interesting idea, but this is a breaking
> change.
> > >>> Out of
> > >>> > > scope for now, but maybe worth discussing in the future.
> > >>> > >
> > >>> > > Hope this explains some of the decisions,
> > >>> > >
> > >>> > > Justine
> > >>> > >
> > >>> > >
> > >>> > >
> > >>> > > On Fri, Sep 11, 2020 at 8:27 AM Gokul Ramanan Subramanian <
> > >>> > > gokul2411s@gmail.com> wrote:
> > >>> > >
> > >>> > > > Hi.
> > >>> > > >
> > >>> > > > Thanks for the KIP.
> > >>> > > >
> > >>> > > > Have you thought about whether it makes sense to support
> > >>> authorizing a
> > >>> > > > principal for a topic ID rather than a topic name to achieve
> > >>> tighter
> > >>> > > > security?
> > >>> > > >
> > >>> > > > Or is the topic ID fundamentally an internal detail similar to
> > >>> epochs
> > >>> > used
> > >>> > > > in a bunch of other places in Kafka?
> > >>> > > >
> > >>> > > > Thanks.
> > >>> > > >
> > >>> > > > On Fri, Sep 11, 2020 at 4:06 PM John Roesler <
> > vvcephei@apache.org>
> > >>> > wrote:
> > >>> > > >
> > >>> > > > > Hello Justine,
> > >>> > > > >
> > >>> > > > > Thanks for the KIP!
> > >>> > > > >
> > >>> > > > > I happen to have been confronted recently with the need to
> keep
> > >>> > track of
> > >>> > > > a
> > >>> > > > > large number of topics as compactly as possible. I was going
> to
> > >>> come
> > >>> > up
> > >>> > > > > with some way to dictionary encode the topic names as
> integers,
> > >>> but
> > >>> > this
> > >>> > > > > seems much better!
> > >>> > > > >
> > >>> > > > > Apologies if this has been raised before, but I’m wondering
> > >>> about the
> > >>> > > > > choice of UUID vs sequence number for the ids. Typically,
> I’ve
> > >>> seen
> > >>> > UUIDs
> > >>> > > > > in two situations:
> > >>> > > > > 1. When processes need to generate non-colliding identifiers
> > >>> without
> > >>> > > > > coordination.
> > >>> > > > > 2. When the identifier needs to be “universally unique”;
> I.e.,
> > >>> the
> > >>> > > > > identifier needs to distinguish the entity from all other
> > >>> entities
> > >>> > that
> > >>> > > > > could ever exist. This is useful in cases where entities from
> > all
> > >>> > kinds
> > >>> > > > of
> > >>> > > > > systems get mixed together, such as when dumping logs from
> all
> > >>> > processes
> > >>> > > > in
> > >>> > > > > a company into a common system.
> > >>> > > > >
> > >>> > > > > Maybe I’m being short-sighted, but it doesn’t seem like
> either
> > >>> really
> > >>> > > > > applies here. It seems like the brokers could and would
> achieve
> > >>> > consensus
> > >>> > > > > when creating a topic anyway, which is all that’s required to
> > >>> > generate
> > >>> > > > > non-colliding sequence ids. For the second, as you mention,
> we
> > >>> could
> > >>> > > > assign
> > >>> > > > > a UUID to the cluster as a whole, which would render any
> > resource
> > >>> > scoped
> > >>> > > > to
> > >>> > > > > the broker universally unique as well.
> > >>> > > > >
> > >>> > > > > The reason I mention this is that, although a UUID is way
> more
> > >>> > compact
> > >>> > > > > than topic names, it’s still 16 bytes. In contrast, a 4-byte
> > >>> integer
> > >>> > > > > sequence id would give us 4 billion unique topics per
> cluster,
> > >>> which
> > >>> > > > seems
> > >>> > > > > like enough ;)
> > >>> > > > >
> > >>> > > > > Considering the number of different times these topic
> > >>> identifiers are
> > >>> > > > sent
> > >>> > > > > over the wire or stored in memory, it seems like it might be
> > >>> worth
> > >>> > the
> > >>> > > > > additional 4x space savings.
> > >>> > > > >
> > >>> > > > > What do you think about this?
> > >>> > > > >
> > >>> > > > > Thanks,
> > >>> > > > > John
> > >>> > > > >
> > >>> > > > > On Fri, Sep 11, 2020, at 03:20, Tom Bentley wrote:
> > >>> > > > > > Hi Justine,
> > >>> > > > > >
> > >>> > > > > > This looks like a very welcome improvement. Thanks!
> > >>> > > > > >
> > >>> > > > > > Maybe I missed it, but the KIP doesn't seem to mention
> > changing
> > >>> > > > > > DeleteTopicsRequest to identify the topic using an id.
> Maybe
> > >>> > that's out
> > >>> > > > > of
> > >>> > > > > > scope, but DeleteTopicsRequest is not listed among the
> Future
> > >>> Work
> > >>> > APIs
> > >>> > > > > > either.
> > >>> > > > > >
> > >>> > > > > > Kind regards,
> > >>> > > > > >
> > >>> > > > > > Tom
> > >>> > > > > >
> > >>> > > > > > On Thu, Sep 10, 2020 at 3:59 PM Satish Duggana <
> > >>> > > > satish.duggana@gmail.com
> > >>> > > > > > wrote:
> > >>> > > > > >
> > >>> > > > > > > Thanks Lucas/Justine for the nice KIP.
> > >>> > > > > > >
> > >>> > > > > > > It has several benefits which also include simplifying
> the
> > >>> topic
> > >>> > > > > > > deletion process by controller and logs cleanup by
> brokers
> > in
> > >>> > corner
> > >>> > > > > > > cases.
> > >>> > > > > > >
> > >>> > > > > > > Best,
> > >>> > > > > > > Satish.
> > >>> > > > > > >
> > >>> > > > > > > On Wed, Sep 9, 2020 at 10:07 PM Justine Olshan <
> > >>> > jolshan@confluent.io
> > >>> > > > > > > wrote:
> > >>> > > > > > > > Hello all, it's been almost a year! I've made some
> > changes
> > >>> to
> > >>> > this
> > >>> > > > > KIP
> > >>> > > > > > > and hope to continue the discussion.
> > >>> > > > > > > > One of the main changes I've added is now the metadata
> > >>> response
> > >>> > > > will
> > >>> > > > > > > include the topic ID (as Colin suggested). Clients can
> > >>> obtain the
> > >>> > > > > topicID
> > >>> > > > > > > of a given topic through a TopicDescription. The topicId
> > will
> > >>> > also be
> > >>> > > > > > > included with the UpdateMetadata request.
> > >>> > > > > > > > Let me know what you all think.
> > >>> > > > > > > > Thank you,
> > >>> > > > > > > > Justine
> > >>> > > > > > > >
> > >>> > > > > > > > On 2019/09/13 16:38:26, "Colin McCabe" <
> > cmccabe@apache.org
> > >>> >
> > >>> > wrote:
> > >>> > > > > > > > > Hi Lucas,
> > >>> > > > > > > > >
> > >>> > > > > > > > > Thanks for tackling this.  Topic IDs are a great
> idea,
> > >>> and
> > >>> > this
> > >>> > > > is
> > >>> > > > > a
> > >>> > > > > > > really good writeup.
> > >>> > > > > > > > > For /brokers/topics/[topic], the schema version
> should
> > be
> > >>> > bumped
> > >>> > > > to
> > >>> > > > > > > version 3, rather than 2.  KIP-455 bumped the version of
> > this
> > >>> > znode
> > >>> > > > to
> > >>> > > > > 2
> > >>> > > > > > > already :)
> > >>> > > > > > > > > Given that we're going to be seeing these things as
> > >>> strings
> > >>> > as
> > >>> > > > lot
> > >>> > > > > (in
> > >>> > > > > > > logs, in ZooKeeper, on the command-line, etc.), does it
> > make
> > >>> > sense to
> > >>> > > > > use
> > >>> > > > > > > base64 when converting them to strings?
> > >>> > > > > > > > > Here is an example of the hex representation:
> > >>> > > > > > > > > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> > >>> > > > > > > > >
> > >>> > > > > > > > > And here is an example in base64.
> > >>> > > > > > > > > b8tRS7h4TJ2Vt43Dp85v2A
> > >>> > > > > > > > >
> > >>> > > > > > > > > The base64 version saves 15 letters (to be fair, 4 of
> > >>> those
> > >>> > were
> > >>> > > > > > > dashes that we could have elided in the hex
> > representation.)
> > >>> > > > > > > > > Another thing to consider is that we should specify
> > that
> > >>> the
> > >>> > > > > > > all-zeroes UUID is not a valid topic UUID.   We can't use
> > >>> null
> > >>> > for
> > >>> > > > this
> > >>> > > > > > > because we can't pass a null UUID over the RPC protocol
> > >>> (there
> > >>> > is no
> > >>> > > > > > > special pattern for null, nor do we want to waste space
> > >>> reserving
> > >>> > > > such
> > >>> > > > > a
> > >>> > > > > > > pattern.)
> > >>> > > > > > > > > Maybe I missed it, but did you describe "migration
> > of...
> > >>> > existing
> > >>> > > > > > > topic[s] without topic IDs" in detail in any section?  It
> > >>> seems
> > >>> > like
> > >>> > > > > when
> > >>> > > > > > > the new controller becomes active, it should just
> generate
> > >>> random
> > >>> > > > > UUIDs for
> > >>> > > > > > > these, and write the random UUIDs back to ZooKeeper.  It
> > >>> would be
> > >>> > > > good
> > >>> > > > > to
> > >>> > > > > > > spell that out.  We should make it clear that this
> happens
> > >>> > regardless
> > >>> > > > > of
> > >>> > > > > > > the inter-broker protocol version (it's a compatible
> > change).
> > >>> > > > > > > > > "LeaderAndIsrRequests including an is_every_partition
> > >>> flag"
> > >>> > > > seems a
> > >>> > > > > > > bit wordy.  Can we just call these "full
> > >>> LeaderAndIsrRequests"?
> > >>> > Then
> > >>> > > > > the
> > >>> > > > > > > RPC field could be named "full".  Also, it would probably
> > be
> > >>> > better
> > >>> > > > > for the
> > >>> > > > > > > RPC field to be an enum of { UNSPECIFIED, INCREMENTAL,
> FULL
> > >>> }, so
> > >>> > > > that
> > >>> > > > > we
> > >>> > > > > > > can cleanly handle old versions (by treating them as
> > >>> UNSPECIFIED)
> > >>> > > > > > > > > In the LeaderAndIsrRequest section, you write "A
> final
> > >>> > deletion
> > >>> > > > > event
> > >>> > > > > > > will be secheduled for X ms after the LeaderAndIsrRequest
> > was
> > >>> > first
> > >>> > > > > > > received..."  I guess the X was a placeholder that you
> > >>> intended
> > >>> > to
> > >>> > > > > replace
> > >>> > > > > > > before posting? :)  In any case, this seems like the kind
> > of
> > >>> > thing
> > >>> > > > we'd
> > >>> > > > > > > want a configuration for.  Let's describe that
> > configuration
> > >>> key
> > >>> > > > > somewhere
> > >>> > > > > > > in this KIP, including what its default value is.
> > >>> > > > > > > > > We should probably also log a bunch of messages at
> WARN
> > >>> level
> > >>> > > > when
> > >>> > > > > > > something is scheduled for deletion, as well.  (Maybe
> this
> > >>> was
> > >>> > > > > assumed, but
> > >>> > > > > > > it would be good to mention it).
> > >>> > > > > > > > > I feel like there are a few sections that should be
> > >>> moved to
> > >>> > > > > "rejected
> > >>> > > > > > > alternatives."  For example, in the DeleteTopics section,
> > >>> since
> > >>> > we're
> > >>> > > > > not
> > >>> > > > > > > going to do option 1 or 2, these should be moved into
> > >>> "rejected
> > >>> > > > > > > alternatives,"  rather than appearing inline.  Another
> case
> > >>> is
> > >>> > the
> > >>> > > > > "Should
> > >>> > > > > > > we remove topic name from the protocol where possible"
> > >>> section.
> > >>> > This
> > >>> > > > > is
> > >>> > > > > > > clearly discussing a design alternative that we're not
> > >>> proposing
> > >>> > to
> > >>> > > > > > > implement: removing the topic name from those protocols.
> > >>> > > > > > > > > Is it really necessary to have a new
> > >>> > /admin/delete_topics_by_id
> > >>> > > > > path
> > >>> > > > > > > in ZooKeeper?  It seems like we don't really need this.
> > >>> Whenever
> > >>> > > > > there is
> > >>> > > > > > > a new controller, we'll send out full
> LeaderAndIsrRequests
> > >>> which
> > >>> > will
> > >>> > > > > > > trigger the stale topics to be cleaned up.   The active
> > >>> > controller
> > >>> > > > will
> > >>> > > > > > > also send the full LeaderAndIsrRequest to brokers that
> are
> > >>> just
> > >>> > > > > starting
> > >>> > > > > > > up.    So we don't really need this kind of two-phase
> > commit
> > >>> > (send
> > >>> > > > out
> > >>> > > > > > > StopReplicasRequest, get ACKs from all nodes, commit by
> > >>> removing
> > >>> > > > > > > /admin/delete_topics node) any more.
> > >>> > > > > > > > > You mention that FetchRequest will now include UUID
> to
> > >>> avoid
> > >>> > > > issues
> > >>> > > > > > > where requests are made to stale partitions.  However,
> > >>> adding a
> > >>> > UUID
> > >>> > > > to
> > >>> > > > > > > MetadataRequest is listed as future work, out of scope
> for
> > >>> this
> > >>> > KIP.
> > >>> > > > > How
> > >>> > > > > > > will the client learn what the topic UUID is, if the
> > metadata
> > >>> > > > response
> > >>> > > > > > > doesn't include that information?  It seems like adding
> the
> > >>> UUID
> > >>> > to
> > >>> > > > > > > MetadataResponse would be an improvement here that might
> > not
> > >>> be
> > >>> > too
> > >>> > > > > hard to
> > >>> > > > > > > make.
> > >>> > > > > > > > > best,
> > >>> > > > > > > > > Colin
> > >>> > > > > > > > >
> > >>> > > > > > > > >
> > >>> > > > > > > > > On Mon, Sep 9, 2019, at 17:48, Ryanne Dolan wrote:
> > >>> > > > > > > > > > Lucas, this would be great. I've run into issues
> with
> > >>> > topics
> > >>> > > > > being
> > >>> > > > > > > > > > resurrected accidentally, since a client cannot
> > easily
> > >>> > > > > distinguish
> > >>> > > > > > > between
> > >>> > > > > > > > > > a deleted topic and a new topic with the same name.
> > I'd
> > >>> > need
> > >>> > > > the
> > >>> > > > > ID
> > >>> > > > > > > > > > accessible from the client to solve that issue, but
> > >>> this
> > >>> > is a
> > >>> > > > > good
> > >>> > > > > > > first
> > >>> > > > > > > > > > step.
> > >>> > > > > > > > > >
> > >>> > > > > > > > > > Ryanne
> > >>> > > > > > > > > >
> > >>> > > > > > > > > > On Wed, Sep 4, 2019 at 1:41 PM Lucas Bradstreet <
> > >>> > > > > lucas@confluent.io>
> > >>> > > > > > > wrote:
> > >>> > > > > > > > > > > Hi all,
> > >>> > > > > > > > > > >
> > >>> > > > > > > > > > > I would like to kick off discussion of KIP-516,
> an
> > >>> > > > > implementation
> > >>> > > > > > > of topic
> > >>> > > > > > > > > > > IDs for Kafka. Topic IDs aim to solve topic
> > >>> uniqueness
> > >>> > > > > problems in
> > >>> > > > > > > Kafka,
> > >>> > > > > > > > > > > where referring to a topic by name alone is
> > >>> insufficient.
> > >>> > > > Such
> > >>> > > > > > > cases
> > >>> > > > > > > > > > > include when a topic has been deleted and
> recreated
> > >>> with
> > >>> > the
> > >>> > > > > same
> > >>> > > > > > > name.
> > >>> > > > > > > > > > > Unique identifiers will help simplify and improve
> > >>> Kafka's
> > >>> > > > topic
> > >>> > > > > > > deletion
> > >>> > > > > > > > > > > process, as well as prevent cases where brokers
> may
> > >>> > > > incorrectly
> > >>> > > > > > > interact
> > >>> > > > > > > > > > > with stale versions of topics.
> > >>> > > > > > > > > > >
> > >>> > > > > > > > > > >
> > >>> > > > > > > > > > >
> > >>> > > >
> > >>> >
> > >>>
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > >>> > > > > > > > > > > Looking forward to your thoughts.
> > >>> > > > > > > > > > >
> > >>> > > > > > > > > > > Lucas
> > >>> > > > > > > > > > >
> > >>> >
> > >>> >
> > >>>
> > >>
> >
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Jun Rao <ju...@confluent.io>]

Hi, Justine,

Thanks for the reply. A couple of more comments below.

12. ListOffset and OffsetForLeader currently don't support flexible fields.
So, we have to bump up the version number and use IBP at least for these
two requests. Note that it seems 2.7.0 will require IBP anyway because of
changes in KAFKA-10435. Also, it seems that the version for
LeaderAndIsrRequest and StopReplica are bumped even though we only added a
tagged field. But since IBP is needed anyway, we may want to revisit the
overall tagged field choice.

13. The only downside is the potential confusion on which one is the source
of truth if they don't match. Another option is to include those fields in
the metadata file when we actually change the directory structure.

Thanks,

Jun

On Thu, Sep 17, 2020 at 2:01 PM Justine Olshan <jo...@confluent.io> wrote:

> Hello all,
>
> I've thought some more about removing the topic name field from some of the
> requests. On closer inspection of the requests/responses, it seems that the
> internal changes would be much larger than I expected. Some protocols
> involve clients, so they would require changes too. I'm thinking that for
> now, removing the topic name from these requests and responses are out of
> scope.
>
> I have decided to just keep the change LeaderAndIsrResponse to remove the
> topic name, and have updated the KIP to reflect this change. I have also
> mentioned the other requests and responses in future work.
>
> I'm hoping to start the voting process soon, so let me know if there is
> anything else we should discuss.
>
> Thank you,
> Justine
>
> On Tue, Sep 15, 2020 at 3:57 PM Justine Olshan <jo...@confluent.io>
> wrote:
>
> > Hello again,
> > To follow up on some of the other comments:
> >
> > 10/11) We can remove the topic name from these requests/responses, and
> > that means we will just have to make a few internal changes to make
> > partitions accessible by topic id and partition. I can update the KIP to
> > remove them unless anyone thinks they should stay.
> >
> > 12) Addressed in the previous email. I've updated the KIP to include
> > tagged fields for the requests and responses. (More on that below)
> >
> > 13) I think part of the idea for including this information is to prepare
> > for future changes. Perhaps the directory structure might change from
> > topicName_partitionNumber to something like topicID_partitionNumber. Then
> > it would be useful to have the topic name in the file since it would not
> be
> > in the directory structure. Supporting topic renames might be easier if
> the
> > other fields are included. Would there be any downsides to including this
> > information?
> >
> > 14)  Yes, we would need to copy the partition metadata file in this
> > process. I've updated the KIP to include this.
> >
> > 15) I believe Lucas meant v1 and v2 here. He was referring to how the
> > requests would fall under different IBP and meant that older brokers
> would
> > have to use the older version of the request and the existing topic
> > deletion process. At first, it seemed like tagged fields would resolve
> > the IBP issue. However, we may need IBP for this request after all since
> > the controller handles the topic deletion differently depending on the
> IBP
> > version. In an older version, we can't just send a StopReplica delete the
> > topic immediately like we'd want to for this KIP.
> >
> > This makes me wonder if we want tagged fields on all the requests after
> > all. Let me know your thoughts!
> >
> > Justine
> >
> > On Tue, Sep 15, 2020 at 1:03 PM Justine Olshan <jo...@confluent.io>
> > wrote:
> >
> >> Hi all,
> >> Jun brought up a good point in his last email about tagged fields, and
> >> I've updated the KIP to reflect that the changes to requests and
> responses
> >> will be in the form of tagged fields to avoid changing IBP.
> >>
> >> Jun: I plan on sending a followup email to address some of the other
> >> points.
> >>
> >> Thanks,
> >> Justine
> >>
> >> On Mon, Sep 14, 2020 at 4:25 PM Jun Rao <ju...@confluent.io> wrote:
> >>
> >>> Hi, Justine,
> >>>
> >>> Thanks for the updated KIP. A few comments below.
> >>>
> >>> 10. LeaderAndIsr Response: Do we need the topic name?
> >>>
> >>> 11. For the changed request/response, other than LeaderAndIsr,
> >>> UpdateMetadata, Metadata, do we need to include the topic name?
> >>>
> >>> 12. It seems that upgrades don't require IBP. Does that mean the new
> >>> fields
> >>> in all the request/response are added as tagged fields without bumping
> up
> >>> the request version? It would be useful to make that clear.
> >>>
> >>> 13. Partition Metadata file: Do we need to include the topic name and
> the
> >>> partition id since they are implied in the directory name?
> >>>
> >>> 14. In the JBOD mode, we support moving a partition's data from one
> disk
> >>> to
> >>> another. Will the new partition metadata file be copied during that
> >>> process?
> >>>
> >>> 15. The KIP says "Remove deleted topics from replicas by sending
> >>> StopReplicaRequest V2 for any topics which do not contain a topic ID,
> and
> >>> V3 for any topics which do contain a topic ID.". However, it seems the
> >>> updated controller will create all missing topic IDs first before doing
> >>> other actions. So, is StopReplicaRequest V2 needed?
> >>>
> >>> Jun
> >>>
> >>> On Fri, Sep 11, 2020 at 10:31 AM John Roesler <vv...@apache.org>
> >>> wrote:
> >>>
> >>> > Thanks, Justine!
> >>> >
> >>> > Your response seems compelling to me.
> >>> >
> >>> > -John
> >>> >
> >>> > On Fri, 2020-09-11 at 10:17 -0700, Justine Olshan wrote:
> >>> > > Hello all,
> >>> > > Thanks for continuing the discussion! I have a few responses to
> your
> >>> > points.
> >>> > >
> >>> > > Tom: You are correct in that this KIP has not mentioned the
> >>> > > DeleteTopicsRequest. I think that this would be out of scope for
> >>> now, but
> >>> > > may be something worth adding in the future.
> >>> > >
> >>> > > John: We did consider sequence ids, but there are a few reasons to
> >>> favor
> >>> > > UUIDs. There are several cases where topics from different clusters
> >>> may
> >>> > > interact now and in the future. For example, Mirror Maker 2 may
> >>> benefit
> >>> > > from being able to detect when a cluster being mirrored is deleted
> >>> and
> >>> > > recreated and globally unique identifiers would make resolving
> issues
> >>> > > easier than sequence IDs which may collide between clusters.
> KIP-405
> >>> > > (tiered storage) will also benefit from globally unique IDs as
> shared
> >>> > > buckets may be used between clusters.
> >>> > >
> >>> > > Globally unique IDs would also make functionality like moving
> topics
> >>> > > between disparate clusters easier in the future, simplify any
> future
> >>> > > implementations of backups and restores, and more. In general,
> >>> unique IDs
> >>> > > would ensure that the source cluster topics do not conflict with
> the
> >>> > > destination cluster topics.
> >>> > >
> >>> > > If we were to use sequence ids, we would need sufficiently large
> >>> cluster
> >>> > > ids to be stored with the topic identifiers or we run the risk of
> >>> > > collisions. This will give up any advantage in compactness that
> >>> sequence
> >>> > > numbers may bring. Given these advantages I think it makes sense to
> >>> use
> >>> > > UUIDs.
> >>> > >
> >>> > > Gokul: This is an interesting idea, but this is a breaking change.
> >>> Out of
> >>> > > scope for now, but maybe worth discussing in the future.
> >>> > >
> >>> > > Hope this explains some of the decisions,
> >>> > >
> >>> > > Justine
> >>> > >
> >>> > >
> >>> > >
> >>> > > On Fri, Sep 11, 2020 at 8:27 AM Gokul Ramanan Subramanian <
> >>> > > gokul2411s@gmail.com> wrote:
> >>> > >
> >>> > > > Hi.
> >>> > > >
> >>> > > > Thanks for the KIP.
> >>> > > >
> >>> > > > Have you thought about whether it makes sense to support
> >>> authorizing a
> >>> > > > principal for a topic ID rather than a topic name to achieve
> >>> tighter
> >>> > > > security?
> >>> > > >
> >>> > > > Or is the topic ID fundamentally an internal detail similar to
> >>> epochs
> >>> > used
> >>> > > > in a bunch of other places in Kafka?
> >>> > > >
> >>> > > > Thanks.
> >>> > > >
> >>> > > > On Fri, Sep 11, 2020 at 4:06 PM John Roesler <
> vvcephei@apache.org>
> >>> > wrote:
> >>> > > >
> >>> > > > > Hello Justine,
> >>> > > > >
> >>> > > > > Thanks for the KIP!
> >>> > > > >
> >>> > > > > I happen to have been confronted recently with the need to keep
> >>> > track of
> >>> > > > a
> >>> > > > > large number of topics as compactly as possible. I was going to
> >>> come
> >>> > up
> >>> > > > > with some way to dictionary encode the topic names as integers,
> >>> but
> >>> > this
> >>> > > > > seems much better!
> >>> > > > >
> >>> > > > > Apologies if this has been raised before, but I’m wondering
> >>> about the
> >>> > > > > choice of UUID vs sequence number for the ids. Typically, I’ve
> >>> seen
> >>> > UUIDs
> >>> > > > > in two situations:
> >>> > > > > 1. When processes need to generate non-colliding identifiers
> >>> without
> >>> > > > > coordination.
> >>> > > > > 2. When the identifier needs to be “universally unique”; I.e.,
> >>> the
> >>> > > > > identifier needs to distinguish the entity from all other
> >>> entities
> >>> > that
> >>> > > > > could ever exist. This is useful in cases where entities from
> all
> >>> > kinds
> >>> > > > of
> >>> > > > > systems get mixed together, such as when dumping logs from all
> >>> > processes
> >>> > > > in
> >>> > > > > a company into a common system.
> >>> > > > >
> >>> > > > > Maybe I’m being short-sighted, but it doesn’t seem like either
> >>> really
> >>> > > > > applies here. It seems like the brokers could and would achieve
> >>> > consensus
> >>> > > > > when creating a topic anyway, which is all that’s required to
> >>> > generate
> >>> > > > > non-colliding sequence ids. For the second, as you mention, we
> >>> could
> >>> > > > assign
> >>> > > > > a UUID to the cluster as a whole, which would render any
> resource
> >>> > scoped
> >>> > > > to
> >>> > > > > the broker universally unique as well.
> >>> > > > >
> >>> > > > > The reason I mention this is that, although a UUID is way more
> >>> > compact
> >>> > > > > than topic names, it’s still 16 bytes. In contrast, a 4-byte
> >>> integer
> >>> > > > > sequence id would give us 4 billion unique topics per cluster,
> >>> which
> >>> > > > seems
> >>> > > > > like enough ;)
> >>> > > > >
> >>> > > > > Considering the number of different times these topic
> >>> identifiers are
> >>> > > > sent
> >>> > > > > over the wire or stored in memory, it seems like it might be
> >>> worth
> >>> > the
> >>> > > > > additional 4x space savings.
> >>> > > > >
> >>> > > > > What do you think about this?
> >>> > > > >
> >>> > > > > Thanks,
> >>> > > > > John
> >>> > > > >
> >>> > > > > On Fri, Sep 11, 2020, at 03:20, Tom Bentley wrote:
> >>> > > > > > Hi Justine,
> >>> > > > > >
> >>> > > > > > This looks like a very welcome improvement. Thanks!
> >>> > > > > >
> >>> > > > > > Maybe I missed it, but the KIP doesn't seem to mention
> changing
> >>> > > > > > DeleteTopicsRequest to identify the topic using an id. Maybe
> >>> > that's out
> >>> > > > > of
> >>> > > > > > scope, but DeleteTopicsRequest is not listed among the Future
> >>> Work
> >>> > APIs
> >>> > > > > > either.
> >>> > > > > >
> >>> > > > > > Kind regards,
> >>> > > > > >
> >>> > > > > > Tom
> >>> > > > > >
> >>> > > > > > On Thu, Sep 10, 2020 at 3:59 PM Satish Duggana <
> >>> > > > satish.duggana@gmail.com
> >>> > > > > > wrote:
> >>> > > > > >
> >>> > > > > > > Thanks Lucas/Justine for the nice KIP.
> >>> > > > > > >
> >>> > > > > > > It has several benefits which also include simplifying the
> >>> topic
> >>> > > > > > > deletion process by controller and logs cleanup by brokers
> in
> >>> > corner
> >>> > > > > > > cases.
> >>> > > > > > >
> >>> > > > > > > Best,
> >>> > > > > > > Satish.
> >>> > > > > > >
> >>> > > > > > > On Wed, Sep 9, 2020 at 10:07 PM Justine Olshan <
> >>> > jolshan@confluent.io
> >>> > > > > > > wrote:
> >>> > > > > > > > Hello all, it's been almost a year! I've made some
> changes
> >>> to
> >>> > this
> >>> > > > > KIP
> >>> > > > > > > and hope to continue the discussion.
> >>> > > > > > > > One of the main changes I've added is now the metadata
> >>> response
> >>> > > > will
> >>> > > > > > > include the topic ID (as Colin suggested). Clients can
> >>> obtain the
> >>> > > > > topicID
> >>> > > > > > > of a given topic through a TopicDescription. The topicId
> will
> >>> > also be
> >>> > > > > > > included with the UpdateMetadata request.
> >>> > > > > > > > Let me know what you all think.
> >>> > > > > > > > Thank you,
> >>> > > > > > > > Justine
> >>> > > > > > > >
> >>> > > > > > > > On 2019/09/13 16:38:26, "Colin McCabe" <
> cmccabe@apache.org
> >>> >
> >>> > wrote:
> >>> > > > > > > > > Hi Lucas,
> >>> > > > > > > > >
> >>> > > > > > > > > Thanks for tackling this.  Topic IDs are a great idea,
> >>> and
> >>> > this
> >>> > > > is
> >>> > > > > a
> >>> > > > > > > really good writeup.
> >>> > > > > > > > > For /brokers/topics/[topic], the schema version should
> be
> >>> > bumped
> >>> > > > to
> >>> > > > > > > version 3, rather than 2.  KIP-455 bumped the version of
> this
> >>> > znode
> >>> > > > to
> >>> > > > > 2
> >>> > > > > > > already :)
> >>> > > > > > > > > Given that we're going to be seeing these things as
> >>> strings
> >>> > as
> >>> > > > lot
> >>> > > > > (in
> >>> > > > > > > logs, in ZooKeeper, on the command-line, etc.), does it
> make
> >>> > sense to
> >>> > > > > use
> >>> > > > > > > base64 when converting them to strings?
> >>> > > > > > > > > Here is an example of the hex representation:
> >>> > > > > > > > > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> >>> > > > > > > > >
> >>> > > > > > > > > And here is an example in base64.
> >>> > > > > > > > > b8tRS7h4TJ2Vt43Dp85v2A
> >>> > > > > > > > >
> >>> > > > > > > > > The base64 version saves 15 letters (to be fair, 4 of
> >>> those
> >>> > were
> >>> > > > > > > dashes that we could have elided in the hex
> representation.)
> >>> > > > > > > > > Another thing to consider is that we should specify
> that
> >>> the
> >>> > > > > > > all-zeroes UUID is not a valid topic UUID.   We can't use
> >>> null
> >>> > for
> >>> > > > this
> >>> > > > > > > because we can't pass a null UUID over the RPC protocol
> >>> (there
> >>> > is no
> >>> > > > > > > special pattern for null, nor do we want to waste space
> >>> reserving
> >>> > > > such
> >>> > > > > a
> >>> > > > > > > pattern.)
> >>> > > > > > > > > Maybe I missed it, but did you describe "migration
> of...
> >>> > existing
> >>> > > > > > > topic[s] without topic IDs" in detail in any section?  It
> >>> seems
> >>> > like
> >>> > > > > when
> >>> > > > > > > the new controller becomes active, it should just generate
> >>> random
> >>> > > > > UUIDs for
> >>> > > > > > > these, and write the random UUIDs back to ZooKeeper.  It
> >>> would be
> >>> > > > good
> >>> > > > > to
> >>> > > > > > > spell that out.  We should make it clear that this happens
> >>> > regardless
> >>> > > > > of
> >>> > > > > > > the inter-broker protocol version (it's a compatible
> change).
> >>> > > > > > > > > "LeaderAndIsrRequests including an is_every_partition
> >>> flag"
> >>> > > > seems a
> >>> > > > > > > bit wordy.  Can we just call these "full
> >>> LeaderAndIsrRequests"?
> >>> > Then
> >>> > > > > the
> >>> > > > > > > RPC field could be named "full".  Also, it would probably
> be
> >>> > better
> >>> > > > > for the
> >>> > > > > > > RPC field to be an enum of { UNSPECIFIED, INCREMENTAL, FULL
> >>> }, so
> >>> > > > that
> >>> > > > > we
> >>> > > > > > > can cleanly handle old versions (by treating them as
> >>> UNSPECIFIED)
> >>> > > > > > > > > In the LeaderAndIsrRequest section, you write "A final
> >>> > deletion
> >>> > > > > event
> >>> > > > > > > will be secheduled for X ms after the LeaderAndIsrRequest
> was
> >>> > first
> >>> > > > > > > received..."  I guess the X was a placeholder that you
> >>> intended
> >>> > to
> >>> > > > > replace
> >>> > > > > > > before posting? :)  In any case, this seems like the kind
> of
> >>> > thing
> >>> > > > we'd
> >>> > > > > > > want a configuration for.  Let's describe that
> configuration
> >>> key
> >>> > > > > somewhere
> >>> > > > > > > in this KIP, including what its default value is.
> >>> > > > > > > > > We should probably also log a bunch of messages at WARN
> >>> level
> >>> > > > when
> >>> > > > > > > something is scheduled for deletion, as well.  (Maybe this
> >>> was
> >>> > > > > assumed, but
> >>> > > > > > > it would be good to mention it).
> >>> > > > > > > > > I feel like there are a few sections that should be
> >>> moved to
> >>> > > > > "rejected
> >>> > > > > > > alternatives."  For example, in the DeleteTopics section,
> >>> since
> >>> > we're
> >>> > > > > not
> >>> > > > > > > going to do option 1 or 2, these should be moved into
> >>> "rejected
> >>> > > > > > > alternatives,"  rather than appearing inline.  Another case
> >>> is
> >>> > the
> >>> > > > > "Should
> >>> > > > > > > we remove topic name from the protocol where possible"
> >>> section.
> >>> > This
> >>> > > > > is
> >>> > > > > > > clearly discussing a design alternative that we're not
> >>> proposing
> >>> > to
> >>> > > > > > > implement: removing the topic name from those protocols.
> >>> > > > > > > > > Is it really necessary to have a new
> >>> > /admin/delete_topics_by_id
> >>> > > > > path
> >>> > > > > > > in ZooKeeper?  It seems like we don't really need this.
> >>> Whenever
> >>> > > > > there is
> >>> > > > > > > a new controller, we'll send out full LeaderAndIsrRequests
> >>> which
> >>> > will
> >>> > > > > > > trigger the stale topics to be cleaned up.   The active
> >>> > controller
> >>> > > > will
> >>> > > > > > > also send the full LeaderAndIsrRequest to brokers that are
> >>> just
> >>> > > > > starting
> >>> > > > > > > up.    So we don't really need this kind of two-phase
> commit
> >>> > (send
> >>> > > > out
> >>> > > > > > > StopReplicasRequest, get ACKs from all nodes, commit by
> >>> removing
> >>> > > > > > > /admin/delete_topics node) any more.
> >>> > > > > > > > > You mention that FetchRequest will now include UUID to
> >>> avoid
> >>> > > > issues
> >>> > > > > > > where requests are made to stale partitions.  However,
> >>> adding a
> >>> > UUID
> >>> > > > to
> >>> > > > > > > MetadataRequest is listed as future work, out of scope for
> >>> this
> >>> > KIP.
> >>> > > > > How
> >>> > > > > > > will the client learn what the topic UUID is, if the
> metadata
> >>> > > > response
> >>> > > > > > > doesn't include that information?  It seems like adding the
> >>> UUID
> >>> > to
> >>> > > > > > > MetadataResponse would be an improvement here that might
> not
> >>> be
> >>> > too
> >>> > > > > hard to
> >>> > > > > > > make.
> >>> > > > > > > > > best,
> >>> > > > > > > > > Colin
> >>> > > > > > > > >
> >>> > > > > > > > >
> >>> > > > > > > > > On Mon, Sep 9, 2019, at 17:48, Ryanne Dolan wrote:
> >>> > > > > > > > > > Lucas, this would be great. I've run into issues with
> >>> > topics
> >>> > > > > being
> >>> > > > > > > > > > resurrected accidentally, since a client cannot
> easily
> >>> > > > > distinguish
> >>> > > > > > > between
> >>> > > > > > > > > > a deleted topic and a new topic with the same name.
> I'd
> >>> > need
> >>> > > > the
> >>> > > > > ID
> >>> > > > > > > > > > accessible from the client to solve that issue, but
> >>> this
> >>> > is a
> >>> > > > > good
> >>> > > > > > > first
> >>> > > > > > > > > > step.
> >>> > > > > > > > > >
> >>> > > > > > > > > > Ryanne
> >>> > > > > > > > > >
> >>> > > > > > > > > > On Wed, Sep 4, 2019 at 1:41 PM Lucas Bradstreet <
> >>> > > > > lucas@confluent.io>
> >>> > > > > > > wrote:
> >>> > > > > > > > > > > Hi all,
> >>> > > > > > > > > > >
> >>> > > > > > > > > > > I would like to kick off discussion of KIP-516, an
> >>> > > > > implementation
> >>> > > > > > > of topic
> >>> > > > > > > > > > > IDs for Kafka. Topic IDs aim to solve topic
> >>> uniqueness
> >>> > > > > problems in
> >>> > > > > > > Kafka,
> >>> > > > > > > > > > > where referring to a topic by name alone is
> >>> insufficient.
> >>> > > > Such
> >>> > > > > > > cases
> >>> > > > > > > > > > > include when a topic has been deleted and recreated
> >>> with
> >>> > the
> >>> > > > > same
> >>> > > > > > > name.
> >>> > > > > > > > > > > Unique identifiers will help simplify and improve
> >>> Kafka's
> >>> > > > topic
> >>> > > > > > > deletion
> >>> > > > > > > > > > > process, as well as prevent cases where brokers may
> >>> > > > incorrectly
> >>> > > > > > > interact
> >>> > > > > > > > > > > with stale versions of topics.
> >>> > > > > > > > > > >
> >>> > > > > > > > > > >
> >>> > > > > > > > > > >
> >>> > > >
> >>> >
> >>>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> >>> > > > > > > > > > > Looking forward to your thoughts.
> >>> > > > > > > > > > >
> >>> > > > > > > > > > > Lucas
> >>> > > > > > > > > > >
> >>> >
> >>> >
> >>>
> >>
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Justine Olshan <jo...@confluent.io>]

Hello all,

I've thought some more about removing the topic name field from some of the
requests. On closer inspection of the requests/responses, it seems that the
internal changes would be much larger than I expected. Some protocols
involve clients, so they would require changes too. I'm thinking that for
now, removing the topic name from these requests and responses are out of
scope.

I have decided to just keep the change LeaderAndIsrResponse to remove the
topic name, and have updated the KIP to reflect this change. I have also
mentioned the other requests and responses in future work.

I'm hoping to start the voting process soon, so let me know if there is
anything else we should discuss.

Thank you,
Justine

On Tue, Sep 15, 2020 at 3:57 PM Justine Olshan <jo...@confluent.io> wrote:

> Hello again,
> To follow up on some of the other comments:
>
> 10/11) We can remove the topic name from these requests/responses, and
> that means we will just have to make a few internal changes to make
> partitions accessible by topic id and partition. I can update the KIP to
> remove them unless anyone thinks they should stay.
>
> 12) Addressed in the previous email. I've updated the KIP to include
> tagged fields for the requests and responses. (More on that below)
>
> 13) I think part of the idea for including this information is to prepare
> for future changes. Perhaps the directory structure might change from
> topicName_partitionNumber to something like topicID_partitionNumber. Then
> it would be useful to have the topic name in the file since it would not be
> in the directory structure. Supporting topic renames might be easier if the
> other fields are included. Would there be any downsides to including this
> information?
>
> 14)  Yes, we would need to copy the partition metadata file in this
> process. I've updated the KIP to include this.
>
> 15) I believe Lucas meant v1 and v2 here. He was referring to how the
> requests would fall under different IBP and meant that older brokers would
> have to use the older version of the request and the existing topic
> deletion process. At first, it seemed like tagged fields would resolve
> the IBP issue. However, we may need IBP for this request after all since
> the controller handles the topic deletion differently depending on the IBP
> version. In an older version, we can't just send a StopReplica delete the
> topic immediately like we'd want to for this KIP.
>
> This makes me wonder if we want tagged fields on all the requests after
> all. Let me know your thoughts!
>
> Justine
>
> On Tue, Sep 15, 2020 at 1:03 PM Justine Olshan <jo...@confluent.io>
> wrote:
>
>> Hi all,
>> Jun brought up a good point in his last email about tagged fields, and
>> I've updated the KIP to reflect that the changes to requests and responses
>> will be in the form of tagged fields to avoid changing IBP.
>>
>> Jun: I plan on sending a followup email to address some of the other
>> points.
>>
>> Thanks,
>> Justine
>>
>> On Mon, Sep 14, 2020 at 4:25 PM Jun Rao <ju...@confluent.io> wrote:
>>
>>> Hi, Justine,
>>>
>>> Thanks for the updated KIP. A few comments below.
>>>
>>> 10. LeaderAndIsr Response: Do we need the topic name?
>>>
>>> 11. For the changed request/response, other than LeaderAndIsr,
>>> UpdateMetadata, Metadata, do we need to include the topic name?
>>>
>>> 12. It seems that upgrades don't require IBP. Does that mean the new
>>> fields
>>> in all the request/response are added as tagged fields without bumping up
>>> the request version? It would be useful to make that clear.
>>>
>>> 13. Partition Metadata file: Do we need to include the topic name and the
>>> partition id since they are implied in the directory name?
>>>
>>> 14. In the JBOD mode, we support moving a partition's data from one disk
>>> to
>>> another. Will the new partition metadata file be copied during that
>>> process?
>>>
>>> 15. The KIP says "Remove deleted topics from replicas by sending
>>> StopReplicaRequest V2 for any topics which do not contain a topic ID, and
>>> V3 for any topics which do contain a topic ID.". However, it seems the
>>> updated controller will create all missing topic IDs first before doing
>>> other actions. So, is StopReplicaRequest V2 needed?
>>>
>>> Jun
>>>
>>> On Fri, Sep 11, 2020 at 10:31 AM John Roesler <vv...@apache.org>
>>> wrote:
>>>
>>> > Thanks, Justine!
>>> >
>>> > Your response seems compelling to me.
>>> >
>>> > -John
>>> >
>>> > On Fri, 2020-09-11 at 10:17 -0700, Justine Olshan wrote:
>>> > > Hello all,
>>> > > Thanks for continuing the discussion! I have a few responses to your
>>> > points.
>>> > >
>>> > > Tom: You are correct in that this KIP has not mentioned the
>>> > > DeleteTopicsRequest. I think that this would be out of scope for
>>> now, but
>>> > > may be something worth adding in the future.
>>> > >
>>> > > John: We did consider sequence ids, but there are a few reasons to
>>> favor
>>> > > UUIDs. There are several cases where topics from different clusters
>>> may
>>> > > interact now and in the future. For example, Mirror Maker 2 may
>>> benefit
>>> > > from being able to detect when a cluster being mirrored is deleted
>>> and
>>> > > recreated and globally unique identifiers would make resolving issues
>>> > > easier than sequence IDs which may collide between clusters. KIP-405
>>> > > (tiered storage) will also benefit from globally unique IDs as shared
>>> > > buckets may be used between clusters.
>>> > >
>>> > > Globally unique IDs would also make functionality like moving topics
>>> > > between disparate clusters easier in the future, simplify any future
>>> > > implementations of backups and restores, and more. In general,
>>> unique IDs
>>> > > would ensure that the source cluster topics do not conflict with the
>>> > > destination cluster topics.
>>> > >
>>> > > If we were to use sequence ids, we would need sufficiently large
>>> cluster
>>> > > ids to be stored with the topic identifiers or we run the risk of
>>> > > collisions. This will give up any advantage in compactness that
>>> sequence
>>> > > numbers may bring. Given these advantages I think it makes sense to
>>> use
>>> > > UUIDs.
>>> > >
>>> > > Gokul: This is an interesting idea, but this is a breaking change.
>>> Out of
>>> > > scope for now, but maybe worth discussing in the future.
>>> > >
>>> > > Hope this explains some of the decisions,
>>> > >
>>> > > Justine
>>> > >
>>> > >
>>> > >
>>> > > On Fri, Sep 11, 2020 at 8:27 AM Gokul Ramanan Subramanian <
>>> > > gokul2411s@gmail.com> wrote:
>>> > >
>>> > > > Hi.
>>> > > >
>>> > > > Thanks for the KIP.
>>> > > >
>>> > > > Have you thought about whether it makes sense to support
>>> authorizing a
>>> > > > principal for a topic ID rather than a topic name to achieve
>>> tighter
>>> > > > security?
>>> > > >
>>> > > > Or is the topic ID fundamentally an internal detail similar to
>>> epochs
>>> > used
>>> > > > in a bunch of other places in Kafka?
>>> > > >
>>> > > > Thanks.
>>> > > >
>>> > > > On Fri, Sep 11, 2020 at 4:06 PM John Roesler <vv...@apache.org>
>>> > wrote:
>>> > > >
>>> > > > > Hello Justine,
>>> > > > >
>>> > > > > Thanks for the KIP!
>>> > > > >
>>> > > > > I happen to have been confronted recently with the need to keep
>>> > track of
>>> > > > a
>>> > > > > large number of topics as compactly as possible. I was going to
>>> come
>>> > up
>>> > > > > with some way to dictionary encode the topic names as integers,
>>> but
>>> > this
>>> > > > > seems much better!
>>> > > > >
>>> > > > > Apologies if this has been raised before, but I’m wondering
>>> about the
>>> > > > > choice of UUID vs sequence number for the ids. Typically, I’ve
>>> seen
>>> > UUIDs
>>> > > > > in two situations:
>>> > > > > 1. When processes need to generate non-colliding identifiers
>>> without
>>> > > > > coordination.
>>> > > > > 2. When the identifier needs to be “universally unique”; I.e.,
>>> the
>>> > > > > identifier needs to distinguish the entity from all other
>>> entities
>>> > that
>>> > > > > could ever exist. This is useful in cases where entities from all
>>> > kinds
>>> > > > of
>>> > > > > systems get mixed together, such as when dumping logs from all
>>> > processes
>>> > > > in
>>> > > > > a company into a common system.
>>> > > > >
>>> > > > > Maybe I’m being short-sighted, but it doesn’t seem like either
>>> really
>>> > > > > applies here. It seems like the brokers could and would achieve
>>> > consensus
>>> > > > > when creating a topic anyway, which is all that’s required to
>>> > generate
>>> > > > > non-colliding sequence ids. For the second, as you mention, we
>>> could
>>> > > > assign
>>> > > > > a UUID to the cluster as a whole, which would render any resource
>>> > scoped
>>> > > > to
>>> > > > > the broker universally unique as well.
>>> > > > >
>>> > > > > The reason I mention this is that, although a UUID is way more
>>> > compact
>>> > > > > than topic names, it’s still 16 bytes. In contrast, a 4-byte
>>> integer
>>> > > > > sequence id would give us 4 billion unique topics per cluster,
>>> which
>>> > > > seems
>>> > > > > like enough ;)
>>> > > > >
>>> > > > > Considering the number of different times these topic
>>> identifiers are
>>> > > > sent
>>> > > > > over the wire or stored in memory, it seems like it might be
>>> worth
>>> > the
>>> > > > > additional 4x space savings.
>>> > > > >
>>> > > > > What do you think about this?
>>> > > > >
>>> > > > > Thanks,
>>> > > > > John
>>> > > > >
>>> > > > > On Fri, Sep 11, 2020, at 03:20, Tom Bentley wrote:
>>> > > > > > Hi Justine,
>>> > > > > >
>>> > > > > > This looks like a very welcome improvement. Thanks!
>>> > > > > >
>>> > > > > > Maybe I missed it, but the KIP doesn't seem to mention changing
>>> > > > > > DeleteTopicsRequest to identify the topic using an id. Maybe
>>> > that's out
>>> > > > > of
>>> > > > > > scope, but DeleteTopicsRequest is not listed among the Future
>>> Work
>>> > APIs
>>> > > > > > either.
>>> > > > > >
>>> > > > > > Kind regards,
>>> > > > > >
>>> > > > > > Tom
>>> > > > > >
>>> > > > > > On Thu, Sep 10, 2020 at 3:59 PM Satish Duggana <
>>> > > > satish.duggana@gmail.com
>>> > > > > > wrote:
>>> > > > > >
>>> > > > > > > Thanks Lucas/Justine for the nice KIP.
>>> > > > > > >
>>> > > > > > > It has several benefits which also include simplifying the
>>> topic
>>> > > > > > > deletion process by controller and logs cleanup by brokers in
>>> > corner
>>> > > > > > > cases.
>>> > > > > > >
>>> > > > > > > Best,
>>> > > > > > > Satish.
>>> > > > > > >
>>> > > > > > > On Wed, Sep 9, 2020 at 10:07 PM Justine Olshan <
>>> > jolshan@confluent.io
>>> > > > > > > wrote:
>>> > > > > > > > Hello all, it's been almost a year! I've made some changes
>>> to
>>> > this
>>> > > > > KIP
>>> > > > > > > and hope to continue the discussion.
>>> > > > > > > > One of the main changes I've added is now the metadata
>>> response
>>> > > > will
>>> > > > > > > include the topic ID (as Colin suggested). Clients can
>>> obtain the
>>> > > > > topicID
>>> > > > > > > of a given topic through a TopicDescription. The topicId will
>>> > also be
>>> > > > > > > included with the UpdateMetadata request.
>>> > > > > > > > Let me know what you all think.
>>> > > > > > > > Thank you,
>>> > > > > > > > Justine
>>> > > > > > > >
>>> > > > > > > > On 2019/09/13 16:38:26, "Colin McCabe" <cmccabe@apache.org
>>> >
>>> > wrote:
>>> > > > > > > > > Hi Lucas,
>>> > > > > > > > >
>>> > > > > > > > > Thanks for tackling this.  Topic IDs are a great idea,
>>> and
>>> > this
>>> > > > is
>>> > > > > a
>>> > > > > > > really good writeup.
>>> > > > > > > > > For /brokers/topics/[topic], the schema version should be
>>> > bumped
>>> > > > to
>>> > > > > > > version 3, rather than 2.  KIP-455 bumped the version of this
>>> > znode
>>> > > > to
>>> > > > > 2
>>> > > > > > > already :)
>>> > > > > > > > > Given that we're going to be seeing these things as
>>> strings
>>> > as
>>> > > > lot
>>> > > > > (in
>>> > > > > > > logs, in ZooKeeper, on the command-line, etc.), does it make
>>> > sense to
>>> > > > > use
>>> > > > > > > base64 when converting them to strings?
>>> > > > > > > > > Here is an example of the hex representation:
>>> > > > > > > > > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
>>> > > > > > > > >
>>> > > > > > > > > And here is an example in base64.
>>> > > > > > > > > b8tRS7h4TJ2Vt43Dp85v2A
>>> > > > > > > > >
>>> > > > > > > > > The base64 version saves 15 letters (to be fair, 4 of
>>> those
>>> > were
>>> > > > > > > dashes that we could have elided in the hex representation.)
>>> > > > > > > > > Another thing to consider is that we should specify that
>>> the
>>> > > > > > > all-zeroes UUID is not a valid topic UUID.   We can't use
>>> null
>>> > for
>>> > > > this
>>> > > > > > > because we can't pass a null UUID over the RPC protocol
>>> (there
>>> > is no
>>> > > > > > > special pattern for null, nor do we want to waste space
>>> reserving
>>> > > > such
>>> > > > > a
>>> > > > > > > pattern.)
>>> > > > > > > > > Maybe I missed it, but did you describe "migration of...
>>> > existing
>>> > > > > > > topic[s] without topic IDs" in detail in any section?  It
>>> seems
>>> > like
>>> > > > > when
>>> > > > > > > the new controller becomes active, it should just generate
>>> random
>>> > > > > UUIDs for
>>> > > > > > > these, and write the random UUIDs back to ZooKeeper.  It
>>> would be
>>> > > > good
>>> > > > > to
>>> > > > > > > spell that out.  We should make it clear that this happens
>>> > regardless
>>> > > > > of
>>> > > > > > > the inter-broker protocol version (it's a compatible change).
>>> > > > > > > > > "LeaderAndIsrRequests including an is_every_partition
>>> flag"
>>> > > > seems a
>>> > > > > > > bit wordy.  Can we just call these "full
>>> LeaderAndIsrRequests"?
>>> > Then
>>> > > > > the
>>> > > > > > > RPC field could be named "full".  Also, it would probably be
>>> > better
>>> > > > > for the
>>> > > > > > > RPC field to be an enum of { UNSPECIFIED, INCREMENTAL, FULL
>>> }, so
>>> > > > that
>>> > > > > we
>>> > > > > > > can cleanly handle old versions (by treating them as
>>> UNSPECIFIED)
>>> > > > > > > > > In the LeaderAndIsrRequest section, you write "A final
>>> > deletion
>>> > > > > event
>>> > > > > > > will be secheduled for X ms after the LeaderAndIsrRequest was
>>> > first
>>> > > > > > > received..."  I guess the X was a placeholder that you
>>> intended
>>> > to
>>> > > > > replace
>>> > > > > > > before posting? :)  In any case, this seems like the kind of
>>> > thing
>>> > > > we'd
>>> > > > > > > want a configuration for.  Let's describe that configuration
>>> key
>>> > > > > somewhere
>>> > > > > > > in this KIP, including what its default value is.
>>> > > > > > > > > We should probably also log a bunch of messages at WARN
>>> level
>>> > > > when
>>> > > > > > > something is scheduled for deletion, as well.  (Maybe this
>>> was
>>> > > > > assumed, but
>>> > > > > > > it would be good to mention it).
>>> > > > > > > > > I feel like there are a few sections that should be
>>> moved to
>>> > > > > "rejected
>>> > > > > > > alternatives."  For example, in the DeleteTopics section,
>>> since
>>> > we're
>>> > > > > not
>>> > > > > > > going to do option 1 or 2, these should be moved into
>>> "rejected
>>> > > > > > > alternatives,"  rather than appearing inline.  Another case
>>> is
>>> > the
>>> > > > > "Should
>>> > > > > > > we remove topic name from the protocol where possible"
>>> section.
>>> > This
>>> > > > > is
>>> > > > > > > clearly discussing a design alternative that we're not
>>> proposing
>>> > to
>>> > > > > > > implement: removing the topic name from those protocols.
>>> > > > > > > > > Is it really necessary to have a new
>>> > /admin/delete_topics_by_id
>>> > > > > path
>>> > > > > > > in ZooKeeper?  It seems like we don't really need this.
>>> Whenever
>>> > > > > there is
>>> > > > > > > a new controller, we'll send out full LeaderAndIsrRequests
>>> which
>>> > will
>>> > > > > > > trigger the stale topics to be cleaned up.   The active
>>> > controller
>>> > > > will
>>> > > > > > > also send the full LeaderAndIsrRequest to brokers that are
>>> just
>>> > > > > starting
>>> > > > > > > up.    So we don't really need this kind of two-phase commit
>>> > (send
>>> > > > out
>>> > > > > > > StopReplicasRequest, get ACKs from all nodes, commit by
>>> removing
>>> > > > > > > /admin/delete_topics node) any more.
>>> > > > > > > > > You mention that FetchRequest will now include UUID to
>>> avoid
>>> > > > issues
>>> > > > > > > where requests are made to stale partitions.  However,
>>> adding a
>>> > UUID
>>> > > > to
>>> > > > > > > MetadataRequest is listed as future work, out of scope for
>>> this
>>> > KIP.
>>> > > > > How
>>> > > > > > > will the client learn what the topic UUID is, if the metadata
>>> > > > response
>>> > > > > > > doesn't include that information?  It seems like adding the
>>> UUID
>>> > to
>>> > > > > > > MetadataResponse would be an improvement here that might not
>>> be
>>> > too
>>> > > > > hard to
>>> > > > > > > make.
>>> > > > > > > > > best,
>>> > > > > > > > > Colin
>>> > > > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > On Mon, Sep 9, 2019, at 17:48, Ryanne Dolan wrote:
>>> > > > > > > > > > Lucas, this would be great. I've run into issues with
>>> > topics
>>> > > > > being
>>> > > > > > > > > > resurrected accidentally, since a client cannot easily
>>> > > > > distinguish
>>> > > > > > > between
>>> > > > > > > > > > a deleted topic and a new topic with the same name. I'd
>>> > need
>>> > > > the
>>> > > > > ID
>>> > > > > > > > > > accessible from the client to solve that issue, but
>>> this
>>> > is a
>>> > > > > good
>>> > > > > > > first
>>> > > > > > > > > > step.
>>> > > > > > > > > >
>>> > > > > > > > > > Ryanne
>>> > > > > > > > > >
>>> > > > > > > > > > On Wed, Sep 4, 2019 at 1:41 PM Lucas Bradstreet <
>>> > > > > lucas@confluent.io>
>>> > > > > > > wrote:
>>> > > > > > > > > > > Hi all,
>>> > > > > > > > > > >
>>> > > > > > > > > > > I would like to kick off discussion of KIP-516, an
>>> > > > > implementation
>>> > > > > > > of topic
>>> > > > > > > > > > > IDs for Kafka. Topic IDs aim to solve topic
>>> uniqueness
>>> > > > > problems in
>>> > > > > > > Kafka,
>>> > > > > > > > > > > where referring to a topic by name alone is
>>> insufficient.
>>> > > > Such
>>> > > > > > > cases
>>> > > > > > > > > > > include when a topic has been deleted and recreated
>>> with
>>> > the
>>> > > > > same
>>> > > > > > > name.
>>> > > > > > > > > > > Unique identifiers will help simplify and improve
>>> Kafka's
>>> > > > topic
>>> > > > > > > deletion
>>> > > > > > > > > > > process, as well as prevent cases where brokers may
>>> > > > incorrectly
>>> > > > > > > interact
>>> > > > > > > > > > > with stale versions of topics.
>>> > > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > > >
>>> > > >
>>> >
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
>>> > > > > > > > > > > Looking forward to your thoughts.
>>> > > > > > > > > > >
>>> > > > > > > > > > > Lucas
>>> > > > > > > > > > >
>>> >
>>> >
>>>
>>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Justine Olshan <jo...@confluent.io>]

Hello again,
To follow up on some of the other comments:

10/11) We can remove the topic name from these requests/responses, and that
means we will just have to make a few internal changes to make partitions
accessible by topic id and partition. I can update the KIP to remove them
unless anyone thinks they should stay.

12) Addressed in the previous email. I've updated the KIP to include tagged
fields for the requests and responses. (More on that below)

13) I think part of the idea for including this information is to prepare
for future changes. Perhaps the directory structure might change from
topicName_partitionNumber to something like topicID_partitionNumber. Then
it would be useful to have the topic name in the file since it would not be
in the directory structure. Supporting topic renames might be easier if the
other fields are included. Would there be any downsides to including this
information?

14)  Yes, we would need to copy the partition metadata file in this
process. I've updated the KIP to include this.

15) I believe Lucas meant v1 and v2 here. He was referring to how the
requests would fall under different IBP and meant that older brokers would
have to use the older version of the request and the existing topic
deletion process. At first, it seemed like tagged fields would resolve
the IBP issue. However, we may need IBP for this request after all since
the controller handles the topic deletion differently depending on the IBP
version. In an older version, we can't just send a StopReplica delete the
topic immediately like we'd want to for this KIP.

This makes me wonder if we want tagged fields on all the requests after
all. Let me know your thoughts!

Justine

On Tue, Sep 15, 2020 at 1:03 PM Justine Olshan <jo...@confluent.io> wrote:

> Hi all,
> Jun brought up a good point in his last email about tagged fields, and
> I've updated the KIP to reflect that the changes to requests and responses
> will be in the form of tagged fields to avoid changing IBP.
>
> Jun: I plan on sending a followup email to address some of the other
> points.
>
> Thanks,
> Justine
>
> On Mon, Sep 14, 2020 at 4:25 PM Jun Rao <ju...@confluent.io> wrote:
>
>> Hi, Justine,
>>
>> Thanks for the updated KIP. A few comments below.
>>
>> 10. LeaderAndIsr Response: Do we need the topic name?
>>
>> 11. For the changed request/response, other than LeaderAndIsr,
>> UpdateMetadata, Metadata, do we need to include the topic name?
>>
>> 12. It seems that upgrades don't require IBP. Does that mean the new
>> fields
>> in all the request/response are added as tagged fields without bumping up
>> the request version? It would be useful to make that clear.
>>
>> 13. Partition Metadata file: Do we need to include the topic name and the
>> partition id since they are implied in the directory name?
>>
>> 14. In the JBOD mode, we support moving a partition's data from one disk
>> to
>> another. Will the new partition metadata file be copied during that
>> process?
>>
>> 15. The KIP says "Remove deleted topics from replicas by sending
>> StopReplicaRequest V2 for any topics which do not contain a topic ID, and
>> V3 for any topics which do contain a topic ID.". However, it seems the
>> updated controller will create all missing topic IDs first before doing
>> other actions. So, is StopReplicaRequest V2 needed?
>>
>> Jun
>>
>> On Fri, Sep 11, 2020 at 10:31 AM John Roesler <vv...@apache.org>
>> wrote:
>>
>> > Thanks, Justine!
>> >
>> > Your response seems compelling to me.
>> >
>> > -John
>> >
>> > On Fri, 2020-09-11 at 10:17 -0700, Justine Olshan wrote:
>> > > Hello all,
>> > > Thanks for continuing the discussion! I have a few responses to your
>> > points.
>> > >
>> > > Tom: You are correct in that this KIP has not mentioned the
>> > > DeleteTopicsRequest. I think that this would be out of scope for now,
>> but
>> > > may be something worth adding in the future.
>> > >
>> > > John: We did consider sequence ids, but there are a few reasons to
>> favor
>> > > UUIDs. There are several cases where topics from different clusters
>> may
>> > > interact now and in the future. For example, Mirror Maker 2 may
>> benefit
>> > > from being able to detect when a cluster being mirrored is deleted and
>> > > recreated and globally unique identifiers would make resolving issues
>> > > easier than sequence IDs which may collide between clusters. KIP-405
>> > > (tiered storage) will also benefit from globally unique IDs as shared
>> > > buckets may be used between clusters.
>> > >
>> > > Globally unique IDs would also make functionality like moving topics
>> > > between disparate clusters easier in the future, simplify any future
>> > > implementations of backups and restores, and more. In general, unique
>> IDs
>> > > would ensure that the source cluster topics do not conflict with the
>> > > destination cluster topics.
>> > >
>> > > If we were to use sequence ids, we would need sufficiently large
>> cluster
>> > > ids to be stored with the topic identifiers or we run the risk of
>> > > collisions. This will give up any advantage in compactness that
>> sequence
>> > > numbers may bring. Given these advantages I think it makes sense to
>> use
>> > > UUIDs.
>> > >
>> > > Gokul: This is an interesting idea, but this is a breaking change.
>> Out of
>> > > scope for now, but maybe worth discussing in the future.
>> > >
>> > > Hope this explains some of the decisions,
>> > >
>> > > Justine
>> > >
>> > >
>> > >
>> > > On Fri, Sep 11, 2020 at 8:27 AM Gokul Ramanan Subramanian <
>> > > gokul2411s@gmail.com> wrote:
>> > >
>> > > > Hi.
>> > > >
>> > > > Thanks for the KIP.
>> > > >
>> > > > Have you thought about whether it makes sense to support
>> authorizing a
>> > > > principal for a topic ID rather than a topic name to achieve tighter
>> > > > security?
>> > > >
>> > > > Or is the topic ID fundamentally an internal detail similar to
>> epochs
>> > used
>> > > > in a bunch of other places in Kafka?
>> > > >
>> > > > Thanks.
>> > > >
>> > > > On Fri, Sep 11, 2020 at 4:06 PM John Roesler <vv...@apache.org>
>> > wrote:
>> > > >
>> > > > > Hello Justine,
>> > > > >
>> > > > > Thanks for the KIP!
>> > > > >
>> > > > > I happen to have been confronted recently with the need to keep
>> > track of
>> > > > a
>> > > > > large number of topics as compactly as possible. I was going to
>> come
>> > up
>> > > > > with some way to dictionary encode the topic names as integers,
>> but
>> > this
>> > > > > seems much better!
>> > > > >
>> > > > > Apologies if this has been raised before, but I’m wondering about
>> the
>> > > > > choice of UUID vs sequence number for the ids. Typically, I’ve
>> seen
>> > UUIDs
>> > > > > in two situations:
>> > > > > 1. When processes need to generate non-colliding identifiers
>> without
>> > > > > coordination.
>> > > > > 2. When the identifier needs to be “universally unique”; I.e., the
>> > > > > identifier needs to distinguish the entity from all other entities
>> > that
>> > > > > could ever exist. This is useful in cases where entities from all
>> > kinds
>> > > > of
>> > > > > systems get mixed together, such as when dumping logs from all
>> > processes
>> > > > in
>> > > > > a company into a common system.
>> > > > >
>> > > > > Maybe I’m being short-sighted, but it doesn’t seem like either
>> really
>> > > > > applies here. It seems like the brokers could and would achieve
>> > consensus
>> > > > > when creating a topic anyway, which is all that’s required to
>> > generate
>> > > > > non-colliding sequence ids. For the second, as you mention, we
>> could
>> > > > assign
>> > > > > a UUID to the cluster as a whole, which would render any resource
>> > scoped
>> > > > to
>> > > > > the broker universally unique as well.
>> > > > >
>> > > > > The reason I mention this is that, although a UUID is way more
>> > compact
>> > > > > than topic names, it’s still 16 bytes. In contrast, a 4-byte
>> integer
>> > > > > sequence id would give us 4 billion unique topics per cluster,
>> which
>> > > > seems
>> > > > > like enough ;)
>> > > > >
>> > > > > Considering the number of different times these topic identifiers
>> are
>> > > > sent
>> > > > > over the wire or stored in memory, it seems like it might be worth
>> > the
>> > > > > additional 4x space savings.
>> > > > >
>> > > > > What do you think about this?
>> > > > >
>> > > > > Thanks,
>> > > > > John
>> > > > >
>> > > > > On Fri, Sep 11, 2020, at 03:20, Tom Bentley wrote:
>> > > > > > Hi Justine,
>> > > > > >
>> > > > > > This looks like a very welcome improvement. Thanks!
>> > > > > >
>> > > > > > Maybe I missed it, but the KIP doesn't seem to mention changing
>> > > > > > DeleteTopicsRequest to identify the topic using an id. Maybe
>> > that's out
>> > > > > of
>> > > > > > scope, but DeleteTopicsRequest is not listed among the Future
>> Work
>> > APIs
>> > > > > > either.
>> > > > > >
>> > > > > > Kind regards,
>> > > > > >
>> > > > > > Tom
>> > > > > >
>> > > > > > On Thu, Sep 10, 2020 at 3:59 PM Satish Duggana <
>> > > > satish.duggana@gmail.com
>> > > > > > wrote:
>> > > > > >
>> > > > > > > Thanks Lucas/Justine for the nice KIP.
>> > > > > > >
>> > > > > > > It has several benefits which also include simplifying the
>> topic
>> > > > > > > deletion process by controller and logs cleanup by brokers in
>> > corner
>> > > > > > > cases.
>> > > > > > >
>> > > > > > > Best,
>> > > > > > > Satish.
>> > > > > > >
>> > > > > > > On Wed, Sep 9, 2020 at 10:07 PM Justine Olshan <
>> > jolshan@confluent.io
>> > > > > > > wrote:
>> > > > > > > > Hello all, it's been almost a year! I've made some changes
>> to
>> > this
>> > > > > KIP
>> > > > > > > and hope to continue the discussion.
>> > > > > > > > One of the main changes I've added is now the metadata
>> response
>> > > > will
>> > > > > > > include the topic ID (as Colin suggested). Clients can obtain
>> the
>> > > > > topicID
>> > > > > > > of a given topic through a TopicDescription. The topicId will
>> > also be
>> > > > > > > included with the UpdateMetadata request.
>> > > > > > > > Let me know what you all think.
>> > > > > > > > Thank you,
>> > > > > > > > Justine
>> > > > > > > >
>> > > > > > > > On 2019/09/13 16:38:26, "Colin McCabe" <cm...@apache.org>
>> > wrote:
>> > > > > > > > > Hi Lucas,
>> > > > > > > > >
>> > > > > > > > > Thanks for tackling this.  Topic IDs are a great idea, and
>> > this
>> > > > is
>> > > > > a
>> > > > > > > really good writeup.
>> > > > > > > > > For /brokers/topics/[topic], the schema version should be
>> > bumped
>> > > > to
>> > > > > > > version 3, rather than 2.  KIP-455 bumped the version of this
>> > znode
>> > > > to
>> > > > > 2
>> > > > > > > already :)
>> > > > > > > > > Given that we're going to be seeing these things as
>> strings
>> > as
>> > > > lot
>> > > > > (in
>> > > > > > > logs, in ZooKeeper, on the command-line, etc.), does it make
>> > sense to
>> > > > > use
>> > > > > > > base64 when converting them to strings?
>> > > > > > > > > Here is an example of the hex representation:
>> > > > > > > > > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
>> > > > > > > > >
>> > > > > > > > > And here is an example in base64.
>> > > > > > > > > b8tRS7h4TJ2Vt43Dp85v2A
>> > > > > > > > >
>> > > > > > > > > The base64 version saves 15 letters (to be fair, 4 of
>> those
>> > were
>> > > > > > > dashes that we could have elided in the hex representation.)
>> > > > > > > > > Another thing to consider is that we should specify that
>> the
>> > > > > > > all-zeroes UUID is not a valid topic UUID.   We can't use null
>> > for
>> > > > this
>> > > > > > > because we can't pass a null UUID over the RPC protocol (there
>> > is no
>> > > > > > > special pattern for null, nor do we want to waste space
>> reserving
>> > > > such
>> > > > > a
>> > > > > > > pattern.)
>> > > > > > > > > Maybe I missed it, but did you describe "migration of...
>> > existing
>> > > > > > > topic[s] without topic IDs" in detail in any section?  It
>> seems
>> > like
>> > > > > when
>> > > > > > > the new controller becomes active, it should just generate
>> random
>> > > > > UUIDs for
>> > > > > > > these, and write the random UUIDs back to ZooKeeper.  It
>> would be
>> > > > good
>> > > > > to
>> > > > > > > spell that out.  We should make it clear that this happens
>> > regardless
>> > > > > of
>> > > > > > > the inter-broker protocol version (it's a compatible change).
>> > > > > > > > > "LeaderAndIsrRequests including an is_every_partition
>> flag"
>> > > > seems a
>> > > > > > > bit wordy.  Can we just call these "full
>> LeaderAndIsrRequests"?
>> > Then
>> > > > > the
>> > > > > > > RPC field could be named "full".  Also, it would probably be
>> > better
>> > > > > for the
>> > > > > > > RPC field to be an enum of { UNSPECIFIED, INCREMENTAL, FULL
>> }, so
>> > > > that
>> > > > > we
>> > > > > > > can cleanly handle old versions (by treating them as
>> UNSPECIFIED)
>> > > > > > > > > In the LeaderAndIsrRequest section, you write "A final
>> > deletion
>> > > > > event
>> > > > > > > will be secheduled for X ms after the LeaderAndIsrRequest was
>> > first
>> > > > > > > received..."  I guess the X was a placeholder that you
>> intended
>> > to
>> > > > > replace
>> > > > > > > before posting? :)  In any case, this seems like the kind of
>> > thing
>> > > > we'd
>> > > > > > > want a configuration for.  Let's describe that configuration
>> key
>> > > > > somewhere
>> > > > > > > in this KIP, including what its default value is.
>> > > > > > > > > We should probably also log a bunch of messages at WARN
>> level
>> > > > when
>> > > > > > > something is scheduled for deletion, as well.  (Maybe this was
>> > > > > assumed, but
>> > > > > > > it would be good to mention it).
>> > > > > > > > > I feel like there are a few sections that should be moved
>> to
>> > > > > "rejected
>> > > > > > > alternatives."  For example, in the DeleteTopics section,
>> since
>> > we're
>> > > > > not
>> > > > > > > going to do option 1 or 2, these should be moved into
>> "rejected
>> > > > > > > alternatives,"  rather than appearing inline.  Another case is
>> > the
>> > > > > "Should
>> > > > > > > we remove topic name from the protocol where possible"
>> section.
>> > This
>> > > > > is
>> > > > > > > clearly discussing a design alternative that we're not
>> proposing
>> > to
>> > > > > > > implement: removing the topic name from those protocols.
>> > > > > > > > > Is it really necessary to have a new
>> > /admin/delete_topics_by_id
>> > > > > path
>> > > > > > > in ZooKeeper?  It seems like we don't really need this.
>> Whenever
>> > > > > there is
>> > > > > > > a new controller, we'll send out full LeaderAndIsrRequests
>> which
>> > will
>> > > > > > > trigger the stale topics to be cleaned up.   The active
>> > controller
>> > > > will
>> > > > > > > also send the full LeaderAndIsrRequest to brokers that are
>> just
>> > > > > starting
>> > > > > > > up.    So we don't really need this kind of two-phase commit
>> > (send
>> > > > out
>> > > > > > > StopReplicasRequest, get ACKs from all nodes, commit by
>> removing
>> > > > > > > /admin/delete_topics node) any more.
>> > > > > > > > > You mention that FetchRequest will now include UUID to
>> avoid
>> > > > issues
>> > > > > > > where requests are made to stale partitions.  However, adding
>> a
>> > UUID
>> > > > to
>> > > > > > > MetadataRequest is listed as future work, out of scope for
>> this
>> > KIP.
>> > > > > How
>> > > > > > > will the client learn what the topic UUID is, if the metadata
>> > > > response
>> > > > > > > doesn't include that information?  It seems like adding the
>> UUID
>> > to
>> > > > > > > MetadataResponse would be an improvement here that might not
>> be
>> > too
>> > > > > hard to
>> > > > > > > make.
>> > > > > > > > > best,
>> > > > > > > > > Colin
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > On Mon, Sep 9, 2019, at 17:48, Ryanne Dolan wrote:
>> > > > > > > > > > Lucas, this would be great. I've run into issues with
>> > topics
>> > > > > being
>> > > > > > > > > > resurrected accidentally, since a client cannot easily
>> > > > > distinguish
>> > > > > > > between
>> > > > > > > > > > a deleted topic and a new topic with the same name. I'd
>> > need
>> > > > the
>> > > > > ID
>> > > > > > > > > > accessible from the client to solve that issue, but this
>> > is a
>> > > > > good
>> > > > > > > first
>> > > > > > > > > > step.
>> > > > > > > > > >
>> > > > > > > > > > Ryanne
>> > > > > > > > > >
>> > > > > > > > > > On Wed, Sep 4, 2019 at 1:41 PM Lucas Bradstreet <
>> > > > > lucas@confluent.io>
>> > > > > > > wrote:
>> > > > > > > > > > > Hi all,
>> > > > > > > > > > >
>> > > > > > > > > > > I would like to kick off discussion of KIP-516, an
>> > > > > implementation
>> > > > > > > of topic
>> > > > > > > > > > > IDs for Kafka. Topic IDs aim to solve topic uniqueness
>> > > > > problems in
>> > > > > > > Kafka,
>> > > > > > > > > > > where referring to a topic by name alone is
>> insufficient.
>> > > > Such
>> > > > > > > cases
>> > > > > > > > > > > include when a topic has been deleted and recreated
>> with
>> > the
>> > > > > same
>> > > > > > > name.
>> > > > > > > > > > > Unique identifiers will help simplify and improve
>> Kafka's
>> > > > topic
>> > > > > > > deletion
>> > > > > > > > > > > process, as well as prevent cases where brokers may
>> > > > incorrectly
>> > > > > > > interact
>> > > > > > > > > > > with stale versions of topics.
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
>> > > > > > > > > > > Looking forward to your thoughts.
>> > > > > > > > > > >
>> > > > > > > > > > > Lucas
>> > > > > > > > > > >
>> >
>> >
>>
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Justine Olshan <jo...@confluent.io>]

Hi all,
Jun brought up a good point in his last email about tagged fields, and I've
updated the KIP to reflect that the changes to requests and responses will
be in the form of tagged fields to avoid changing IBP.

Jun: I plan on sending a followup email to address some of the other points.

Thanks,
Justine

On Mon, Sep 14, 2020 at 4:25 PM Jun Rao <ju...@confluent.io> wrote:

> Hi, Justine,
>
> Thanks for the updated KIP. A few comments below.
>
> 10. LeaderAndIsr Response: Do we need the topic name?
>
> 11. For the changed request/response, other than LeaderAndIsr,
> UpdateMetadata, Metadata, do we need to include the topic name?
>
> 12. It seems that upgrades don't require IBP. Does that mean the new fields
> in all the request/response are added as tagged fields without bumping up
> the request version? It would be useful to make that clear.
>
> 13. Partition Metadata file: Do we need to include the topic name and the
> partition id since they are implied in the directory name?
>
> 14. In the JBOD mode, we support moving a partition's data from one disk to
> another. Will the new partition metadata file be copied during that
> process?
>
> 15. The KIP says "Remove deleted topics from replicas by sending
> StopReplicaRequest V2 for any topics which do not contain a topic ID, and
> V3 for any topics which do contain a topic ID.". However, it seems the
> updated controller will create all missing topic IDs first before doing
> other actions. So, is StopReplicaRequest V2 needed?
>
> Jun
>
> On Fri, Sep 11, 2020 at 10:31 AM John Roesler <vv...@apache.org> wrote:
>
> > Thanks, Justine!
> >
> > Your response seems compelling to me.
> >
> > -John
> >
> > On Fri, 2020-09-11 at 10:17 -0700, Justine Olshan wrote:
> > > Hello all,
> > > Thanks for continuing the discussion! I have a few responses to your
> > points.
> > >
> > > Tom: You are correct in that this KIP has not mentioned the
> > > DeleteTopicsRequest. I think that this would be out of scope for now,
> but
> > > may be something worth adding in the future.
> > >
> > > John: We did consider sequence ids, but there are a few reasons to
> favor
> > > UUIDs. There are several cases where topics from different clusters may
> > > interact now and in the future. For example, Mirror Maker 2 may benefit
> > > from being able to detect when a cluster being mirrored is deleted and
> > > recreated and globally unique identifiers would make resolving issues
> > > easier than sequence IDs which may collide between clusters. KIP-405
> > > (tiered storage) will also benefit from globally unique IDs as shared
> > > buckets may be used between clusters.
> > >
> > > Globally unique IDs would also make functionality like moving topics
> > > between disparate clusters easier in the future, simplify any future
> > > implementations of backups and restores, and more. In general, unique
> IDs
> > > would ensure that the source cluster topics do not conflict with the
> > > destination cluster topics.
> > >
> > > If we were to use sequence ids, we would need sufficiently large
> cluster
> > > ids to be stored with the topic identifiers or we run the risk of
> > > collisions. This will give up any advantage in compactness that
> sequence
> > > numbers may bring. Given these advantages I think it makes sense to use
> > > UUIDs.
> > >
> > > Gokul: This is an interesting idea, but this is a breaking change. Out
> of
> > > scope for now, but maybe worth discussing in the future.
> > >
> > > Hope this explains some of the decisions,
> > >
> > > Justine
> > >
> > >
> > >
> > > On Fri, Sep 11, 2020 at 8:27 AM Gokul Ramanan Subramanian <
> > > gokul2411s@gmail.com> wrote:
> > >
> > > > Hi.
> > > >
> > > > Thanks for the KIP.
> > > >
> > > > Have you thought about whether it makes sense to support authorizing
> a
> > > > principal for a topic ID rather than a topic name to achieve tighter
> > > > security?
> > > >
> > > > Or is the topic ID fundamentally an internal detail similar to epochs
> > used
> > > > in a bunch of other places in Kafka?
> > > >
> > > > Thanks.
> > > >
> > > > On Fri, Sep 11, 2020 at 4:06 PM John Roesler <vv...@apache.org>
> > wrote:
> > > >
> > > > > Hello Justine,
> > > > >
> > > > > Thanks for the KIP!
> > > > >
> > > > > I happen to have been confronted recently with the need to keep
> > track of
> > > > a
> > > > > large number of topics as compactly as possible. I was going to
> come
> > up
> > > > > with some way to dictionary encode the topic names as integers, but
> > this
> > > > > seems much better!
> > > > >
> > > > > Apologies if this has been raised before, but I’m wondering about
> the
> > > > > choice of UUID vs sequence number for the ids. Typically, I’ve seen
> > UUIDs
> > > > > in two situations:
> > > > > 1. When processes need to generate non-colliding identifiers
> without
> > > > > coordination.
> > > > > 2. When the identifier needs to be “universally unique”; I.e., the
> > > > > identifier needs to distinguish the entity from all other entities
> > that
> > > > > could ever exist. This is useful in cases where entities from all
> > kinds
> > > > of
> > > > > systems get mixed together, such as when dumping logs from all
> > processes
> > > > in
> > > > > a company into a common system.
> > > > >
> > > > > Maybe I’m being short-sighted, but it doesn’t seem like either
> really
> > > > > applies here. It seems like the brokers could and would achieve
> > consensus
> > > > > when creating a topic anyway, which is all that’s required to
> > generate
> > > > > non-colliding sequence ids. For the second, as you mention, we
> could
> > > > assign
> > > > > a UUID to the cluster as a whole, which would render any resource
> > scoped
> > > > to
> > > > > the broker universally unique as well.
> > > > >
> > > > > The reason I mention this is that, although a UUID is way more
> > compact
> > > > > than topic names, it’s still 16 bytes. In contrast, a 4-byte
> integer
> > > > > sequence id would give us 4 billion unique topics per cluster,
> which
> > > > seems
> > > > > like enough ;)
> > > > >
> > > > > Considering the number of different times these topic identifiers
> are
> > > > sent
> > > > > over the wire or stored in memory, it seems like it might be worth
> > the
> > > > > additional 4x space savings.
> > > > >
> > > > > What do you think about this?
> > > > >
> > > > > Thanks,
> > > > > John
> > > > >
> > > > > On Fri, Sep 11, 2020, at 03:20, Tom Bentley wrote:
> > > > > > Hi Justine,
> > > > > >
> > > > > > This looks like a very welcome improvement. Thanks!
> > > > > >
> > > > > > Maybe I missed it, but the KIP doesn't seem to mention changing
> > > > > > DeleteTopicsRequest to identify the topic using an id. Maybe
> > that's out
> > > > > of
> > > > > > scope, but DeleteTopicsRequest is not listed among the Future
> Work
> > APIs
> > > > > > either.
> > > > > >
> > > > > > Kind regards,
> > > > > >
> > > > > > Tom
> > > > > >
> > > > > > On Thu, Sep 10, 2020 at 3:59 PM Satish Duggana <
> > > > satish.duggana@gmail.com
> > > > > > wrote:
> > > > > >
> > > > > > > Thanks Lucas/Justine for the nice KIP.
> > > > > > >
> > > > > > > It has several benefits which also include simplifying the
> topic
> > > > > > > deletion process by controller and logs cleanup by brokers in
> > corner
> > > > > > > cases.
> > > > > > >
> > > > > > > Best,
> > > > > > > Satish.
> > > > > > >
> > > > > > > On Wed, Sep 9, 2020 at 10:07 PM Justine Olshan <
> > jolshan@confluent.io
> > > > > > > wrote:
> > > > > > > > Hello all, it's been almost a year! I've made some changes to
> > this
> > > > > KIP
> > > > > > > and hope to continue the discussion.
> > > > > > > > One of the main changes I've added is now the metadata
> response
> > > > will
> > > > > > > include the topic ID (as Colin suggested). Clients can obtain
> the
> > > > > topicID
> > > > > > > of a given topic through a TopicDescription. The topicId will
> > also be
> > > > > > > included with the UpdateMetadata request.
> > > > > > > > Let me know what you all think.
> > > > > > > > Thank you,
> > > > > > > > Justine
> > > > > > > >
> > > > > > > > On 2019/09/13 16:38:26, "Colin McCabe" <cm...@apache.org>
> > wrote:
> > > > > > > > > Hi Lucas,
> > > > > > > > >
> > > > > > > > > Thanks for tackling this.  Topic IDs are a great idea, and
> > this
> > > > is
> > > > > a
> > > > > > > really good writeup.
> > > > > > > > > For /brokers/topics/[topic], the schema version should be
> > bumped
> > > > to
> > > > > > > version 3, rather than 2.  KIP-455 bumped the version of this
> > znode
> > > > to
> > > > > 2
> > > > > > > already :)
> > > > > > > > > Given that we're going to be seeing these things as strings
> > as
> > > > lot
> > > > > (in
> > > > > > > logs, in ZooKeeper, on the command-line, etc.), does it make
> > sense to
> > > > > use
> > > > > > > base64 when converting them to strings?
> > > > > > > > > Here is an example of the hex representation:
> > > > > > > > > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> > > > > > > > >
> > > > > > > > > And here is an example in base64.
> > > > > > > > > b8tRS7h4TJ2Vt43Dp85v2A
> > > > > > > > >
> > > > > > > > > The base64 version saves 15 letters (to be fair, 4 of those
> > were
> > > > > > > dashes that we could have elided in the hex representation.)
> > > > > > > > > Another thing to consider is that we should specify that
> the
> > > > > > > all-zeroes UUID is not a valid topic UUID.   We can't use null
> > for
> > > > this
> > > > > > > because we can't pass a null UUID over the RPC protocol (there
> > is no
> > > > > > > special pattern for null, nor do we want to waste space
> reserving
> > > > such
> > > > > a
> > > > > > > pattern.)
> > > > > > > > > Maybe I missed it, but did you describe "migration of...
> > existing
> > > > > > > topic[s] without topic IDs" in detail in any section?  It seems
> > like
> > > > > when
> > > > > > > the new controller becomes active, it should just generate
> random
> > > > > UUIDs for
> > > > > > > these, and write the random UUIDs back to ZooKeeper.  It would
> be
> > > > good
> > > > > to
> > > > > > > spell that out.  We should make it clear that this happens
> > regardless
> > > > > of
> > > > > > > the inter-broker protocol version (it's a compatible change).
> > > > > > > > > "LeaderAndIsrRequests including an is_every_partition flag"
> > > > seems a
> > > > > > > bit wordy.  Can we just call these "full LeaderAndIsrRequests"?
> > Then
> > > > > the
> > > > > > > RPC field could be named "full".  Also, it would probably be
> > better
> > > > > for the
> > > > > > > RPC field to be an enum of { UNSPECIFIED, INCREMENTAL, FULL },
> so
> > > > that
> > > > > we
> > > > > > > can cleanly handle old versions (by treating them as
> UNSPECIFIED)
> > > > > > > > > In the LeaderAndIsrRequest section, you write "A final
> > deletion
> > > > > event
> > > > > > > will be secheduled for X ms after the LeaderAndIsrRequest was
> > first
> > > > > > > received..."  I guess the X was a placeholder that you intended
> > to
> > > > > replace
> > > > > > > before posting? :)  In any case, this seems like the kind of
> > thing
> > > > we'd
> > > > > > > want a configuration for.  Let's describe that configuration
> key
> > > > > somewhere
> > > > > > > in this KIP, including what its default value is.
> > > > > > > > > We should probably also log a bunch of messages at WARN
> level
> > > > when
> > > > > > > something is scheduled for deletion, as well.  (Maybe this was
> > > > > assumed, but
> > > > > > > it would be good to mention it).
> > > > > > > > > I feel like there are a few sections that should be moved
> to
> > > > > "rejected
> > > > > > > alternatives."  For example, in the DeleteTopics section, since
> > we're
> > > > > not
> > > > > > > going to do option 1 or 2, these should be moved into "rejected
> > > > > > > alternatives,"  rather than appearing inline.  Another case is
> > the
> > > > > "Should
> > > > > > > we remove topic name from the protocol where possible" section.
> > This
> > > > > is
> > > > > > > clearly discussing a design alternative that we're not
> proposing
> > to
> > > > > > > implement: removing the topic name from those protocols.
> > > > > > > > > Is it really necessary to have a new
> > /admin/delete_topics_by_id
> > > > > path
> > > > > > > in ZooKeeper?  It seems like we don't really need this.
> Whenever
> > > > > there is
> > > > > > > a new controller, we'll send out full LeaderAndIsrRequests
> which
> > will
> > > > > > > trigger the stale topics to be cleaned up.   The active
> > controller
> > > > will
> > > > > > > also send the full LeaderAndIsrRequest to brokers that are just
> > > > > starting
> > > > > > > up.    So we don't really need this kind of two-phase commit
> > (send
> > > > out
> > > > > > > StopReplicasRequest, get ACKs from all nodes, commit by
> removing
> > > > > > > /admin/delete_topics node) any more.
> > > > > > > > > You mention that FetchRequest will now include UUID to
> avoid
> > > > issues
> > > > > > > where requests are made to stale partitions.  However, adding a
> > UUID
> > > > to
> > > > > > > MetadataRequest is listed as future work, out of scope for this
> > KIP.
> > > > > How
> > > > > > > will the client learn what the topic UUID is, if the metadata
> > > > response
> > > > > > > doesn't include that information?  It seems like adding the
> UUID
> > to
> > > > > > > MetadataResponse would be an improvement here that might not be
> > too
> > > > > hard to
> > > > > > > make.
> > > > > > > > > best,
> > > > > > > > > Colin
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Mon, Sep 9, 2019, at 17:48, Ryanne Dolan wrote:
> > > > > > > > > > Lucas, this would be great. I've run into issues with
> > topics
> > > > > being
> > > > > > > > > > resurrected accidentally, since a client cannot easily
> > > > > distinguish
> > > > > > > between
> > > > > > > > > > a deleted topic and a new topic with the same name. I'd
> > need
> > > > the
> > > > > ID
> > > > > > > > > > accessible from the client to solve that issue, but this
> > is a
> > > > > good
> > > > > > > first
> > > > > > > > > > step.
> > > > > > > > > >
> > > > > > > > > > Ryanne
> > > > > > > > > >
> > > > > > > > > > On Wed, Sep 4, 2019 at 1:41 PM Lucas Bradstreet <
> > > > > lucas@confluent.io>
> > > > > > > wrote:
> > > > > > > > > > > Hi all,
> > > > > > > > > > >
> > > > > > > > > > > I would like to kick off discussion of KIP-516, an
> > > > > implementation
> > > > > > > of topic
> > > > > > > > > > > IDs for Kafka. Topic IDs aim to solve topic uniqueness
> > > > > problems in
> > > > > > > Kafka,
> > > > > > > > > > > where referring to a topic by name alone is
> insufficient.
> > > > Such
> > > > > > > cases
> > > > > > > > > > > include when a topic has been deleted and recreated
> with
> > the
> > > > > same
> > > > > > > name.
> > > > > > > > > > > Unique identifiers will help simplify and improve
> Kafka's
> > > > topic
> > > > > > > deletion
> > > > > > > > > > > process, as well as prevent cases where brokers may
> > > > incorrectly
> > > > > > > interact
> > > > > > > > > > > with stale versions of topics.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > > > > > > > > > > Looking forward to your thoughts.
> > > > > > > > > > >
> > > > > > > > > > > Lucas
> > > > > > > > > > >
> >
> >
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Jun Rao <ju...@confluent.io>]

Hi, Justine,

Thanks for the updated KIP. A few comments below.

10. LeaderAndIsr Response: Do we need the topic name?

11. For the changed request/response, other than LeaderAndIsr,
UpdateMetadata, Metadata, do we need to include the topic name?

12. It seems that upgrades don't require IBP. Does that mean the new fields
in all the request/response are added as tagged fields without bumping up
the request version? It would be useful to make that clear.

13. Partition Metadata file: Do we need to include the topic name and the
partition id since they are implied in the directory name?

14. In the JBOD mode, we support moving a partition's data from one disk to
another. Will the new partition metadata file be copied during that process?

15. The KIP says "Remove deleted topics from replicas by sending
StopReplicaRequest V2 for any topics which do not contain a topic ID, and
V3 for any topics which do contain a topic ID.". However, it seems the
updated controller will create all missing topic IDs first before doing
other actions. So, is StopReplicaRequest V2 needed?

Jun

On Fri, Sep 11, 2020 at 10:31 AM John Roesler <vv...@apache.org> wrote:

> Thanks, Justine!
>
> Your response seems compelling to me.
>
> -John
>
> On Fri, 2020-09-11 at 10:17 -0700, Justine Olshan wrote:
> > Hello all,
> > Thanks for continuing the discussion! I have a few responses to your
> points.
> >
> > Tom: You are correct in that this KIP has not mentioned the
> > DeleteTopicsRequest. I think that this would be out of scope for now, but
> > may be something worth adding in the future.
> >
> > John: We did consider sequence ids, but there are a few reasons to favor
> > UUIDs. There are several cases where topics from different clusters may
> > interact now and in the future. For example, Mirror Maker 2 may benefit
> > from being able to detect when a cluster being mirrored is deleted and
> > recreated and globally unique identifiers would make resolving issues
> > easier than sequence IDs which may collide between clusters. KIP-405
> > (tiered storage) will also benefit from globally unique IDs as shared
> > buckets may be used between clusters.
> >
> > Globally unique IDs would also make functionality like moving topics
> > between disparate clusters easier in the future, simplify any future
> > implementations of backups and restores, and more. In general, unique IDs
> > would ensure that the source cluster topics do not conflict with the
> > destination cluster topics.
> >
> > If we were to use sequence ids, we would need sufficiently large cluster
> > ids to be stored with the topic identifiers or we run the risk of
> > collisions. This will give up any advantage in compactness that sequence
> > numbers may bring. Given these advantages I think it makes sense to use
> > UUIDs.
> >
> > Gokul: This is an interesting idea, but this is a breaking change. Out of
> > scope for now, but maybe worth discussing in the future.
> >
> > Hope this explains some of the decisions,
> >
> > Justine
> >
> >
> >
> > On Fri, Sep 11, 2020 at 8:27 AM Gokul Ramanan Subramanian <
> > gokul2411s@gmail.com> wrote:
> >
> > > Hi.
> > >
> > > Thanks for the KIP.
> > >
> > > Have you thought about whether it makes sense to support authorizing a
> > > principal for a topic ID rather than a topic name to achieve tighter
> > > security?
> > >
> > > Or is the topic ID fundamentally an internal detail similar to epochs
> used
> > > in a bunch of other places in Kafka?
> > >
> > > Thanks.
> > >
> > > On Fri, Sep 11, 2020 at 4:06 PM John Roesler <vv...@apache.org>
> wrote:
> > >
> > > > Hello Justine,
> > > >
> > > > Thanks for the KIP!
> > > >
> > > > I happen to have been confronted recently with the need to keep
> track of
> > > a
> > > > large number of topics as compactly as possible. I was going to come
> up
> > > > with some way to dictionary encode the topic names as integers, but
> this
> > > > seems much better!
> > > >
> > > > Apologies if this has been raised before, but I’m wondering about the
> > > > choice of UUID vs sequence number for the ids. Typically, I’ve seen
> UUIDs
> > > > in two situations:
> > > > 1. When processes need to generate non-colliding identifiers without
> > > > coordination.
> > > > 2. When the identifier needs to be “universally unique”; I.e., the
> > > > identifier needs to distinguish the entity from all other entities
> that
> > > > could ever exist. This is useful in cases where entities from all
> kinds
> > > of
> > > > systems get mixed together, such as when dumping logs from all
> processes
> > > in
> > > > a company into a common system.
> > > >
> > > > Maybe I’m being short-sighted, but it doesn’t seem like either really
> > > > applies here. It seems like the brokers could and would achieve
> consensus
> > > > when creating a topic anyway, which is all that’s required to
> generate
> > > > non-colliding sequence ids. For the second, as you mention, we could
> > > assign
> > > > a UUID to the cluster as a whole, which would render any resource
> scoped
> > > to
> > > > the broker universally unique as well.
> > > >
> > > > The reason I mention this is that, although a UUID is way more
> compact
> > > > than topic names, it’s still 16 bytes. In contrast, a 4-byte integer
> > > > sequence id would give us 4 billion unique topics per cluster, which
> > > seems
> > > > like enough ;)
> > > >
> > > > Considering the number of different times these topic identifiers are
> > > sent
> > > > over the wire or stored in memory, it seems like it might be worth
> the
> > > > additional 4x space savings.
> > > >
> > > > What do you think about this?
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > > On Fri, Sep 11, 2020, at 03:20, Tom Bentley wrote:
> > > > > Hi Justine,
> > > > >
> > > > > This looks like a very welcome improvement. Thanks!
> > > > >
> > > > > Maybe I missed it, but the KIP doesn't seem to mention changing
> > > > > DeleteTopicsRequest to identify the topic using an id. Maybe
> that's out
> > > > of
> > > > > scope, but DeleteTopicsRequest is not listed among the Future Work
> APIs
> > > > > either.
> > > > >
> > > > > Kind regards,
> > > > >
> > > > > Tom
> > > > >
> > > > > On Thu, Sep 10, 2020 at 3:59 PM Satish Duggana <
> > > satish.duggana@gmail.com
> > > > > wrote:
> > > > >
> > > > > > Thanks Lucas/Justine for the nice KIP.
> > > > > >
> > > > > > It has several benefits which also include simplifying the topic
> > > > > > deletion process by controller and logs cleanup by brokers in
> corner
> > > > > > cases.
> > > > > >
> > > > > > Best,
> > > > > > Satish.
> > > > > >
> > > > > > On Wed, Sep 9, 2020 at 10:07 PM Justine Olshan <
> jolshan@confluent.io
> > > > > > wrote:
> > > > > > > Hello all, it's been almost a year! I've made some changes to
> this
> > > > KIP
> > > > > > and hope to continue the discussion.
> > > > > > > One of the main changes I've added is now the metadata response
> > > will
> > > > > > include the topic ID (as Colin suggested). Clients can obtain the
> > > > topicID
> > > > > > of a given topic through a TopicDescription. The topicId will
> also be
> > > > > > included with the UpdateMetadata request.
> > > > > > > Let me know what you all think.
> > > > > > > Thank you,
> > > > > > > Justine
> > > > > > >
> > > > > > > On 2019/09/13 16:38:26, "Colin McCabe" <cm...@apache.org>
> wrote:
> > > > > > > > Hi Lucas,
> > > > > > > >
> > > > > > > > Thanks for tackling this.  Topic IDs are a great idea, and
> this
> > > is
> > > > a
> > > > > > really good writeup.
> > > > > > > > For /brokers/topics/[topic], the schema version should be
> bumped
> > > to
> > > > > > version 3, rather than 2.  KIP-455 bumped the version of this
> znode
> > > to
> > > > 2
> > > > > > already :)
> > > > > > > > Given that we're going to be seeing these things as strings
> as
> > > lot
> > > > (in
> > > > > > logs, in ZooKeeper, on the command-line, etc.), does it make
> sense to
> > > > use
> > > > > > base64 when converting them to strings?
> > > > > > > > Here is an example of the hex representation:
> > > > > > > > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> > > > > > > >
> > > > > > > > And here is an example in base64.
> > > > > > > > b8tRS7h4TJ2Vt43Dp85v2A
> > > > > > > >
> > > > > > > > The base64 version saves 15 letters (to be fair, 4 of those
> were
> > > > > > dashes that we could have elided in the hex representation.)
> > > > > > > > Another thing to consider is that we should specify that the
> > > > > > all-zeroes UUID is not a valid topic UUID.   We can't use null
> for
> > > this
> > > > > > because we can't pass a null UUID over the RPC protocol (there
> is no
> > > > > > special pattern for null, nor do we want to waste space reserving
> > > such
> > > > a
> > > > > > pattern.)
> > > > > > > > Maybe I missed it, but did you describe "migration of...
> existing
> > > > > > topic[s] without topic IDs" in detail in any section?  It seems
> like
> > > > when
> > > > > > the new controller becomes active, it should just generate random
> > > > UUIDs for
> > > > > > these, and write the random UUIDs back to ZooKeeper.  It would be
> > > good
> > > > to
> > > > > > spell that out.  We should make it clear that this happens
> regardless
> > > > of
> > > > > > the inter-broker protocol version (it's a compatible change).
> > > > > > > > "LeaderAndIsrRequests including an is_every_partition flag"
> > > seems a
> > > > > > bit wordy.  Can we just call these "full LeaderAndIsrRequests"?
> Then
> > > > the
> > > > > > RPC field could be named "full".  Also, it would probably be
> better
> > > > for the
> > > > > > RPC field to be an enum of { UNSPECIFIED, INCREMENTAL, FULL }, so
> > > that
> > > > we
> > > > > > can cleanly handle old versions (by treating them as UNSPECIFIED)
> > > > > > > > In the LeaderAndIsrRequest section, you write "A final
> deletion
> > > > event
> > > > > > will be secheduled for X ms after the LeaderAndIsrRequest was
> first
> > > > > > received..."  I guess the X was a placeholder that you intended
> to
> > > > replace
> > > > > > before posting? :)  In any case, this seems like the kind of
> thing
> > > we'd
> > > > > > want a configuration for.  Let's describe that configuration key
> > > > somewhere
> > > > > > in this KIP, including what its default value is.
> > > > > > > > We should probably also log a bunch of messages at WARN level
> > > when
> > > > > > something is scheduled for deletion, as well.  (Maybe this was
> > > > assumed, but
> > > > > > it would be good to mention it).
> > > > > > > > I feel like there are a few sections that should be moved to
> > > > "rejected
> > > > > > alternatives."  For example, in the DeleteTopics section, since
> we're
> > > > not
> > > > > > going to do option 1 or 2, these should be moved into "rejected
> > > > > > alternatives,"  rather than appearing inline.  Another case is
> the
> > > > "Should
> > > > > > we remove topic name from the protocol where possible" section.
> This
> > > > is
> > > > > > clearly discussing a design alternative that we're not proposing
> to
> > > > > > implement: removing the topic name from those protocols.
> > > > > > > > Is it really necessary to have a new
> /admin/delete_topics_by_id
> > > > path
> > > > > > in ZooKeeper?  It seems like we don't really need this.  Whenever
> > > > there is
> > > > > > a new controller, we'll send out full LeaderAndIsrRequests which
> will
> > > > > > trigger the stale topics to be cleaned up.   The active
> controller
> > > will
> > > > > > also send the full LeaderAndIsrRequest to brokers that are just
> > > > starting
> > > > > > up.    So we don't really need this kind of two-phase commit
> (send
> > > out
> > > > > > StopReplicasRequest, get ACKs from all nodes, commit by removing
> > > > > > /admin/delete_topics node) any more.
> > > > > > > > You mention that FetchRequest will now include UUID to avoid
> > > issues
> > > > > > where requests are made to stale partitions.  However, adding a
> UUID
> > > to
> > > > > > MetadataRequest is listed as future work, out of scope for this
> KIP.
> > > > How
> > > > > > will the client learn what the topic UUID is, if the metadata
> > > response
> > > > > > doesn't include that information?  It seems like adding the UUID
> to
> > > > > > MetadataResponse would be an improvement here that might not be
> too
> > > > hard to
> > > > > > make.
> > > > > > > > best,
> > > > > > > > Colin
> > > > > > > >
> > > > > > > >
> > > > > > > > On Mon, Sep 9, 2019, at 17:48, Ryanne Dolan wrote:
> > > > > > > > > Lucas, this would be great. I've run into issues with
> topics
> > > > being
> > > > > > > > > resurrected accidentally, since a client cannot easily
> > > > distinguish
> > > > > > between
> > > > > > > > > a deleted topic and a new topic with the same name. I'd
> need
> > > the
> > > > ID
> > > > > > > > > accessible from the client to solve that issue, but this
> is a
> > > > good
> > > > > > first
> > > > > > > > > step.
> > > > > > > > >
> > > > > > > > > Ryanne
> > > > > > > > >
> > > > > > > > > On Wed, Sep 4, 2019 at 1:41 PM Lucas Bradstreet <
> > > > lucas@confluent.io>
> > > > > > wrote:
> > > > > > > > > > Hi all,
> > > > > > > > > >
> > > > > > > > > > I would like to kick off discussion of KIP-516, an
> > > > implementation
> > > > > > of topic
> > > > > > > > > > IDs for Kafka. Topic IDs aim to solve topic uniqueness
> > > > problems in
> > > > > > Kafka,
> > > > > > > > > > where referring to a topic by name alone is insufficient.
> > > Such
> > > > > > cases
> > > > > > > > > > include when a topic has been deleted and recreated with
> the
> > > > same
> > > > > > name.
> > > > > > > > > > Unique identifiers will help simplify and improve Kafka's
> > > topic
> > > > > > deletion
> > > > > > > > > > process, as well as prevent cases where brokers may
> > > incorrectly
> > > > > > interact
> > > > > > > > > > with stale versions of topics.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > > > > > > > > > Looking forward to your thoughts.
> > > > > > > > > >
> > > > > > > > > > Lucas
> > > > > > > > > >
>
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[John Roesler <vv...@apache.org>]

Thanks, Justine!

Your response seems compelling to me.

-John

On Fri, 2020-09-11 at 10:17 -0700, Justine Olshan wrote:
> Hello all,
> Thanks for continuing the discussion! I have a few responses to your points.
> 
> Tom: You are correct in that this KIP has not mentioned the
> DeleteTopicsRequest. I think that this would be out of scope for now, but
> may be something worth adding in the future.
> 
> John: We did consider sequence ids, but there are a few reasons to favor
> UUIDs. There are several cases where topics from different clusters may
> interact now and in the future. For example, Mirror Maker 2 may benefit
> from being able to detect when a cluster being mirrored is deleted and
> recreated and globally unique identifiers would make resolving issues
> easier than sequence IDs which may collide between clusters. KIP-405
> (tiered storage) will also benefit from globally unique IDs as shared
> buckets may be used between clusters.
> 
> Globally unique IDs would also make functionality like moving topics
> between disparate clusters easier in the future, simplify any future
> implementations of backups and restores, and more. In general, unique IDs
> would ensure that the source cluster topics do not conflict with the
> destination cluster topics.
> 
> If we were to use sequence ids, we would need sufficiently large cluster
> ids to be stored with the topic identifiers or we run the risk of
> collisions. This will give up any advantage in compactness that sequence
> numbers may bring. Given these advantages I think it makes sense to use
> UUIDs.
> 
> Gokul: This is an interesting idea, but this is a breaking change. Out of
> scope for now, but maybe worth discussing in the future.
> 
> Hope this explains some of the decisions,
> 
> Justine
> 
> 
> 
> On Fri, Sep 11, 2020 at 8:27 AM Gokul Ramanan Subramanian <
> gokul2411s@gmail.com> wrote:
> 
> > Hi.
> > 
> > Thanks for the KIP.
> > 
> > Have you thought about whether it makes sense to support authorizing a
> > principal for a topic ID rather than a topic name to achieve tighter
> > security?
> > 
> > Or is the topic ID fundamentally an internal detail similar to epochs used
> > in a bunch of other places in Kafka?
> > 
> > Thanks.
> > 
> > On Fri, Sep 11, 2020 at 4:06 PM John Roesler <vv...@apache.org> wrote:
> > 
> > > Hello Justine,
> > > 
> > > Thanks for the KIP!
> > > 
> > > I happen to have been confronted recently with the need to keep track of
> > a
> > > large number of topics as compactly as possible. I was going to come up
> > > with some way to dictionary encode the topic names as integers, but this
> > > seems much better!
> > > 
> > > Apologies if this has been raised before, but I’m wondering about the
> > > choice of UUID vs sequence number for the ids. Typically, I’ve seen UUIDs
> > > in two situations:
> > > 1. When processes need to generate non-colliding identifiers without
> > > coordination.
> > > 2. When the identifier needs to be “universally unique”; I.e., the
> > > identifier needs to distinguish the entity from all other entities that
> > > could ever exist. This is useful in cases where entities from all kinds
> > of
> > > systems get mixed together, such as when dumping logs from all processes
> > in
> > > a company into a common system.
> > > 
> > > Maybe I’m being short-sighted, but it doesn’t seem like either really
> > > applies here. It seems like the brokers could and would achieve consensus
> > > when creating a topic anyway, which is all that’s required to generate
> > > non-colliding sequence ids. For the second, as you mention, we could
> > assign
> > > a UUID to the cluster as a whole, which would render any resource scoped
> > to
> > > the broker universally unique as well.
> > > 
> > > The reason I mention this is that, although a UUID is way more compact
> > > than topic names, it’s still 16 bytes. In contrast, a 4-byte integer
> > > sequence id would give us 4 billion unique topics per cluster, which
> > seems
> > > like enough ;)
> > > 
> > > Considering the number of different times these topic identifiers are
> > sent
> > > over the wire or stored in memory, it seems like it might be worth the
> > > additional 4x space savings.
> > > 
> > > What do you think about this?
> > > 
> > > Thanks,
> > > John
> > > 
> > > On Fri, Sep 11, 2020, at 03:20, Tom Bentley wrote:
> > > > Hi Justine,
> > > > 
> > > > This looks like a very welcome improvement. Thanks!
> > > > 
> > > > Maybe I missed it, but the KIP doesn't seem to mention changing
> > > > DeleteTopicsRequest to identify the topic using an id. Maybe that's out
> > > of
> > > > scope, but DeleteTopicsRequest is not listed among the Future Work APIs
> > > > either.
> > > > 
> > > > Kind regards,
> > > > 
> > > > Tom
> > > > 
> > > > On Thu, Sep 10, 2020 at 3:59 PM Satish Duggana <
> > satish.duggana@gmail.com
> > > > wrote:
> > > > 
> > > > > Thanks Lucas/Justine for the nice KIP.
> > > > > 
> > > > > It has several benefits which also include simplifying the topic
> > > > > deletion process by controller and logs cleanup by brokers in corner
> > > > > cases.
> > > > > 
> > > > > Best,
> > > > > Satish.
> > > > > 
> > > > > On Wed, Sep 9, 2020 at 10:07 PM Justine Olshan <jolshan@confluent.io
> > > > > wrote:
> > > > > > Hello all, it's been almost a year! I've made some changes to this
> > > KIP
> > > > > and hope to continue the discussion.
> > > > > > One of the main changes I've added is now the metadata response
> > will
> > > > > include the topic ID (as Colin suggested). Clients can obtain the
> > > topicID
> > > > > of a given topic through a TopicDescription. The topicId will also be
> > > > > included with the UpdateMetadata request.
> > > > > > Let me know what you all think.
> > > > > > Thank you,
> > > > > > Justine
> > > > > > 
> > > > > > On 2019/09/13 16:38:26, "Colin McCabe" <cm...@apache.org> wrote:
> > > > > > > Hi Lucas,
> > > > > > > 
> > > > > > > Thanks for tackling this.  Topic IDs are a great idea, and this
> > is
> > > a
> > > > > really good writeup.
> > > > > > > For /brokers/topics/[topic], the schema version should be bumped
> > to
> > > > > version 3, rather than 2.  KIP-455 bumped the version of this znode
> > to
> > > 2
> > > > > already :)
> > > > > > > Given that we're going to be seeing these things as strings as
> > lot
> > > (in
> > > > > logs, in ZooKeeper, on the command-line, etc.), does it make sense to
> > > use
> > > > > base64 when converting them to strings?
> > > > > > > Here is an example of the hex representation:
> > > > > > > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> > > > > > > 
> > > > > > > And here is an example in base64.
> > > > > > > b8tRS7h4TJ2Vt43Dp85v2A
> > > > > > > 
> > > > > > > The base64 version saves 15 letters (to be fair, 4 of those were
> > > > > dashes that we could have elided in the hex representation.)
> > > > > > > Another thing to consider is that we should specify that the
> > > > > all-zeroes UUID is not a valid topic UUID.   We can't use null for
> > this
> > > > > because we can't pass a null UUID over the RPC protocol (there is no
> > > > > special pattern for null, nor do we want to waste space reserving
> > such
> > > a
> > > > > pattern.)
> > > > > > > Maybe I missed it, but did you describe "migration of... existing
> > > > > topic[s] without topic IDs" in detail in any section?  It seems like
> > > when
> > > > > the new controller becomes active, it should just generate random
> > > UUIDs for
> > > > > these, and write the random UUIDs back to ZooKeeper.  It would be
> > good
> > > to
> > > > > spell that out.  We should make it clear that this happens regardless
> > > of
> > > > > the inter-broker protocol version (it's a compatible change).
> > > > > > > "LeaderAndIsrRequests including an is_every_partition flag"
> > seems a
> > > > > bit wordy.  Can we just call these "full LeaderAndIsrRequests"?  Then
> > > the
> > > > > RPC field could be named "full".  Also, it would probably be better
> > > for the
> > > > > RPC field to be an enum of { UNSPECIFIED, INCREMENTAL, FULL }, so
> > that
> > > we
> > > > > can cleanly handle old versions (by treating them as UNSPECIFIED)
> > > > > > > In the LeaderAndIsrRequest section, you write "A final deletion
> > > event
> > > > > will be secheduled for X ms after the LeaderAndIsrRequest was first
> > > > > received..."  I guess the X was a placeholder that you intended to
> > > replace
> > > > > before posting? :)  In any case, this seems like the kind of thing
> > we'd
> > > > > want a configuration for.  Let's describe that configuration key
> > > somewhere
> > > > > in this KIP, including what its default value is.
> > > > > > > We should probably also log a bunch of messages at WARN level
> > when
> > > > > something is scheduled for deletion, as well.  (Maybe this was
> > > assumed, but
> > > > > it would be good to mention it).
> > > > > > > I feel like there are a few sections that should be moved to
> > > "rejected
> > > > > alternatives."  For example, in the DeleteTopics section, since we're
> > > not
> > > > > going to do option 1 or 2, these should be moved into "rejected
> > > > > alternatives,"  rather than appearing inline.  Another case is the
> > > "Should
> > > > > we remove topic name from the protocol where possible" section.  This
> > > is
> > > > > clearly discussing a design alternative that we're not proposing to
> > > > > implement: removing the topic name from those protocols.
> > > > > > > Is it really necessary to have a new /admin/delete_topics_by_id
> > > path
> > > > > in ZooKeeper?  It seems like we don't really need this.  Whenever
> > > there is
> > > > > a new controller, we'll send out full LeaderAndIsrRequests which will
> > > > > trigger the stale topics to be cleaned up.   The active controller
> > will
> > > > > also send the full LeaderAndIsrRequest to brokers that are just
> > > starting
> > > > > up.    So we don't really need this kind of two-phase commit (send
> > out
> > > > > StopReplicasRequest, get ACKs from all nodes, commit by removing
> > > > > /admin/delete_topics node) any more.
> > > > > > > You mention that FetchRequest will now include UUID to avoid
> > issues
> > > > > where requests are made to stale partitions.  However, adding a UUID
> > to
> > > > > MetadataRequest is listed as future work, out of scope for this KIP.
> > > How
> > > > > will the client learn what the topic UUID is, if the metadata
> > response
> > > > > doesn't include that information?  It seems like adding the UUID to
> > > > > MetadataResponse would be an improvement here that might not be too
> > > hard to
> > > > > make.
> > > > > > > best,
> > > > > > > Colin
> > > > > > > 
> > > > > > > 
> > > > > > > On Mon, Sep 9, 2019, at 17:48, Ryanne Dolan wrote:
> > > > > > > > Lucas, this would be great. I've run into issues with topics
> > > being
> > > > > > > > resurrected accidentally, since a client cannot easily
> > > distinguish
> > > > > between
> > > > > > > > a deleted topic and a new topic with the same name. I'd need
> > the
> > > ID
> > > > > > > > accessible from the client to solve that issue, but this is a
> > > good
> > > > > first
> > > > > > > > step.
> > > > > > > > 
> > > > > > > > Ryanne
> > > > > > > > 
> > > > > > > > On Wed, Sep 4, 2019 at 1:41 PM Lucas Bradstreet <
> > > lucas@confluent.io>
> > > > > wrote:
> > > > > > > > > Hi all,
> > > > > > > > > 
> > > > > > > > > I would like to kick off discussion of KIP-516, an
> > > implementation
> > > > > of topic
> > > > > > > > > IDs for Kafka. Topic IDs aim to solve topic uniqueness
> > > problems in
> > > > > Kafka,
> > > > > > > > > where referring to a topic by name alone is insufficient.
> > Such
> > > > > cases
> > > > > > > > > include when a topic has been deleted and recreated with the
> > > same
> > > > > name.
> > > > > > > > > Unique identifiers will help simplify and improve Kafka's
> > topic
> > > > > deletion
> > > > > > > > > process, as well as prevent cases where brokers may
> > incorrectly
> > > > > interact
> > > > > > > > > with stale versions of topics.
> > > > > > > > > 
> > > > > > > > > 
> > > > > > > > > 
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > > > > > > > > Looking forward to your thoughts.
> > > > > > > > > 
> > > > > > > > > Lucas
> > > > > > > > > 

Re: [DISCUSS] KIP-516: Topic Identifiers

[Justine Olshan <jo...@confluent.io>]

Hello all,
Thanks for continuing the discussion! I have a few responses to your points.

Tom: You are correct in that this KIP has not mentioned the
DeleteTopicsRequest. I think that this would be out of scope for now, but
may be something worth adding in the future.

John: We did consider sequence ids, but there are a few reasons to favor
UUIDs. There are several cases where topics from different clusters may
interact now and in the future. For example, Mirror Maker 2 may benefit
from being able to detect when a cluster being mirrored is deleted and
recreated and globally unique identifiers would make resolving issues
easier than sequence IDs which may collide between clusters. KIP-405
(tiered storage) will also benefit from globally unique IDs as shared
buckets may be used between clusters.

Globally unique IDs would also make functionality like moving topics
between disparate clusters easier in the future, simplify any future
implementations of backups and restores, and more. In general, unique IDs
would ensure that the source cluster topics do not conflict with the
destination cluster topics.

If we were to use sequence ids, we would need sufficiently large cluster
ids to be stored with the topic identifiers or we run the risk of
collisions. This will give up any advantage in compactness that sequence
numbers may bring. Given these advantages I think it makes sense to use
UUIDs.

Gokul: This is an interesting idea, but this is a breaking change. Out of
scope for now, but maybe worth discussing in the future.

Hope this explains some of the decisions,

Justine



On Fri, Sep 11, 2020 at 8:27 AM Gokul Ramanan Subramanian <
gokul2411s@gmail.com> wrote:

> Hi.
>
> Thanks for the KIP.
>
> Have you thought about whether it makes sense to support authorizing a
> principal for a topic ID rather than a topic name to achieve tighter
> security?
>
> Or is the topic ID fundamentally an internal detail similar to epochs used
> in a bunch of other places in Kafka?
>
> Thanks.
>
> On Fri, Sep 11, 2020 at 4:06 PM John Roesler <vv...@apache.org> wrote:
>
> > Hello Justine,
> >
> > Thanks for the KIP!
> >
> > I happen to have been confronted recently with the need to keep track of
> a
> > large number of topics as compactly as possible. I was going to come up
> > with some way to dictionary encode the topic names as integers, but this
> > seems much better!
> >
> > Apologies if this has been raised before, but I’m wondering about the
> > choice of UUID vs sequence number for the ids. Typically, I’ve seen UUIDs
> > in two situations:
> > 1. When processes need to generate non-colliding identifiers without
> > coordination.
> > 2. When the identifier needs to be “universally unique”; I.e., the
> > identifier needs to distinguish the entity from all other entities that
> > could ever exist. This is useful in cases where entities from all kinds
> of
> > systems get mixed together, such as when dumping logs from all processes
> in
> > a company into a common system.
> >
> > Maybe I’m being short-sighted, but it doesn’t seem like either really
> > applies here. It seems like the brokers could and would achieve consensus
> > when creating a topic anyway, which is all that’s required to generate
> > non-colliding sequence ids. For the second, as you mention, we could
> assign
> > a UUID to the cluster as a whole, which would render any resource scoped
> to
> > the broker universally unique as well.
> >
> > The reason I mention this is that, although a UUID is way more compact
> > than topic names, it’s still 16 bytes. In contrast, a 4-byte integer
> > sequence id would give us 4 billion unique topics per cluster, which
> seems
> > like enough ;)
> >
> > Considering the number of different times these topic identifiers are
> sent
> > over the wire or stored in memory, it seems like it might be worth the
> > additional 4x space savings.
> >
> > What do you think about this?
> >
> > Thanks,
> > John
> >
> > On Fri, Sep 11, 2020, at 03:20, Tom Bentley wrote:
> > > Hi Justine,
> > >
> > > This looks like a very welcome improvement. Thanks!
> > >
> > > Maybe I missed it, but the KIP doesn't seem to mention changing
> > > DeleteTopicsRequest to identify the topic using an id. Maybe that's out
> > of
> > > scope, but DeleteTopicsRequest is not listed among the Future Work APIs
> > > either.
> > >
> > > Kind regards,
> > >
> > > Tom
> > >
> > > On Thu, Sep 10, 2020 at 3:59 PM Satish Duggana <
> satish.duggana@gmail.com
> > >
> > > wrote:
> > >
> > > > Thanks Lucas/Justine for the nice KIP.
> > > >
> > > > It has several benefits which also include simplifying the topic
> > > > deletion process by controller and logs cleanup by brokers in corner
> > > > cases.
> > > >
> > > > Best,
> > > > Satish.
> > > >
> > > > On Wed, Sep 9, 2020 at 10:07 PM Justine Olshan <jolshan@confluent.io
> >
> > > > wrote:
> > > > >
> > > > > Hello all, it's been almost a year! I've made some changes to this
> > KIP
> > > > and hope to continue the discussion.
> > > > >
> > > > > One of the main changes I've added is now the metadata response
> will
> > > > include the topic ID (as Colin suggested). Clients can obtain the
> > topicID
> > > > of a given topic through a TopicDescription. The topicId will also be
> > > > included with the UpdateMetadata request.
> > > > >
> > > > > Let me know what you all think.
> > > > > Thank you,
> > > > > Justine
> > > > >
> > > > > On 2019/09/13 16:38:26, "Colin McCabe" <cm...@apache.org> wrote:
> > > > > > Hi Lucas,
> > > > > >
> > > > > > Thanks for tackling this.  Topic IDs are a great idea, and this
> is
> > a
> > > > really good writeup.
> > > > > >
> > > > > > For /brokers/topics/[topic], the schema version should be bumped
> to
> > > > version 3, rather than 2.  KIP-455 bumped the version of this znode
> to
> > 2
> > > > already :)
> > > > > >
> > > > > > Given that we're going to be seeing these things as strings as
> lot
> > (in
> > > > logs, in ZooKeeper, on the command-line, etc.), does it make sense to
> > use
> > > > base64 when converting them to strings?
> > > > > >
> > > > > > Here is an example of the hex representation:
> > > > > > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> > > > > >
> > > > > > And here is an example in base64.
> > > > > > b8tRS7h4TJ2Vt43Dp85v2A
> > > > > >
> > > > > > The base64 version saves 15 letters (to be fair, 4 of those were
> > > > dashes that we could have elided in the hex representation.)
> > > > > >
> > > > > > Another thing to consider is that we should specify that the
> > > > all-zeroes UUID is not a valid topic UUID.   We can't use null for
> this
> > > > because we can't pass a null UUID over the RPC protocol (there is no
> > > > special pattern for null, nor do we want to waste space reserving
> such
> > a
> > > > pattern.)
> > > > > >
> > > > > > Maybe I missed it, but did you describe "migration of... existing
> > > > topic[s] without topic IDs" in detail in any section?  It seems like
> > when
> > > > the new controller becomes active, it should just generate random
> > UUIDs for
> > > > these, and write the random UUIDs back to ZooKeeper.  It would be
> good
> > to
> > > > spell that out.  We should make it clear that this happens regardless
> > of
> > > > the inter-broker protocol version (it's a compatible change).
> > > > > >
> > > > > > "LeaderAndIsrRequests including an is_every_partition flag"
> seems a
> > > > bit wordy.  Can we just call these "full LeaderAndIsrRequests"?  Then
> > the
> > > > RPC field could be named "full".  Also, it would probably be better
> > for the
> > > > RPC field to be an enum of { UNSPECIFIED, INCREMENTAL, FULL }, so
> that
> > we
> > > > can cleanly handle old versions (by treating them as UNSPECIFIED)
> > > > > >
> > > > > > In the LeaderAndIsrRequest section, you write "A final deletion
> > event
> > > > will be secheduled for X ms after the LeaderAndIsrRequest was first
> > > > received..."  I guess the X was a placeholder that you intended to
> > replace
> > > > before posting? :)  In any case, this seems like the kind of thing
> we'd
> > > > want a configuration for.  Let's describe that configuration key
> > somewhere
> > > > in this KIP, including what its default value is.
> > > > > >
> > > > > > We should probably also log a bunch of messages at WARN level
> when
> > > > something is scheduled for deletion, as well.  (Maybe this was
> > assumed, but
> > > > it would be good to mention it).
> > > > > >
> > > > > > I feel like there are a few sections that should be moved to
> > "rejected
> > > > alternatives."  For example, in the DeleteTopics section, since we're
> > not
> > > > going to do option 1 or 2, these should be moved into "rejected
> > > > alternatives,"  rather than appearing inline.  Another case is the
> > "Should
> > > > we remove topic name from the protocol where possible" section.  This
> > is
> > > > clearly discussing a design alternative that we're not proposing to
> > > > implement: removing the topic name from those protocols.
> > > > > >
> > > > > > Is it really necessary to have a new /admin/delete_topics_by_id
> > path
> > > > in ZooKeeper?  It seems like we don't really need this.  Whenever
> > there is
> > > > a new controller, we'll send out full LeaderAndIsrRequests which will
> > > > trigger the stale topics to be cleaned up.   The active controller
> will
> > > > also send the full LeaderAndIsrRequest to brokers that are just
> > starting
> > > > up.    So we don't really need this kind of two-phase commit (send
> out
> > > > StopReplicasRequest, get ACKs from all nodes, commit by removing
> > > > /admin/delete_topics node) any more.
> > > > > >
> > > > > > You mention that FetchRequest will now include UUID to avoid
> issues
> > > > where requests are made to stale partitions.  However, adding a UUID
> to
> > > > MetadataRequest is listed as future work, out of scope for this KIP.
> > How
> > > > will the client learn what the topic UUID is, if the metadata
> response
> > > > doesn't include that information?  It seems like adding the UUID to
> > > > MetadataResponse would be an improvement here that might not be too
> > hard to
> > > > make.
> > > > > >
> > > > > > best,
> > > > > > Colin
> > > > > >
> > > > > >
> > > > > > On Mon, Sep 9, 2019, at 17:48, Ryanne Dolan wrote:
> > > > > > > Lucas, this would be great. I've run into issues with topics
> > being
> > > > > > > resurrected accidentally, since a client cannot easily
> > distinguish
> > > > between
> > > > > > > a deleted topic and a new topic with the same name. I'd need
> the
> > ID
> > > > > > > accessible from the client to solve that issue, but this is a
> > good
> > > > first
> > > > > > > step.
> > > > > > >
> > > > > > > Ryanne
> > > > > > >
> > > > > > > On Wed, Sep 4, 2019 at 1:41 PM Lucas Bradstreet <
> > lucas@confluent.io>
> > > > wrote:
> > > > > > >
> > > > > > > > Hi all,
> > > > > > > >
> > > > > > > > I would like to kick off discussion of KIP-516, an
> > implementation
> > > > of topic
> > > > > > > > IDs for Kafka. Topic IDs aim to solve topic uniqueness
> > problems in
> > > > Kafka,
> > > > > > > > where referring to a topic by name alone is insufficient.
> Such
> > > > cases
> > > > > > > > include when a topic has been deleted and recreated with the
> > same
> > > > name.
> > > > > > > >
> > > > > > > > Unique identifiers will help simplify and improve Kafka's
> topic
> > > > deletion
> > > > > > > > process, as well as prevent cases where brokers may
> incorrectly
> > > > interact
> > > > > > > > with stale versions of topics.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > > > > > > >
> > > > > > > > Looking forward to your thoughts.
> > > > > > > >
> > > > > > > > Lucas
> > > > > > > >
> > > > > > >
> > > > > >
> > > >
> > > >
> > >
> >
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Gokul Ramanan Subramanian <go...@gmail.com>]

Hi.

Thanks for the KIP.

Have you thought about whether it makes sense to support authorizing a
principal for a topic ID rather than a topic name to achieve tighter
security?

Or is the topic ID fundamentally an internal detail similar to epochs used
in a bunch of other places in Kafka?

Thanks.

On Fri, Sep 11, 2020 at 4:06 PM John Roesler <vv...@apache.org> wrote:

> Hello Justine,
>
> Thanks for the KIP!
>
> I happen to have been confronted recently with the need to keep track of a
> large number of topics as compactly as possible. I was going to come up
> with some way to dictionary encode the topic names as integers, but this
> seems much better!
>
> Apologies if this has been raised before, but I’m wondering about the
> choice of UUID vs sequence number for the ids. Typically, I’ve seen UUIDs
> in two situations:
> 1. When processes need to generate non-colliding identifiers without
> coordination.
> 2. When the identifier needs to be “universally unique”; I.e., the
> identifier needs to distinguish the entity from all other entities that
> could ever exist. This is useful in cases where entities from all kinds of
> systems get mixed together, such as when dumping logs from all processes in
> a company into a common system.
>
> Maybe I’m being short-sighted, but it doesn’t seem like either really
> applies here. It seems like the brokers could and would achieve consensus
> when creating a topic anyway, which is all that’s required to generate
> non-colliding sequence ids. For the second, as you mention, we could assign
> a UUID to the cluster as a whole, which would render any resource scoped to
> the broker universally unique as well.
>
> The reason I mention this is that, although a UUID is way more compact
> than topic names, it’s still 16 bytes. In contrast, a 4-byte integer
> sequence id would give us 4 billion unique topics per cluster, which seems
> like enough ;)
>
> Considering the number of different times these topic identifiers are sent
> over the wire or stored in memory, it seems like it might be worth the
> additional 4x space savings.
>
> What do you think about this?
>
> Thanks,
> John
>
> On Fri, Sep 11, 2020, at 03:20, Tom Bentley wrote:
> > Hi Justine,
> >
> > This looks like a very welcome improvement. Thanks!
> >
> > Maybe I missed it, but the KIP doesn't seem to mention changing
> > DeleteTopicsRequest to identify the topic using an id. Maybe that's out
> of
> > scope, but DeleteTopicsRequest is not listed among the Future Work APIs
> > either.
> >
> > Kind regards,
> >
> > Tom
> >
> > On Thu, Sep 10, 2020 at 3:59 PM Satish Duggana <satish.duggana@gmail.com
> >
> > wrote:
> >
> > > Thanks Lucas/Justine for the nice KIP.
> > >
> > > It has several benefits which also include simplifying the topic
> > > deletion process by controller and logs cleanup by brokers in corner
> > > cases.
> > >
> > > Best,
> > > Satish.
> > >
> > > On Wed, Sep 9, 2020 at 10:07 PM Justine Olshan <jo...@confluent.io>
> > > wrote:
> > > >
> > > > Hello all, it's been almost a year! I've made some changes to this
> KIP
> > > and hope to continue the discussion.
> > > >
> > > > One of the main changes I've added is now the metadata response will
> > > include the topic ID (as Colin suggested). Clients can obtain the
> topicID
> > > of a given topic through a TopicDescription. The topicId will also be
> > > included with the UpdateMetadata request.
> > > >
> > > > Let me know what you all think.
> > > > Thank you,
> > > > Justine
> > > >
> > > > On 2019/09/13 16:38:26, "Colin McCabe" <cm...@apache.org> wrote:
> > > > > Hi Lucas,
> > > > >
> > > > > Thanks for tackling this.  Topic IDs are a great idea, and this is
> a
> > > really good writeup.
> > > > >
> > > > > For /brokers/topics/[topic], the schema version should be bumped to
> > > version 3, rather than 2.  KIP-455 bumped the version of this znode to
> 2
> > > already :)
> > > > >
> > > > > Given that we're going to be seeing these things as strings as lot
> (in
> > > logs, in ZooKeeper, on the command-line, etc.), does it make sense to
> use
> > > base64 when converting them to strings?
> > > > >
> > > > > Here is an example of the hex representation:
> > > > > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> > > > >
> > > > > And here is an example in base64.
> > > > > b8tRS7h4TJ2Vt43Dp85v2A
> > > > >
> > > > > The base64 version saves 15 letters (to be fair, 4 of those were
> > > dashes that we could have elided in the hex representation.)
> > > > >
> > > > > Another thing to consider is that we should specify that the
> > > all-zeroes UUID is not a valid topic UUID.   We can't use null for this
> > > because we can't pass a null UUID over the RPC protocol (there is no
> > > special pattern for null, nor do we want to waste space reserving such
> a
> > > pattern.)
> > > > >
> > > > > Maybe I missed it, but did you describe "migration of... existing
> > > topic[s] without topic IDs" in detail in any section?  It seems like
> when
> > > the new controller becomes active, it should just generate random
> UUIDs for
> > > these, and write the random UUIDs back to ZooKeeper.  It would be good
> to
> > > spell that out.  We should make it clear that this happens regardless
> of
> > > the inter-broker protocol version (it's a compatible change).
> > > > >
> > > > > "LeaderAndIsrRequests including an is_every_partition flag" seems a
> > > bit wordy.  Can we just call these "full LeaderAndIsrRequests"?  Then
> the
> > > RPC field could be named "full".  Also, it would probably be better
> for the
> > > RPC field to be an enum of { UNSPECIFIED, INCREMENTAL, FULL }, so that
> we
> > > can cleanly handle old versions (by treating them as UNSPECIFIED)
> > > > >
> > > > > In the LeaderAndIsrRequest section, you write "A final deletion
> event
> > > will be secheduled for X ms after the LeaderAndIsrRequest was first
> > > received..."  I guess the X was a placeholder that you intended to
> replace
> > > before posting? :)  In any case, this seems like the kind of thing we'd
> > > want a configuration for.  Let's describe that configuration key
> somewhere
> > > in this KIP, including what its default value is.
> > > > >
> > > > > We should probably also log a bunch of messages at WARN level when
> > > something is scheduled for deletion, as well.  (Maybe this was
> assumed, but
> > > it would be good to mention it).
> > > > >
> > > > > I feel like there are a few sections that should be moved to
> "rejected
> > > alternatives."  For example, in the DeleteTopics section, since we're
> not
> > > going to do option 1 or 2, these should be moved into "rejected
> > > alternatives,"  rather than appearing inline.  Another case is the
> "Should
> > > we remove topic name from the protocol where possible" section.  This
> is
> > > clearly discussing a design alternative that we're not proposing to
> > > implement: removing the topic name from those protocols.
> > > > >
> > > > > Is it really necessary to have a new /admin/delete_topics_by_id
> path
> > > in ZooKeeper?  It seems like we don't really need this.  Whenever
> there is
> > > a new controller, we'll send out full LeaderAndIsrRequests which will
> > > trigger the stale topics to be cleaned up.   The active controller will
> > > also send the full LeaderAndIsrRequest to brokers that are just
> starting
> > > up.    So we don't really need this kind of two-phase commit (send out
> > > StopReplicasRequest, get ACKs from all nodes, commit by removing
> > > /admin/delete_topics node) any more.
> > > > >
> > > > > You mention that FetchRequest will now include UUID to avoid issues
> > > where requests are made to stale partitions.  However, adding a UUID to
> > > MetadataRequest is listed as future work, out of scope for this KIP.
> How
> > > will the client learn what the topic UUID is, if the metadata response
> > > doesn't include that information?  It seems like adding the UUID to
> > > MetadataResponse would be an improvement here that might not be too
> hard to
> > > make.
> > > > >
> > > > > best,
> > > > > Colin
> > > > >
> > > > >
> > > > > On Mon, Sep 9, 2019, at 17:48, Ryanne Dolan wrote:
> > > > > > Lucas, this would be great. I've run into issues with topics
> being
> > > > > > resurrected accidentally, since a client cannot easily
> distinguish
> > > between
> > > > > > a deleted topic and a new topic with the same name. I'd need the
> ID
> > > > > > accessible from the client to solve that issue, but this is a
> good
> > > first
> > > > > > step.
> > > > > >
> > > > > > Ryanne
> > > > > >
> > > > > > On Wed, Sep 4, 2019 at 1:41 PM Lucas Bradstreet <
> lucas@confluent.io>
> > > wrote:
> > > > > >
> > > > > > > Hi all,
> > > > > > >
> > > > > > > I would like to kick off discussion of KIP-516, an
> implementation
> > > of topic
> > > > > > > IDs for Kafka. Topic IDs aim to solve topic uniqueness
> problems in
> > > Kafka,
> > > > > > > where referring to a topic by name alone is insufficient. Such
> > > cases
> > > > > > > include when a topic has been deleted and recreated with the
> same
> > > name.
> > > > > > >
> > > > > > > Unique identifiers will help simplify and improve Kafka's topic
> > > deletion
> > > > > > > process, as well as prevent cases where brokers may incorrectly
> > > interact
> > > > > > > with stale versions of topics.
> > > > > > >
> > > > > > >
> > > > > > >
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > > > > > >
> > > > > > > Looking forward to your thoughts.
> > > > > > >
> > > > > > > Lucas
> > > > > > >
> > > > > >
> > > > >
> > >
> > >
> >
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[John Roesler <vv...@apache.org>]

Hello Justine,

Thanks for the KIP!

I happen to have been confronted recently with the need to keep track of a large number of topics as compactly as possible. I was going to come up with some way to dictionary encode the topic names as integers, but this seems much better!

Apologies if this has been raised before, but I’m wondering about the choice of UUID vs sequence number for the ids. Typically, I’ve seen UUIDs in two situations:
1. When processes need to generate non-colliding identifiers without coordination. 
2. When the identifier needs to be “universally unique”; I.e., the identifier needs to distinguish the entity from all other entities that could ever exist. This is useful in cases where entities from all kinds of systems get mixed together, such as when dumping logs from all processes in a company into a common system. 

Maybe I’m being short-sighted, but it doesn’t seem like either really applies here. It seems like the brokers could and would achieve consensus when creating a topic anyway, which is all that’s required to generate non-colliding sequence ids. For the second, as you mention, we could assign a UUID to the cluster as a whole, which would render any resource scoped to the broker universally unique as well. 

The reason I mention this is that, although a UUID is way more compact than topic names, it’s still 16 bytes. In contrast, a 4-byte integer sequence id would give us 4 billion unique topics per cluster, which seems like enough ;)

Considering the number of different times these topic identifiers are sent over the wire or stored in memory, it seems like it might be worth the additional 4x space savings. 

What do you think about this?

Thanks,
John

On Fri, Sep 11, 2020, at 03:20, Tom Bentley wrote:
> Hi Justine,
> 
> This looks like a very welcome improvement. Thanks!
> 
> Maybe I missed it, but the KIP doesn't seem to mention changing
> DeleteTopicsRequest to identify the topic using an id. Maybe that's out of
> scope, but DeleteTopicsRequest is not listed among the Future Work APIs
> either.
> 
> Kind regards,
> 
> Tom
> 
> On Thu, Sep 10, 2020 at 3:59 PM Satish Duggana <sa...@gmail.com>
> wrote:
> 
> > Thanks Lucas/Justine for the nice KIP.
> >
> > It has several benefits which also include simplifying the topic
> > deletion process by controller and logs cleanup by brokers in corner
> > cases.
> >
> > Best,
> > Satish.
> >
> > On Wed, Sep 9, 2020 at 10:07 PM Justine Olshan <jo...@confluent.io>
> > wrote:
> > >
> > > Hello all, it's been almost a year! I've made some changes to this KIP
> > and hope to continue the discussion.
> > >
> > > One of the main changes I've added is now the metadata response will
> > include the topic ID (as Colin suggested). Clients can obtain the topicID
> > of a given topic through a TopicDescription. The topicId will also be
> > included with the UpdateMetadata request.
> > >
> > > Let me know what you all think.
> > > Thank you,
> > > Justine
> > >
> > > On 2019/09/13 16:38:26, "Colin McCabe" <cm...@apache.org> wrote:
> > > > Hi Lucas,
> > > >
> > > > Thanks for tackling this.  Topic IDs are a great idea, and this is a
> > really good writeup.
> > > >
> > > > For /brokers/topics/[topic], the schema version should be bumped to
> > version 3, rather than 2.  KIP-455 bumped the version of this znode to 2
> > already :)
> > > >
> > > > Given that we're going to be seeing these things as strings as lot (in
> > logs, in ZooKeeper, on the command-line, etc.), does it make sense to use
> > base64 when converting them to strings?
> > > >
> > > > Here is an example of the hex representation:
> > > > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> > > >
> > > > And here is an example in base64.
> > > > b8tRS7h4TJ2Vt43Dp85v2A
> > > >
> > > > The base64 version saves 15 letters (to be fair, 4 of those were
> > dashes that we could have elided in the hex representation.)
> > > >
> > > > Another thing to consider is that we should specify that the
> > all-zeroes UUID is not a valid topic UUID.   We can't use null for this
> > because we can't pass a null UUID over the RPC protocol (there is no
> > special pattern for null, nor do we want to waste space reserving such a
> > pattern.)
> > > >
> > > > Maybe I missed it, but did you describe "migration of... existing
> > topic[s] without topic IDs" in detail in any section?  It seems like when
> > the new controller becomes active, it should just generate random UUIDs for
> > these, and write the random UUIDs back to ZooKeeper.  It would be good to
> > spell that out.  We should make it clear that this happens regardless of
> > the inter-broker protocol version (it's a compatible change).
> > > >
> > > > "LeaderAndIsrRequests including an is_every_partition flag" seems a
> > bit wordy.  Can we just call these "full LeaderAndIsrRequests"?  Then the
> > RPC field could be named "full".  Also, it would probably be better for the
> > RPC field to be an enum of { UNSPECIFIED, INCREMENTAL, FULL }, so that we
> > can cleanly handle old versions (by treating them as UNSPECIFIED)
> > > >
> > > > In the LeaderAndIsrRequest section, you write "A final deletion event
> > will be secheduled for X ms after the LeaderAndIsrRequest was first
> > received..."  I guess the X was a placeholder that you intended to replace
> > before posting? :)  In any case, this seems like the kind of thing we'd
> > want a configuration for.  Let's describe that configuration key somewhere
> > in this KIP, including what its default value is.
> > > >
> > > > We should probably also log a bunch of messages at WARN level when
> > something is scheduled for deletion, as well.  (Maybe this was assumed, but
> > it would be good to mention it).
> > > >
> > > > I feel like there are a few sections that should be moved to "rejected
> > alternatives."  For example, in the DeleteTopics section, since we're not
> > going to do option 1 or 2, these should be moved into "rejected
> > alternatives,"  rather than appearing inline.  Another case is the "Should
> > we remove topic name from the protocol where possible" section.  This is
> > clearly discussing a design alternative that we're not proposing to
> > implement: removing the topic name from those protocols.
> > > >
> > > > Is it really necessary to have a new /admin/delete_topics_by_id path
> > in ZooKeeper?  It seems like we don't really need this.  Whenever there is
> > a new controller, we'll send out full LeaderAndIsrRequests which will
> > trigger the stale topics to be cleaned up.   The active controller will
> > also send the full LeaderAndIsrRequest to brokers that are just starting
> > up.    So we don't really need this kind of two-phase commit (send out
> > StopReplicasRequest, get ACKs from all nodes, commit by removing
> > /admin/delete_topics node) any more.
> > > >
> > > > You mention that FetchRequest will now include UUID to avoid issues
> > where requests are made to stale partitions.  However, adding a UUID to
> > MetadataRequest is listed as future work, out of scope for this KIP.  How
> > will the client learn what the topic UUID is, if the metadata response
> > doesn't include that information?  It seems like adding the UUID to
> > MetadataResponse would be an improvement here that might not be too hard to
> > make.
> > > >
> > > > best,
> > > > Colin
> > > >
> > > >
> > > > On Mon, Sep 9, 2019, at 17:48, Ryanne Dolan wrote:
> > > > > Lucas, this would be great. I've run into issues with topics being
> > > > > resurrected accidentally, since a client cannot easily distinguish
> > between
> > > > > a deleted topic and a new topic with the same name. I'd need the ID
> > > > > accessible from the client to solve that issue, but this is a good
> > first
> > > > > step.
> > > > >
> > > > > Ryanne
> > > > >
> > > > > On Wed, Sep 4, 2019 at 1:41 PM Lucas Bradstreet <lu...@confluent.io>
> > wrote:
> > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > I would like to kick off discussion of KIP-516, an implementation
> > of topic
> > > > > > IDs for Kafka. Topic IDs aim to solve topic uniqueness problems in
> > Kafka,
> > > > > > where referring to a topic by name alone is insufficient. Such
> > cases
> > > > > > include when a topic has been deleted and recreated with the same
> > name.
> > > > > >
> > > > > > Unique identifiers will help simplify and improve Kafka's topic
> > deletion
> > > > > > process, as well as prevent cases where brokers may incorrectly
> > interact
> > > > > > with stale versions of topics.
> > > > > >
> > > > > >
> > > > > >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > > > > >
> > > > > > Looking forward to your thoughts.
> > > > > >
> > > > > > Lucas
> > > > > >
> > > > >
> > > >
> >
> >
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Tom Bentley <tb...@redhat.com>]

Hi Justine,

This looks like a very welcome improvement. Thanks!

Maybe I missed it, but the KIP doesn't seem to mention changing
DeleteTopicsRequest to identify the topic using an id. Maybe that's out of
scope, but DeleteTopicsRequest is not listed among the Future Work APIs
either.

Kind regards,

Tom

On Thu, Sep 10, 2020 at 3:59 PM Satish Duggana <sa...@gmail.com>
wrote:

> Thanks Lucas/Justine for the nice KIP.
>
> It has several benefits which also include simplifying the topic
> deletion process by controller and logs cleanup by brokers in corner
> cases.
>
> Best,
> Satish.
>
> On Wed, Sep 9, 2020 at 10:07 PM Justine Olshan <jo...@confluent.io>
> wrote:
> >
> > Hello all, it's been almost a year! I've made some changes to this KIP
> and hope to continue the discussion.
> >
> > One of the main changes I've added is now the metadata response will
> include the topic ID (as Colin suggested). Clients can obtain the topicID
> of a given topic through a TopicDescription. The topicId will also be
> included with the UpdateMetadata request.
> >
> > Let me know what you all think.
> > Thank you,
> > Justine
> >
> > On 2019/09/13 16:38:26, "Colin McCabe" <cm...@apache.org> wrote:
> > > Hi Lucas,
> > >
> > > Thanks for tackling this.  Topic IDs are a great idea, and this is a
> really good writeup.
> > >
> > > For /brokers/topics/[topic], the schema version should be bumped to
> version 3, rather than 2.  KIP-455 bumped the version of this znode to 2
> already :)
> > >
> > > Given that we're going to be seeing these things as strings as lot (in
> logs, in ZooKeeper, on the command-line, etc.), does it make sense to use
> base64 when converting them to strings?
> > >
> > > Here is an example of the hex representation:
> > > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> > >
> > > And here is an example in base64.
> > > b8tRS7h4TJ2Vt43Dp85v2A
> > >
> > > The base64 version saves 15 letters (to be fair, 4 of those were
> dashes that we could have elided in the hex representation.)
> > >
> > > Another thing to consider is that we should specify that the
> all-zeroes UUID is not a valid topic UUID.   We can't use null for this
> because we can't pass a null UUID over the RPC protocol (there is no
> special pattern for null, nor do we want to waste space reserving such a
> pattern.)
> > >
> > > Maybe I missed it, but did you describe "migration of... existing
> topic[s] without topic IDs" in detail in any section?  It seems like when
> the new controller becomes active, it should just generate random UUIDs for
> these, and write the random UUIDs back to ZooKeeper.  It would be good to
> spell that out.  We should make it clear that this happens regardless of
> the inter-broker protocol version (it's a compatible change).
> > >
> > > "LeaderAndIsrRequests including an is_every_partition flag" seems a
> bit wordy.  Can we just call these "full LeaderAndIsrRequests"?  Then the
> RPC field could be named "full".  Also, it would probably be better for the
> RPC field to be an enum of { UNSPECIFIED, INCREMENTAL, FULL }, so that we
> can cleanly handle old versions (by treating them as UNSPECIFIED)
> > >
> > > In the LeaderAndIsrRequest section, you write "A final deletion event
> will be secheduled for X ms after the LeaderAndIsrRequest was first
> received..."  I guess the X was a placeholder that you intended to replace
> before posting? :)  In any case, this seems like the kind of thing we'd
> want a configuration for.  Let's describe that configuration key somewhere
> in this KIP, including what its default value is.
> > >
> > > We should probably also log a bunch of messages at WARN level when
> something is scheduled for deletion, as well.  (Maybe this was assumed, but
> it would be good to mention it).
> > >
> > > I feel like there are a few sections that should be moved to "rejected
> alternatives."  For example, in the DeleteTopics section, since we're not
> going to do option 1 or 2, these should be moved into "rejected
> alternatives,"  rather than appearing inline.  Another case is the "Should
> we remove topic name from the protocol where possible" section.  This is
> clearly discussing a design alternative that we're not proposing to
> implement: removing the topic name from those protocols.
> > >
> > > Is it really necessary to have a new /admin/delete_topics_by_id path
> in ZooKeeper?  It seems like we don't really need this.  Whenever there is
> a new controller, we'll send out full LeaderAndIsrRequests which will
> trigger the stale topics to be cleaned up.   The active controller will
> also send the full LeaderAndIsrRequest to brokers that are just starting
> up.    So we don't really need this kind of two-phase commit (send out
> StopReplicasRequest, get ACKs from all nodes, commit by removing
> /admin/delete_topics node) any more.
> > >
> > > You mention that FetchRequest will now include UUID to avoid issues
> where requests are made to stale partitions.  However, adding a UUID to
> MetadataRequest is listed as future work, out of scope for this KIP.  How
> will the client learn what the topic UUID is, if the metadata response
> doesn't include that information?  It seems like adding the UUID to
> MetadataResponse would be an improvement here that might not be too hard to
> make.
> > >
> > > best,
> > > Colin
> > >
> > >
> > > On Mon, Sep 9, 2019, at 17:48, Ryanne Dolan wrote:
> > > > Lucas, this would be great. I've run into issues with topics being
> > > > resurrected accidentally, since a client cannot easily distinguish
> between
> > > > a deleted topic and a new topic with the same name. I'd need the ID
> > > > accessible from the client to solve that issue, but this is a good
> first
> > > > step.
> > > >
> > > > Ryanne
> > > >
> > > > On Wed, Sep 4, 2019 at 1:41 PM Lucas Bradstreet <lu...@confluent.io>
> wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > I would like to kick off discussion of KIP-516, an implementation
> of topic
> > > > > IDs for Kafka. Topic IDs aim to solve topic uniqueness problems in
> Kafka,
> > > > > where referring to a topic by name alone is insufficient. Such
> cases
> > > > > include when a topic has been deleted and recreated with the same
> name.
> > > > >
> > > > > Unique identifiers will help simplify and improve Kafka's topic
> deletion
> > > > > process, as well as prevent cases where brokers may incorrectly
> interact
> > > > > with stale versions of topics.
> > > > >
> > > > >
> > > > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > > > >
> > > > > Looking forward to your thoughts.
> > > > >
> > > > > Lucas
> > > > >
> > > >
> > >
>
>

Re: [DISCUSS] KIP-516: Topic Identifiers

[Satish Duggana <sa...@gmail.com>]

Thanks Lucas/Justine for the nice KIP.

It has several benefits which also include simplifying the topic
deletion process by controller and logs cleanup by brokers in corner
cases.

Best,
Satish.

On Wed, Sep 9, 2020 at 10:07 PM Justine Olshan <jo...@confluent.io> wrote:
>
> Hello all, it's been almost a year! I've made some changes to this KIP and hope to continue the discussion.
>
> One of the main changes I've added is now the metadata response will include the topic ID (as Colin suggested). Clients can obtain the topicID of a given topic through a TopicDescription. The topicId will also be included with the UpdateMetadata request.
>
> Let me know what you all think.
> Thank you,
> Justine
>
> On 2019/09/13 16:38:26, "Colin McCabe" <cm...@apache.org> wrote:
> > Hi Lucas,
> >
> > Thanks for tackling this.  Topic IDs are a great idea, and this is a really good writeup.
> >
> > For /brokers/topics/[topic], the schema version should be bumped to version 3, rather than 2.  KIP-455 bumped the version of this znode to 2 already :)
> >
> > Given that we're going to be seeing these things as strings as lot (in logs, in ZooKeeper, on the command-line, etc.), does it make sense to use base64 when converting them to strings?
> >
> > Here is an example of the hex representation:
> > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8
> >
> > And here is an example in base64.
> > b8tRS7h4TJ2Vt43Dp85v2A
> >
> > The base64 version saves 15 letters (to be fair, 4 of those were dashes that we could have elided in the hex representation.)
> >
> > Another thing to consider is that we should specify that the all-zeroes UUID is not a valid topic UUID.   We can't use null for this because we can't pass a null UUID over the RPC protocol (there is no special pattern for null, nor do we want to waste space reserving such a pattern.)
> >
> > Maybe I missed it, but did you describe "migration of... existing topic[s] without topic IDs" in detail in any section?  It seems like when the new controller becomes active, it should just generate random UUIDs for these, and write the random UUIDs back to ZooKeeper.  It would be good to spell that out.  We should make it clear that this happens regardless of the inter-broker protocol version (it's a compatible change).
> >
> > "LeaderAndIsrRequests including an is_every_partition flag" seems a bit wordy.  Can we just call these "full LeaderAndIsrRequests"?  Then the RPC field could be named "full".  Also, it would probably be better for the RPC field to be an enum of { UNSPECIFIED, INCREMENTAL, FULL }, so that we can cleanly handle old versions (by treating them as UNSPECIFIED)
> >
> > In the LeaderAndIsrRequest section, you write "A final deletion event will be secheduled for X ms after the LeaderAndIsrRequest was first received..."  I guess the X was a placeholder that you intended to replace before posting? :)  In any case, this seems like the kind of thing we'd want a configuration for.  Let's describe that configuration key somewhere in this KIP, including what its default value is.
> >
> > We should probably also log a bunch of messages at WARN level when something is scheduled for deletion, as well.  (Maybe this was assumed, but it would be good to mention it).
> >
> > I feel like there are a few sections that should be moved to "rejected alternatives."  For example, in the DeleteTopics section, since we're not going to do option 1 or 2, these should be moved into "rejected alternatives,"  rather than appearing inline.  Another case is the "Should we remove topic name from the protocol where possible" section.  This is clearly discussing a design alternative that we're not proposing to implement: removing the topic name from those protocols.
> >
> > Is it really necessary to have a new /admin/delete_topics_by_id path in ZooKeeper?  It seems like we don't really need this.  Whenever there is a new controller, we'll send out full LeaderAndIsrRequests which will trigger the stale topics to be cleaned up.   The active controller will also send the full LeaderAndIsrRequest to brokers that are just starting up.    So we don't really need this kind of two-phase commit (send out StopReplicasRequest, get ACKs from all nodes, commit by removing /admin/delete_topics node) any more.
> >
> > You mention that FetchRequest will now include UUID to avoid issues where requests are made to stale partitions.  However, adding a UUID to MetadataRequest is listed as future work, out of scope for this KIP.  How will the client learn what the topic UUID is, if the metadata response doesn't include that information?  It seems like adding the UUID to MetadataResponse would be an improvement here that might not be too hard to make.
> >
> > best,
> > Colin
> >
> >
> > On Mon, Sep 9, 2019, at 17:48, Ryanne Dolan wrote:
> > > Lucas, this would be great. I've run into issues with topics being
> > > resurrected accidentally, since a client cannot easily distinguish between
> > > a deleted topic and a new topic with the same name. I'd need the ID
> > > accessible from the client to solve that issue, but this is a good first
> > > step.
> > >
> > > Ryanne
> > >
> > > On Wed, Sep 4, 2019 at 1:41 PM Lucas Bradstreet <lu...@confluent.io> wrote:
> > >
> > > > Hi all,
> > > >
> > > > I would like to kick off discussion of KIP-516, an implementation of topic
> > > > IDs for Kafka. Topic IDs aim to solve topic uniqueness problems in Kafka,
> > > > where referring to a topic by name alone is insufficient. Such cases
> > > > include when a topic has been deleted and recreated with the same name.
> > > >
> > > > Unique identifiers will help simplify and improve Kafka's topic deletion
> > > > process, as well as prevent cases where brokers may incorrectly interact
> > > > with stale versions of topics.
> > > >
> > > >
> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers
> > > >
> > > > Looking forward to your thoughts.
> > > >
> > > > Lucas
> > > >
> > >
> >