[jira] [Comment Edited] (KAFKA-12457) Implications of KIP-516 for quorum controller

["Justine Olshan (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-12457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17299936#comment-17299936 ] 

Justine Olshan edited comment on KAFKA-12457 at 3/11/21, 11:43 PM:
-------------------------------------------------------------------

Thanks for creating this JIRA. There is some discussion on sentinel topic IDs in KIP-516 as well, but it has been a while. I think the original plan was to use the sentinel ID as a sort of bootstrap for Vote requests and then to generate a unique topic ID for the metadata topic. Very much open to discussion and potential changes though.
 [https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-CompatibilitywithKIP-500]

 


was (Author: jolshan):
Thanks for creating this JIRA. There is some discussion on sentinel topic IDs in KIP-516 as well, but it has been a while since we discussed this. I think the original plan was to use the sentinel ID as a sort of bootstrap for Vote requests and then to generate a unique topic ID for the metadata topic. Very much open to discussion and potential changes though.
[https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-CompatibilitywithKIP-500]

 

> Implications of KIP-516 for quorum controller
> ---------------------------------------------
>
>                 Key: KAFKA-12457
>                 URL: https://issues.apache.org/jira/browse/KAFKA-12457
>             Project: Kafka
>          Issue Type: Improvement
>            Reporter: Jason Gustafson
>            Priority: Major
>              Labels: kip-500
>
> KIP-516 introduces topic IDs to Kafka. We are in the process of updating many of the protocols to support them. In most cases, we are dropping the topic name entirely from new API versions. I think there are two open questions for KIP-500 in regard to this:
> 1. Can we assume topic ID existence in KIP-500? 
> I think the answer here is yes, and the existing code already assumes it. The nice thing is that KIP-516 brings with it the logic to create topic IDs for existing topics. We can rely on this ability in the bridge release to ensure that all topics have topic IDs. And we can add it to pre-upgrade validations.
> 2. What topic ID should be used for `@metadata`? 
> There are basically two options for this: either use a sentinel topic ID or let the controller generate a new one and write it into a `TopicRecord` when the cluster first initializes. If we assume long term that we won't be able to use topic names in the inter-broker protocol, then a sentinel might really be a necessity since brokers would need to know the topic ID before they can send fetches. In other words, if we generate a unique ID, then we probably still need some sentinel so that followers can fetch the initial `TopicRecord` which contains the ID.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)