[jira] [Created] (KAFKA-12457) Implications of KIP-516 for quorum controller

["Jason Gustafson (Jira)" <ji...@apache.org>]

Jason Gustafson created KAFKA-12457:
---------------------------------------

             Summary: Implications of KIP-516 for quorum controller
                 Key: KAFKA-12457
                 URL: https://issues.apache.org/jira/browse/KAFKA-12457
             Project: Kafka
          Issue Type: Improvement
            Reporter: Jason Gustafson


KIP-516 introduces topic IDs to Kafka. We are in the process of updating many of the protocols to support them. In most cases, we are dropping the topic name entirely from new API versions. I think there are two open questions for KIP-500 in regard to this:

1. Can we assume topic ID existence in KIP-500? 

I think the answer here is yes, and the existing code already assumes it. The nice thing is that KIP-516 brings with it the logic to create topic IDs for existing topics. We can rely on this ability in the bridge release to ensure that all topics have topic IDs. And we can add it to pre-upgrade validations.

2. What topic ID should be used for `@metadata`? 

There are basically two options for this: either use a sentinel topic ID or let the controller generate a new one and write it into a `TopicRecord` when the cluster first initializes. If we assume long term that we won't be able to use topic names in the inter-broker protocol, then a sentinel might really be a necessity since brokers would need to know the topic ID before they can send fetches. In other words, if we generate a unique ID, then we probably still need some sentinel so that followers can fetch the initial `TopicRecord` which contains the ID.






--
This message was sent by Atlassian Jira
(v8.3.4#803005)