[jira] [Commented] (KAFKA-10606) Auto create non-existent topics when fetching metadata for all topics

["huxihx (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-10606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213813#comment-17213813 ] 

huxihx commented on KAFKA-10606:
--------------------------------

I am wondering if we could change the default value for _AllowAutoTopicCreation_ to _false_ in MetadataRequest.json.

> Auto create non-existent topics when fetching metadata for all topics
> ---------------------------------------------------------------------
>
>                 Key: KAFKA-10606
>                 URL: https://issues.apache.org/jira/browse/KAFKA-10606
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Lincong Li
>            Priority: Major
>
> The "allow auto topic creation" flag is hardcoded to be true for the fetch-all-topic metadata request:
> https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/common/requests/MetadataRequest.java#L37
> In the below code, annotation claims that "*This never causes auto-creation*". It it NOT true and auto topic creation still gets triggered under some circumstances. So, this is a bug that needs to be fixed.
> https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/common/requests/MetadataRequest.java#L68
> For example, the bug could be manifested in the below situation:
> A topic T is being deleted and a request to fetch metadata for all topics gets sent to one broker. The broker reads names of all topics from its metadata cache (shown below).
> https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/server/KafkaApis.scala#L1196
> Then the broker authorizes all topics and makes sure that they are allowed to be described. Then the broker tries to get metadata for every authorized topic by reading the metadata cache again, once for every topic (show below).
> https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/server/KafkaApis.scala#L1240
> However, the metadata cache could have been updated while the broker was authorizing all topics and topic T and its metadata no longer exist in the cache since the topic got deleted and metadata update requests eventually got propagated from the controller to all brokers. So, at this point, when the broker tries to get metadata for topic T from its cache, it realizes that it does not exist and the broker tries to "auto create" topic T since the allow-auto-topic-creation flag was set to true in all the fetch-all-topic metadata requests.
> I think this bug exists since "*metadataRequest.allowAutoTopicCreation*" was introduced.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)