[jira] [Updated] (KAFKA-4385) producer is sending too many unnecessary meta data request if the meta data for a topic is not available and "auto.create.topics.enable" =false

["Jun Yao (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-4385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jun Yao updated KAFKA-4385:
---------------------------
    Summary: producer is sending too many unnecessary meta data request if the meta data for a topic is not available and "auto.create.topics.enable" =false  (was: producer is sending too many unnecessary meta data request if the meta data for a topic is not available)

> producer is sending too many unnecessary meta data request if the meta data for a topic is not available and "auto.create.topics.enable" =false
> -----------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: KAFKA-4385
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4385
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Jun Yao
>
> All current kafka-client producer implementation (<= 0.10.1.0),
> When sending a msg to a topic, it will first check if meta data for this topic is available or not, 
> when not available, it will set "metadata.requestUpdate()" and wait for meta data from brokers, 
> The thing is inside "org.apache.kafka.clients.Metadata.awaitUpdate()", it's already doing a "while (this.version <= lastVersion)" loop waiting for new version response, 
> So the loop inside "org.apache.kafka.clients.producer.KafkaProducer.waitOnMetadata() is not needed, 
> When "auto.create.topics.enable" is false, sending msgs to a non-exist topic will trigger too many meta requests, everytime a metadata response is returned, because it does not contain the metadata for the topic, it's going to try again until TimeoutException is thrown; 
> This is a waste and sometimes causes too much overhead when unexpected msgs are arrived. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)