[jira] [Created] (KAFKA-2975) The newtorkClient should request a metadata update after it gets an error in the handleResponse()

["Mayuresh Gharat (JIRA)" <ji...@apache.org>]

Mayuresh Gharat created KAFKA-2975:
--------------------------------------

             Summary: The newtorkClient should request a metadata update after it gets an error in the handleResponse()
                 Key: KAFKA-2975
                 URL: https://issues.apache.org/jira/browse/KAFKA-2975
             Project: Kafka
          Issue Type: Bug
          Components: clients
            Reporter: Mayuresh Gharat
            Assignee: Mayuresh Gharat


Currently in data pipeline, 
1) Lets say Mirror Maker requestTimeout is set to 2 min and metadataExpiry is set to 5 min
2) We delete a topic, the Mirror Maker get UNKNOWN_TOPIC_PARTITION and tries torefresh its Metadata.
3) It gets LeaderNotAvailableException, may be because the topic is not created yet.
4) Now its metadata does not have any information about that topic.
5) It will wait for 5 min to do the next refresh.
6) In the mean tie the batches sitting in the accumulator will expire and the mirror makers die.

To overcome this we need to refresh the metadata after 3) before the timeout kicks in.

Well there is an alternative solution to have the metadataExpiry set to be less then requestTimeout, but this will mean we make more metadataRequest over the wire in normal scenario as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)