[jira] [Commented] (KAFKA-7738) Track partition leader epochs in client metadata

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-7738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16723351#comment-16723351 ] 

ASF GitHub Bot commented on KAFKA-7738:
---------------------------------------

mumrah opened a new pull request #6045: KAFKA-7738 Track leader epochs in Metadata
URL: https://github.com/apache/kafka/pull/6045
 
 
   Track the last seen partition epoch in the Metadata class. When handling metadata updates, check that the partition info being received is for the last seen epoch or a newer one. This prevents stale metadata from being loaded into the client.
   
   ### Committer Checklist (excluded from commit message)
   - [ ] Verify design and implementation 
   - [ ] Verify test coverage and CI build status
   - [ ] Verify documentation (including upgrade notes)
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> Track partition leader epochs in client metadata
> ------------------------------------------------
>
>                 Key: KAFKA-7738
>                 URL: https://issues.apache.org/jira/browse/KAFKA-7738
>             Project: Kafka
>          Issue Type: Improvement
>            Reporter: David Arthur
>            Assignee: David Arthur
>            Priority: Minor
>
> The Metadata API now exposes the current leader epoch of each partition. We can leverage this information to be smarter in how we fetch metadata. For example, when we receive a NOT_LEADER_FOR_PARTITION, we know to look for an epoch bump before resuming whatever operation we were trying to do. Additionally, this gives us a way to detect stale metadata.
> This requires a little more sophistication in how we track metadata in the client. For example, we may be not be able to assume metadata updates are monotonic. In other words, they may mix stale metadata for one partition and fresh metadata for another. I am not sure we have any strong guarantees on the order in which metadata updates are seen on each broker.
> It may be helpful in this context to control the topics that we fetch metadata for at a finer granularity. Potentially we could even extend the Metadata API to specify a subset of the partitions that the client is interested in.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)