You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "wilmerdooley (via GitHub)" <gi...@apache.org> on 2026/06/13 02:04:33 UTC

[PR] KAFKA-19902: fix(consumer): use latest leader epoch when committing offsets [kafka]

wilmerdooley opened a new pull request, #22558:
URL: https://github.com/apache/kafka/pull/22558

   When the partition leader changes, the consumer's `FetchPosition.currentLeader` can become stale because it is inherited from the previous position and only refreshed when leader-change errors are observed. `SubscriptionState.allConsumed()` then committed `position.offsetEpoch` (the epoch of the last consumed batch) instead, which can reference log segments that have since been deleted by retention, causing `OFFSET_OUT_OF_RANGE` on the next session and an `auto.offset.reset` jump.
   
   This change keeps `currentLeader` fresh by looking it up in metadata every time the fetch position is advanced in `FetchCollector`, and makes `SubscriptionState.allConsumed()` use the maximum of `offsetEpoch` and `currentLeader.epoch` (falling back to whichever is present) so that the committed offset epoch matches the most recent leader epoch known to the client.
   
   ---
   
   Delete this text and replace it with a detailed description of your change. The
   PR title and body will become the squashed commit message.
   
   If you would like to tag individuals, add some commentary, upload images, or
   include other supplemental information that should not be part of the eventual
   commit message, please use a separate comment.
   
   If applicable, please include a summary of the testing strategy (including
   rationale) for the proposed change. Unit and/or integration tests are expected
   for any behavior change and system tests should be considered for larger
   changes.
   
   ### Testing strategy
   
   - Updated `SubscriptionStateTest.testAllConsumedUsesLatestAvailableLeaderEpoch` covers the new `allConsumed()` epoch selection across the four combinations of present/absent `offsetEpoch` and `currentLeader.epoch`.
   - Added `FetchCollectorTest.testPositionUpdateUsesCurrentLeaderEpoch` to verify that advancing the fetch position refreshes `currentLeader.epoch` from the metadata cache.
   
   - [x] Unit tests added/updated
   
   JIRA: https://issues.apache.org/jira/browse/KAFKA-19902


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19902: fix(consumer): use latest leader epoch when committing offsets [kafka]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22558:
URL: https://github.com/apache/kafka/pull/22558#issuecomment-4756475013

   A label of 'needs-attention' was automatically added to this PR in order to raise the
   attention of the committers. Once this issue has been triaged, the `triage` label
   should be removed to prevent this automation from happening again.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19902: fix(consumer): use latest leader epoch when committing offsets [kafka]

Posted by "github-actions[bot] (via GitHub) via jira" <ji...@kafka.apache.org>.
github-actions[bot] commented on PR #22558:
URL: https://github.com/apache/kafka/pull/22558#issuecomment-6538278508

   A label of 'needs-attention' was automatically added to this PR in order to raise the
   attention of the committers. Once this issue has been triaged, the `triage` label
   should be removed to prevent this automation from happening again.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-19902: fix(consumer): use latest leader epoch when committing offsets [kafka]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22558:
URL: https://github.com/apache/kafka/pull/22558#issuecomment-6487203472

   This PR is being marked as stale since it has not had any activity in 90 days. If you
   would like to keep this PR alive, please leave a comment asking for a review. If the PR has 
   merge conflicts, update it with the latest from the base branch.
   <p>
   If you are having difficulty finding a reviewer, please reach out on the 
   [mailing list](https://kafka.apache.org/contact).
   <p>
   If this PR is no longer valid or desired, please feel free to close it. If no activity
   occurs in the next 30 days, it will be automatically closed.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org