[jira] [Updated] (KAFKA-21191) Rebuild the remote log aux state when a follower's leader epoch cache does not cover the remote log range

["Doyeon Kim (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-21191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Doyeon Kim updated KAFKA-21191:
-------------------------------
    Priority: Major  (was: Minor)

> Rebuild the remote log aux state when a follower's leader epoch cache does not cover the remote log range
> ---------------------------------------------------------------------------------------------------------
>
>                 Key: KAFKA-21191
>                 URL: https://issues.apache.org/jira/browse/KAFKA-21191
>             Project: Kafka
>          Issue Type: Improvement
>            Reporter: Doyeon Kim
>            Assignee: Doyeon Kim
>            Priority: Major
>
> A follower hit by KAFKA-17249 claims the remote log range [logStartOffset, localLogStartOffset) without leader epoch entries for it, and nothing rebuilds them because its fetch offset is already inside the leader's local log.
> When a replica becomes a follower and detects this, it should rebuild the remote log aux state from scratch. Resetting the fetch offset to logStartOffset is not enough: after a restart the follower's logStartOffset can lag the leader's, so the leader returns OFFSET_OUT_OF_RANGE and the fetcher resumes from the local log end offset, skipping the rebuild. Truncating the local log to logStartOffset first avoids that.
> If the latest remote segment was uploaded by a leader with the same gap, the rebuilt cache has the gap again, so the rebuild should not be retried in a loop.
> Discussion: https://github.com/apache/kafka/pull/23569#discussion_r4100224664



--
This message was sent by Atlassian Jira
(v8.20.10#820010)