You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Kamal Chandraprakash (Jira)" <ji...@apache.org> on 2023/10/27 04:35:00 UTC

[jira] [Commented] (KAFKA-15695) Local log start offset is not updated on the follower after rebuilding remote log auxiliary state

    [ https://issues.apache.org/jira/browse/KAFKA-15695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17780166#comment-17780166 ] 

Kamal Chandraprakash commented on KAFKA-15695:
----------------------------------------------

[~nikramakrishnan]
I remember fixing this issue in [#14328|https://github.com/apache/kafka/pull/14328] and covered it via integration test. Can you come up with a test to reproduce this issue? Thanks!

> Local log start offset is not updated on the follower after rebuilding remote log auxiliary state
> -------------------------------------------------------------------------------------------------
>
>                 Key: KAFKA-15695
>                 URL: https://issues.apache.org/jira/browse/KAFKA-15695
>             Project: Kafka
>          Issue Type: Bug
>          Components: replication, Tiered-Storage
>    Affects Versions: 3.6.0
>            Reporter: Nikhil Ramakrishnan
>            Assignee: Nikhil Ramakrishnan
>            Priority: Major
>              Labels: KIP-405, tiered-storage
>             Fix For: 3.7.0
>
>
> In 3.6, the local log start offset is not updated when reconstructing the auxiliary state of the remote log on a follower.
> The impact of this bug is significant because, if this follower becomes the leader before the local log start offset has had a change to be updated, reads from any offset between [wrong log start offset; actual log start offset] will be routed on the local storage, which does not contain the corresponding data. Consumer reads will in this case never be satisfied.
>  
> Reproduction case:
>  # Create a cluster with 2 brokers, broker 0 and broker 1.
>  # Create a topic topicA with RF=2, 1 partition (topicA-0) and 2 batches per segment, with broker 0 as the leader.
>  # Stop broker 1, and produce 3 records to topicA, such that segment 1 with the first two records are copied to remote and deleted from local storage.
>  # Start broker 1, let it catch up with broker 0.
>  # Stop broker 0 such that broker 1 is elected as the leader, and try to consume from the beginning of topicA-0.
> This consumer read will not be satisfied because the local log start offset is not updated on broker 1 when it builds the auxiliary state of the remote log segments.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)