You are viewing a plain text version of this content. The canonical link for it is here.

Posted to jira@kafka.apache.org by "Jun Rao (JIRA)" <ji...@apache.org> on 2018/11/27 21:49:00 UTC

[jira] [Commented] (KAFKA-7680) fetching a refilled chunk of log can cause log divergence

    [ https://issues.apache.org/jira/browse/KAFKA-7680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16701084#comment-16701084 ] 

Jun Rao commented on KAFKA-7680:
--------------------------------

One way to fix this is the following. We already include the expected leaderEpoch for each partition in the fetch response. So, the follower shouldn't see message sets with leaderEpoch higher than the expected one. If the follower does see such message sets, it's the result that the log is refilled in the middle of a transfer and the follower should just ignore this response and retry.

cc [~hachikuji]

> fetching a refilled chunk of log can cause log divergence
> ---------------------------------------------------------
>
>                 Key: KAFKA-7680
>                 URL: https://issues.apache.org/jira/browse/KAFKA-7680
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Jun Rao
>            Priority: Major
>
> We use FileRecords.writeTo to send a fetch response for a follower. A log could be truncated and refilled in the middle of the send process (due to leader change). Then it's possible for the follower to append some uncommitted messages followed by committed messages. Those uncommitted messages may never be removed, causing log divergence.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)