You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@kafka.apache.org by "Jun Rao (JIRA)" <ji...@apache.org> on 2017/05/16 16:00:06 UTC

[jira] [Commented] (KAFKA-977) Implement generation/term per leader to reconcile messages correctly

    [ https://issues.apache.org/jira/browse/KAFKA-977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16012637#comment-16012637 ] 

Jun Rao commented on KAFKA-977:
-------------------------------

Just to clarify. KAFKA-1211 added leader epoch in message set to prevent data losses. However, it didn't address the log divergency issue due to unclean leader election. The complexity is mostly on compacted topics. When a log is compacted, it's possible for all messages in a given leader epoch to be deleted. Therefore, it's a bit tricky to fully reconcile the log when an unclean leader election happens. This is less an issue since unclean leader election will be turned off by default from 0.11.0.

> Implement generation/term per leader to reconcile messages correctly
> --------------------------------------------------------------------
>
>                 Key: KAFKA-977
>                 URL: https://issues.apache.org/jira/browse/KAFKA-977
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Sriram Subramanian
>            Assignee: Sriram Subramanian
>             Fix For: 0.11.0.0
>
>
> During unclean leader election, the log messages can diverge and when the followers come back up Kafka does not reconcile correctly. To implement it correctly, we need to add a term/generation to each message and use that to reconcile.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)