[jira] [Commented] (KAFKA-977) Implement generation/term per leader to reconcile messages correctly

["Sriram Subramanian (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14142990#comment-14142990 ] 

Sriram Subramanian commented on KAFKA-977:
------------------------------------------

I would like to bring this issue to discussion again. Kafka is used a lot more now for use cases other than just moving data from point A to point B. For example, consider the case where Kafka acts as the log and materialized views are created by consuming these logs. In such scenarios, it is important that the logs are consistent and do not diverge even under unclean leader elections (Replaying these replicas should create the same view). Having a generation/term is essential for log replication and it would be great for Kafka to have the same guarantees as other log replication protocols.  I would be happy to give more detailed examples for this but would want to know if we think this is an issue to address soon.

> Implement generation/term per leader to reconcile messages correctly
> --------------------------------------------------------------------
>
>                 Key: KAFKA-977
>                 URL: https://issues.apache.org/jira/browse/KAFKA-977
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Sriram Subramanian
>            Assignee: Sriram Subramanian
>
> During unclean leader election, the log messages can diverge and when the followers come back up Kafka does not reconcile correctly. To implement it correctly, we need to add a term/generation to each message and use that to reconcile.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)