[jira] [Commented] (KAFKA-1148) Delayed fetch/producer requests should be satisfied on a leader change

["Xing Huang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-1148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15042760#comment-15042760 ] 

Xing Huang commented on KAFKA-1148:
-----------------------------------

Recently, we have lost a set of messages. After checking the log, we think it's very likely caused by this issue.

We have a broker A, when it was the leader, it appended a MessageSet to local log, and waiting for acks, soon it became a follower, and it truncated local log, deleted that message set. But, soon after that, it became leader for that partition again, shrunk ISR and increased its HW, so the previous DelayedProduce be satisfied and the producer client received a successful produce response. But, actually, the messages was lost.

I think Kafka should check leader epoch before sending produce response, to make sure the leader was not changed since it append the produce data to log

> Delayed fetch/producer requests should be satisfied on a leader change
> ----------------------------------------------------------------------
>
>                 Key: KAFKA-1148
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1148
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Joel Koshy
>
> Somewhat related to KAFKA-1016.
> This would be an issue only if max.wait is set to a very high value. When a leader change occurs we should remove the delayed request from the purgatory - either satisfy with error/expire - whichever makes more sense.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)