[jira] [Created] (KAFKA-851) Broken handling of leader and isr request leads to incorrect high watermark checkpoint file

["Neha Narkhede (JIRA)" <ji...@apache.org>]

Neha Narkhede created KAFKA-851:
-----------------------------------

             Summary: Broken handling of leader and isr request leads to incorrect high watermark checkpoint file
                 Key: KAFKA-851
                 URL: https://issues.apache.org/jira/browse/KAFKA-851
             Project: Kafka
          Issue Type: Bug
          Components: replication
    Affects Versions: 0.8
            Reporter: Neha Narkhede
            Assignee: Neha Narkhede
            Priority: Blocker


The broker depends on receiving a list of *all* partitions from the controller on startup. It uses this information to create a list of partitions that will get check pointed to the high watermark checkpoint file. However, during a make follower operation, it adds a partition to the high watermark checkpoint list only if its leader is alive. Due to this, even if the controller sends a full list of partitions to the broker, replica manager filters it to keep only those partitions whose leader is alive. This leads to the high watermark value for the rest of those partitions to reset to 0. Which, in turn, leads to the follower to fetch from the beginning of leader's log instead of min(log end offset, high watermark). The effect of this is very long lag on the replica fetchers leading to high under replicated partition count

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira