[jira] [Commented] (KAFKA-1631) ReplicationFactor and under-replicated partitions incorrect during reassignment

["Neha Narkhede (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-1631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159388#comment-14159388 ] 

Neha Narkhede commented on KAFKA-1631:
--------------------------------------

The behavior of partition reassignment being old set -> old set + new set -> new set is just an implementation detail that users don't need to know and understand. However, there are 2 ways to report under replicated partitions today and this solution fixes one but not the other. For instance, if partitions being reassigned are not reported as under replicated through the topics tool (with this patch) but are reported by the broker's mbean, users would get confused. An ideal long term solution would be to define partition states as being one of the following - new, initializing, ready, migrating, under replicated (maybe more or less) and expose the partition's state as being one of these through the topic tool as well as JMX. It is possible to get away without having these states if there are maybe just 2 possible states that the partition lives in, but as the # of states increases, it is worth exposing those explicitly. One of these states is under-replicated and partitions being reassigned should belong to a separate "migrating" state, not "under replicated". 

> ReplicationFactor and under-replicated partitions incorrect during reassignment
> -------------------------------------------------------------------------------
>
>                 Key: KAFKA-1631
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1631
>             Project: Kafka
>          Issue Type: Bug
>    Affects Versions: 0.8.1.1
>            Reporter: Ryan Berdeen
>              Labels: newbie
>         Attachments: KAFKA-1631-v1.patch
>
>
> We have a topic with a replication factor of 3. We monitor UnderReplicatedPartitions as recommended by the documentation.
> During a partition reassignment, partitions being reassigned are reported as under-replicated. Running a describe shows:
> {code}
> Topic:activity-wal-1    PartitionCount:15       ReplicationFactor:5     Configs:
>         Topic: activity-wal-1   Partition: 0    Leader: 14      Replicas: 14,13,12,11,15        Isr: 14,12,11,13
>         Topic: activity-wal-1   Partition: 1    Leader: 14      Replicas: 15,14,11      Isr: 14,11
>         Topic: activity-wal-1   Partition: 2    Leader: 11      Replicas: 11,15,12      Isr: 12,11,15
> ...
> {code}
> It looks like the displayed replication factor, 5, is simply the number of replicas listed for the first partition, which includes both brokers in the current list and those onto which the partition is being reassigned. Partition 0 is also included in the list when using the `--under-replicated-partitions` option, even though it is replicated to more partitions than the true replication factor.
> During a reassignment, the under-replicated partitions metric is not usable, meaning that actual under-replicated partitions can go unnoticed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)