[jira] [Resolved] (KAFKA-2471) Replicas Order and Leader out of sync

["Manikumar (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-2471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Manikumar resolved KAFKA-2471.
------------------------------
    Resolution: Auto Closed

Closing inactive issue.  Please reopen if the issue still exists in newer versions.

> Replicas Order and Leader out of sync
> -------------------------------------
>
>                 Key: KAFKA-2471
>                 URL: https://issues.apache.org/jira/browse/KAFKA-2471
>             Project: Kafka
>          Issue Type: Bug
>          Components: replication
>    Affects Versions: 0.8.2.1
>            Reporter: Manish Sharma
>            Priority: Major
>
> Our 2 kafka brokers ( 1 & 5) were rebooted due to hypervisor going down and I think we encountered a similar
> issue that was discussed in thread "Problem with node after restart no partitions?".  The resulting JIRA is closed without conclusions or
> recovery steps. 
> Our Brokers 5 and 1 were also running zookeeper of our cluster (along with broker 2),
> we are running kafka version 0.8.2.1
> After doing a controlled restarts over all brokers a few times our cluster seems ok now.
> But there are a some topics that have replicas out of sync with Leaders.
> Partition 2 below has Leader 5 and replicas order should be 5,1 
> {code}
> Topic:2015-01-12        PartitionCount:3        ReplicationFactor:2     Configs:
>         Topic: 2015-01-12       Partition: 0    Leader: 4       Replicas: 4,3   Isr: 3,4
>         Topic: 2015-01-12       Partition: 1    Leader: 0       Replicas: 0,4   Isr: 0,4
>         Topic: 2015-01-12       Partition: 2    Leader: 5       Replicas: 1,5   Isr: 5
> {code}
> I tried reassigning partition 2 replicas to broker 5 (leader) and broker : 0
> Now partition reassignment is stuck for more than a day. 
> %) /usr/local/kafka/bin/kafka-reassign-partitions.sh --zookeeper kafka-trgt05:2182 --reassignment-json-file 2015-01-12_2.json --verify
> Status of partition reassignment:
> Reassignment of partition [2015-01-12,2] is still in progress
> And In zookeeper, reassign_partitions is empty..
> [zk: kafka-trgt05:2182(CONNECTED) 2] ls /admin/reassign_partitions
> []
> This seems like a bug being triggered, that leaves the cluster in unhealthy state.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)