You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "Ding Haifeng (JIRA)" <ji...@apache.org> on 2014/08/19 12:31:18 UTC

[jira] [Created] (KAFKA-1600) Controller failover not working correctly.

Ding Haifeng created KAFKA-1600:
-----------------------------------

             Summary: Controller failover not working correctly.
                 Key: KAFKA-1600
                 URL: https://issues.apache.org/jira/browse/KAFKA-1600
             Project: Kafka
          Issue Type: Bug
          Components: controller
    Affects Versions: 0.8.1
         Environment: Linux 3.2.0-4-amd64 #1 SMP Debian 3.2.46-1 x86_64 GNU/Linux
java version "1.7.0_03"
            Reporter: Ding Haifeng
            Assignee: Neha Narkhede


We are running a 10 node Kafka 0.8.1 cluster and experienced a failure as following. 
At some time, broker A stopped acting as controller any more. We see this by kafka.controller - KafkaController - ActiveControllerCount in JMX metrics jumped from 1 to 0.
In the meanwhile, broker A was still running and registering itself in the zookeeper /kafka/controller node. So no other brokers could be elected as new controller.
Since that the cluster was running without controller. Producers and consumers still worked. But functions requiring a controller such as new topic leader election and topic leader failover were not working any more.
A force restart of broker A could lead to a controller election and bring the cluster back to a correct state.
Here is our brief observations. I can provide more necessary informations if needed.





--
This message was sent by Atlassian JIRA
(v6.2#6252)