[jira] [Issue Comment Deleted] (KAFKA-4447) Controller resigned but it also acts as a controller for a long time

["Json Tu (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-4447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Json Tu updated KAFKA-4447:
---------------------------
    Comment: was deleted

(was: [~skarface] thanks for your reply.
the latest release version 0.10.1.0，handleNewSession()'s implemention is as below,
  def handleNewSession() {
      info("ZK expired; shut down all controller components and try to re-elect")
      inLock(controllerContext.controllerLock) {
        onControllerResignation()
        controllerElector.elect
      }
    }

so deregisterIsrChangeNotificationListener() is also with the controllerlock. the lock is out of the onControllerResignation(). and this is a bug which was reported at https://issues.apache.org/jira/browse/KAFKA-4360.

my version is 0.9.0.1, so it is not bugfixed,  so we can image it as below.
1. ZK expired callback queue is fired. and he get controllerLock first. then start to execute onControllerResignation .
2. at that time IsrChangeNotificationListener、PartitionsReassignedListener and so on are all fired very compact. 
3. then the onControllerResignation() start to exectue  de-register listeners.

as we know，the zkclient callback thread is single thread，so the listener fired after zk expired only can be executed after handleNewSession(),
may be this is make sense.)

> Controller resigned but it also acts as a controller for a long time 
> ---------------------------------------------------------------------
>
>                 Key: KAFKA-4447
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4447
>             Project: Kafka
>          Issue Type: Improvement
>          Components: controller
>    Affects Versions: 0.9.0.0, 0.9.0.1, 0.10.0.0, 0.10.0.1
>         Environment: Linux Os
>            Reporter: Json Tu
>         Attachments: log.tar.gz
>
>
> We have a cluster with 10 nodes，and we execute following operation as below.
> 1.we execute some topic partition reassign from one node to other 9 nodes in the cluster, and which triggered controller.
> 2.controller invoke PartitionsReassignedListener's handleDataChange and read all partition reassign rules from the zk path, and executed all onPartitionReassignment for all partition that match conditions.
> 3.but the controller is expired from zk, after what some nodes of 9 nodes also expired from zk.
> 5.then controller invoke onControllerResignation to resigned as the controller.
> we found after the controller is resigned, it acts as controller for about 3 minutes, which can be found in my attachment.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)