[jira] [Created] (KAFKA-6584) Session expiration concurrent with ZooKeeper leadership failover may lead to broker registration failure

["Chris Thunes (JIRA)" <ji...@apache.org>]

Chris Thunes created KAFKA-6584:
-----------------------------------

             Summary: Session expiration concurrent with ZooKeeper leadership failover may lead to broker registration failure
                 Key: KAFKA-6584
                 URL: https://issues.apache.org/jira/browse/KAFKA-6584
             Project: Kafka
          Issue Type: Bug
    Affects Versions: 1.0.0
            Reporter: Chris Thunes


It seems that an edge case exists which can lead to sessions "un-expiring" during a ZooKeeper leadership failover. Additional details can be found in ZOOKEEPER-2985.

This leads to a NODEXISTS error when attempting to re-create the ephemeral brokers/ids/\{id} node in ZkUtils.registerBrokerInZk. We experienced this issue on each node within a 3-node Kafka cluster running 1.0.0. All three nodes continued running (producers and consumers appeared unaffected), but none of the nodes were considered online and partition leadership could be not re-assigned.

I took a quick look at trunk and I believe the issue is still present, but has moved into KafkaZkClient.checkedEphemeralCreate which will [raise an error|https://github.com/apache/kafka/blob/90e0bbe/core/src/main/scala/kafka/zk/KafkaZkClient.scala#L1512] when it finds that the broker/ids/\{id} node exists, but belongs to the old (believed expired) session.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)