[jira] [Created] (KAFKA-7987) a broker's ZK session may die on transient auth failure

["Jun Rao (JIRA)" <ji...@apache.org>]

Jun Rao created KAFKA-7987:
------------------------------

             Summary: a broker's ZK session may die on transient auth failure
                 Key: KAFKA-7987
                 URL: https://issues.apache.org/jira/browse/KAFKA-7987
             Project: Kafka
          Issue Type: Improvement
            Reporter: Jun Rao


After a transient network issue, we saw the following log in a broker.
{code:java}
[23:37:02,102] ERROR SASL authentication with Zookeeper Quorum member failed: javax.security.sasl.SaslException: An error: (java.security.PrivilegedActionException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7))]) occurred when evaluating Zookeeper Quorum Member's received SASL token. Zookeeper Client will go to AUTH_FAILED state. (org.apache.zookeeper.ClientCnxn)
[23:37:02,102] ERROR [ZooKeeperClient] Auth failed. (kafka.zookeeper.ZooKeeperClient)
{code}
The network issue prevented the broker from communicating to ZK. The broker's ZK session then expired, but the broker didn't know that yet since it couldn't establish a connection to ZK. When the network was back, the broker tried to establish a connection to ZK, but failed due to auth failure (likely due to a transient KDC issue). The current logic just ignores the auth failure without trying to create a new ZK session. Then the broker will be permanently in a state that it's alive, but not registered in ZK.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)