[jira] [Created] (KAFKA-7954) When broker IP addresses change client never resolves new addresses to fetch metadata

["Robert Pofuk (JIRA)" <ji...@apache.org>]

Robert Pofuk created KAFKA-7954:
-----------------------------------

             Summary: When broker IP addresses change client never resolves new addresses to fetch metadata
                 Key: KAFKA-7954
                 URL: https://issues.apache.org/jira/browse/KAFKA-7954
             Project: Kafka
          Issue Type: Bug
          Components: clients
    Affects Versions: 2.1.0
            Reporter: Robert Pofuk


I'm running Kafka on AWS autoscaling group. Currently we are having infrastructure immutable and we have no fixed IP addresses. 

When we recreate out Kafka cluster all brokers get new IP addresses, even in case when all nodes die horrible death simultaneously. 

Looking at the code IP addresses are resolved and added ClusterConnectionStates class line 119: 

 

nodeState.put(id, new NodeConnectionState(ConnectionState.CONNECTING, now,
 this.reconnectBackoffInitMs, ClientUtils.resolve(host, clientDnsLookup)));

At this point ClientUtils.resolve resolves IP addreses. If brokers receive new IP addresses each subsequent metadata fetch will fail because none of the nodes will be addressable by IP addresses that where resolved on previous metadata fetch. 

 

Since addresses list will never be evicted old IP addresses will stay there forever. 

 

 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)