[jira] [Created] (KAFKA-14330) The Kafka Connection Is High During Service Startup

["flybird (Jira)" <ji...@apache.org>]

flybird created KAFKA-14330:
-------------------------------

             Summary: The Kafka Connection Is High During Service Startup
                 Key: KAFKA-14330
                 URL: https://issues.apache.org/jira/browse/KAFKA-14330
             Project: Kafka
          Issue Type: Improvement
          Components: consumer
    Affects Versions: 2.8.1
            Reporter: flybird


The Kafka Connection Is High During Service Startup

During service restart, the number of Kafka connections suddenly increases for a period of time. The actual number of process connections is 328. However, the number of consumer threads and producer threads of the process is changed by querying the number of consumer threads and producer threads. The number of consumer threads x 2 + producer threads = 283. After a while, the number of connections changes back to 283.

The TCP connection is created when the KafkaConsumer.poll method is invoked. There are three times to create a TCP connection inside the poll method.
1. Created when the findCoordinator request is initiated to determine the coordinator and obtain the cluster metadata.
2. When the coordinator is connected, enable normal group coordination operations.
3. Obtain the actual message when consuming data.
When a Type 3 TCP connection is successfully created, the consumer program discards the Type 1 TCP connection and then uses the Type 3 TCP connection instead when periodically requesting metadata. For a consumer program that has been running for a while, there are only the latter two types of TCP connections.

In summary, the producer creates one connection, and the consumer creates three connections. However, the first type of connection is closed after the third type of connection is created. In the end, there are only two connections.

The first type of connection analysis is a temporary connection, which causes a high number of connections. Why not disable the third type of connection immediately after it is created?



--
This message was sent by Atlassian Jira
(v8.20.10#820010)