[jira] [Updated] (KAFKA-18139) AsyncKafkaConsumer's ConsumerNetworkThread loop requires further optimization

["Kirk True (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-18139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kirk True updated KAFKA-18139:
------------------------------
    Labels: async-kafka-consumer-performance consumer-threading-refactor performance  (was: consumer-threading-refactor performance)

> AsyncKafkaConsumer's ConsumerNetworkThread loop requires further optimization
> -----------------------------------------------------------------------------
>
>                 Key: KAFKA-18139
>                 URL: https://issues.apache.org/jira/browse/KAFKA-18139
>             Project: Kafka
>          Issue Type: Bug
>          Components: clients, consumer
>            Reporter: Philip Nee
>            Assignee: Kirk True
>            Priority: Major
>              Labels: async-kafka-consumer-performance, consumer-threading-refactor, performance
>             Fix For: 4.2.0
>
>
> We've noticed the AsyncKafkaConsumer polls the NetworkClient at a much higher cadence than the ClassicKafkaConsumer.  Using the `java-producer-consumer-demo.sh` example, the ClassicKafkaConsumer polls networkClients almost twice as much as the old consumer and also results in some side effects such as higher CPU usage (>10% more) and potentially affecting the request latency.  For example, using the example, we measured ClassicKafkaConsumer's average latency is 10.5ms and AsyncKafkaConsumer is ~12ms.
>  
> We suspect the added time delay is mostly caused by the inefficient NetworkClient poll.
>  
> During the demo run produce and consuming 100,000,000 records results in ~400k network client poll calls for the classic consumer and ~800k for the async consumer using the default client configuration provided by the example.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)