[jira] [Created] (KAFKA-20915) Unneeded app thread wakeup when a fetch already in flight and nothing has changed in AsyncConsumer

["Lianet Magrans (Jira)" <ji...@apache.org>]

Lianet Magrans created KAFKA-20915:
--------------------------------------

             Summary: Unneeded app thread wakeup when a fetch already in flight and nothing has changed in AsyncConsumer
                 Key: KAFKA-20915
                 URL: https://issues.apache.org/jira/browse/KAFKA-20915
             Project: Kafka
          Issue Type: Bug
          Components: clients, consumer
            Reporter: Lianet Magrans
            Assignee: Lianet Magrans
             Fix For: 4.4.0


On the async consumer, the fetch req manager has logic to wake up the fetch buffer if it doesn't generate any fetch request to send. 

[https://github.com/apache/kafka/blob/dacff9c85a29806fcafe617feb2db0c9677e6de7/clients/src/main/java/org/apache/kafka/clients/consumer/internals/FetchRequestManager.java#L149-L151] 

That wake up is needed in some cases (to unblock app thread if the case was no fetch req generated because partitions were paused, not subscribed etc)., but the wakeup is useless and harmful if the case was that no requests was generated because they were already in-flight.
 * Useless because nothing has changed yet regarding data (it is the fetch response for the inflights that should wake up the app thread)
 * harmful because it leads to a spin (high CPU) between the app thread and the background (background wakes ups app thread when nothing really changed regarding data, we're just waiting for in-flight requests that will themselves wake up the buffer then complete).

This wakeup behaviour was introduced in 4.2, but it's impact on CPU due to unneeded app thread - background thread spin was revealed with a recent correctness fix https://issues.apache.org/jira/browse/KAFKA-20780



--
This message was sent by Atlassian Jira
(v8.20.10#820010)