[jira] [Created] (KAFKA-15141) High CPU usage with log4j2

["Gaurav Narula (Jira)" <ji...@apache.org>]

Gaurav Narula created KAFKA-15141:
-------------------------------------

             Summary: High CPU usage with log4j2
                 Key: KAFKA-15141
                 URL: https://issues.apache.org/jira/browse/KAFKA-15141
             Project: Kafka
          Issue Type: Improvement
          Components: core
    Affects Versions: 3.4.1, 3.5.0, 3.3.2
            Reporter: Gaurav Narula
            Assignee: Gaurav Narula


Kafka brokers make use of the [Logging trait|https://github.com/apache/kafka/blob/1f4cbc5d53259031123b6e9e6bb9a5bbe1e084e8/core/src/main/scala/kafka/utils/Logging.scala#L41] which instantiates a Logger object for every instantiation of the class using the trait by default.

When using log4j2 as the logging implementation, the instantiation of a Logger object requires a stack traversal [[1]|https://github.com/apache/logging-log4j2/blob/2.x/log4j-api/src/main/java/org/apache/logging/log4j/spi/AbstractLoggerAdapter.java#L121] and [[2]|https://github.com/apache/logging-log4j2/blob/83bba1bc322e80e7e95edbebc2383f2724dbe0de/log4j-slf4j-impl/src/main/java/org/apache/logging/slf4j/Log4jLoggerFactory.java#L54].

While LOG4J2-2940 ensures stack is not traversed unless required, the default {{{}ContextSelector{}}}, [ClassLoaderContextSelector|https://logging.apache.org/log4j/2.x/log4j-core/apidocs/org/apache/logging/log4j/core/selector/ClassLoaderContextSelector.html] causes a stack traversal.

These stack traversals are frequent and quite CPU intensive and profiling suggests they consume ~5% CPU time (of the CPU stacks we have profiled on a sample of clusters). While log4j2 users can potentially avoid this by changing the context selector in their configuration, it is easy to overlook and the default configurations results in high CPU usage inadvertently.

An easy fix would be to instantiate the loggers statically for some commonly instantiated classes in Kafka which make use of the Logging trait.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)