[jira] [Commented] (KAFKA-14088) KafkaChannel memory leak

["Gao Fei (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-14088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17634689#comment-17634689 ] 

Gao Fei commented on KAFKA-14088:
---------------------------------

Is it the following question? Here only the version above 2.8 is involved, and the old versions are not involved？
h2. [CVE-2022-34917|https://nvd.nist.gov/vuln/detail/CVE-2022-34917] UNAUTHENTICATED CLIENTS MAY CAUSE OUTOFMEMORYERROR ON BROKERS

> KafkaChannel memory leak
> ------------------------
>
>                 Key: KAFKA-14088
>                 URL: https://issues.apache.org/jira/browse/KAFKA-14088
>             Project: Kafka
>          Issue Type: Bug
>          Components: network
>    Affects Versions: 2.2.1, 2.4.1, 2.5.1, 2.6.1, 2.7.1, 2.8.1, 3.1.1, 3.2.1
>         Environment: Current system environment:
> kafka version: 2.2.1
> openjdk(openj9): jdk1.8
> Heap memory: 6.4GB
> MaxDirectSize: 8GB
> Total number of topics: about 150+, each with about 3 partitions
>            Reporter: Gao Fei
>            Priority: Minor
>
> The kafka broker reports OutOfMemoryError: Java heap space and OutOfMemoryError: Direct buffer memory at the same time. Through the memory dump, it is found that the most occupied objects are KafkaChannel->NetworkReceive->HeapByteBuffer, there are about 4 such KafkaChannels, each about 1.5GB Around, and the total heap memory allocation is only 6.4GB.
> It's strange why a KafkaChannel occupies so much heap memory. Isn't each batch request slowly written to disk through the RequestHandler thread? Normally, this memory in KafkaChannel should be released continuously, but it is not released.
> I am curious why there is such a large HeapByteBuffer object in KafkaChannel? What does this object store? Shouldn't the socket communication here use a lot of direct memory? Instead, why a lot of heap memory is used, and why is it not released?
> The business data is not very large, the business data of each customer is different, and some customers have this OOM in the environment, and some customers with large business data do not appear OOM.
> java.lang.OutOfMemoryError: Direct buffer memory
>     at java.nio.Bits.reserveMemory(Bits.java:693)
>     at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
>     at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
>     at sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:174)
>     at sun.nio.ch.IOUtil.read(IOUtil.java:195)
>     at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
>     at org.apache.kafka.common.network.PlaintextTransportLayer.read(PlaintextTransportLayer.java:103)
>     at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:117)
>     at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:424)
>     at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:385)
>     at org.apache.kafka.common.network.Selector.attemptRead(Selector.java:651)
>     at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:572)
>     at org.apache.kafka.common.network.Selector.poll(Selector.java:483)
>     at kafka.network.Processor.poll(SocketServer.scala:863)
>     at kafka.network.Processor.run(SocketServer.scala:762)
>     at java.lang.Thread.run(Thread.java:745)
> java.lang.OutOfMemoryError: Java heap space
>     at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
>     at java.nio.ByteBuffer.allocate(ByteBuffer.java:335)
>     at org.apache.kafka.common.MemoryPool$1.tryAllocate(MemoryPool.java:30)
>     at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:112)
>     at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:424)
>     at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:385)
>     at org.apache.kafka.common.network.Selector.attemptRead(Selector.java:651)
>     at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:572)
>     at org.apache.kafka.common.network.Selector.poll(Selector.java:483)
>     at kafka.network.Processor.poll(SocketServer.scala:863)
>     at kafka.network.Processor.run(SocketServer.scala:762)
>     at java.lang.Thread.run(Thread.java:745)



--
This message was sent by Atlassian Jira
(v8.20.10#820010)