You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Philip Nee (Jira)" <ji...@apache.org> on 2024/05/07 16:44:00 UTC
[jira] [Commented] (KAFKA-16687) Native memory leak by Unsafe_allocatememory in Kafka Clients 3.7.0
[ https://issues.apache.org/jira/browse/KAFKA-16687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844379#comment-17844379 ]
Philip Nee commented on KAFKA-16687:
------------------------------------
Hey [~fortherightous] - we also observed some (possible) memory leak. The symptom is particularly obvious when working with large number of partitions. Have you tried kafka client version 3.6 or earlier?
> Native memory leak by Unsafe_allocatememory in Kafka Clients 3.7.0
> --------------------------------------------------------------------
>
> Key: KAFKA-16687
> URL: https://issues.apache.org/jira/browse/KAFKA-16687
> Project: Kafka
> Issue Type: Bug
> Components: clients, consumer
> Affects Versions: 3.7.0
> Reporter: FTR
> Priority: Major
>
> I am building a Java Project which using Maven dependency Kafka-clients with 3.7.0 version.
> My Java application logic is to use Kafka Consumer to poll Kakfa broker topic continuously.
> I have configured my Java application with JVM options with -Xms8G -Xmx8G -XX:MaxMetaspaceSize=4G, and then run it.
> Also, there are 16G physical memory on my virtual machine.
> After my Java application running a long time, I have found that resident memory of the Java Process was being grown to more than 14G.
> In the end, the Java process ate Swap space.
> I checked it with jmap -heap pid, and found heap memory usage is Ok.
> Also with Native Memory Tracking [jcmd pid Native.memory detail.diff], I found that it's caused by [NMT Internal] memory, which created by Unsafe_allocatememory xxx.
> In my Java application, I don't use any NIO DirectByteBuffer to allocate memory.
> And I check it the Kafka-clients source code, it have codes with use "sun.misc.unsafe" to allocate memory. And MaxMetaspaceSize not work for it .
> Could you help to check it? How could I to stop this growing native memory to avoid my System hang?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)