You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flink.apache.org by Antti Kaikkonen <an...@protonmail.com> on 2025/05/06 18:44:40 UTC

Help with Memory Leak in DataStream API V2

Hi all!

I'm trying to use the new DataStream API V2, but I have noticed that the Java heap usage keeps increasing infinitely. Upon analyzing a heap dump, I found that a HashSet named 'keySet' is consuming 97% of the heap.

Here is the relevant field in the source code I traced it to: https://github.com/apache/flink/blob/b8f8d6e602897b7f275f1db36478f5e8f1d6604e/flink-datastream/src/main/java/org/apache/flink/datastream/impl/operators/BaseKeyedTwoInputNonBroadcastProcessOperator.java#L43
I tried to investigate the source code a little bit but could not find any logic for removing elements from the Set.

I have attached my implementation of the TwoInputNonBroadcastEventTimeStreamProcessFunction which apprears to trigger this behavior.

For context, I'm using the Docker image flink:2.0.0-scala_2.12-java11 with the RocksDB state backend.

Any advice or insights would be greatly appreciated!