[jira] [Created] (KAFKA-17499) Memory leak when using ReadOnlyKeyValueStore `all()` method

["James Cook (Jira)" <ji...@apache.org>]

James Cook created KAFKA-17499:
----------------------------------

             Summary: Memory leak when using ReadOnlyKeyValueStore `all()` method
                 Key: KAFKA-17499
                 URL: https://issues.apache.org/jira/browse/KAFKA-17499
             Project: Kafka
          Issue Type: Bug
          Components: streams
    Affects Versions: 3.8.0
            Reporter: James Cook
         Attachments: Screenshot 2024-09-09 at 08.31.57.png, Screenshot 2024-09-09 at 08.32.16.png, Screenshot 2024-09-09 at 08.32.25.png, Screenshot 2024-09-09 at 08.35.07.png

There is a memory leak when using the `.all()` method of ReadOnlyKeyValueStore (specifically MeteredTimestampedKeyValueStore).

I have recreated the problem is a small app here: [https://github.com/jamescookie/kafka-stream-memory-leak]

When I run the test method using 3.8.0, I get this memory footprint:

!Screenshot 2024-09-09 at 08.31.57.png!

In a little over 90 seconds the number of `java.util.TreeMap$Entry` grows to over 10 million:

!Screenshot 2024-09-09 at 08.32.16.png!

!Screenshot 2024-09-09 at 08.32.25.png!

 

When running against 3.7.1, this does not happen:

!Screenshot 2024-09-09 at 08.35.07.png!

 

The workaround (and correct way of coding) is to wrap a try-with-resources statement around the `store.all()`.

I think this has been caused by this [commit|https://github.com/apache/kafka/pull/15219/files] to fix KAFKA-15770

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)