[jira] [Created] (KAFKA-12500) Cache memory is leaked after removing/replacing a StreamThread

["A. Sophie Blee-Goldman (Jira)" <ji...@apache.org>]

A. Sophie Blee-Goldman created KAFKA-12500:
----------------------------------------------

             Summary: Cache memory is leaked after removing/replacing a StreamThread
                 Key: KAFKA-12500
                 URL: https://issues.apache.org/jira/browse/KAFKA-12500
             Project: Kafka
          Issue Type: Bug
          Components: streams
            Reporter: A. Sophie Blee-Goldman
             Fix For: 2.8.0


We currently leak the memory given to a StreamThread for its cache after it gets shutdown due to being removed or replaced. 

Removal:
If the StreamThread is removing itself, or if a thread is being removed by an external caller but fails to shutdown within the allotted time, it won't be removed from the `threads` list to avoid freeing up it's thread id while still running. If the thread hasn't reached the DEAD state when we resize the cache, the new cache size per thread is computed based on the number of live threads, which includes the removed thread at this point.

Replacement:
When a thread is replaced, we first shutdown that thread but hold off on removing it from the threads list until it's DEAD. Immediately after the shutdown we call addStreamThread to start up a new thread, which resizes the cache according to num-live-threads + 1.

In both of these cases, the cache memory of the shutting down thread won't be made available to the remaining threads.

Note: in general this leak may not be permanent, as a subsequent event once the thread has reached DEAD and been removed from the threads list will cause it's memory to be released. OF course if the subsequent event is another thread removal or replacement, then we just have a new memory leak.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)