[jira] [Commented] (IGNITE-967) Internal thread locals are not always cleaned

["Scott Baugher (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IGNITE-967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15335107#comment-15335107 ] 

Scott Baugher commented on IGNITE-967:
--------------------------------------

I also see the same issue with Ignite.  I am currently using it only as a distributed cache.  The problem is particularly acute during development, as each redeployment of the app causes a memory leak.  Both Glassfish/Payara and Tomcat report:

16-Jun-2016 18:24:16.127 SEVERE [RMI TCP Connection(6)-127.0.0.1] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [ICM_Service] created a ThreadLocal with key of type [org.apache.ignite.internal.binary.BinaryThreadLocalContext$1] (value [org.apache.ignite.internal.binary.BinaryThreadLocalContext$1@796c8a3a]) and a value of type [org.apache.ignite.internal.binary.BinaryThreadLocalContext] (value [org.apache.ignite.internal.binary.BinaryThreadLocalContext@2f791267]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
16-Jun-2016 18:24:16.127 SEVERE [RMI TCP Connection(6)-127.0.0.1] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [ICM_Service] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@490055ca]) and a value of type [org.apache.ignite.internal.binary.streams.BinaryMemoryAllocatorChunk] (value [org.apache.ignite.internal.binary.streams.BinaryMemoryAllocatorChunk@54203b8a]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
16-Jun-2016 18:24:16.127 SEVERE [RMI TCP Connection(6)-127.0.0.1] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [ICM_Service] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@33adf143]) and a value of type [org.apache.ignite.marshaller.optimized.OptimizedObjectStreamRegistry.StreamHolder] (value [org.apache.ignite.marshaller.optimized.OptimizedObjectStreamRegistry$StreamHolder@9fd7094]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.

Based on my experiences so far, neither one eventually cleans things up.

> Internal thread locals are not always cleaned
> ---------------------------------------------
>
>                 Key: IGNITE-967
>                 URL: https://issues.apache.org/jira/browse/IGNITE-967
>             Project: Ignite
>          Issue Type: Bug
>          Components: general
>    Affects Versions: sprint-4
>            Reporter: Valentin Kulichenko
>            Assignee: Alexei Scherbakov
>            Priority: Critical
>              Labels: important
>             Fix For: 1.6
>
>
> One of our users reported that he sees warnings in Tomcat's log when the application that's running Ignite in embedded mode is undeployed:
> {code}
> SEVERE: The web application [/XXX] created a ThreadLocal with key of type [org.apache.ignite.internal.util.GridSpinReadWriteLock$1] (value [org.apache.ignite.internal.util.GridSpinReadWriteLock$1@2c2858af]) and a value of type [java.lang.Integer] (value [0]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
> {code}
> There is also the similar warning for {{GridToStringBuilder.threadCache}}. 
> While it's usually OK not to clean thread locals on standalone node, in app server it can cause a memory leak.
> To avoid such issues I suggest to add a special step after all test suites that will check thread locals in test runner thread. If we have this check in CI, we will fix it once and for always.
> Thread local values can be introspected through {{Thread.threadLocals}} variable. It would also be a good idea to check Tomcat's sources on how it's done there.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)