You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Mark Eggers <it...@yahoo.com.INVALID> on 2021/02/01 05:39:41 UTC

Memory leak with Jersey 2.33

Folks,

This is probably not a Tomcat issue, but any thoughts on how to resolve 
this would be greatly appreciated.

I am running into an apparent ClassLoader leak with the following 
configuration:

Windows 10 Professional (64 bit, latest updates)
OpenJDK 11.0.10
Apache Tomcat 7.0.107
Jersey Rest 2.33

I have a simple application:

ApplicationConfig:

@ApplicationPath("/service")
public class ApplicationConfig extends Application {

     public ApplicationConfig() {
     }

     @Override
     public Set<Class<?>> getClasses(){
         Set<Class<?>> resources = new java.util.HashSet<>();

 
resources.add(org.mdeggers.cplanapi.resource.CostInfoResource.class);
 
resources.add(org.mdeggers.cplanapi.resource.InstituteInfoResource.class);
 
resources.add(org.mdeggers.cplanapi.resource.InstituteTypeInfoResource.class);

 
resources.add(org.mdeggers.cplanapi.mapper.NotFoundExceptionMapper.class);

         return resources;
     }
}

One of three resources:

@Path("/v1/inst")
public class InstituteInfoResource {

     public InstituteInfoResource() {
     }

     @GET
     @Path("/info")
     @Produces({MediaType.APPLICATION_JSON})
     public InstituteContainer getNationalList() {
         InstituteInfo instituteinfo = new InstituteInfo();
         return instituteinfo.getNational();
     }

     @GET
     @Path("/info/{ state : [A-Z]{2} }")
     @Produces({MediaType.APPLICATION_JSON})
     public InstituteContainer getStateList(@PathParam("state") String 
state) {
         InstituteInfo instituteinfo = new InstituteInfo();
         return instituteinfo.getState(state);
     }
}

The InstituteInfo class is a DAO that makes requests into a MySQL 
database and returns a container with the information.

This all works, but on undeploying the Tomcat Manager complains that 
there is a leak after unloading the web application.

Doing a heap dump and looking at it with the following OQL in Eclipse 
MAT shows that the application still resides in memory, with started=false.

SELECT wcl.contextName.toString() AS contextName,
    wcl.started AS started,
    wcl.@retainedHeapSize AS retainedSize
FROM org.apache.catalina.loader.WebappClassLoader wcl

contextName   started      retainedSize
/cplanapi     false        586,720
/docs         true          46,664
/host-manager true          47,288
/manager      true          47,392
/examples     true          85,432
               true          46,912

GC roots for cplanapi are:

class com.sun.naming.internal.ResourceManager
'- propertiesCache java.util.WeakHashMap
    '- table java.util.WeakHashMap$Entry[16]
       '- java.util.WeakHashMap$Entry
          '- referent org.apache.catalina.loader.WebappClassLoader

This occurs on both Windows and Linux, and with the above JRE as well as 
Java 1.8.0_202.

. . . just my two cents
/mde/


Re: Memory leak with Jersey 2.33

Posted by Mark Eggers <it...@yahoo.com.INVALID>.
Mark,

On 2/1/2021 1:33 AM, Mark Thomas wrote:
> Is the GC root above the only one? I've seen similar behaviour in the
> past where weak references appear to be the cause of a leak but closer
> inspection uncovers a strong reference.
> 
> Mark

Thanks for putting me on the correct track. I'm using log4j2 2.14.0 and 
Jersey REST 2.33 which includes Jackson 2.11.3.

Analyzing the heap dump using Eclipse MAT and excluding weak references 
led me down a fun rabbit hole.

Leak 1:
https://github.com/FasterXML/jackson-core/issues/400

Fix:
in setenv.(bat/sh) add
-Dcom.fasterxml.jackson.core.util.BufferRecyclers.trackReusableBuffers=true

In a servlet context listener add
int released = releaseBuffers(); // with the proper import, of course

Leak 2:
https://issues.apache.org/jira/browse/LOG4J2-578

Although it's marked as fixed, it apparently is not.

Fix (for now):
in setenv(bat/sh) add
-Dlog4j2.disable.jmx=true

With all of that done, undeploying the web application and doing a heap 
dump shows no traces of the offending app (all WebappClassLoaders have 
started=true).

Again, thanks for the tip.

. . . just my two cents.


Re: Memory leak with Jersey 2.33

Posted by Mark Thomas <ma...@apache.org>.
On 01/02/2021 06:37, Mark Eggers wrote:
> On 1/31/2021 9:39 PM, Mark Eggers wrote:

<snip/>

>> GC roots for cplanapi are:
>>
>> class com.sun.naming.internal.ResourceManager
>> '- propertiesCache java.util.WeakHashMap
>>     '- table java.util.WeakHashMap$Entry[16]
>>        '- java.util.WeakHashMap$Entry
>>           '- referent org.apache.catalina.loader.WebappClassLoader
>>
>> This occurs on both Windows and Linux, and with the above JRE as well
>> as Java 1.8.0_202.
>>
>> . . . just my two cents
>> /mde/
>>
> 
> Please ignore this. Doing two rounds of GC through the manager removed
> the application from memory (as confirmed by a heap dump).
> 
> Now I need to figure out how to force a GC, since we had an OOM
> Metaspace issue with repeated deployments of the application.

Is the GC root above the only one? I've seen similar behaviour in the
past where weak references appear to be the cause of a leak but closer
inspection uncovers a strong reference.

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Memory leak with Jersey 2.33

Posted by Mark Eggers <it...@yahoo.com.INVALID>.
On 1/31/2021 9:39 PM, Mark Eggers wrote:
> Folks,
> 
> This is probably not a Tomcat issue, but any thoughts on how to resolve 
> this would be greatly appreciated.
> 
> I am running into an apparent ClassLoader leak with the following 
> configuration:
> 
> Windows 10 Professional (64 bit, latest updates)
> OpenJDK 11.0.10
> Apache Tomcat 7.0.107
> Jersey Rest 2.33
> 
> I have a simple application:
> 
> ApplicationConfig:
> 
> @ApplicationPath("/service")
> public class ApplicationConfig extends Application {
> 
>      public ApplicationConfig() {
>      }
> 
>      @Override
>      public Set<Class<?>> getClasses(){
>          Set<Class<?>> resources = new java.util.HashSet<>();
> 
> 
> resources.add(org.mdeggers.cplanapi.resource.CostInfoResource.class);
> 
> resources.add(org.mdeggers.cplanapi.resource.InstituteInfoResource.class);
> 
> resources.add(org.mdeggers.cplanapi.resource.InstituteTypeInfoResource.class); 
> 
> 
> 
> resources.add(org.mdeggers.cplanapi.mapper.NotFoundExceptionMapper.class);
> 
>          return resources;
>      }
> }
> 
> One of three resources:
> 
> @Path("/v1/inst")
> public class InstituteInfoResource {
> 
>      public InstituteInfoResource() {
>      }
> 
>      @GET
>      @Path("/info")
>      @Produces({MediaType.APPLICATION_JSON})
>      public InstituteContainer getNationalList() {
>          InstituteInfo instituteinfo = new InstituteInfo();
>          return instituteinfo.getNational();
>      }
> 
>      @GET
>      @Path("/info/{ state : [A-Z]{2} }")
>      @Produces({MediaType.APPLICATION_JSON})
>      public InstituteContainer getStateList(@PathParam("state") String 
> state) {
>          InstituteInfo instituteinfo = new InstituteInfo();
>          return instituteinfo.getState(state);
>      }
> }
> 
> The InstituteInfo class is a DAO that makes requests into a MySQL 
> database and returns a container with the information.
> 
> This all works, but on undeploying the Tomcat Manager complains that 
> there is a leak after unloading the web application.
> 
> Doing a heap dump and looking at it with the following OQL in Eclipse 
> MAT shows that the application still resides in memory, with started=false.
> 
> SELECT wcl.contextName.toString() AS contextName,
>     wcl.started AS started,
>     wcl.@retainedHeapSize AS retainedSize
> FROM org.apache.catalina.loader.WebappClassLoader wcl
> 
> contextName   started      retainedSize
> /cplanapi     false        586,720
> /docs         true          46,664
> /host-manager true          47,288
> /manager      true          47,392
> /examples     true          85,432
>                true          46,912
> 
> GC roots for cplanapi are:
> 
> class com.sun.naming.internal.ResourceManager
> '- propertiesCache java.util.WeakHashMap
>     '- table java.util.WeakHashMap$Entry[16]
>        '- java.util.WeakHashMap$Entry
>           '- referent org.apache.catalina.loader.WebappClassLoader
> 
> This occurs on both Windows and Linux, and with the above JRE as well as 
> Java 1.8.0_202.
> 
> . . . just my two cents
> /mde/
> 

Please ignore this. Doing two rounds of GC through the manager removed 
the application from memory (as confirmed by a heap dump).

Now I need to figure out how to force a GC, since we had an OOM 
Metaspace issue with repeated deployments of the application.

. . . just my two cents
/mde/