You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Ognjen Blagojevic <og...@etf.bg.ac.rs> on 2010/04/28 17:39:01 UTC

Debugging PermGen problems

Hi,

My co-worker and I decided to try to debug PermGen problems after 
detailed reading of mailing list archives and Tomcat wiki.

We followed the procedure described at the wiki [1], and got one 
inactive WebappClassLoader (started = false), with 18 GC root paths. 
They are attached to the message (relevant ones). In the attached list 
there isn't a single class developed in-house, but are mostly from 
third-party jars.

The first one, org.postgresql.Driver I think I understand.

postgresql-*.jar is located in Tomcat's lib folder, so it is loaded with 
Tomcat class loader in order to create DB pool. Then, it is used by some 
of webapp libraries, which I guess is OpenJPA. However, I don't have a 
clue how to deregister it. I tried to do it in contextDestroyed of 
registered ServletContextListener, but no success. It seems that I 
cannot access registered drivers from different classloader. 
org.postgresql.Driver is loaded from Tomcat class loader, and 
ServletContextListener is being loaded from webapp class loader, so no 
luck. Any ideas how to clean up the driver on application unload?

The second root path is even worse. UploadedFile (which is very simple 
class part of MyFaces) seems to be referenced in static HashTable 
"registry" in class PropertyEditorManager. I can't find who made that 
registration, but I believe it is not in my code. I would like to ask 
developers to fix the problem (or try to fix it myself) but I am unable 
to pin-point what is the cause of the problem. Any ideas with this one?

Environment: Tomcat 6.0.14, Windows XP SP2.

Regards,
Ognjen

[1] http://wiki.apache.org/tomcat/OutOfMemory


------
GC root paths (relevant part):

Paths from "GC Roots" to "Object WebappClassLoader #054435C8"

+----------------------------------------------------------------------
|                                                                 Name
+----------------------------------------------------------------------
|  +---org.apache.catalina.loader.WebappClassLoader
|    |
|    +---<loader> of org.postgresql.Driver
|    | |
|    | +---driverClass of java.sql.DriverInfo
|    |   |
|    |   +---[1] of java.lang.Object[10]
|    |     |
|    |     +---elementData of java.util.Vector
|    |       |
|    |       +---writeDrivers of java.sql.DriverManager [Class]
|    |
|    +---<loader> of org.apache.myfaces.custom.fileupload.UploadedFile
|    | |
|    | +---key of java.util.Hashtable$Entry
|    |   |
|    |   +---[10] of java.util.Hashtable$Entry[23]
|    |     |
|    |     +---table of java.util.Hashtable
|    |       |
|    |       +---registry of java.beans.PropertyEditorManager [Class]


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


RE: Debugging PermGen problems

Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: Martin Gainty [mailto:mgainty@hotmail.com]
> Subject: RE: Debugging PermGen problems
> 
> if the database is only being referenced by that webapp then put the
> postgresql-*.jars into WEB-INF/lib

You can't do that if Tomcat is managing the DB connection pool - the driver jar must be placed in Tomcat's lib directory.  You may place the driver jar in the webapp's WEB-INF/lib directory if and only if the webapp is managing the connections.

 - Chuck


THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


RE: Debugging PermGen problems

Posted by Martin Gainty <mg...@hotmail.com>.


> Hi,
> 
> My co-worker and I decided to try to debug PermGen problems after 
> detailed reading of mailing list archives and Tomcat wiki.
> 
> We followed the procedure described at the wiki [1], and got one 
> inactive WebappClassLoader (started = false), with 18 GC root paths. 
> They are attached to the message (relevant ones). In the attached list 
> there isn't a single class developed in-house, but are mostly from 
> third-party jars.
> 
> The first one, org.postgresql.Driver I think I understand.
> 
> postgresql-*.jar is located in Tomcat's lib folder, so it is loaded with 
> Tomcat class loader in order to create DB pool. Then, it is used by some 
> of webapp libraries, which I guess is OpenJPA. However, I don't have a 
> clue how to deregister it. I tried to do it in contextDestroyed of 
> registered ServletContextListener, but no success. It seems that I 
> cannot access registered drivers from different classloader. 
> org.postgresql.Driver is loaded from Tomcat class loader, and 
> ServletContextListener is being loaded from webapp class loader, so no 
> luck. Any ideas how to clean up the driver on application unload?

if the database is only being referenced by that webapp then put the postgresql-*.jars into WEB-INF/lib

> 
> The second root path is even worse. UploadedFile (which is very simple 
> class part of MyFaces) seems to be referenced in static HashTable 
> "registry" in class PropertyEditorManager. I can't find who made that 
> registration, but I believe it is not in my code. I would like to ask 
> developers to fix the problem (or try to fix it myself) but I am unable 
> to pin-point what is the cause of the problem. Any ideas with this one?

declare UploadedFile class as @Embeddable class inside PropertyEditorManager class
http://java.sun.com/javaee/5/docs/api/javax/persistence/Embeddable.html
 
> Environment: Tomcat 6.0.14, Windows XP SP2.
> 
> Regards,
> Ognjen
> 
> [1] http://wiki.apache.org/tomcat/OutOfMemory
> 
> 
> ------
> GC root paths (relevant part):
> 
> Paths from "GC Roots" to "Object WebappClassLoader #054435C8"
> 
> +----------------------------------------------------------------------
> |                                                                 Name
> +----------------------------------------------------------------------
> |  +---org.apache.catalina.loader.WebappClassLoader
> |    |
> |    +---<loader> of org.postgresql.Driver
> |    | |
> |    | +---driverClass of java.sql.DriverInfo
> |    |   |
> |    |   +---[1] of java.lang.Object[10]
> |    |     |
> |    |     +---elementData of java.util.Vector
> |    |       |
> |    |       +---writeDrivers of java.sql.DriverManager [Class]
> |    |
> |    +---<loader> of org.apache.myfaces.custom.fileupload.UploadedFile
> |    | |
> |    | +---key of java.util.Hashtable$Entry
> |    |   |
> |    |   +---[10] of java.util.Hashtable$Entry[23]
> |    |     |
> |    |     +---table of java.util.Hashtable
> |    |       |
> |    |       +---registry of java.beans.PropertyEditorManager [Class]
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
> 
 		 	   		  
_________________________________________________________________
The New Busy is not the old busy. Search, chat and e-mail from your inbox.
http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_3

Re: Debugging PermGen problems

Posted by Ognjen Blagojevic <og...@etf.bg.ac.rs>.
Caldarale, Charles R wrote:
>> From: Ognjen Blagojevic [mailto:ognjen@etf.bg.ac.rs]
>> Subject: Debugging PermGen problems
>>
>> Environment: Tomcat 6.0.14, Windows XP SP2.
> 
> Before doing *anything* else, move up to the current level (6.0.26).  Besides having numerous stability and security fixes beyond your nearly three-year old version, it has several enhancements to catch and work around these memory leakers.  It's pointless to continue experimentation with 6.0.14.

Thank you for hint. We upgraded to 6.0.26, and DriverManager problems 
are now gone.

A bit of research of the other GC root paths revealed that one culprit 
is jsf-impl jar, so we upgraded from mojarra 1.2_08 to latest 1.2_14 and 
solved part of the problem with PropertyEditorManager.

Now, it seems that there is no more problematic static references, but 
few threadLocal references are still there. Those seems to be related to 
JasperReports 2.0.5 library. From the root paths attached below, I guess 
that the thread "http-8443-6" holds reference to the class loaded by 
webapp class loader, and therefore it is not possible to GC that webapp 
class loader.

If my assumption is correct, next thing we need to find is how to 
inspect and clean up thread local after JasperReport servlet finished 
his work. Right?

Regards,
Ognjen

Paths from "GC Roots" to "Object WebappClassLoader #04068C80"

+-------------------------------------------------------------------------------------------+-----------------+----------------+
|                                           Name 
                     |  Retained Size  |  Shallow Size  |
+-------------------------------------------------------------------------------------------+-----------------+----------------+
|  +---org.apache.catalina.loader.WebappClassLoader 
                     |      8,109,432  |           168  |
|    | 
                     |                 |                |
|    +---<loader> of net.sf.jasperreports.engine.util.JRClassLoader 
                     |             64  |            64  |
|    | | 
                     |                 |                |
|    | +---<class> of net.sf.jasperreports.engine.util.JRClassLoader 
                     |            632  |            56  |
|    |   | 
                     |                 |                |
|    |   +---<loader> of 
stu_zavrsni_rad_prijava_detail_1269596557540_786829                | 
         696  |            64  |
|    |     | 
                     |                 |                |
|    |     +---value of java.lang.ThreadLocal$ThreadLocalMap$Entry 
                     |            728  |            32  |
|    |       | 
                     |                 |                |
|    |       +---[52] of java.lang.ThreadLocal$ThreadLocalMap$Entry[64] 
                     |          5,776  |           272  |
|    |         | 
                     |                 |                |
|    |         +---table of java.lang.ThreadLocal$ThreadLocalMap 
                     |          5,800  |            24  |
|    |           | 
                     |                 |                |
|    |           +---threadLocals of java.lang.Thread [Stack Local, 
Thread]  "http-8443-6"  |          6,088  |           104  |
|    | 
                     |                 |                |
|    +---classloader of java.security.ProtectionDomain 
                     |            920  |            32  |
|    | | 
                     |                 |                |
|    | +---<protection domain> of 
stu_zavrsni_rad_prijava_detail_1269596557540_786829       | 
696  |            64  |
|    |   | 
                     |                 |                |
|    |   +---value of java.lang.ThreadLocal$ThreadLocalMap$Entry 
                     |            728  |            32  |
|    |     | 
                     |                 |                |
|    |     +---[52] of java.lang.ThreadLocal$ThreadLocalMap$Entry[64] 
                     |          5,776  |           272  |
|    |       | 
                     |                 |                |
|    |       +---table of java.lang.ThreadLocal$ThreadLocalMap 
                     |          5,800  |            24  |
|    |         | 
                     |                 |                |
|    |         +---threadLocals of java.lang.Thread [Stack Local, 
Thread]  "http-8443-6"    |          6,088  |           104  |
|    | 
                     |                 |                |
|    +---contextClassLoader of java.lang.Thread [Stack Local, Thread] 
"Java2D Disposer"    |            320  |           104  |
|    | 
                     |                 |                |
|    +---parent of net.sf.jasperreports.engine.util.JRClassLoader 
                     |            632  |            56  |
|      | 
                     |                 |                |
|      +---<loader> of 
stu_zavrsni_rad_prijava_detail_1269596557540_786829                  | 
           696  |            64  |
|        | 
                     |                 |                |
|        +---value of java.lang.ThreadLocal$ThreadLocalMap$Entry 
                     |            728  |            32  |
|          | 
                     |                 |                |
|          +---[52] of java.lang.ThreadLocal$ThreadLocalMap$Entry[64] 
                     |          5,776  |           272  |
|            | 
                     |                 |                |
|            +---table of java.lang.ThreadLocal$ThreadLocalMap 
                     |          5,800  |            24  |
|              | 
                     |                 |                |
|              +---threadLocals of java.lang.Thread [Stack Local, 
Thread]  "http-8443-6"    |          6,088  |           104  |
+-------------------------------------------------------------------------------------------+-----------------+----------------+


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


RE: Debugging PermGen problems

Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: Ognjen Blagojevic [mailto:ognjen@etf.bg.ac.rs]
> Subject: Debugging PermGen problems
> 
> Environment: Tomcat 6.0.14, Windows XP SP2.

Before doing *anything* else, move up to the current level (6.0.26).  Besides having numerous stability and security fixes beyond your nearly three-year old version, it has several enhancements to catch and work around these memory leakers.  It's pointless to continue experimentation with 6.0.14.

 - Chuck


THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org