You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@tomcat.apache.org by Ian Marsh <ia...@sportplan.com> on 2011/07/22 18:26:31 UTC

Upgrading from Tomcat 7.0.8 to 7.0.10 and higher causes Old Generation memory problems

Hi,

I am in charge of running a Apache-2, Tomcat-7, Ubuntu-10.04 set up
for which we have to be PCI Compliant. We recently upgraded to
Apache-2.2.17 and Tomcat-7.0.8 (from Apache-2.0.x and Tomcat 5.0.28)
in order to comply with the requirements of the PCI Compliance checks
and ironed out any issues to get us back to a satisfactory running
state.

We have now received warning, from our PCI Compliance monitor service,
that further updates are required to remain compliant, those being to
use Apache-2.2.19 and Tomcat-7.0.14 (or higher).

When using Tomcat-7.0.8, we experience healthy memory cycling. The Old
Generation slowly increases until garbage collection runs and clears
out the Old Gen memory level, dropping considerably as expected.

So to the upgrading! Upgrading Apache has been successful and without
problem, however upgrading Tomcat has caused memory problems which, as
yet, I cannot find similar reported cases or clear possibilities for
why its happening.

I've searched post archives and web results, looked through the change
log of Tomcat and tried using tools like jmap and the YourKit monitor
to discover some answers, although I am quite inexperienced with these
tools, but unfortunately I'm not progressing so far.

I have tried running with...
- Tomcat-7.0.16 (being the latest available)
- Tomcat-7.0.14 (being the minimum required for PCI Compliance)
- Tomcat-7.0.10 (being the next available release after 7.0.8)

Tomcat-7.0.10 obviously has the fewest number changes from
Tomcat-7.0.8 but, all of these have resulted in seeing a rather sharp
rise in the Old Generation memory usage, reaching 90%+ (of the
available 1.25GB) in under an hour. Requesting forced garbage
collection gives little effect, if not none at all, and eventually the
Old Gen memory becomes full and the site stops functioning.

There have been no changes to the jvm settings, so I doubt they are
the cause, but for your information they are as follows...

-Djava.awt.headless=true
-Xmx1536m
-Xms1536m
-XX:NewSize=256m
-XX:MaxNewSize=256M
-XX:SurvivorRatio=6
-XX:ReservedCodeCacheSize=64m
-XX:MaxPermSize=512m
-XX:PermSize=512m

I have used jmap to look into what objects are being held in memory
for the problematic versions of Tomcat. For Tomcat-7.0.10, the top 20
listed entries are...

 num     #instances         #bytes  class name
----------------------------------------------
  1:       8170940      471172552  [C
  2:       8501813      272058016  java.lang.String
  3:       5388341      215533640  javax.servlet.jsp.tagext.TagAttributeInfo
  4:        581051       35164552  [Ljava.lang.Object;
  5:        170658       34508384  <constMethodKlass>
  6:        528746       33839744  javax.servlet.jsp.tagext.TagInfo
  7:        528746       31094960  [Ljavax.servlet.jsp.tagext.TagAttributeInfo;
  8:         75231       25469312  [B
  9:        170658       23223888  <methodKlass>
 10:        395025       22121400  org.apache.jasper.compiler.Mark
 11:         11737       21889840  <constantPoolKlass>
 12:        281224       20248128  org.apache.jasper.compiler.Node$TemplateText
 13:        229740       19444304  [Ljava.util.HashMap$Entry;
 14:        594062       19009984  java.util.HashMap$Entry
 15:        220856       15973856  <symbolKlass>
 16:        110747       13909808  [Ljava.lang.String;
 17:         10561       13740840  <constantPoolCacheKlass>
 18:        400243       12807776  java.util.Stack
 19:        224826       10791648  java.util.HashMap
 20:         11737       10042552  <instanceKlassKlass>

When running under Tomcat-7.0.8 after approximately a similar time of
about an hour, the top 20 entries are...

 num     #instances         #bytes  class name
----------------------------------------------
  1:        592439       86974504  [C
  2:         47888       73416688  [I
  3:        189957       39457512  <constMethodKlass>
  4:         88307       27806528  [B
  5:         13444       26059552  <constantPoolKlass>
  6:        189957       25848632  <methodKlass>
  7:        619859       19835488  java.lang.String
  8:        242174       19462792  <symbolKlass>
  9:         12260       16256800  <constantPoolCacheKlass>
 10:         13444       11624240  <instanceKlassKlass>
 11:        236662        9466480  java.math.BigDecimal
 12:        184843        7393720  javax.servlet.jsp.tagext.TagAttributeInfo
 13:         83238        5795576  [Ljava.lang.Object;
 14:         59168        5611728  [Ljava.util.HashMap$Entry;
 15:        132465        4238880  java.util.HashMap$Entry
 16:          5287        3630456  <methodDataKlass>
 17:         54218        2602464  java.util.HashMap
 18:         26369        2320472  java.lang.reflect.Method
 19:         39714        1588560  java.util.TreeMap$Entry
 20:         14143        1470872  java.lang.Class

It seems that the character arrays [C, java.lang.String and
javax.servlet.jsp.tagext.TagAttributeInfo entries are considerably
higher in Tomcat-7.0.10 than in Tomcat-7.0.8 and I am wondering if
this could lead to an explanation for the difference.

Would anyone know of any changes between the two versions, possibly
linked to those memory entries, that could lead to such behaviour?

Any help or suggestions is greatly appreciated! I'm sorry for a long
post, but hopefully its got the information needed to help diagnosis.

Thanks in advance,

Ian

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org