Re: Tomcat randomly hangs - new info

[Brian Cross <br...@tigernet.com>]

Well after turning on vebose garbage collect I found out what the "hang" 
really is.  For some reason the garbage collector gets in a state where 
it tries to do a full GC but cannot for some unknown reason.  Here is 
what the verbose gc looks like on the hang:

7007.950: [Full GC [PSYoungGen: 157503K->157492K(206656K)] [PSOldGen: 
699071K->699071K(699072K)
] 856575K->856564K(905728K) [PSPermGen: 115222K->115222K(115520K)], 
1.1234380 secs]
7009.075: [Full GC [PSYoungGen: 157504K->157504K(206656K)] [PSOldGen: 
699071K->699071K(699072K)
] 856575K->856575K(905728K) [PSPermGen: 115222K->115222K(115520K)], 
1.1220980 secs]
7010.198: [Full GC [PSYoungGen: 157504K->157504K(206656K)] [PSOldGen: 
699071K->699071K(699072K)
] 856575K->856575K(905728K) [PSPermGen: 115222K->115222K(115520K)], 
1.1215690 secs]

this repeats forever...

I was actually able to get a thread dump but I don't really know what to 
look for.  I realize this is almost certainly not a tomcat issue so I 
won't post anymore about it, but I thought I would just give an 
update.   Thanks for all the suggestions.

--Brian.

Brian Cross wrote:
> Hello and thanks in advance for any advice.
>
> We have Tomcat 5.5.20 running standalone on the following system:
>
> Dual AMD Opteron
> 4 gig of memory
> Debian Linux 2.6.8 amd64 smp kernel
> latest sun java 1.5, 1.6, ibm 1.5 (happens on all 3 VM's)
> Tomcat 5.5.20 (originally happened on 5.5.16, upgrade didn't help)
>
> Randomly it will hang and stop accepting connections on port 80, 
> sometimes even late at night when there is not much traffic.
>
> When the hang occurs:
>
> * Can't access any web app on tomcat including the tomcat manager
> * Telnet to port 80 hangs(Coyote running on port 80)
> * load goes up, but only slightly (from < 1 to maybe 3 or 4)
> * sometimes after a couple of minutes it starts responding again, 
> sometimes it doesn't.
> * when I run shutdown.sh the java process continues to run and I have 
> to kill -9 to stop it
> * I try to do a kill -3 or kill -QUIT on the java process to get a 
> thread dump, but it does not work
>
> I have tried many different jvm settings and tomcat connector 
> settings, here is what I have currently:
>
> <Connector port="80" maxHttpHeaderSize="8192"
>           maxThreads="1000" minSpareThreads="50" maxSpareThreads="100"
>           enableLookups="false" redirectPort="8443" acceptCount="100"
>           connectionTimeout="20000" strategy="ms" 
> disableUploadTimeout="false" />
>
> CATALINA_OPTS="-server -XX:MaxPermSize=256m -ms512m -mx1280m -Xss96k 
> -Djava.awt.headless=true"
>
> I would really appreciate any advice, I really don't know where to go 
> from here to try and figure it out.  If I can't figure it out soon my 
> plan is to use apache2 as the webserver and connect to tomcat via JK.
>
> Thanks so much for your expert advice!
>
> --Brian.
>
> ---------------------------------------------------------------------
> To start a new topic, e-mail: users@tomcat.apache.org
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>

---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

RE: Tomcat randomly hangs - new info

["Caldarale, Charles R" <Ch...@unisys.com>]

> From: Brian Cross [mailto:brian@tigernet.com] 
> Subject: Re: Tomcat randomly hangs - new info
> 
> 7010.198: [Full GC
> [PSYoungGen: 157504K->157504K(206656K)]
> [PSOldGen: 699071K->699071K(699072K)]
> 856575K->856575K(905728K)
> [PSPermGen: 115222K->115222K(115520K)],
> 1.1215690 secs]

You're simply running out of heap space, so even a full GC can't free up
enough space to keep running.  The most likely cause is a memory leak
within your app(s), so a memory profiler would probably be the next
step.  It's also possible that you simply don't have enough PermGen
space available within the total heap to contain all the classes you
need, so you could try expanding that; if you really have a memory leak,
it will only postpone the inevitable.

There's some useful info here:
http://tomcat.apache.org/faq/memory.html

 - Chuck


THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
MATERIAL and is thus for use only by the intended recipient. If you
received this in error, please contact the sender and delete the e-mail
and its attachments from all computers.

---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org