You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by jnedzel <jn...@broad.mit.edu> on 2007/11/27 19:58:24 UTC
Tomcat becomes non-response for ~30 seconds
Folks:
We're having an intermittent problem with Tomcat becoming non-responsive for
a while (between 30 seconds and several minutes) and then recovering without
any intervention. There are no error messages in the Tomcat logs.
Any ideas what might be causing this or where to look?
We're running Tomcat 5.5.20 on linux.
Thanks,
Jared
--
View this message in context: http://www.nabble.com/Tomcat-becomes-non-response-for-%7E30-seconds-tf4883561.html#a13976711
Sent from the Tomcat - User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat becomes non-response for ~30 seconds
Posted by Jim Cox <sh...@gmail.com>.
On Nov 27, 2007 3:30 PM, jnedzel <jn...@broad.mit.edu> wrote:
>
> Thanks, I'll look at maxing out the logging.
>
A simple thing to do would be to grab a stack dump with a "kill -QUIT
<tomcat pid>" during one of these "slow periods" and see if anything
jumps out at you, compare it to one from a "normal period", etc.
Shouldn't require any modifications to the running system.
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat becomes non-response for ~30 seconds
Posted by jnedzel <jn...@broad.mit.edu>.
Thanks, I'll look at maxing out the logging.
There's only one webapp per Tomcat instance and none of them are heavily
loaded. In the case of one system, the only webapp does hardly any updating
(only tracks usage) and is basically query-only type of app that currently
has very light usage.
We are will be looking into our system monitoring tools to check if the
machines are getting overloaded (memory, i/o, cpu, etc.), but our suspicion
is that the machines are lightly loaded.
mgainty wrote:
>
> you'll want to max your logging to understand whats going on
> http://tomcat.apache.org/tomcat-5.5-doc/logging.html
> I concur with using JConsole *and logging*
> JMX is good for point in time System wide configuration snapshots
> If you are doing something like rendering giant PDF's or updating 1000's
> of
> records in a DB then you're exhausting resources
> Try a new TC install and add on one webapp each day..
> this way you can at least determine that the last loaded webapp caused the
> slowdown
--
View this message in context: http://www.nabble.com/Tomcat-becomes-non-response-for-%7E30-seconds-tf4883561.html#a13979134
Sent from the Tomcat - User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat becomes non-response for ~30 seconds
Posted by Delian Krustev <to...@krustev.net>.
On Tue, 27 Nov 2007 10:58:24 -0800 (PST) jnedzel wrote:
> We're having an intermittent problem with Tomcat becoming non-responsive
> for a while (between 30 seconds and several minutes) and then recovering
> without any intervention. There are no error messages in the Tomcat logs.
>
> Any ideas what might be causing this or where to look?
>
> We're running Tomcat 5.5.20 on linux.
Try running the VM with the -verbosegc option.
May be the garbage collector and the memory settings need some tuning.
You can also read about JMX and jconsole. They might prove to be helpful.
Cheers
--
Delian
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat becomes non-response for ~30 seconds
Posted by Ronald Klop <ro...@base.nl>.
Are you swapping during garbage collection?
Ronald.
On Tue Nov 27 19:58:24 CET 2007 Tomcat Users List <us...@tomcat.apache.org> wrote:
>
> Folks:
>
> We're having an intermittent problem with Tomcat becoming non-responsive for
> a while (between 30 seconds and several minutes) and then recovering without
> any intervention. There are no error messages in the Tomcat logs.
>
> Any ideas what might be causing this or where to look?
>
> We're running Tomcat 5.5.20 on linux.
>
> Thanks,
>
> Jared
> --
> View this message in context: http://www.nabble.com/Tomcat-becomes-non-response-for-%7E30-seconds-tf4883561.html#a13976711
> Sent from the Tomcat - User mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To start a new topic, e-mail: users@tomcat.apache.org
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
Re: Tomcat becomes non-response for ~30 seconds
Posted by Mark Deneen <md...@gmail.com>.
If the CPU is maxed out, the thread dump will provide lightweight
process numbers. Using a tool like "prstat -L" on Solaris, you can
figure out which lightweight process (thread) it is, and at least
confirm that it is the GC thread. the output from prstat is indexed
at 1, but the stack dump is indexed at 0, so you need to either add
one to the stack dump lwp id, or subtract one from the prstat output.
I'm sure there is an equivalent command for prstat in other operating
systems, but I do not know what it is.
Mark
On Nov 28, 2007 12:04 PM, Christopher Schultz
<ch...@christopherschultz.net> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Jared,
>
> jnedzel wrote:
> > We're having an intermittent problem with Tomcat becoming non-responsive for
> > a while (between 30 seconds and several minutes) and then recovering without
> > any intervention. There are no error messages in the Tomcat logs.
>
> Here's a silly question: what does a thread-dump during those pauses
> look like?
>
> - -chris
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.7 (MingW32)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
>
> iD8DBQFHTZ+49CaO5/Lv0PARArxDAKCVsJw7mtwHxZZxEZBYIgOUZ4yobwCZAVjK
> SKXAqTz7BBCMxXs6j2tDPWA=
> =1UjQ
> -----END PGP SIGNATURE-----
>
>
> ---------------------------------------------------------------------
> To start a new topic, e-mail: users@tomcat.apache.org
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat becomes non-response for ~30 seconds
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Jared,
jnedzel wrote:
> We're having an intermittent problem with Tomcat becoming non-responsive for
> a while (between 30 seconds and several minutes) and then recovering without
> any intervention. There are no error messages in the Tomcat logs.
Here's a silly question: what does a thread-dump during those pauses
look like?
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFHTZ+49CaO5/Lv0PARArxDAKCVsJw7mtwHxZZxEZBYIgOUZ4yobwCZAVjK
SKXAqTz7BBCMxXs6j2tDPWA=
=1UjQ
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat becomes non-response for ~30 seconds
Posted by Hassan Schroeder <ha...@gmail.com>.
On Nov 27, 2007 12:25 PM, jnedzel <jn...@broad.mit.edu> wrote:
> We are adding more monitoring to see if could be caused by auto-mounter
> problems.
You're NFS-mounting resources that Tomcat uses via automounter??
I'd think that would be, er, problematic from the get-go :-)
--
Hassan Schroeder ------------------------ hassan.schroeder@gmail.com
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Tomcat becomes non-response for ~30 seconds
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: jnedzel [mailto:jnedzel@broad.mit.edu]
> Subject: Re: Tomcat becomes non-response for ~30 seconds
>
> Are there any cases where everything is working at the OS and
> network level but Tomcat itself hangs for 30 seconds or so?
What JVM are you using? Older ones have been known to have long pauses
for garbage collection, but even then 30 seconds would be quite unusual.
Regarding the logging: additional logging may make the problem worse,
depending on what's actually causing the problem. Strongly suggest
poking around with JConsole first to see what's going on, paying close
attention to the state of the heap and looking at stack traces of the
worker threads.
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
MATERIAL and is thus for use only by the intended recipient. If you
received this in error, please contact the sender and delete the e-mail
and its attachments from all computers.
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat becomes non-response for ~30 seconds
Posted by jnedzel <jn...@broad.mit.edu>.
Our systems group is adding additional monitoring, but based on the
monitoring currently in place we don't think it is due to network problems.
We are adding more monitoring to see if could be caused by auto-mounter
problems.
Are there any cases where everything is working at the OS and network level
but Tomcat itself hangs for 30 seconds or so?
--
View this message in context: http://www.nabble.com/Tomcat-becomes-non-response-for-%7E30-seconds-tf4883561.html#a13979112
Sent from the Tomcat - User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat becomes non-response for ~30 seconds
Posted by Alexey Solofnenko <A....@mdl.com>.
Theoretically, the behaviour can be caused by network problems -
[reverse] DNS could not be resolved (maybe for logging), file share
becomes unavailable, etc.
- Alexey.
jnedzel wrote:
> Folks:
>
> We're having an intermittent problem with Tomcat becoming non-responsive for
> a while (between 30 seconds and several minutes) and then recovering without
> any intervention. There are no error messages in the Tomcat logs.
>
> Any ideas what might be causing this or where to look?
>
> We're running Tomcat 5.5.20 on linux.
>
> Thanks,
>
> Jared
>
--
------------------------------------------------------------------------
Alexey N. Solofnenko <http://trelony.cjb.net/>
Pleasant Hill, CA (GMT-8 usually)