You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by bu...@apache.org on 2010/11/04 18:06:16 UTC
DO NOT REPLY [Bug 50214] New: Tomcat threads caught in "S" state and
spiralling out of control
https://issues.apache.org/bugzilla/show_bug.cgi?id=50214
Summary: Tomcat threads caught in "S" state and spiralling out
of control
Product: Tomcat 6
Version: 6.0.29
Platform: PC
OS/Version: Linux
Status: NEW
Severity: critical
Priority: P2
Component: Catalina
AssignedTo: dev@tomcat.apache.org
ReportedBy: paulm@sitescout.com
Hello,
I'm dealing with a bug that I have seen on a few other posts but to date has no
solution.
Basically, connections come in to the nodes and periodically hang forever and
cause the thread to spiral out of control. If left untouched they keep
happening until I restart tomcat.
Looks like this on tomcat manager:
Stage Time B Sent B Recv
S 11115068 ms 0 KB 0 KB
Exact behavior is also described in this old thread:
- http://old.nabble.com/httpd-JK-Tomcat-hung-connections-td10403182.html
- Mind you we dont use JK -- just standalone tomcat on its own
getting requests from a LB
- more recently: http://forums.terracotta.org/forums/posts/list/3264.page
First off, let me explain my setup. We run a high performance ad network that
processes & bids on real time bidding information. We run two clusters with two
different ISPs: one on the west coast and one on the east coast. (East coast
one is amazon).
I originally thought this was a bug caused by virtualization since I would only
see this issue on nodes running on the west coast. At the time the east coast
and the west coast shared the exact same setup (same software) but different os
images., We use CentOS on the west and the Amazon Flavor linux distro on east -
both 32bit.
After fighting for weeks with my west coast provider I finally realized it
wasn't them. The tell tale was the fact that I started seeing this issue on
west coast (yesterday) when i launched 64bit instance(only with 64bit does it
occur on east coast now).
Also, to note we were originally running with 6.0.24 and then recently upgraded
all to 6.0.29 and still occurring. I tried yesterday running the 64bit instance
with the APR JNI libs and also still occurs.
We don't use sessions, and there are no errors on any nodes when this happens.
They just block and spiral forever. Luckily this doesn't really affect the
cluster. The node is still able to function normally, however eating all the
CPU.
On the west coast where this has happened for months now (since launching) we
are running 32bit CentOS and I have tried everything including upgrading the
kernel to latest. Up until yesterday I had never seen this issue on the east
coast where it shares _identical_ setup (binaries/java/configuration/db) until
we attempted to run a 64bit node where it was immediately noticed.
Can anyone point me in the right direction ? It's definitely occurring on the
latest version of tomcat 6.
Thanks,
-Paul
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
DO NOT REPLY [Bug 50214] Tomcat threads caught in "S" state and
spiralling out of control
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=50214
Paul M <pa...@sitescout.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|RESOLVED |REOPENED
Resolution|INVALID |
--- Comment #2 from Paul M <pa...@sitescout.com> 2010-11-04 13:28:15 EDT ---
I will provide you with a thread dump on the next occurrence.
How could it be application level when the hanging thread is unable to even
process the underlying request?
Please note that these hanging threads are hanging on normal HTTP connector
threads (using keepalive mind you) and not using any external connector.
With that said, I am without doubt that this is not an application level issue.
Nothing external should cause tomcat to hang like this. Which is why I thought
it was a virtualization issue (which I have now ruled out).
Can this be reopened please? Would really like to resolv this.
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
DO NOT REPLY [Bug 50214] Tomcat threads caught in "S" state and
spiralling out of control
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=50214
Chuck Caldarale <ch...@unisys.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|REOPENED |RESOLVED
Resolution| |INVALID
--- Comment #3 from Chuck Caldarale <ch...@unisys.com> 2010-11-04 13:36:18 EDT ---
(In reply to comment #2)
> I will provide you with a thread dump on the next occurrence.
> Can this be reopened please? Would really like to resolv this.
Bugzilla is *not* a support forum. Use the Tomcat support mailing list.
- Chuck
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
DO NOT REPLY [Bug 50214] Tomcat threads caught in "S" state and
spiralling out of control
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=50214
Mark Thomas <ma...@apache.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Resolution| |INVALID
--- Comment #1 from Mark Thomas <ma...@apache.org> 2010-11-04 13:10:42 EDT ---
Without a thread dump that shows the thread is stuck in Tomcat code, this looks
like an application issue. The issue from the terracotta forums is definitely
an app or terracotta issue, not a Tomcat one.
The users mailing list is the place to get help.
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org