You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Anurag Kapur <an...@gmail.com> on 2010/02/10 03:35:11 UTC

Tomcat HTTP Connector Threads Hung

*Problem Statement

*While load testing a web application, several tomcat http connector threads
get stuck. Ultimatelty all the available http connector threads get
exhausted and the application stops responding completely.

*Setup*

I have an apache http server in front of tomcat which proxies requests to
it. I am using mod_proxy_http module(I do not know the exact version. Is
there an easy way to find this out?) to proxy requests to tomcat.

Relevant snippet from apache httpd.conf that shows the proxy setting:

ProxyPass                 http://localhost:8080/ retry=0
ProxyPassReverse          http://myserver/

*Observations*

Since there is no explicit timeout set along with the proxy directive, it
assumes a default value of 300s. Thus, after 300s, if apache does not get
back a response from tomcat, it servers a HTTP 502 error code back to the
user.

However, the corresponding http connector thread does not terminate and
continues waiting for a response. As a result, with time a lot of http
connector threads get stuck and ultimately the whole pool gets exhausted.

Http connector settings from tomcat server.xml:
<Connector port="8080" maxHttpHeaderSize="8192"
               maxThreads="300" minSpareThreads="25" maxSpareThreads="77"
               enableLookups="false" redirectPort="8443" acceptCount="256"
               connectionTimeout="300000" disableUploadTimeout="true" />

*Questions*

1. What does the connectionTimeout attribute in the tomcat http connector
do? Does it not terminate the thread if it is still processing the request
once the timeout expires? I assume it does NOT and that is why I am seeing
this problem.

2. Can some other configuration be added that forces the http connector
thread to terminate after a certain time interval has elapsed?

3. Any other thoughts or advice to solve this problem? I know I should
investigate why the http connector thread does not return a response even in
as long as 300s, but assuming there is a bug in the application code
shouldn't there be a settings in tomcat that prevents such application bugs
from exhausting all tomcat connector threads?

*Other System Information*

Server version: Apache Tomcat/5.5.27
Server built:   Aug 28 2008 10:08:26
Server number:  5.5.27.0
OS Name:        SunOS
OS Version:     5.10
Architecture:   x86
JVM Version:    1.5.0_14-b03
JVM Vendor:     Sun Microsystems Inc.

Apache Server version: Apache/2.2.9 (Unix)

Your thoughts and advice on this would be much appreciated.

Thanks
Anurag

------------------------------------------------------------------
Anurag Kapur
Associate - Technology,
Sapient Corporation.

http://www.linkedin.com/in/anuragkapur

http://www.google.com/profiles/anuragkapur
------------------------------------------------------------------

Re: Tomcat HTTP Connector Threads Hung

Posted by Anurag Kapur <an...@gmail.com>.
Thank you for your response Chuck. Got your point and have started
investigating this problem at the web app level.

Regards
Anurag

On Wed, Feb 10, 2010 at 3:45 AM, Caldarale, Charles R <
Chuck.Caldarale@unisys.com> wrote:

> > From: Anurag Kapur [mailto:anuragkapur@gmail.com]
> > Subject: Tomcat HTTP Connector Threads Hung
> >
> > 1. What does the connectionTimeout attribute in the tomcat http
> > connector do?
>
> Exactly what the documentation says:
>
> "The number of milliseconds this Connector will wait, after accepting a
> connection, for the request URI line to be presented."
>
> > Does it not terminate the thread if it is still processing the
> > request once the timeout expires?
>
> No - it is the *connectionTimeout*; if the client doesn't deliver the URI
> in that time period, the connection will be dropped.  If the client does
> deliver the URI, the servicing thread is off and running.
>
> > 2. Can some other configuration be added that forces the http connector
> > thread to terminate after a certain time interval has elapsed?
>
> No - that's up to the logic of your webapp.
>
> > I know I should investigate why the http connector thread does
> > not return a response even in as long as 300s
>
> Yes, you should.  Start with a thread dump and find out why your webapp is
> sitting still.
>
> > shouldn't there be a settings in tomcat that prevents such application
> > bugs from exhausting all tomcat connector threads?
>
> No - there is no mechanism in Java to stop an arbitrary thread without the
> consent of that thread.  There's nothing in the servlet spec that provides
> for such a mechanism, so there's nothing Tomcat can do.  If a thread of your
> webapp is off contemplating its navel, it's up to your webapp to police that
> behavior.
>
>  - Chuck
>
>
> THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
> MATERIAL and is thus for use only by the intended recipient. If you received
> this in error, please contact the sender and delete the e-mail and its
> attachments from all computers.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>

RE: Tomcat HTTP Connector Threads Hung

Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: Anurag Kapur [mailto:anuragkapur@gmail.com]
> Subject: Tomcat HTTP Connector Threads Hung
> 
> 1. What does the connectionTimeout attribute in the tomcat http
> connector do?

Exactly what the documentation says:

"The number of milliseconds this Connector will wait, after accepting a connection, for the request URI line to be presented."

> Does it not terminate the thread if it is still processing the
> request once the timeout expires?

No - it is the *connectionTimeout*; if the client doesn't deliver the URI in that time period, the connection will be dropped.  If the client does deliver the URI, the servicing thread is off and running.

> 2. Can some other configuration be added that forces the http connector
> thread to terminate after a certain time interval has elapsed?

No - that's up to the logic of your webapp.

> I know I should investigate why the http connector thread does 
> not return a response even in as long as 300s

Yes, you should.  Start with a thread dump and find out why your webapp is sitting still.
 
> shouldn't there be a settings in tomcat that prevents such application
> bugs from exhausting all tomcat connector threads?

No - there is no mechanism in Java to stop an arbitrary thread without the consent of that thread.  There's nothing in the servlet spec that provides for such a mechanism, so there's nothing Tomcat can do.  If a thread of your webapp is off contemplating its navel, it's up to your webapp to police that behavior.

 - Chuck


THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org