Tomcat status 75 error

[Paul Beattie <pb...@orbisuk.com>]

Hi,

I have a series of relatively high transaction volume tomcat servers
which are sporadically hitting thread issues.  We run with with tomcat
4.1.27 at the current time.  The behaviour we notice is after several
days of around one hundred thousand transactions our tomcat instances
hang with the catalina.out message of:

SEVERE: All threads are busy, waiting. Please increase maxThreads or
check the servlet status75 75
Stopping service Tomcat-Standalone

I've done a quite a lot of reading on this and in some cases there is an
indication that moving to a later tomcat version alleviates the issue. 
However in many others it indicates that this problem is due solely to
the code whereby the threads are not exiting cleanly.  We have a classic
3-tier architecture and we can see the db connections closing.  I have
tired reproducing this error on a tomcat 5 environment and I get a
similar but not identical error:

org.apache.tomcat.util.threads.ThreadPool logFull
SEVERE: All threads (15) are currently busy, waiting. Increase
maxThreads (15) or check the servlet status

The tomcat install continues to function correctly after this has
occurred.   What I would like to know is do the 2 error messages
correspond to the same underlying issue.  I strongly suspect it is an
application code issue so I would like to be able to count Tomcat out of
the loop.

Thanks,

Paul

---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

RE: Tomcat status 75 error

["Caldarale, Charles R" <Ch...@unisys.com>]

> From: Martin Gainty [mailto:mgainty@hotmail.com] 
> Subject: RE: Tomcat status 75 error
> 
> Good CallIf you can upgrade to TC 6 you can use this shared 
> threadPool (being mindful that you need to configure before Connector)
> http://tomcat.apache.org/tomcat-6.0-doc/config/executor.html

Don't do that.  The perfomance of the executor does not measure up (nor
does using an executor have any bearing on the problem):
http://marc.info/?l=tomcat-user&m=118002259402411&w=2

 - Chuck


THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
MATERIAL and is thus for use only by the intended recipient. If you
received this in error, please contact the sender and delete the e-mail
and its attachments from all computers.

---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

RE: Tomcat status 75 error

[Martin Gainty <mg...@hotmail.com>]

Good CallIf you can upgrade to TC 6 you can use this shared threadPool (being mindful that you need to configure before Connector)
http://tomcat.apache.org/tomcat-6.0-doc/config/executor.html
In any event the statement of taking tomcat out of the loop requires clarification
Martin______________________________________________Disclaimer and confidentiality noteEverything in this e-mail and any attachments relates to the official business of Sender. This transmission is of a confidential nature and Sender does not endorse distribution to any party other than intended recipient. Sender does not necessarily endorse content contained within this transmission.> Date: Wed, 2 Jan 2008 19:24:18 +0100> From: rainer.jung@kippdata.de> To: users@tomcat.apache.org> Subject: Re: Tomcat status 75 error> > Hi Paul,> > Paul Beattie schrieb:> > Hi,> > > > I have a series of relatively high transaction volume tomcat servers> > which are sporadically hitting thread issues. We run with with tomcat> > 4.1.27 at the current time. The behaviour we notice is after several> > days of around one hundred thousand transactions our tomcat instances> > hang with the catalina.out message of:> > > > SEVERE: All threads are busy, waiting. Please increase maxThreads or> > check the servlet status75 75> > Stopping service Tomcat-Standalone> > > > I've done a quite a lot of reading on this and in some cases there is an> > indication that moving to a later tomcat version alleviates the issue. > > However in many others it indicates that this problem is due solely to> > the code whereby the threads are not exiting cleanly. We have a classic> > 3-tier architecture and we can see the db connections closing. I have> > tired reproducing this error on a tomcat 5 environment and I get a> > similar but not identical error:> > > > org.apache.tomcat.util.threads.ThreadPool logFull> > SEVERE: All threads (15) are currently busy, waiting. Increase> > maxThreads (15) or check the servlet status> > If all threads are busy, you can do a thread dump (kill -QUIT) for the> Java process to find out, what they are actually doing. The result goes> into catalina.out. A thread dump contains the full stack for each thread> in the JVM. The Java process will proceed with the usual work after> writing out the stacks. If all threads are busy, either the load is> higher than expected, or the things your threads are doing are running> slower than expected. In the stack you could see e.g. if threads are> waiting for database results etc.> > The above message also indicates, that you have only 15 threads> configured. It's quite possible, that this is a somewhat small thread pool.> > B.T.W.: 4.1.27 is a little old. If you need to stick with 4.1, there is> 4.1.36 already. I'm not saying, that an update will help, but it could> be a good general cleanup thing.> > > The tomcat install continues to function correctly after this has> > occurred. What I would like to know is do the 2 error messages> > correspond to the same underlying issue. I strongly suspect it is an> > application code issue so I would like to be able to count Tomcat out of> > the loop.> > > > Thanks,> > > > Paul> > Regards,> > Rainer> > ---------------------------------------------------------------------> To start a new topic, e-mail: users@tomcat.apache.org> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org> For additional commands, e-mail: users-help@tomcat.apache.org> 
_________________________________________________________________
Get the power of Windows + Web with the new Windows Live.
http://www.windowslive.com?ocid=TXT_TAGHM_Wave2_powerofwindows_122007

Re: Tomcat status 75 error

[Rainer Jung <ra...@kippdata.de>]

Hi Paul,

Paul Beattie schrieb:
> Hi,
> 
> I have a series of relatively high transaction volume tomcat servers
> which are sporadically hitting thread issues.  We run with with tomcat
> 4.1.27 at the current time.  The behaviour we notice is after several
> days of around one hundred thousand transactions our tomcat instances
> hang with the catalina.out message of:
> 
> SEVERE: All threads are busy, waiting. Please increase maxThreads or
> check the servlet status75 75
> Stopping service Tomcat-Standalone
> 
> I've done a quite a lot of reading on this and in some cases there is an
> indication that moving to a later tomcat version alleviates the issue. 
> However in many others it indicates that this problem is due solely to
> the code whereby the threads are not exiting cleanly.  We have a classic
> 3-tier architecture and we can see the db connections closing.  I have
> tired reproducing this error on a tomcat 5 environment and I get a
> similar but not identical error:
> 
> org.apache.tomcat.util.threads.ThreadPool logFull
> SEVERE: All threads (15) are currently busy, waiting. Increase
> maxThreads (15) or check the servlet status

If all threads are busy, you can do a thread dump (kill -QUIT) for the
Java process to find out, what they are actually doing. The result goes
into catalina.out. A thread dump contains the full stack for each thread
in the JVM. The Java process will proceed with the usual work after
writing out the stacks. If all threads are busy, either the load is
higher than expected, or the things your threads are doing are running
slower than expected. In the stack you could see e.g. if threads are
waiting for database results etc.

The above message also indicates, that you have only 15 threads
configured. It's quite possible, that this is a somewhat small thread pool.

B.T.W.: 4.1.27 is a little old. If you need to stick with 4.1, there is
4.1.36 already. I'm not saying, that an update will help, but it could
be a good general cleanup thing.

> The tomcat install continues to function correctly after this has
> occurred.   What I would like to know is do the 2 error messages
> correspond to the same underlying issue.  I strongly suspect it is an
> application code issue so I would like to be able to count Tomcat out of
> the loop.
> 
> Thanks,
> 
> Paul

Regards,

Rainer

---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org