Re: Tomcat Thread utilization growing very large causing a hung tomcat instance

[Peter Crowther <pe...@melandra.com>]

What happens if you *reduce* the allocated heap size - do you actually need
that much Java heap?  Native threads are created outside of the Java heap.
You have 2G for (4 * 400) = 1600 threads plus the OS, or about 1 Mbyte per
thread (this assumes no swap space).  That may not be enough.

- Peter

On 3 August 2010 15:21, Tomcat Novice <to...@gmail.com> wrote:

> Hi,
>
> I hope someone can help me. I am new to tomcat and I have been encountering
> issues where a particular tomcat instance/s are having large thread count
> during load testing and results in a hung state. The error shown on the
> browser is: java.lang.OutOfMemoryError: unable to create new native thread
> (stack trace attached)
>
> Environment:
>
> OS: Windows 2003 64 bit
> JDK 1.5
> 8GB Total Machine RAM
> 4 Tomcat 5.5 instances running as a service on one Physical Server (each
> Tomcat configured to use 1.5GB of Heap) 4 instances x 1.5GB = 6GB of Heap
> 4 Identical servers configured the same way and load balanced
> IIS Web Server front end (AJP Port configured) no maxThreads property is
> set on server.xml for AJP port declaration. Per documentation (
> http://tomcat.apache.org/tomcat-5.5-doc/config/ajp.html), if no value is
> set maxThreads are set to 200
>
> Not all 4 servers are experiencing this problem.
>
> 1. Any ideas how I can resolve/isolate this problem?
> 2. I know I need to do a thread dump but I am not familiar on how to
> analyze thread dumps from Tomcat. Can you point me to documententation on
> how I can analyze thread dumps from Tomcat?
> 3. Also, can someone confirm if the thread dumps from JDK 1.5 are the same
> as the Thread Dumps from a JDK 1.6? I seem to be getting different formats.
> Can someone just confirm?
> 4. If the maxThread is set to 200, why are the Tomcat instances utilizing
> up to 400 Threads (see screenshot)
>
> Regards,
>
> Tomcat Novice
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>

Re: Tomcat Thread utilization growing very large causing a hung tomcat instance

[Tomcat Novice <to...@gmail.com>]

Hi Peter,

Thanks for your resonse. I believe the issue is why is the Tomcat instance
utilizing around 400+ threads. It should be just utilizing only 200 threads
because of the default value for maxThreads? On the other servers, Thread
utilization is at 200-250 on normal operations which is still over 200.
The system was designed by the solution architech to use 1.5GB for heap and
we have observed the memory usage to go as high as 1.2GB per tomcat
instance. The proposal is to shutdown one of the Tomcat instances to free up
memory but I think the root issue is why we are reaching up to 400 threads.

We are planning another round of load tests but I am not familiar with how
to analyze thread dumps. Any ideas or documentation I can look into?

Also apologies if you got my first email multiple times. Might have been a
gmail issue as I only sent out one email to the list but was notified by one
of the people in the list of the multiple messsages.
Best regards,

On Tue, Aug 3, 2010 at 11:04 PM, Peter Crowther <peter.crowther@melandra.com
> wrote:

> What happens if you *reduce* the allocated heap size - do you actually need
> that much Java heap?  Native threads are created outside of the Java heap.
> You have 2G for (4 * 400) = 1600 threads plus the OS, or about 1 Mbyte per
> thread (this assumes no swap space).  That may not be enough.
>
> - Peter
>
> On 3 August 2010 15:21, Tomcat Novice <to...@gmail.com> wrote:
>
> > Hi,
> >
> > I hope someone can help me. I am new to tomcat and I have been
> encountering
> > issues where a particular tomcat instance/s are having large thread count
> > during load testing and results in a hung state. The error shown on the
> > browser is: java.lang.OutOfMemoryError: unable to create new native
> thread
> > (stack trace attached)
> >
> > Environment:
> >
> > OS: Windows 2003 64 bit
> > JDK 1.5
> > 8GB Total Machine RAM
> > 4 Tomcat 5.5 instances running as a service on one Physical Server (each
> > Tomcat configured to use 1.5GB of Heap) 4 instances x 1.5GB = 6GB of Heap
> > 4 Identical servers configured the same way and load balanced
> > IIS Web Server front end (AJP Port configured) no maxThreads property is
> > set on server.xml for AJP port declaration. Per documentation (
> > http://tomcat.apache.org/tomcat-5.5-doc/config/ajp.html), if no value is
> > set maxThreads are set to 200
> >
> > Not all 4 servers are experiencing this problem.
> >
> > 1. Any ideas how I can resolve/isolate this problem?
> > 2. I know I need to do a thread dump but I am not familiar on how to
> > analyze thread dumps from Tomcat. Can you point me to documententation on
> > how I can analyze thread dumps from Tomcat?
> > 3. Also, can someone confirm if the thread dumps from JDK 1.5 are the
> same
> > as the Thread Dumps from a JDK 1.6? I seem to be getting different
> formats.
> > Can someone just confirm?
> > 4. If the maxThread is set to 200, why are the Tomcat instances utilizing
> > up to 400 Threads (see screenshot)
> >
> > Regards,
> >
> > Tomcat Novice
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> > For additional commands, e-mail: users-help@tomcat.apache.org
> >
>