Deadlock problem

["EXT-Mezey, Peter" <Pe...@PSS.Boeing.com>]

I have tomcat 3.2.1 with Apache 1.3.12 running on a Solaris (2.6) box.  Everything works just fine for normal use.  However, when I crank up a Verity search engine (vspider) to walk through a document bundle of about 4,500 JSP pages I start to run into trouble.  Since every document in the bundle is a JSP page, the vspider program is generating a considerable load on the Apache/Tomcat installation.  After quite a bit of successful activity (say about a half hour or more of processing), everything seems to just freeze up.  Looking at top, you can see that the system is at a 100% iowait state.  Prior to that happening, you can see the tomcat process slowly allocating more and more memory as it digests all the requests.  When it gets to about 200MB, that's when the system goes into 100% iowait state and tomcat basically stops.

I have increased the memory to tomcat via -Xms and -Xmx flags to 1GB.  I know it's not running out of memory because I don't see that error message.  I've tried tinkering with the PoolTcpConnector settings in server.xml to increase the number of threads to 200.  That seemed to make it run faster, but still the deadlock problem.  One bit of confusion here is the tomcat.properties file.  It has a "pool=false" setting in there along with a comment that says the Thread pooling may cause deadlock problems.  I tried it with both true and false settings here, but to no avail.  Oh, one other piece of information: I'm using the J2SDK 1.3.0.  Any help, hints or pointers would be greatly appreciated.

thanks,

peter

Re: Deadlock problem

[Chris Janicki <Ja...@ia-inc.com>]

Check out bug# 4293268 on Sun's Java bug parade.  It described a 1.3 
threading bug that involves lost locks.  I've had similar deadlock 
problem on a different app (not Tomcat) that went away when I downgraded 
to Java 1.2.2.5. It is an rare problem that most occurs when a system is 
under load.  

By the way, please cast your vote for this bug (if you're on the Java Dev 
Connection).  Java 1.3 is a nice improvement, but the thread/lock bug is 
a killer.

Chris



>>>>>>>>>>>>>>>>>> Original Message <<<<<<<<<<<<<<<<<<

On 2/6/01, 4:58:25 PM, "EXT-Mezey, Peter" <Pe...@PSS.Boeing.com> 
wrote regarding Deadlock problem:


> I have tomcat 3.2.1 with Apache 1.3.12 running on a Solaris (2.6) box.  
Everything works just fine for normal use.  However, when I crank up a 
Verity search engine (vspider) to walk through a document bundle of about 
4,500 JSP pages I start to run into trouble.  Since every document in the 
bundle is a JSP page, the vspider program is generating a considerable load 
on the Apache/Tomcat installation.  After quite a bit of successful 
activity (say about a half hour or more of processing), everything seems to 
just freeze up.  Looking at top, you can see that the system is at a 100% 
iowait state.  Prior to that happening, you can see the tomcat process 
slowly allocating more and more memory as it digests all the requests.  
When it gets to about 200MB, that's when the system goes into 100% iowait 
state and tomcat basically stops.

> I have increased the memory to tomcat via -Xms and -Xmx flags to 1GB.  I 
know it's not running out of memory because I don't see that error message. 
 I've tried tinkering with the PoolTcpConnector settings in server.xml to 
increase the number of threads to 200.  That seemed to make it run faster, 
but still the deadlock problem.  One bit of confusion here is the 
tomcat.properties file.  It has a "pool=false" setting in there along with 
a comment that says the Thread pooling may cause deadlock problems.  I 
tried it with both true and false settings here, but to no avail.  Oh, one 
other piece of information: I'm using the J2SDK 1.3.0.  Any help, hints or 
pointers would be greatly appreciated.

> thanks,

> peter

Re: Deadlock problem

[Kurt Bernhard Pruenner <le...@gmx.at>]

"EXT-Mezey, Peter" wrote:
[Memory gobbled up fast by a search engine walking over JSP pages]

Well, if it's the same problem we had here, I'd say it's the spider
creating a new session with every request as it doesn't send back the
session-id cookie; and since it does this much faster than the sessions
will time out (by default, they expire after 30 minutes), the JVM's
memory will be used up by oodles of session objects that can't be
garbage collected yet... :(

We solved this problem by stopping our spider from traversing JSP pages,
but that won't help you, of course...

-- 
Kurt Bernhard Pruenner --- Haendelstrasse 17 --- 4020 Linz --- Austria
Music: http://www.mp3.com/Leak --- Work: http://www.ssw.uni-linz.ac.at
.......It might be written "Mindfuck", but it's spelt "L-A-I-N".......
np: Vladislav Delay - Huone (Multila)