You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Andrew McDonald <an...@hotcopper.com.au> on 2001/04/23 07:56:04 UTC
2 Critical Problems with Tomcat 3.2.x
Hi there,
First I'd like to thank the developers for a great product. Tomcat is a
nice piece of software. And like all great OSS, very easy to raise a
purchase order for ;) Thanks guys.
Unfortunately I have run into a couple of serious show-stoppers when
deploying TC in production.
A bit of background:
We have been running various incarnations of JServ and Tomcat over the
past 12 months or so.
We have 3 Application Servers: Two Compaq Deskpro PII/450s and one
PIII/500 with 128MB Ram each. They are running Linux, kernel is 2.2.19,
glibc-2.1.3. We are currently running various incarnations of TC3.2.x
We run a Stock Trading Forum, which is made up of Servlets and JSPs.
You can see it at http://www.hotcopper.com.au if you're interested at
all. We are one of the most hit websites in Australia according to
www.top100.com.au.
We have tested Tomcat with the Solaris 1.3 JVM, IBM 1.3 JVM and
Blackdown 1.3 JVM (FCS). I'm happiest with Blackdown. It *seems* to
have the best performance.
On to the problems:
1. We dubbed this one "The Apache White Screen" problem. We connected
Tomcat to various versions of Apache <=1.14 in the standard way
(mod_jk.so) Symptom: People using Internet Explorer to browse our site
will intermittently get a white/ blank screen instead of results from
our application. When this happens, we believe the request doesn't even
enter the java code, so it may be a problem with the connector. I seem
to recall (sorry this was about 8 months ago) Apache returning a 304 or
a 302. I haven't re-tested this in a while, I was just wondering
whether people were aware of this problem in the past and had already
looked into it and possibly fixed it.
Our "solution" was to run Tomcat standalone, which seemed to solve this
issue.
2. We dubbed this one "The 132 Thread" Problem, even though it doesn't
always happen with 132 threads; sometimes 128, 130, or if using the
Blackdown JVM, 134. (wierd). We use NATIVE threads. Symptom: Tomcat
stops responding to http requests after a certain number (~132) of
threads is reached. When this happens it will either deadlock or
segfault. During this time the box is fine, and to solve it temporarily
I just need to kill all the java processes and restart Tomcat. This
problem occurs regardless of JVM and the version of the Linux Kernel
doesn't seem to make a difference. It happens once or twice a week.
Assistance with either of these problems would be greatly appreciated.
Number 2 is the really important one but information regarding number 1
would be great as well.
Of course if you need more information please get back to me.
Oh, and btw, yes I realise that running Compaq desktops in production
isn't the best idea ;) That will be attended to.
Thanks in advance
Cheers
--
Andrew McDonald "We cannot reason ourselves out of our basic
Systems Administration irrationality. All we can do is learn the art
HOTcopper Australia of being irrational in a reasonable way."
http://www.hotcopper.com.au - Huxley
Re: 2 Critical Problems with Tomcat 3.2.x
Posted by Andrew McDonald <an...@hotcopper.com.au>.
Indeed. The documentation I've found doesn't seem to clarify this :( but I'll
be testing and posting results here on the list.
Cheers
AndyM
Jeff Kilbride wrote:
> Hi Andy,
>
> Sorry, I was assuming you were using Tomcat with Apache. I'm not sure if you
> can configure thread pooling the same way in standalone mode or not. The
> configurations I was talking about were for the PoolTCPConnector class, I
> believe. It's worth a look, but I'm not sure it will apply in your case.
>
> Thanks,
> --jeff
>
> ----- Original Message -----
> From: "Andrew McDonald" <an...@hotcopper.com.au>
> To: <to...@jakarta.apache.org>
> Cc: <bi...@spotlife.com>
> Sent: Friday, April 26, 2024 10:23 PM
> Subject: Re: 2 Critical Problems with Tomcat 3.2.x
>
> > Thank you very much for the advice Jeff. I'll have a go at that. We
> *are*
> > using a thread pool
> > to run threads that make connections to our database but we have Tomcat in
> > standalone mode.
> > The Tomcat documentation describes setting the max_threads, etc of the
> ajpv12
> > connector but
> > i'd suppose the other connectors would have the same/similar syntax.
> >
> > Cheers
> >
> > AndyM
> >
> >
> > Jeff Kilbride wrote:
> >
> > > I saw something on the dev list about Tomcat's default thread limit and
> how,
> > > under load, Apache processes could get "stuck" waiting for a Tomcat
> thread
> > > to become available.
> > >
> > > Have you changed any of the threading defaults in your server.xml file?
> > > Check out the Tomcat Users Guide here:
> > >
> > > http://jakarta.apache.org/tomcat/jakarta-tomcat/src/doc/index.html
> > >
> > > and do a find for the phrase "max_threads". It shows you how to
> configure
> > > min, max, and spare threads in Tomcat's thread pool and talks about the
> > > default values. If you up these values, maybe that will help.
> > >
> > > Thanks,
> > > --jeff
> > >
> > > ----- Original Message -----
> > > From: "Bill Graham" <bi...@spotlife.com>
> > > To: <to...@jakarta.apache.org>
> > > Sent: Thursday, April 26, 2001 11:50 AM
> > > Subject: RE: 2 Critical Problems with Tomcat 3.2.x
> > >
> > > > I've also been experienceing a lot of the Number 2 problem described
> > > > where Tomcat stops responding to http requests and I can't figure out
> > > > what's causing it. I've been able to reproduce the problem with
> multiple
> > > > VMs on both Linux and Solaris machines. I applied the patch for Tomcat
> > > > bug #1006 and the hanging still occurs.
> > > >
> > > > I haven't come across any responses to Andy's first post so I thought
> > > > I'd ask again. Anyone have any idea what's causing this?
> > > >
> > > > thanks,
> > > > Bill
> > > >
> > > > -----Original Message-----
> > > > From: andym@cleopatra.rubiconproject.com
> > > > [mailto:andym@cleopatra.rubiconproject.com]On Behalf Of Andrew
> McDonald
> > > > Sent: Sunday, April 22, 2001 10:56 PM
> > > > To: tomcat-user@jakarta.apache.org
> > > > Subject: 2 Critical Problems with Tomcat 3.2.x
> > > >
> > > >
> > > > Hi there,
> > > >
> > > > First I'd like to thank the developers for a great product. Tomcat is
> a
> > > > nice piece of software. And like all great OSS, very easy to raise a
> > > > purchase order for ;) Thanks guys.
> > > > Unfortunately I have run into a couple of serious show-stoppers when
> > > > deploying TC in production.
> > > > A bit of background:
> > > > We have been running various incarnations of JServ and Tomcat over the
> > > > past 12 months or so.
> > > > We have 3 Application Servers: Two Compaq Deskpro PII/450s and one
> > > > PIII/500 with 128MB Ram each. They are running Linux, kernel is
> 2.2.19,
> > > > glibc-2.1.3. We are currently running various incarnations of TC3.2.x
> > > > We run a Stock Trading Forum, which is made up of Servlets and JSPs.
> > > > You can see it at http://www.hotcopper.com.au if you're interested at
> > > > all. We are one of the most hit websites in Australia according to
> > > > www.top100.com.au.
> > > >
> > > > We have tested Tomcat with the Solaris 1.3 JVM, IBM 1.3 JVM and
> > > > Blackdown 1.3 JVM (FCS). I'm happiest with Blackdown. It *seems* to
> > > > have the best performance.
> > > >
> > > > On to the problems:
> > > > 1. We dubbed this one "The Apache White Screen" problem. We
> connected
> > > > Tomcat to various versions of Apache <=1.14 in the standard way
> > > > (mod_jk.so) Symptom: People using Internet Explorer to browse our
> site
> > > > will intermittently get a white/ blank screen instead of results from
> > > > our application. When this happens, we believe the request doesn't
> even
> > > > enter the java code, so it may be a problem with the connector. I
> seem
> > > > to recall (sorry this was about 8 months ago) Apache returning a 304
> or
> > > > a 302. I haven't re-tested this in a while, I was just wondering
> > > > whether people were aware of this problem in the past and had already
> > > > looked into it and possibly fixed it.
> > > > Our "solution" was to run Tomcat standalone, which seemed to solve
> this
> > > > issue.
> > > >
> > > > 2. We dubbed this one "The 132 Thread" Problem, even though it
> doesn't
> > > > always happen with 132 threads; sometimes 128, 130, or if using the
> > > > Blackdown JVM, 134. (wierd). We use NATIVE threads. Symptom:
> Tomcat
> > > > stops responding to http requests after a certain number (~132) of
> > > > threads is reached. When this happens it will either deadlock or
> > > > segfault. During this time the box is fine, and to solve it
> temporarily
> > > > I just need to kill all the java processes and restart Tomcat. This
> > > > problem occurs regardless of JVM and the version of the Linux Kernel
> > > > doesn't seem to make a difference. It happens once or twice a week.
> > > >
> > > > Assistance with either of these problems would be greatly appreciated.
> > > > Number 2 is the really important one but information regarding number
> 1
> > > > would be great as well.
> > > >
> > > > Of course if you need more information please get back to me.
> > > >
> > > > Oh, and btw, yes I realise that running Compaq desktops in production
> > > > isn't the best idea ;) That will be attended to.
> > > >
> > > > Thanks in advance
> > > >
> > > > Cheers
> > > >
> > > > --
> > > > Andrew McDonald "We cannot reason ourselves out of our
> > > basic
> > > > Systems Administration irrationality. All we can do is learn
> the
> > > art
> > > > HOTcopper Australia of being irrational in a reasonable
> way."
> > > > http://www.hotcopper.com.au - Huxley
> > > >
> > > >
> >
Re: 2 Critical Problems with Tomcat 3.2.x
Posted by Andrew McDonald <an...@hotcopper.com.au>.
Thank you very much for the advice Jeff. I'll have a go at that. We *are*
using a thread pool
to run threads that make connections to our database but we have Tomcat in
standalone mode.
The Tomcat documentation describes setting the max_threads, etc of the ajpv12
connector but
i'd suppose the other connectors would have the same/similar syntax.
Cheers
AndyM
Jeff Kilbride wrote:
> I saw something on the dev list about Tomcat's default thread limit and how,
> under load, Apache processes could get "stuck" waiting for a Tomcat thread
> to become available.
>
> Have you changed any of the threading defaults in your server.xml file?
> Check out the Tomcat Users Guide here:
>
> http://jakarta.apache.org/tomcat/jakarta-tomcat/src/doc/index.html
>
> and do a find for the phrase "max_threads". It shows you how to configure
> min, max, and spare threads in Tomcat's thread pool and talks about the
> default values. If you up these values, maybe that will help.
>
> Thanks,
> --jeff
>
> ----- Original Message -----
> From: "Bill Graham" <bi...@spotlife.com>
> To: <to...@jakarta.apache.org>
> Sent: Thursday, April 26, 2001 11:50 AM
> Subject: RE: 2 Critical Problems with Tomcat 3.2.x
>
> > I've also been experienceing a lot of the Number 2 problem described
> > where Tomcat stops responding to http requests and I can't figure out
> > what's causing it. I've been able to reproduce the problem with multiple
> > VMs on both Linux and Solaris machines. I applied the patch for Tomcat
> > bug #1006 and the hanging still occurs.
> >
> > I haven't come across any responses to Andy's first post so I thought
> > I'd ask again. Anyone have any idea what's causing this?
> >
> > thanks,
> > Bill
> >
> > -----Original Message-----
> > From: andym@cleopatra.rubiconproject.com
> > [mailto:andym@cleopatra.rubiconproject.com]On Behalf Of Andrew McDonald
> > Sent: Sunday, April 22, 2001 10:56 PM
> > To: tomcat-user@jakarta.apache.org
> > Subject: 2 Critical Problems with Tomcat 3.2.x
> >
> >
> > Hi there,
> >
> > First I'd like to thank the developers for a great product. Tomcat is a
> > nice piece of software. And like all great OSS, very easy to raise a
> > purchase order for ;) Thanks guys.
> > Unfortunately I have run into a couple of serious show-stoppers when
> > deploying TC in production.
> > A bit of background:
> > We have been running various incarnations of JServ and Tomcat over the
> > past 12 months or so.
> > We have 3 Application Servers: Two Compaq Deskpro PII/450s and one
> > PIII/500 with 128MB Ram each. They are running Linux, kernel is 2.2.19,
> > glibc-2.1.3. We are currently running various incarnations of TC3.2.x
> > We run a Stock Trading Forum, which is made up of Servlets and JSPs.
> > You can see it at http://www.hotcopper.com.au if you're interested at
> > all. We are one of the most hit websites in Australia according to
> > www.top100.com.au.
> >
> > We have tested Tomcat with the Solaris 1.3 JVM, IBM 1.3 JVM and
> > Blackdown 1.3 JVM (FCS). I'm happiest with Blackdown. It *seems* to
> > have the best performance.
> >
> > On to the problems:
> > 1. We dubbed this one "The Apache White Screen" problem. We connected
> > Tomcat to various versions of Apache <=1.14 in the standard way
> > (mod_jk.so) Symptom: People using Internet Explorer to browse our site
> > will intermittently get a white/ blank screen instead of results from
> > our application. When this happens, we believe the request doesn't even
> > enter the java code, so it may be a problem with the connector. I seem
> > to recall (sorry this was about 8 months ago) Apache returning a 304 or
> > a 302. I haven't re-tested this in a while, I was just wondering
> > whether people were aware of this problem in the past and had already
> > looked into it and possibly fixed it.
> > Our "solution" was to run Tomcat standalone, which seemed to solve this
> > issue.
> >
> > 2. We dubbed this one "The 132 Thread" Problem, even though it doesn't
> > always happen with 132 threads; sometimes 128, 130, or if using the
> > Blackdown JVM, 134. (wierd). We use NATIVE threads. Symptom: Tomcat
> > stops responding to http requests after a certain number (~132) of
> > threads is reached. When this happens it will either deadlock or
> > segfault. During this time the box is fine, and to solve it temporarily
> > I just need to kill all the java processes and restart Tomcat. This
> > problem occurs regardless of JVM and the version of the Linux Kernel
> > doesn't seem to make a difference. It happens once or twice a week.
> >
> > Assistance with either of these problems would be greatly appreciated.
> > Number 2 is the really important one but information regarding number 1
> > would be great as well.
> >
> > Of course if you need more information please get back to me.
> >
> > Oh, and btw, yes I realise that running Compaq desktops in production
> > isn't the best idea ;) That will be attended to.
> >
> > Thanks in advance
> >
> > Cheers
> >
> > --
> > Andrew McDonald "We cannot reason ourselves out of our
> basic
> > Systems Administration irrationality. All we can do is learn the
> art
> > HOTcopper Australia of being irrational in a reasonable way."
> > http://www.hotcopper.com.au - Huxley
> >
> >
Re: 2 Critical Problems with Tomcat 3.2.x
Posted by Jeff Kilbride <je...@kilbride.com>.
I saw something on the dev list about Tomcat's default thread limit and how,
under load, Apache processes could get "stuck" waiting for a Tomcat thread
to become available.
Have you changed any of the threading defaults in your server.xml file?
Check out the Tomcat Users Guide here:
http://jakarta.apache.org/tomcat/jakarta-tomcat/src/doc/index.html
and do a find for the phrase "max_threads". It shows you how to configure
min, max, and spare threads in Tomcat's thread pool and talks about the
default values. If you up these values, maybe that will help.
Thanks,
--jeff
----- Original Message -----
From: "Bill Graham" <bi...@spotlife.com>
To: <to...@jakarta.apache.org>
Sent: Thursday, April 26, 2001 11:50 AM
Subject: RE: 2 Critical Problems with Tomcat 3.2.x
> I've also been experienceing a lot of the Number 2 problem described
> where Tomcat stops responding to http requests and I can't figure out
> what's causing it. I've been able to reproduce the problem with multiple
> VMs on both Linux and Solaris machines. I applied the patch for Tomcat
> bug #1006 and the hanging still occurs.
>
> I haven't come across any responses to Andy's first post so I thought
> I'd ask again. Anyone have any idea what's causing this?
>
> thanks,
> Bill
>
> -----Original Message-----
> From: andym@cleopatra.rubiconproject.com
> [mailto:andym@cleopatra.rubiconproject.com]On Behalf Of Andrew McDonald
> Sent: Sunday, April 22, 2001 10:56 PM
> To: tomcat-user@jakarta.apache.org
> Subject: 2 Critical Problems with Tomcat 3.2.x
>
>
> Hi there,
>
> First I'd like to thank the developers for a great product. Tomcat is a
> nice piece of software. And like all great OSS, very easy to raise a
> purchase order for ;) Thanks guys.
> Unfortunately I have run into a couple of serious show-stoppers when
> deploying TC in production.
> A bit of background:
> We have been running various incarnations of JServ and Tomcat over the
> past 12 months or so.
> We have 3 Application Servers: Two Compaq Deskpro PII/450s and one
> PIII/500 with 128MB Ram each. They are running Linux, kernel is 2.2.19,
> glibc-2.1.3. We are currently running various incarnations of TC3.2.x
> We run a Stock Trading Forum, which is made up of Servlets and JSPs.
> You can see it at http://www.hotcopper.com.au if you're interested at
> all. We are one of the most hit websites in Australia according to
> www.top100.com.au.
>
> We have tested Tomcat with the Solaris 1.3 JVM, IBM 1.3 JVM and
> Blackdown 1.3 JVM (FCS). I'm happiest with Blackdown. It *seems* to
> have the best performance.
>
> On to the problems:
> 1. We dubbed this one "The Apache White Screen" problem. We connected
> Tomcat to various versions of Apache <=1.14 in the standard way
> (mod_jk.so) Symptom: People using Internet Explorer to browse our site
> will intermittently get a white/ blank screen instead of results from
> our application. When this happens, we believe the request doesn't even
> enter the java code, so it may be a problem with the connector. I seem
> to recall (sorry this was about 8 months ago) Apache returning a 304 or
> a 302. I haven't re-tested this in a while, I was just wondering
> whether people were aware of this problem in the past and had already
> looked into it and possibly fixed it.
> Our "solution" was to run Tomcat standalone, which seemed to solve this
> issue.
>
> 2. We dubbed this one "The 132 Thread" Problem, even though it doesn't
> always happen with 132 threads; sometimes 128, 130, or if using the
> Blackdown JVM, 134. (wierd). We use NATIVE threads. Symptom: Tomcat
> stops responding to http requests after a certain number (~132) of
> threads is reached. When this happens it will either deadlock or
> segfault. During this time the box is fine, and to solve it temporarily
> I just need to kill all the java processes and restart Tomcat. This
> problem occurs regardless of JVM and the version of the Linux Kernel
> doesn't seem to make a difference. It happens once or twice a week.
>
> Assistance with either of these problems would be greatly appreciated.
> Number 2 is the really important one but information regarding number 1
> would be great as well.
>
> Of course if you need more information please get back to me.
>
> Oh, and btw, yes I realise that running Compaq desktops in production
> isn't the best idea ;) That will be attended to.
>
> Thanks in advance
>
> Cheers
>
> --
> Andrew McDonald "We cannot reason ourselves out of our
basic
> Systems Administration irrationality. All we can do is learn the
art
> HOTcopper Australia of being irrational in a reasonable way."
> http://www.hotcopper.com.au - Huxley
>
>
Re: 2 Critical Problems with Tomcat 3.2.x
Posted by Andrew McDonald <an...@hotcopper.com.au>.
Thanks Bill for chasing this up, I really didn't want to repost. I was
thinking of posting to the tomcat-dev list wrt these problems but I believe
tomcat-user is the more appropriate list for discussing these sorts of issues.
I thought I better give a few more data points:
- Last night I set the logging to "FATAL" on the tomcat.log, etc. I read on
the tomcat-dev list that having it set on DEBUG is potentially dangerous: the
logger may not be able to keep up under extreme load.
- I was almost *convinced* that I was running across bug #1006 too! However
after installing tc 3.2.2b2 the problem remained.
- Jeff Kilbride suggested I try to cap the max_threads etc of the thread pool,
which may help. I will be trying this soon (but not today: Fridays are bad
days to modify production systems)
We need a solution to this and are willing to help in whatever way we can. We
are testing commercial alternatives such as JRun and BEA despite my
reluctance: Apart from these 2 critical problems I think Tomcat is great and I
love OSS projects such as this. I'm expected to get this fixed through
whatever means possible so any suggestions would be appreciated.
Cheers
AndyM
Bill Graham wrote:
> I've also been experienceing a lot of the Number 2 problem described
> where Tomcat stops responding to http requests and I can't figure out
> what's causing it. I've been able to reproduce the problem with multiple
> VMs on both Linux and Solaris machines. I applied the patch for Tomcat
> bug #1006 and the hanging still occurs.
>
> I haven't come across any responses to Andy's first post so I thought
> I'd ask again. Anyone have any idea what's causing this?
>
> thanks,
> Bill
>
> -----Original Message-----
> From: andym@cleopatra.rubiconproject.com
> [mailto:andym@cleopatra.rubiconproject.com]On Behalf Of Andrew McDonald
> Sent: Sunday, April 22, 2001 10:56 PM
> To: tomcat-user@jakarta.apache.org
> Subject: 2 Critical Problems with Tomcat 3.2.x
>
> Hi there,
>
> First I'd like to thank the developers for a great product. Tomcat is a
> nice piece of software. And like all great OSS, very easy to raise a
> purchase order for ;) Thanks guys.
> Unfortunately I have run into a couple of serious show-stoppers when
> deploying TC in production.
> A bit of background:
> We have been running various incarnations of JServ and Tomcat over the
> past 12 months or so.
> We have 3 Application Servers: Two Compaq Deskpro PII/450s and one
> PIII/500 with 128MB Ram each. They are running Linux, kernel is 2.2.19,
> glibc-2.1.3. We are currently running various incarnations of TC3.2.x
> We run a Stock Trading Forum, which is made up of Servlets and JSPs.
> You can see it at http://www.hotcopper.com.au if you're interested at
> all. We are one of the most hit websites in Australia according to
> www.top100.com.au.
>
> We have tested Tomcat with the Solaris 1.3 JVM, IBM 1.3 JVM and
> Blackdown 1.3 JVM (FCS). I'm happiest with Blackdown. It *seems* to
> have the best performance.
>
> On to the problems:
> 1. We dubbed this one "The Apache White Screen" problem. We connected
> Tomcat to various versions of Apache <=1.14 in the standard way
> (mod_jk.so) Symptom: People using Internet Explorer to browse our site
> will intermittently get a white/ blank screen instead of results from
> our application. When this happens, we believe the request doesn't even
> enter the java code, so it may be a problem with the connector. I seem
> to recall (sorry this was about 8 months ago) Apache returning a 304 or
> a 302. I haven't re-tested this in a while, I was just wondering
> whether people were aware of this problem in the past and had already
> looked into it and possibly fixed it.
> Our "solution" was to run Tomcat standalone, which seemed to solve this
> issue.
>
> 2. We dubbed this one "The 132 Thread" Problem, even though it doesn't
> always happen with 132 threads; sometimes 128, 130, or if using the
> Blackdown JVM, 134. (wierd). We use NATIVE threads. Symptom: Tomcat
> stops responding to http requests after a certain number (~132) of
> threads is reached. When this happens it will either deadlock or
> segfault. During this time the box is fine, and to solve it temporarily
> I just need to kill all the java processes and restart Tomcat. This
> problem occurs regardless of JVM and the version of the Linux Kernel
> doesn't seem to make a difference. It happens once or twice a week.
>
> Assistance with either of these problems would be greatly appreciated.
> Number 2 is the really important one but information regarding number 1
> would be great as well.
>
> Of course if you need more information please get back to me.
>
> Oh, and btw, yes I realise that running Compaq desktops in production
> isn't the best idea ;) That will be attended to.
>
> Thanks in advance
>
> Cheers
>
> --
> Andrew McDonald "We cannot reason ourselves out of our basic
> Systems Administration irrationality. All we can do is learn the art
> HOTcopper Australia of being irrational in a reasonable way."
> http://www.hotcopper.com.au - Huxley
RE: 2 Critical Problems with Tomcat 3.2.x
Posted by Bill Graham <bi...@spotlife.com>.
I've also been experienceing a lot of the Number 2 problem described
where Tomcat stops responding to http requests and I can't figure out
what's causing it. I've been able to reproduce the problem with multiple
VMs on both Linux and Solaris machines. I applied the patch for Tomcat
bug #1006 and the hanging still occurs.
I haven't come across any responses to Andy's first post so I thought
I'd ask again. Anyone have any idea what's causing this?
thanks,
Bill
-----Original Message-----
From: andym@cleopatra.rubiconproject.com
[mailto:andym@cleopatra.rubiconproject.com]On Behalf Of Andrew McDonald
Sent: Sunday, April 22, 2001 10:56 PM
To: tomcat-user@jakarta.apache.org
Subject: 2 Critical Problems with Tomcat 3.2.x
Hi there,
First I'd like to thank the developers for a great product. Tomcat is a
nice piece of software. And like all great OSS, very easy to raise a
purchase order for ;) Thanks guys.
Unfortunately I have run into a couple of serious show-stoppers when
deploying TC in production.
A bit of background:
We have been running various incarnations of JServ and Tomcat over the
past 12 months or so.
We have 3 Application Servers: Two Compaq Deskpro PII/450s and one
PIII/500 with 128MB Ram each. They are running Linux, kernel is 2.2.19,
glibc-2.1.3. We are currently running various incarnations of TC3.2.x
We run a Stock Trading Forum, which is made up of Servlets and JSPs.
You can see it at http://www.hotcopper.com.au if you're interested at
all. We are one of the most hit websites in Australia according to
www.top100.com.au.
We have tested Tomcat with the Solaris 1.3 JVM, IBM 1.3 JVM and
Blackdown 1.3 JVM (FCS). I'm happiest with Blackdown. It *seems* to
have the best performance.
On to the problems:
1. We dubbed this one "The Apache White Screen" problem. We connected
Tomcat to various versions of Apache <=1.14 in the standard way
(mod_jk.so) Symptom: People using Internet Explorer to browse our site
will intermittently get a white/ blank screen instead of results from
our application. When this happens, we believe the request doesn't even
enter the java code, so it may be a problem with the connector. I seem
to recall (sorry this was about 8 months ago) Apache returning a 304 or
a 302. I haven't re-tested this in a while, I was just wondering
whether people were aware of this problem in the past and had already
looked into it and possibly fixed it.
Our "solution" was to run Tomcat standalone, which seemed to solve this
issue.
2. We dubbed this one "The 132 Thread" Problem, even though it doesn't
always happen with 132 threads; sometimes 128, 130, or if using the
Blackdown JVM, 134. (wierd). We use NATIVE threads. Symptom: Tomcat
stops responding to http requests after a certain number (~132) of
threads is reached. When this happens it will either deadlock or
segfault. During this time the box is fine, and to solve it temporarily
I just need to kill all the java processes and restart Tomcat. This
problem occurs regardless of JVM and the version of the Linux Kernel
doesn't seem to make a difference. It happens once or twice a week.
Assistance with either of these problems would be greatly appreciated.
Number 2 is the really important one but information regarding number 1
would be great as well.
Of course if you need more information please get back to me.
Oh, and btw, yes I realise that running Compaq desktops in production
isn't the best idea ;) That will be attended to.
Thanks in advance
Cheers
--
Andrew McDonald "We cannot reason ourselves out of our basic
Systems Administration irrationality. All we can do is learn the art
HOTcopper Australia of being irrational in a reasonable way."
http://www.hotcopper.com.au - Huxley