You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Tomasz Nowak <tn...@netventure.pl> on 2006/02/28 18:02:14 UTC
Sad: Tomcat 5.5.x crashes almost every single day.
Probably important:
===================
- 2x Xeon, 3 GB mem
- Linux 2.4.26
- Java 1.5.0_06-b05
- Apache Tomcat 5.5.15
- CATALINA_OPTS="\
-server \
-Djava.awt.headless=true \
-Dfile.encoding=UTF-8 \
-XX:MaxPermSize=256m \
-Xms1024m -Xmx1024m"
- <Connector port="8009" protocol="AJP/1.3"
maxThreads="200" minSpareThreads="25" maxSpareThreads="50"
backlog="20" connectionTimeout="10000"
enableLookups="true" redirectPort="8443" />
- mod_jk 1.2.15
- Apache Httpd 1.3.34
- 8 Tomcat vhosts
- 8 webapps
- each webapp runs on separate vhost
- each webapp is Apache Cocoon 2.1.7
- system load avarage: usually under 0.5
- http load: not more then 20-30 concurrent req
The pain: crashing Tomcat
======================
Everything works right until Tomcat suddenly crashes,
what happens almost every single day. Sometimes it crashes
with OOME / out of heap (heh, is 1GB not enought!?), but
somtimes NOTHING wrong is logged into the logs - it
just stops responding to requests (hangs).
When Tomcat stops responding to requests JK eats all
of 256 Apache HTTPd MaxClients so other httpd vhosts
stop responding also. Disaster!
Any ideas before abandoning Java technology? ;)
Default Tomcat logging facility seems to be useless to me.
j.u.l. logging.properties semantics looks like a programmers
joke made to users (!). And even with swallowOutput it logs
everything into catalina.out (?) I found none decent
log4j-tomcat-virtual-hosts-logging-manual in Tomcat docs
also. Is Tomcat a production ready reference j2se server
or just a "finish it yourself" proof of concept toy?
I've been crond-restarting Tomcat 4.1 on Java 1.4 every
night for more than a year, and now I have to restart
Tomcat 5.5.15 on Java 1.5 AND apache httpd also almost
every single day. From my -user point of view, Tomcat
doesn't change in a good direction. In fact in it is
probably changing in wrong direction - at least the
4.1.x logger element has been working 'out of the box'.
Come on! Am I the only one that has such problems with Tomcat?
--
T.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Sad: Tomcat 5.5.x crashes almost every single day.
Posted by Tomasz Nowak <tn...@netventure.pl>.
Nathan Smith <na...@labpro2000.com> wrote:
>
> Hi Tomasz,
>
> you mentioned below that you have a MaxPermSize of 256m.
I have not seen "OOME: PermGen" errors on that machine
but after having some of them on other machine with aditonal
10 webapps deployed I've increased it from default 64m
to 256m to be absolutly sure it wont hang on PermGen error.
--
T.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Sad: Tomcat 5.5.x crashes almost every single day.
Posted by Nathan Smith <na...@labpro2000.com>.
Hi Tomasz,
you mentioned below that you have a MaxPermSize of 256m. I would suggest
increasing this value to a much larger amount and even increasing your
-Xms and -Xmx values.
We had a problem with our application in a live environment where it
would regularly get errors like the following;
Exception java.lang.OutOfMemoryError: requested SIZE_FORMAT bytes for
compute_new_size. Out of swap space?
Possible causes:
- not enough swap space left, or
- kernel parameter MAXDSIZ is very small.
( 0) 0xc5bf0474 vm_exit_out_of_memory__FUlPCcb + 0x47c
[/opt/java1.5/jre/lib/PA_RISC2.0/server/libjvm.sl]
( 1) 0xc5e89d78 compute_new_size__9PSPermGenFUl + 0x150
[/opt/java1.5/jre/lib/PA_RISC2.0/server/libjvm.sl]
( 2) 0xc5e86024 invoke_no_policy__11PSMarkSweepSFPbb + 0x244
[/opt/java1.5/jre/lib/PA_RISC2.0/server/libjvm.sl]
( 3) 0xc5e8b814 invoke__10PSScavengeSFPb + 0x154
[/opt/java1.5/jre/lib/PA_RISC2.0/server/libjvm.sl]
( 4) 0xc5e51f4c failed_mem_allocate__20ParallelScavengeHeapFPbUlbT3 +
0x64 [/opt/java1.5/jre/lib/PA_RISC2.0/server/libjvm.sl]
( 5) 0xc5f6a78c doit__29VM_ParallelGCFailedAllocationFv + 0x54
[/opt/java1.5/jre/lib/PA_RISC2.0/server/libjvm.sl]
( 6) 0xc5f6a280 evaluate__12VM_OperationFv + 0xb0
[/opt/java1.5/jre/lib/PA_RISC2.0/server/libjvm.sl]
( 7) 0xc5f691e0 evaluate_operation__8VMThreadFP12VM_Operation + 0x48
[/opt/java1.5/jre/lib/PA_RISC2.0/server/libjvm.sl]
( 8) 0xc5f698a0 loop__8VMThreadFv + 0x558
[/opt/java1.5/jre/lib/PA_RISC2.0/server/libjvm.sl]
( 9) 0xc5f68f24 run__8VMThreadFv + 0xd4
[/opt/java1.5/jre/lib/PA_RISC2.0/server/libjvm.sl]
(10) 0xc5e39b20 _start__FPv + 0x148
[/opt/java1.5/jre/lib/PA_RISC2.0/server/libjvm.sl]
(11) 0xc004b168 __pthread_body + 0x44 [/usr/lib/libpthread.1]
(12) 0xc00549ec __pthread_start + 0x14 [/usr/lib/libpthread.1]
In the end it was the fact that the PermGen space was all used up and
JVM was trying to resize it. The PermGen may have become full because of
the number of class files and JSP class files that were loaded into
memory or other resources that are kept in PermGen by the JVM.
I would say with 8 applications running under Tomcat you would need to
make as much of that 3Gb available to the JVM to use. My 2 cents worth.
Cheers.
Tomasz Nowak wrote:
> Probably important:
> ===================
> - 2x Xeon, 3 GB mem
> - Linux 2.4.26
> - Java 1.5.0_06-b05
> - Apache Tomcat 5.5.15
> - CATALINA_OPTS="\
> -server \
> -Djava.awt.headless=true \
> -Dfile.encoding=UTF-8 \
> -XX:MaxPermSize=256m \
> -Xms1024m -Xmx1024m"
> - <Connector port="8009" protocol="AJP/1.3"
> maxThreads="200" minSpareThreads="25" maxSpareThreads="50"
> backlog="20" connectionTimeout="10000"
> enableLookups="true" redirectPort="8443" />
>
> - mod_jk 1.2.15
> - Apache Httpd 1.3.34
> - 8 Tomcat vhosts
> - 8 webapps
> - each webapp runs on separate vhost
> - each webapp is Apache Cocoon 2.1.7
> - system load avarage: usually under 0.5
> - http load: not more then 20-30 concurrent req
>
> The pain: crashing Tomcat
> ======================
>
> Everything works right until Tomcat suddenly crashes,
> what happens almost every single day. Sometimes it crashes
> with OOME / out of heap (heh, is 1GB not enought!?), but
> somtimes NOTHING wrong is logged into the logs - it
> just stops responding to requests (hangs).
>
> When Tomcat stops responding to requests JK eats all
> of 256 Apache HTTPd MaxClients so other httpd vhosts
> stop responding also. Disaster!
>
> Any ideas before abandoning Java technology? ;)
>
> Default Tomcat logging facility seems to be useless to me.
> j.u.l. logging.properties semantics looks like a programmers
> joke made to users (!). And even with swallowOutput it logs
> everything into catalina.out (?) I found none decent
> log4j-tomcat-virtual-hosts-logging-manual in Tomcat docs
> also. Is Tomcat a production ready reference j2se server
> or just a "finish it yourself" proof of concept toy?
>
> I've been crond-restarting Tomcat 4.1 on Java 1.4 every
> night for more than a year, and now I have to restart
> Tomcat 5.5.15 on Java 1.5 AND apache httpd also almost
> every single day. From my -user point of view, Tomcat
> doesn't change in a good direction. In fact in it is
> probably changing in wrong direction - at least the
> 4.1.x logger element has been working 'out of the box'.
>
> Come on! Am I the only one that has such problems with Tomcat?
>
>
--
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Sad: Tomcat 5.5.x crashes almost every single day.
Posted by Leon Rosenberg <ro...@googlemail.com>.
On 3/1/06, Peter Lin <wo...@gmail.com> wrote:
> On 2/28/06, Tomasz Nowak <tn...@netventure.pl> wrote:
> >
> >
> >
> > I've not noticed more then 400-500 java threads, but I'll monitor
> > that. I've set different ulimits form differnt users, BUT user
> > that runs java/tomcat has no limits but one: -s 2048 (stacksize).
> >
> > AFAIK that one was recommended by RELEASE-NOTES.txt
> >
> > --
> > T.
> >
> >
> >
> that seems odd. in the original message the connector settings was this
>
> <Connector port="8009" protocol="AJP/1.3"
> maxThreads="200" minSpareThreads="25" maxSpareThreads="50"
> backlog="20" connectionTimeout="10000"
> enableLookups="true" redirectPort="8443" />
>
> if there's more than 400 threads, that would suggest the webapp is creating
> threads and not handling them correctly. If the guess is correct, a simple
> way to prove it is to deploy the app in Resin and use JMeter to generate 1-4
> million requests. Resin should also crash, if the webapp is creating threads
> and letting them zombie.
Sorry, but that's not quite true. As far as we've tested, resin is
able to survive an OutOfMemoryError upon thread creation. Tomcat
brings an OOME and stops doing anything useful, resin just prints the
error to the stderr.log, but remaining threads continue working, thus
being more robust.
Tested with 3.0.8 against 5.0.25 under linux 2.4.x in september 2004.
Regards
Leon
>
> ultimately, the webapp needs to be profiled to make sure it behaves
> correctly.
>
> peter
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Sad: Tomcat 5.5.x crashes almost every single day.
Posted by Peter Lin <wo...@gmail.com>.
On 2/28/06, Tomasz Nowak <tn...@netventure.pl> wrote:
>
>
>
> I've not noticed more then 400-500 java threads, but I'll monitor
> that. I've set different ulimits form differnt users, BUT user
> that runs java/tomcat has no limits but one: -s 2048 (stacksize).
>
> AFAIK that one was recommended by RELEASE-NOTES.txt
>
> --
> T.
>
>
>
that seems odd. in the original message the connector settings was this
<Connector port="8009" protocol="AJP/1.3"
maxThreads="200" minSpareThreads="25" maxSpareThreads="50"
backlog="20" connectionTimeout="10000"
enableLookups="true" redirectPort="8443" />
if there's more than 400 threads, that would suggest the webapp is creating
threads and not handling them correctly. If the guess is correct, a simple
way to prove it is to deploy the app in Resin and use JMeter to generate 1-4
million requests. Resin should also crash, if the webapp is creating threads
and letting them zombie.
ultimately, the webapp needs to be profiled to make sure it behaves
correctly.
peter
Re: Sad: Tomcat 5.5.x crashes almost every single day.
Posted by Tomasz Nowak <tn...@netventure.pl>.
Leon Rosenberg <ro...@googlemail.com> wrote:
>
> you should seriously consider upgrading to 2.6.x linux. 2.4.s thread
> support simply sucks! If your tomcat spawns more than allowed threads
> it will get an outofmemoryerror and simply die... And I don't mean
> your 200 tomcat httpworker threads only, but all threads in the jvm,
> including database connection pools and so on. You should be able to
> count currently running threads on a 2.4.x system with ps, I think the
> proper switch was -H. Something like ps auxH | wc -l.
>
> You can calculate your systems thread limitation by this, very simple
> test program, which just creates and starts new Threads until an OOME
> is thrown.
I've not noticed more then 400-500 java threads, but I'll monitor
that. I've set different ulimits form differnt users, BUT user
that runs java/tomcat has no limits but one: -s 2048 (stacksize).
AFAIK that one was recommended by RELEASE-NOTES.txt
--
T.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Sad: Tomcat 5.5.x crashes almost every single day.
Posted by Leon Rosenberg <ro...@googlemail.com>.
you should seriously consider upgrading to 2.6.x linux. 2.4.s thread
support simply sucks! If your tomcat spawns more than allowed threads
it will get an outofmemoryerror and simply die... And I don't mean
your 200 tomcat httpworker threads only, but all threads in the jvm,
including database connection pools and so on. You should be able to
count currently running threads on a 2.4.x system with ps, I think the
proper switch was -H. Something like ps auxH | wc -l.
You can calculate your systems thread limitation by this, very simple
test program, which just creates and starts new Threads until an OOME
is thrown.
regards
Leon
On 3/1/06, Tomasz Nowak <tn...@netventure.pl> wrote:
> Leon Rosenberg <ro...@googlemail.com> wrote:
> >
> > We had some serious problems with tomcat, which are solved now,
> > however, we are on 5.0.25 (with session patch) and not on 5.5.x yet.
> > However, when tomcat dies, what your cpu usage is looking like? 100%?
>
> I've noticed that when tomcat hangs - load avarage increases
> from 0.2-0.4 (normal) to about 1.0. But I'll take a close look
> at the cpu usage next time 5.5.x hangs.
>
> --
> T.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Sad: Tomcat 5.5.x crashes almost every single day.
Posted by Tomasz Nowak <tn...@netventure.pl>.
Leon Rosenberg <ro...@googlemail.com> wrote:
>
> We had some serious problems with tomcat, which are solved now,
> however, we are on 5.0.25 (with session patch) and not on 5.5.x yet.
> However, when tomcat dies, what your cpu usage is looking like? 100%?
I've noticed that when tomcat hangs - load avarage increases
from 0.2-0.4 (normal) to about 1.0. But I'll take a close look
at the cpu usage next time 5.5.x hangs.
--
T.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Sad: Tomcat 5.5.x crashes almost every single day.
Posted by Leon Rosenberg <ro...@googlemail.com>.
[...skiped...]
> I've been crond-restarting Tomcat 4.1 on Java 1.4 every
> night for more than a year, and now I have to restart
> Tomcat 5.5.15 on Java 1.5 AND apache httpd also almost
> every single day. From my -user point of view, Tomcat
> doesn't change in a good direction. In fact in it is
> probably changing in wrong direction - at least the
> 4.1.x logger element has been working 'out of the box'.
If you have to restart tomcat every night, your application seems to
be very buggy...
> Come on! Am I the only one that has such problems with Tomcat?
>
We had some serious problems with tomcat, which are solved now,
however, we are on 5.0.25 (with session patch) and not on 5.5.x yet.
However, when tomcat dies, what your cpu usage is looking like? 100%?
Leon
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat 5.5.x crashes almost every single day.
Posted by Leon Rosenberg <ro...@googlemail.com>.
Tomasz.
could you please stop whining and reply to the suggestions in your
original thread (sad: ...) if you want to get real help...
If you don't need help, and just want to curse around tomcat
developers and/or users... then tell us, so we can filter your
messages out.
regards
Leon
On 2/28/06, Tomasz Nowak <tn...@netventure.pl> wrote:
> John C. Dale <jc...@downinthedesert.com> wrote:
> >
> > PHP is great. So is Zoap and .NET. Careful, though, some of the same
> > problems you would be shunning Java for would also appear in any
> > Internet development/application development platform.
>
> I know the pitfalls of PHP, I been developing php webapps before.
> That's why I moved to Java ;) But as far as I remember:
> - I've never managed to hang httpd with a buggy PHP script :)
> - I could _easly_ diagnose ANY PHP bug just by browsing the log file.
> I'd love Tomcat behave similar.
>
> > There is a correlation between the size of the application and the
> > time it takes to properly test the application. The larger the
> > application, the more it takes to vet application quality. What you
> > have said here is that you don't want to properly test the
> > application because it's too large and too much work, and that you
> > are happy to continually restart the application as a solution.
> > Sounds equitable.
>
> You seems reasonable, but souldn't the appserver at least try
> to help me a bit? I'd really like to see something more than
> "An error occurred. I quitting. Bye!" An error where? Which
> vhost? Which webapp? Which class? Anything, please.
>
> > And with regularity, what steps lead up to Tomcat hanging? This is
> > the question that you should try to answer with increasing accuracy
> > each time Tomcat hangs until you find the issue. Once you do, submit
> > it as a bug, and give back to the OS community that has been
> > supporting your applications for 3 years.
>
> No hangs regularity. Nothing special in the log files.
> No starting point out of the box.
>
> Please, browse the mailinig list topics. I'm not the only
> one here with such problems. Different people post similar
> stories. Are we all wrong?
>
> --
> T.
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat 5.5.x crashes almost every single day.
Posted by Tomasz Nowak <tn...@netventure.pl>.
John C. Dale <jc...@downinthedesert.com> wrote:
>
> PHP is great. So is Zoap and .NET. Careful, though, some of the same
> problems you would be shunning Java for would also appear in any
> Internet development/application development platform.
I know the pitfalls of PHP, I been developing php webapps before.
That's why I moved to Java ;) But as far as I remember:
- I've never managed to hang httpd with a buggy PHP script :)
- I could _easly_ diagnose ANY PHP bug just by browsing the log file.
I'd love Tomcat behave similar.
> There is a correlation between the size of the application and the
> time it takes to properly test the application. The larger the
> application, the more it takes to vet application quality. What you
> have said here is that you don't want to properly test the
> application because it's too large and too much work, and that you
> are happy to continually restart the application as a solution.
> Sounds equitable.
You seems reasonable, but souldn't the appserver at least try
to help me a bit? I'd really like to see something more than
"An error occurred. I quitting. Bye!" An error where? Which
vhost? Which webapp? Which class? Anything, please.
> And with regularity, what steps lead up to Tomcat hanging? This is
> the question that you should try to answer with increasing accuracy
> each time Tomcat hangs until you find the issue. Once you do, submit
> it as a bug, and give back to the OS community that has been
> supporting your applications for 3 years.
No hangs regularity. Nothing special in the log files.
No starting point out of the box.
Please, browse the mailinig list topics. I'm not the only
one here with such problems. Different people post similar
stories. Are we all wrong?
--
T.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Tomcat 5.5.x crashes almost every single day.
Posted by "John C. Dale" <jc...@downinthedesert.com>.
I don't really want to get into a pissing contest over this (mainly because
I have some serious pressure and a rather small aperture, resulting in an
unmatchable pressure/volume ratio), so this is the last thing I will say on
the topic that doesn't relate to a solution for you (which, at the moment,
is not possible given the information you've provided about your issue).
"At least 3 years of giving a chance are not "soon" "
You've only been at it for three years? Written history is 'only' a few
thousand years old and 3 years is small by comparison.
"Operating system is only as good as the applications run on it." True of
false? True... in Windows 95 :-) Few years later even Microsoft handles apps
erros successfully."
If this is what you believe in your heart, then I recommend a wholesale
switch to .NET for you. I wish you all the low switching costs in the
world, and commend you for the faith you put in Microsoft (and their error
messages), as this is a gritty display of loyalty in the face of
overwhelming evidence contradicting your assertion.
"I'm so sad that I think of a _completly_ new code. In PHP!"
PHP is great. So is Zoap and .NET. Careful, though, some of the same
problems you would be shunning Java for would also appear in any Internet
development/application development platform. You know the old adage,
however, if you let it go, and it comes back to you, it's yours forever. If
you switch and you aren't very, very stubborn, you may be Java's forever.
"- the apps have _plenty_ of different urls"
There is a correlation between the size of the application and the time it
takes to properly test the application. The larger the application, the
more it takes to vet application quality. What you have said here is that
you don't want to properly test the application because it's too large and
too much work, and that you are happy to continually restart the application
as a solution. Sounds equitable.
" I can't reproduce such Tomcat hangs on development machine."
And with regularity, what steps lead up to Tomcat hanging? This is the
question that you should try to answer with increasing accuracy each time
Tomcat hangs until you find the issue. Once you do, submit it as a bug, and
give back to the OS community that has been supporting your applications for
3 years.
"- Shall I really examin and monitor the production enviroment?"
Only if you don't care about having server downtime for your users.
"Looks like a "finish it yourself" thing"
try
{
Hacker hacker = new Hacker();
if(hacker.canFinishItHimselfByAskingGoodQuestions())
{
hacker.walkProudly();
hacker.goToBank();
hacker.retireWealthy();
hacker.dieHavingLeftAGoodLegacy();
}
else
{
throw new NotReallyAHackerException();
}
}
catch (NotReallyAHackerException hrahe)
{
return new GoToLawOrBusinessSchoolEvent();
}
Bear Down,
JCD
-----Original Message-----
From: news [mailto:news@sea.gmane.org] On Behalf Of Tomasz Nowak
Sent: Tuesday, February 28, 2006 11:25 AM
To: users@tomcat.apache.org
Subject: Re: Tomcat 5.5.x crashes almost every single day.
John C. Dale <jc...@downinthedesert.com> wrote:
>
> Don't give-up on Tomcat so soon.
At least 3 years of giving a chance are not "soon"... ;]
> Sometimes Tomcat is only as good as the applications deployed within
> her.
"Operating system is only as good as the applications run on it." True of
false? True... in Windows 95 :-) Few years later even Microsoft handles apps
erros successfully.
Tomcat is the only server-type software I know that
_any_user_ can crash so easily and the only
that is so helpless about handling their erros.
Why don't it send 500 status to pages in faulty webapp
and contiunes to serve other webapps normally?
> I write my own apps ;o)
I use Apache Foundation apps ;]
> I would stick with it, and cease development of new code until you
> have eliminated the existing installed code
I'm so sad that I think of a _completly_ new code. In PHP!
> Along the way, it would
> be helpful for you to beat up a separate, mirrored staging instance of
> your production environment using JMeter. Reproduce your problem with
> regularity by hitting each URL many times, then examining the results.
This is probably imposibble because:
- the apps have _plenty_ of different urls.
- I can't reproduce such Tomcat hangs on development machine.
- Shall I really examin and monitor the production enviroment?
What about performance then?
Are there any resource monitors built-in into Tomcat?
I've browsed the JMX management manual.. well is it written
for a human being? Looks like a "finish it yourself" thing. :|
--
Tomasz Nowak
Netventure, http://www.netventure.pl/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat 5.5.x crashes almost every single day.
Posted by Tomasz Nowak <tn...@netventure.pl>.
John C. Dale <jc...@downinthedesert.com> wrote:
>
> Don't give-up on Tomcat so soon.
At least 3 years of giving a chance are not "soon"... ;]
> Sometimes Tomcat is only as good as the applications deployed within
> her.
"Operating system is only as good as the applications run on it."
True of false? True... in Windows 95 :-) Few years later
even Microsoft handles apps erros successfully.
Tomcat is the only server-type software I know that
_any_user_ can crash so easily and the only
that is so helpless about handling their erros.
Why don't it send 500 status to pages in faulty webapp
and contiunes to serve other webapps normally?
> I write my own apps ;o)
I use Apache Foundation apps ;]
> I would stick with it, and cease development of new code until you
> have eliminated the existing installed code
I'm so sad that I think of a _completly_ new code. In PHP!
> Along the way, it would
> be helpful for you to beat up a separate, mirrored staging instance
> of your production environment using JMeter. Reproduce your problem
> with regularity by hitting each URL many times, then examining the
> results.
This is probably imposibble because:
- the apps have _plenty_ of different urls.
- I can't reproduce such Tomcat hangs on development machine.
- Shall I really examin and monitor the production enviroment?
What about performance then?
Are there any resource monitors built-in into Tomcat?
I've browsed the JMX management manual.. well is it written
for a human being? Looks like a "finish it yourself" thing. :|
--
Tomasz Nowak
Netventure, http://www.netventure.pl/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Sad: Tomcat 5.5.x crashes almost every single day.
Posted by Mladen Turk <ml...@jboss.com>.
Mark Eggers wrote:
> I'm confused - but wouldn't an enableLookups on the
> AJP connector return the hostname of the remote Apache
> server sending the request (and not the remote web
> client)?
>
No.
JkOptions +ForwardLocalAddress
Regards,
Mladen.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Sad: Tomcat 5.5.x crashes almost every single day.
Posted by Mark Eggers <it...@yahoo.com>.
I'm confused - but wouldn't an enableLookups on the
AJP connector return the hostname of the remote Apache
server sending the request (and not the remote web
client)?
I don't know since I'm at work, and I always disable
enableLookups since it's a potential performance hit.
/mde/
--- Tomasz Nowak <tn...@netventure.pl> wrote:
> Mark Eggers <it...@yahoo.com> wrote:
> >
> > > enableLookups="true" redirectPort="8443" />
> >
> > Why enableLookups="true" ???
>
> I use request.getRemoteHost in couple of places.
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Sad: Tomcat 5.5.x crashes almost every single day.
Posted by Tomasz Nowak <tn...@netventure.pl>.
Mark Eggers <it...@yahoo.com> wrote:
>
> > enableLookups="true" redirectPort="8443" />
>
> Why enableLookups="true" ???
I use request.getRemoteHost in couple of places.
> Also, Cocoon can be very resource-intensive depending
> on which blocks you've built.
Almost always only:
- databases-block
- lucene-block
- xsp-block
- mail-block
- profiler-block
Nothing more.
> JSP under Cocoon 2.1.7 and JDK 1.5 is also
> problematic, see for example:
>
> http://issues.apache.org/jira/browse/COCOON-1457
>
> I have some ideas as to how to approach this, but I've
> not had the time to investigate.
I don't use JSP. But on my development box JSP works fine.
--
T.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Tomcat 5.5.x crashes almost every single day.
Posted by "John C. Dale" <jc...@downinthedesert.com>.
There are many variables along the path to enlightenment.
Don't give-up on Tomcat so soon.
Sometimes Tomcat is only as good as the applications deployed within her.
Given that Tomcat has been up and running for me for as many as 90 days in a
row with a high-use application accessing a database (of course, I write my
own apps ;o), I would start with a code review of any custom applications
within the beast before pointing to the beast itself.
Honestly, I was just thinking this last night after my struggles with the
default webdav servlet:
"Is Tomcat a production ready reference j2se server or just a "finish it
yourself" proof of concept toy?"
Have you and I been separated at birth?
I would stick with it, and cease development of new code until you have
eliminated the existing installed code base as a source of the error. Look
for connections that aren't being closed, for rampant object creation, for
loitering objects, etc. Along the way, it would be helpful for you to beat
up a separate, mirrored staging instance of your production environment
using JMeter. Reproduce your problem with regularity by hitting each URL
many times, then examining the results.
If this type of application development was easy, we wouldn't all get paid
the big bucks, right?
Keep at it. Open source gets better if people like you and I continue to
point out both the silver linings and the rain in any clouds that should
form on the horizon.
Bear Down.
JCD
-----Original Message-----
From: news [mailto:news@sea.gmane.org] On Behalf Of Tomasz Nowak
Sent: Tuesday, February 28, 2006 10:02 AM
To: users@tomcat.apache.org
Subject: Sad: Tomcat 5.5.x crashes almost every single day.
Probably important:
===================
- 2x Xeon, 3 GB mem
- Linux 2.4.26
- Java 1.5.0_06-b05
- Apache Tomcat 5.5.15
- CATALINA_OPTS="\
-server \
-Djava.awt.headless=true \
-Dfile.encoding=UTF-8 \
-XX:MaxPermSize=256m \
-Xms1024m -Xmx1024m"
- <Connector port="8009" protocol="AJP/1.3"
maxThreads="200" minSpareThreads="25" maxSpareThreads="50"
backlog="20" connectionTimeout="10000"
enableLookups="true" redirectPort="8443" />
- mod_jk 1.2.15
- Apache Httpd 1.3.34
- 8 Tomcat vhosts
- 8 webapps
- each webapp runs on separate vhost
- each webapp is Apache Cocoon 2.1.7
- system load avarage: usually under 0.5
- http load: not more then 20-30 concurrent req
The pain: crashing Tomcat
======================
Everything works right until Tomcat suddenly crashes,
what happens almost every single day. Sometimes it crashes
with OOME / out of heap (heh, is 1GB not enought!?), but somtimes NOTHING
wrong is logged into the logs - it just stops responding to requests
(hangs).
When Tomcat stops responding to requests JK eats all
of 256 Apache HTTPd MaxClients so other httpd vhosts
stop responding also. Disaster!
Any ideas before abandoning Java technology? ;)
Default Tomcat logging facility seems to be useless to me. j.u.l.
logging.properties semantics looks like a programmers joke made to users
(!). And even with swallowOutput it logs everything into catalina.out (?) I
found none decent log4j-tomcat-virtual-hosts-logging-manual in Tomcat docs
also. Is Tomcat a production ready reference j2se server or just a "finish
it yourself" proof of concept toy?
I've been crond-restarting Tomcat 4.1 on Java 1.4 every
night for more than a year, and now I have to restart
Tomcat 5.5.15 on Java 1.5 AND apache httpd also almost
every single day. From my -user point of view, Tomcat
doesn't change in a good direction. In fact in it is
probably changing in wrong direction - at least the
4.1.x logger element has been working 'out of the box'.
Come on! Am I the only one that has such problems with Tomcat?
--
T.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Sad: Tomcat 5.5.x crashes almost every single day.
Posted by Mark Eggers <it...@yahoo.com>.
> enableLookups="true" redirectPort="8443" />
Why enableLookups="true" ???
Also, Cocoon can be very resource-intensive depending
on which blocks you've built.
JSP under Cocoon 2.1.7 and JDK 1.5 is also
problematic, see for example:
http://issues.apache.org/jira/browse/COCOON-1457
I have some ideas as to how to approach this, but I've
not had the time to investigate.
/mde/
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Tomcat 5.5.x crashes almost every single day.
Posted by Abraham Fdez <ab...@fnac.es>.
Hi,
You must look at your java code in order to find any function that are
"deprecated". Sometimes this functions are the problem of memory leaks, high
cpu usage, etc...
Best regards,
Abraham Fernández Álvarez
Dpto. de Informática
FNAC ESPAÑA S.A.
Parque Empresarial La Finca
Pº de la Finca, bloque 11 2ª Planta
28223 Pozuelo de Alarcón (Madrid)
( +34 91 7689211
7 +34 91 7689225
/ abraham.fernandez@fnac.es
: http://www.fnac.es
-----Mensaje original-----
De: news [mailto:news@sea.gmane.org] En nombre de Tomasz Nowak
Enviado el: martes, 28 de febrero de 2006 18:02
Para: users@tomcat.apache.org
Asunto: Sad: Tomcat 5.5.x crashes almost every single day.
Probably important:
===================
- 2x Xeon, 3 GB mem
- Linux 2.4.26
- Java 1.5.0_06-b05
- Apache Tomcat 5.5.15
- CATALINA_OPTS="\
-server \
-Djava.awt.headless=true \
-Dfile.encoding=UTF-8 \
-XX:MaxPermSize=256m \
-Xms1024m -Xmx1024m"
- <Connector port="8009" protocol="AJP/1.3"
maxThreads="200" minSpareThreads="25" maxSpareThreads="50"
backlog="20" connectionTimeout="10000"
enableLookups="true" redirectPort="8443" />
- mod_jk 1.2.15
- Apache Httpd 1.3.34
- 8 Tomcat vhosts
- 8 webapps
- each webapp runs on separate vhost
- each webapp is Apache Cocoon 2.1.7
- system load avarage: usually under 0.5
- http load: not more then 20-30 concurrent req
The pain: crashing Tomcat
======================
Everything works right until Tomcat suddenly crashes,
what happens almost every single day. Sometimes it crashes
with OOME / out of heap (heh, is 1GB not enought!?), but
somtimes NOTHING wrong is logged into the logs - it
just stops responding to requests (hangs).
When Tomcat stops responding to requests JK eats all
of 256 Apache HTTPd MaxClients so other httpd vhosts
stop responding also. Disaster!
Any ideas before abandoning Java technology? ;)
Default Tomcat logging facility seems to be useless to me.
j.u.l. logging.properties semantics looks like a programmers
joke made to users (!). And even with swallowOutput it logs
everything into catalina.out (?) I found none decent
log4j-tomcat-virtual-hosts-logging-manual in Tomcat docs
also. Is Tomcat a production ready reference j2se server
or just a "finish it yourself" proof of concept toy?
I've been crond-restarting Tomcat 4.1 on Java 1.4 every
night for more than a year, and now I have to restart
Tomcat 5.5.15 on Java 1.5 AND apache httpd also almost
every single day. From my -user point of view, Tomcat
doesn't change in a good direction. In fact in it is
probably changing in wrong direction - at least the
4.1.x logger element has been working 'out of the box'.
Come on! Am I the only one that has such problems with Tomcat?
--
T.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org