You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Peter Neu <pe...@gmx.net> on 2006/06/12 08:57:34 UTC
Detect tomcat problems with a custom java reporting apps?
Hello,
I want to build a small custom reporting tool for our tomcat production
server.
First of all is this the right place to ask about this kind of things or is
this more a question for the dev list?
The basic idea is, I want to send queries to tomcat and if one fails three
times send out alert messages to the admins.
My first idea was, to just use the http common client and use the http
status codes. But this is too easy as I suspect because
an out of memory exception wouldn't be considered as an error because it
sends back a html page. So what can I do?
Is JMX the magic word here? I saw some ant tasks in the docs but no way of
detecting a java.lang.outOfMemory Exception.
Cheers,
Pete
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Detect tomcat problems with a custom java reporting apps?
Posted by Leon Rosenberg <ro...@googlemail.com>.
I think you need a multilayer system :-)
1) a http-client app which simply calls the same simple servlet
(simply writing "available" in the response or something like this)
each 10 seconds and check if the server replies at all.
2) monitor cpu load on the machine. If the idle time is 100% the
tomcat is probably dead (This of course expects some continious load
on the server)
3) monitor the logs
4) your simple servlet from 1) could deliver current free memory and
the monitoring application could raise alerts if the amount decreases
(still, would be harder to get PermGenSpace)
Leon
P.S.
btw, an outofmemory in the permgenspace doesn't necessary mean an
OutOfMemory in the logs. The AllThreadsBusy message also happens
because an OutOfMemory. Or any other "strange" log messages. We had
this situation (running of permspace) some time ago, and the errors
were everything but logical: response already commited exceptions or
bean xyz not found for example.
regards
Leon
On 6/12/06, Peter Neu <pe...@gmx.net> wrote:
> You are right Leon. I just did a field test on our test server. The perm gen
> space is relatively small on that machine. So opened a lot sessions and
> did a lot of XSLT/FOP/Excel generation.
>
> Thus the perm gen space was spent very fast. I monitored this with lambda.
> In the end the server did not respond anymore. And here comes the
> frightening news. There is no trace in the log files for that problem.
>
> So it is safe to say when the server does not respond in say 2 minutes the
> game is over.
>
> One question remains after all. Doesn't it pay off to simply look ater the
> Memory space in frequent intervalls to see it bottle neck is ahead? I mean
> it's possible to define certain ranges.
>
> Any ideas?
>
>
> Cheers,
> Swen
>
>
>
> -----Ursprüngliche Nachricht-----
> Von: Leon Rosenberg [mailto:rosenberg.leon@googlemail.com]
> Gesendet: Montag, 12. Juni 2006 14:22
> An: Tomcat Users List
> Betreff: Re: Detect tomcat problems with a custom java reporting apps?
>
> I'm not sure that after something "bad" happens you can access the
> server at all. An OutOfMemory or an AllThreadsBusy Exception normally
> lead to a non-responding tomcat. The best thing you could do is
> monitoring the logs with a separate script on the same machine and
> send mails on failure imho.
>
> regards
> leon
>
> p.s. for the application monitoring of your production server you
> should try moskito :-)
>
> On 6/12/06, Peter Neu <pe...@gmx.net> wrote:
> > Yes, I am using this already. But there is some vital functionality
> missing.
> >
> > When an java.lang.outOf.MemoryException happens I won't get an alert
> unless
> > I happen to be using lambda at the moment. So what we have at hand is a
> > 'Schrödinger Cat' situation.
> >
> > I need to monitor the jmx control layer and have my program send out
> alerts
> > when something bad happens.
> >
> > So far I have 2 problems:
> >
> > 1. How do I find an error? I played around with the JMX Proxy Servlet.
> There
> > is this Mbean: MemoryPool. When I query this I get the following result:
> >
> > Name: java.lang:type=MemoryPool,name=Perm Gen [shared-rw]
> > modelerType: sun.management.MemoryPoolImpl
> > Type: NON_HEAP
> > CollectionUsageThreshold: 0
> > CollectionUsageThresholdExceeded: true
> > MemoryManagerNames: [Ljava.lang.String;@943129
> > CollectionUsageThresholdSupported: true
> > .....
> >
> > If I see this I can't tell if anything is wrong. So what to do?
> >
> > 2. How do I access the jmxproxy servlet from a java app? It requires
> > authentication. :o(
> >
> > Cheers,
> > Pete
> >
> >
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To start a new topic, e-mail: users@tomcat.apache.org
> > To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> > For additional commands, e-mail: users-help@tomcat.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To start a new topic, e-mail: users@tomcat.apache.org
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
>
>
> ---------------------------------------------------------------------
> To start a new topic, e-mail: users@tomcat.apache.org
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
AW: Detect tomcat problems with a custom java reporting apps?
Posted by Peter Neu <pe...@gmx.net>.
You are right Leon. I just did a field test on our test server. The perm gen
space is relatively small on that machine. So opened a lot sessions and
did a lot of XSLT/FOP/Excel generation.
Thus the perm gen space was spent very fast. I monitored this with lambda.
In the end the server did not respond anymore. And here comes the
frightening news. There is no trace in the log files for that problem.
So it is safe to say when the server does not respond in say 2 minutes the
game is over.
One question remains after all. Doesn't it pay off to simply look ater the
Memory space in frequent intervalls to see it bottle neck is ahead? I mean
it's possible to define certain ranges.
Any ideas?
Cheers,
Swen
-----Ursprüngliche Nachricht-----
Von: Leon Rosenberg [mailto:rosenberg.leon@googlemail.com]
Gesendet: Montag, 12. Juni 2006 14:22
An: Tomcat Users List
Betreff: Re: Detect tomcat problems with a custom java reporting apps?
I'm not sure that after something "bad" happens you can access the
server at all. An OutOfMemory or an AllThreadsBusy Exception normally
lead to a non-responding tomcat. The best thing you could do is
monitoring the logs with a separate script on the same machine and
send mails on failure imho.
regards
leon
p.s. for the application monitoring of your production server you
should try moskito :-)
On 6/12/06, Peter Neu <pe...@gmx.net> wrote:
> Yes, I am using this already. But there is some vital functionality
missing.
>
> When an java.lang.outOf.MemoryException happens I won't get an alert
unless
> I happen to be using lambda at the moment. So what we have at hand is a
> 'Schrödinger Cat' situation.
>
> I need to monitor the jmx control layer and have my program send out
alerts
> when something bad happens.
>
> So far I have 2 problems:
>
> 1. How do I find an error? I played around with the JMX Proxy Servlet.
There
> is this Mbean: MemoryPool. When I query this I get the following result:
>
> Name: java.lang:type=MemoryPool,name=Perm Gen [shared-rw]
> modelerType: sun.management.MemoryPoolImpl
> Type: NON_HEAP
> CollectionUsageThreshold: 0
> CollectionUsageThresholdExceeded: true
> MemoryManagerNames: [Ljava.lang.String;@943129
> CollectionUsageThresholdSupported: true
> .....
>
> If I see this I can't tell if anything is wrong. So what to do?
>
> 2. How do I access the jmxproxy servlet from a java app? It requires
> authentication. :o(
>
> Cheers,
> Pete
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To start a new topic, e-mail: users@tomcat.apache.org
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Detect tomcat problems with a custom java reporting apps?
Posted by Leon Rosenberg <ro...@googlemail.com>.
I'm not sure that after something "bad" happens you can access the
server at all. An OutOfMemory or an AllThreadsBusy Exception normally
lead to a non-responding tomcat. The best thing you could do is
monitoring the logs with a separate script on the same machine and
send mails on failure imho.
regards
leon
p.s. for the application monitoring of your production server you
should try moskito :-)
On 6/12/06, Peter Neu <pe...@gmx.net> wrote:
> Yes, I am using this already. But there is some vital functionality missing.
>
> When an java.lang.outOf.MemoryException happens I won't get an alert unless
> I happen to be using lambda at the moment. So what we have at hand is a
> 'Schrödinger Cat' situation.
>
> I need to monitor the jmx control layer and have my program send out alerts
> when something bad happens.
>
> So far I have 2 problems:
>
> 1. How do I find an error? I played around with the JMX Proxy Servlet. There
> is this Mbean: MemoryPool. When I query this I get the following result:
>
> Name: java.lang:type=MemoryPool,name=Perm Gen [shared-rw]
> modelerType: sun.management.MemoryPoolImpl
> Type: NON_HEAP
> CollectionUsageThreshold: 0
> CollectionUsageThresholdExceeded: true
> MemoryManagerNames: [Ljava.lang.String;@943129
> CollectionUsageThresholdSupported: true
> .....
>
> If I see this I can't tell if anything is wrong. So what to do?
>
> 2. How do I access the jmxproxy servlet from a java app? It requires
> authentication. :o(
>
> Cheers,
> Pete
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To start a new topic, e-mail: users@tomcat.apache.org
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
AW: Detect tomcat problems with a custom java reporting apps?
Posted by Peter Neu <pe...@gmx.net>.
Yes, I am using this already. But there is some vital functionality missing.
When an java.lang.outOf.MemoryException happens I won't get an alert unless
I happen to be using lambda at the moment. So what we have at hand is a
'Schrödinger Cat' situation.
I need to monitor the jmx control layer and have my program send out alerts
when something bad happens.
So far I have 2 problems:
1. How do I find an error? I played around with the JMX Proxy Servlet. There
is this Mbean: MemoryPool. When I query this I get the following result:
Name: java.lang:type=MemoryPool,name=Perm Gen [shared-rw]
modelerType: sun.management.MemoryPoolImpl
Type: NON_HEAP
CollectionUsageThreshold: 0
CollectionUsageThresholdExceeded: true
MemoryManagerNames: [Ljava.lang.String;@943129
CollectionUsageThresholdSupported: true
.....
If I see this I can't tell if anything is wrong. So what to do?
2. How do I access the jmxproxy servlet from a java app? It requires
authentication. :o(
Cheers,
Pete
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Detect tomcat problems with a custom java reporting apps?
Posted by Leon Rosenberg <ro...@googlemail.com>.
maybe you should look at existing tools first?
http://www.lambdaprobe.org/d/index.htm
regards
leon
On 6/12/06, Peter Neu <pe...@gmx.net> wrote:
> Hello,
>
> I want to build a small custom reporting tool for our tomcat production
> server.
>
> First of all is this the right place to ask about this kind of things or is
> this more a question for the dev list?
>
> The basic idea is, I want to send queries to tomcat and if one fails three
> times send out alert messages to the admins.
> My first idea was, to just use the http common client and use the http
> status codes. But this is too easy as I suspect because
> an out of memory exception wouldn't be considered as an error because it
> sends back a html page. So what can I do?
>
> Is JMX the magic word here? I saw some ant tasks in the docs but no way of
> detecting a java.lang.outOfMemory Exception.
>
>
> Cheers,
> Pete
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To start a new topic, e-mail: users@tomcat.apache.org
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Detect tomcat problems with a custom java reporting apps?
Posted by Mark Thomas <ma...@apache.org>.
Please do not cross-post between the users and the dev list.
This question belongs on the users list.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Detect tomcat problems with a custom java reporting apps?
Posted by Peter Neu <pe...@gmx.net>.
Hello,
I want to build a small custom reporting tool for our tomcat production
server.
The basic idea is, I want to send queries to tomcat and if one fails three
times send out alert messages to the admins.
My first idea was, to just use the http common client and use the http
status codes. But this is too easy as I suspect because
an out of memory exception wouldn't be considered as an error because it
sends back a html page. So what can I do?
Is JMX the magic word here? I saw some ant tasks in the docs but no way of
detecting a java.lang.outOfMemory Exception.
Cheers,
Pete
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org