You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by André Warnier <aw...@ice-sa.com> on 2011/04/12 12:20:17 UTC
Simple Tomcat monitoring ?
Hi.
I have a Linux application server running Apache + mod_jk + tomcat 5.5 (I know that it is
relatively old; and to compound the heresy, it is installed from a Debian package).
The hardware is quite OK (quad-core fast CPU, 12 GB RAM).
Apache is serving most of the static content, and passing some requests to Tomcat via AJP.
Tomcat runs basically a single application, but this webapp, although itself quite
small, can be heavily used : it is an interface to a text retrieval system (a kind of
database specialised for textual data), and some of the searches made by users can take
several seconds to come back with results, which may themselves be several pages of data.
I can access the host remotely via SSH and via HTTP, and have full control of it.
But it is a productive customer server, so there are limits to what I can do.
I am having performance issues with that system : users at times complain that "they
cannot access the application", and I see things like this in a "top" display :
top - 08:20:02 up 72 days, 9:39, 4 users, load average: 6.83, 5.63, 3.16
Tasks: 265 total, 1 running, 264 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.2%us, 0.1%sy, 0.0%ni, 99.2%id, 0.5%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 12328624k total, 11382976k used, 945648k free, 123400k buffers
Swap: 4882424k total, 860k used, 4881564k free, 7565012k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2023 tomcat55 20 0 627m 184m 9344 S 333 1.5 32:00.47 jsvc
5396 root 20 0 832m 610m 9224 S 100 5.1 12:32.43 java
1 root 20 0 10316 760 624 S 0 0.0 0:33.14 init
(where I presume that if tomcat is showing 333% CPU usage, it is because it is running on
multiple cores; the second task visible above is the interface to the search engine).
Anyway, my question is :
Considering all the above, which would be the easiest/quickest way of starting to figure
out what this tomcat is doing, and where the system/application bottleneck might be ?
My preference would be in an easy tool to install and run, to first get a rough idea, and
then maybe select more specific tools to examine one or the other area more in-depth.
Or maybe I can just first add some command-line switches to the JVM running tomcat, to
give me more information in the logs ?
(My workstation is Windows XP, and I can install anything I want on it.)
Additional details :
Platform : (Linux 64-bit)
# uname -a
Linux vogon2 2.6.26-2-amd64 #1 SMP Thu Nov 25 04:30:55 UTC 2010 x86_64 GNU/Linux
Tomcat and Java :
Using CATALINA_BASE: /usr/share/tomcat5.5
Using CATALINA_HOME: /usr/share/tomcat5.5
Using CATALINA_TMPDIR: /usr/share/tomcat5.5/temp
Using JRE_HOME: /usr/lib/jvm/java-6-sun
Server version: Apache Tomcat/5.5
Server built: Oct 15 2008 12:57:44
Server number: 5.5.26.0
OS Name: Linux
OS Version: 2.6.26-2-amd64
Architecture: amd64
JVM Version: 1.6.0_22-b04
JVM Vendor: Sun Microsystems Inc.
vogon2:/usr/share/tomcat5.5/bin#
Tomcat is started as :
tomcat55 2023 2021 6 Apr11 ? 00:31:58 /usr/bin/jsvc -user tomcat55 -cp
/usr/share/java/commons-daemon.jar:/usr/share/tomcat5.5/bin/bootstrap.jar -outfile SYSLOG
-errfile SYSLOG -pidfile /var/run/tomcat5.5.pid -Djava.awt.headless=true -Xms128M -Xmx128M
-Djava.endorsed.dirs=/usr/share/tomcat5.5/common/endorsed
-Dcatalina.base=/var/lib/tomcat5.5 -Dcatalina.home=/usr/share/tomcat5.5
-Djava.io.tmpdir=/var/lib/tomcat5.5/temp
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
-Djava.util.logging.config.file=/var/lib/tomcat5.5/conf/logging.properties
org.apache.catalina.startup.Bootstrap
(which I don't think is a lot of heap space, which may be a problem here. But before I
change it, I'd like to know why I change it; the server.xml settings are pretty much the
standard issue, all defaults for threads etc..).
Apache and mod_jk :
Apache/2.2.9 (Debian) DAV/2 mod_jk/1.2.26 mod_apreq2-20051231/2.6.0 mod_perl/2.0.4
Perl/v5.10.0 configured
Thanks in advance for any recommendation
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Simple Tomcat monitoring ?
Posted by Matteo Turra <m....@kion.it>.
Try to analyze your java process with
http://java.sun.com/developer/technicalArticles/J2SE/jconsole.html
and
http://visualvm.java.net/
-----Original Message-----
From: André Warnier [mailto:aw@ice-sa.com]
Sent: martedì 12 aprile 2011 15:43
To: Tomcat Users List
Subject: Re: Simple Tomcat monitoring ?
Caldarale, Charles R wrote:
>> From: André Warnier [mailto:aw@ice-sa.com]
>> Subject: Simple Tomcat monitoring ?
>
>> 5396 root 20 0 832m 610m 9224 S 100 5.1 12:32.43 java
>
>> the second task visible above is the interface to the search engine).
>
> Which seems to be single-thread and thus a possible bottleneck. The 100% CPU usage is very suspicious.
I thought so too, but it does not seem to be.
When I use the command
ps -p 5396 -Lf
I see multiple "instances", all running the same command. They all have the same PID, but different numbers in the "LWP" column (Light-Weight-Process ?).
>
>> which would be the easiest/quickest way of starting to figure out
>> what this tomcat is doing, and where the system/application
>> bottleneck might be ?
>
> This has proven to be a useful tool to look inside a running Tomcat:
> http://moskito.anotheria.net/
>
I had a (very) quick look at the website.
From what I gather at this first look though, it seems already a bit "invasive", in the sense that it seems that to start collecting any information at all about a servlet or tomcat, the least one can do (unless one controls the source code of the application, which I do not), is to configure another classloader etc..
That means changing the Tomcat configuration and restarting Tomcat.
That's maybe a bit more than what I am looking for right now.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Simple Tomcat monitoring ?
Posted by André Warnier <aw...@ice-sa.com>.
Caldarale, Charles R wrote:
>> From: André Warnier [mailto:aw@ice-sa.com]
>> Subject: Simple Tomcat monitoring ?
>
>> 5396 root 20 0 832m 610m 9224 S 100 5.1 12:32.43 java
>
>> the second task visible above is the interface to the search engine).
>
> Which seems to be single-thread and thus a possible bottleneck. The 100% CPU usage is very suspicious.
I thought so too, but it does not seem to be.
When I use the command
ps -p 5396 -Lf
I see multiple "instances", all running the same command. They all have the same PID, but
different numbers in the "LWP" column (Light-Weight-Process ?).
>
>> which would be the easiest/quickest way of starting to figure
>> out what this tomcat is doing, and where the system/application
>> bottleneck might be ?
>
> This has proven to be a useful tool to look inside a running Tomcat:
> http://moskito.anotheria.net/
>
I had a (very) quick look at the website.
From what I gather at this first look though, it seems already a bit "invasive", in the
sense that it seems that to start collecting any information at all about a servlet or
tomcat, the least one can do (unless one controls the source code of the application,
which I do not), is to configure another classloader etc..
That means changing the Tomcat configuration and restarting Tomcat.
That's maybe a bit more than what I am looking for right now.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Simple Tomcat monitoring ?
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: André Warnier [mailto:aw@ice-sa.com]
> Subject: Simple Tomcat monitoring ?
> 5396 root 20 0 832m 610m 9224 S 100 5.1 12:32.43 java
> the second task visible above is the interface to the search engine).
Which seems to be single-thread and thus a possible bottleneck. The 100% CPU usage is very suspicious.
> which would be the easiest/quickest way of starting to figure
> out what this tomcat is doing, and where the system/application
> bottleneck might be ?
This has proven to be a useful tool to look inside a running Tomcat:
http://moskito.anotheria.net/
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
Re: Simple Tomcat monitoring ?
Posted by Borut Hadžialić <bo...@gmail.com>.
You could check it apache threads are the problem by using something like:
ps -feH | grep httpd | wc
(or something else instead httpd to hit only apache processes)
If the line number gets close to 150 (Apache (prefork) MaxClients)
then that is the problem.
On Tue, Apr 12, 2011 at 2:18 PM, André Warnier <aw...@ice-sa.com> wrote:
> Borut Hadžialić wrote:
>>
>> I'm gonna try to make a guess here :)
>>
>>> top - 08:20:02 up 72 days, 9:39, 4 users, load average: 6.83, 5.63,
>>> 3.16
>>> Tasks: 265 total, 1 running, 264 sleeping, 0 stopped, 0 zombie
>>> Cpu(s): 0.2%us, 0.1%sy, 0.0%ni, 99.2%id, 0.5%wa, 0.0%hi, 0.0%si,
>>> 0.0%st
>>> Mem: 12328624k total, 11382976k used, 945648k free, 123400k buffers
>>> Swap: 4882424k total, 860k used, 4881564k free, 7565012k cached
>>
>> - somewhat large load average (6.83)
>> - but no io waiting (0.5%wa),
>> - app is heavily used
>> - tomcat requests take long time (sometimes takes few seconds to
>> produce the answer).
>>
>> Could it be a lack of threads at apache or tomcat?
>
> Spot on for the observation.
>
> That is one of the things that puzzle me : overall CPU usage is very low,
> and there is no i/o wait. Yet the load average is high (which should mean
> that a number of processes are constantly waiting for something), and Tomcat
> is using 333% of CPU time (which may be very little time in an absolute
> sense, but is puzzling anyway).
> And the clients are waiting.
> So where are the time/resources being spent ?
>
> About threads :
> - Apache (prefork) MaxClients is set to 150
> - Tomcat AJP Connector does not specify any thread parameters, so defaults
> apply, which should be :
> maxThreads : 200
> maxSpareThreads : 50
> minSpareThreads : 4
>
> Considering that Apache handles most requests for static contents, I think
> that the numbers should be about right.
> But not having so far done any real monitoring, that may be wrong. Which is
> one of the things I'd like to check.
>
> I think I'll start by making the Tomcat AJP Connector parameters explicit,
> and raise the minSpareThreads a bit. And also its Heap size.
> But I don't like to start changing things left and right, without knowing
> exactly why I'm doing it.
>
> Is there for example any parameter or command-line switch that I can use for
> the JVM or Tomcat, which would tell me a bit more (in the logs) about when
> Tomcat is starting a new thread, how many are alive on average etc.. ?
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
--
Why?
Because YES!
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Simple Tomcat monitoring ?
Posted by André Warnier <aw...@ice-sa.com>.
Borut Hadžialić wrote:
> I'm gonna try to make a guess here :)
>
>> top - 08:20:02 up 72 days, 9:39, 4 users, load average: 6.83, 5.63, 3.16
>> Tasks: 265 total, 1 running, 264 sleeping, 0 stopped, 0 zombie
>> Cpu(s): 0.2%us, 0.1%sy, 0.0%ni, 99.2%id, 0.5%wa, 0.0%hi, 0.0%si, 0.0%st
>> Mem: 12328624k total, 11382976k used, 945648k free, 123400k buffers
>> Swap: 4882424k total, 860k used, 4881564k free, 7565012k cached
>
> - somewhat large load average (6.83)
> - but no io waiting (0.5%wa),
> - app is heavily used
> - tomcat requests take long time (sometimes takes few seconds to
> produce the answer).
>
> Could it be a lack of threads at apache or tomcat?
Spot on for the observation.
That is one of the things that puzzle me : overall CPU usage is very low, and there is no
i/o wait. Yet the load average is high (which should mean that a number of processes are
constantly waiting for something), and Tomcat is using 333% of CPU time (which may be very
little time in an absolute sense, but is puzzling anyway).
And the clients are waiting.
So where are the time/resources being spent ?
About threads :
- Apache (prefork) MaxClients is set to 150
- Tomcat AJP Connector does not specify any thread parameters, so defaults apply, which
should be :
maxThreads : 200
maxSpareThreads : 50
minSpareThreads : 4
Considering that Apache handles most requests for static contents, I think that the
numbers should be about right.
But not having so far done any real monitoring, that may be wrong. Which is one of the
things I'd like to check.
I think I'll start by making the Tomcat AJP Connector parameters explicit, and raise the
minSpareThreads a bit. And also its Heap size.
But I don't like to start changing things left and right, without knowing exactly why I'm
doing it.
Is there for example any parameter or command-line switch that I can use for the JVM or
Tomcat, which would tell me a bit more (in the logs) about when Tomcat is starting a new
thread, how many are alive on average etc.. ?
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Simple Tomcat monitoring ?
Posted by Borut Hadžialić <bo...@gmail.com>.
I'm gonna try to make a guess here :)
> top - 08:20:02 up 72 days, 9:39, 4 users, load average: 6.83, 5.63, 3.16
> Tasks: 265 total, 1 running, 264 sleeping, 0 stopped, 0 zombie
> Cpu(s): 0.2%us, 0.1%sy, 0.0%ni, 99.2%id, 0.5%wa, 0.0%hi, 0.0%si, 0.0%st
> Mem: 12328624k total, 11382976k used, 945648k free, 123400k buffers
> Swap: 4882424k total, 860k used, 4881564k free, 7565012k cached
- somewhat large load average (6.83)
- but no io waiting (0.5%wa),
- app is heavily used
- tomcat requests take long time (sometimes takes few seconds to
produce the answer).
Could it be a lack of threads at apache or tomcat?
On Tue, Apr 12, 2011 at 12:20 PM, André Warnier <aw...@ice-sa.com> wrote:
> Hi.
>
> I have a Linux application server running Apache + mod_jk + tomcat 5.5 (I
> know that it is relatively old; and to compound the heresy, it is installed
> from a Debian package).
> The hardware is quite OK (quad-core fast CPU, 12 GB RAM).
>
> Apache is serving most of the static content, and passing some requests to
> Tomcat via AJP. Tomcat runs basically a single application, but this
> webapp, although itself quite small, can be heavily used : it is an
> interface to a text retrieval system (a kind of database specialised for
> textual data), and some of the searches made by users can take several
> seconds to come back with results, which may themselves be several pages of
> data.
>
> I can access the host remotely via SSH and via HTTP, and have full control
> of it.
> But it is a productive customer server, so there are limits to what I can
> do.
>
> I am having performance issues with that system : users at times complain
> that "they cannot access the application", and I see things like this in a
> "top" display :
>
> top - 08:20:02 up 72 days, 9:39, 4 users, load average: 6.83, 5.63, 3.16
> Tasks: 265 total, 1 running, 264 sleeping, 0 stopped, 0 zombie
> Cpu(s): 0.2%us, 0.1%sy, 0.0%ni, 99.2%id, 0.5%wa, 0.0%hi, 0.0%si,
> 0.0%st
> Mem: 12328624k total, 11382976k used, 945648k free, 123400k buffers
> Swap: 4882424k total, 860k used, 4881564k free, 7565012k cached
>
> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
> 2023 tomcat55 20 0 627m 184m 9344 S 333 1.5 32:00.47 jsvc
> 5396 root 20 0 832m 610m 9224 S 100 5.1 12:32.43 java
> 1 root 20 0 10316 760 624 S 0 0.0 0:33.14 init
>
> (where I presume that if tomcat is showing 333% CPU usage, it is because it
> is running on multiple cores; the second task visible above is the interface
> to the search engine).
>
> Anyway, my question is :
>
> Considering all the above, which would be the easiest/quickest way of
> starting to figure out what this tomcat is doing, and where the
> system/application bottleneck might be ?
>
> My preference would be in an easy tool to install and run, to first get a
> rough idea, and then maybe select more specific tools to examine one or the
> other area more in-depth.
> Or maybe I can just first add some command-line switches to the JVM running
> tomcat, to give me more information in the logs ?
>
> (My workstation is Windows XP, and I can install anything I want on it.)
>
> Additional details :
>
> Platform : (Linux 64-bit)
> # uname -a
> Linux vogon2 2.6.26-2-amd64 #1 SMP Thu Nov 25 04:30:55 UTC 2010 x86_64
> GNU/Linux
>
> Tomcat and Java :
>
> Using CATALINA_BASE: /usr/share/tomcat5.5
> Using CATALINA_HOME: /usr/share/tomcat5.5
> Using CATALINA_TMPDIR: /usr/share/tomcat5.5/temp
> Using JRE_HOME: /usr/lib/jvm/java-6-sun
> Server version: Apache Tomcat/5.5
> Server built: Oct 15 2008 12:57:44
> Server number: 5.5.26.0
> OS Name: Linux
> OS Version: 2.6.26-2-amd64
> Architecture: amd64
> JVM Version: 1.6.0_22-b04
> JVM Vendor: Sun Microsystems Inc.
> vogon2:/usr/share/tomcat5.5/bin#
>
> Tomcat is started as :
> tomcat55 2023 2021 6 Apr11 ? 00:31:58 /usr/bin/jsvc -user tomcat55
> -cp
> /usr/share/java/commons-daemon.jar:/usr/share/tomcat5.5/bin/bootstrap.jar
> -outfile SYSLOG -errfile SYSLOG -pidfile /var/run/tomcat5.5.pid
> -Djava.awt.headless=true -Xms128M -Xmx128M
> -Djava.endorsed.dirs=/usr/share/tomcat5.5/common/endorsed
> -Dcatalina.base=/var/lib/tomcat5.5 -Dcatalina.home=/usr/share/tomcat5.5
> -Djava.io.tmpdir=/var/lib/tomcat5.5/temp
> -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
> -Djava.util.logging.config.file=/var/lib/tomcat5.5/conf/logging.properties
> org.apache.catalina.startup.Bootstrap
>
> (which I don't think is a lot of heap space, which may be a problem here.
> But before I change it, I'd like to know why I change it; the server.xml
> settings are pretty much the standard issue, all defaults for threads
> etc..).
>
> Apache and mod_jk :
>
> Apache/2.2.9 (Debian) DAV/2 mod_jk/1.2.26 mod_apreq2-20051231/2.6.0
> mod_perl/2.0.4 Perl/v5.10.0 configured
>
> Thanks in advance for any recommendation
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
--
Why?
Because YES!
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Simple Tomcat monitoring ?
Posted by Francis GALIEGUE <fg...@one2team.com>.
On Tue, Apr 12, 2011 at 14:26, André Warnier <aw...@ice-sa.com> wrote:
> Francis GALIEGUE wrote:
>>
>> On Tue, Apr 12, 2011 at 12:20, André Warnier <aw...@ice-sa.com> wrote:
>> [...]
>>>
>>> Considering all the above, which would be the easiest/quickest way of
>>> starting to figure out what this tomcat is doing, and where the
>>> system/application bottleneck might be ?
>>>
>>
>> We have the following tools, all very easy to setup and run:
>>
>> * SNMP monitoring of the JVM (allows to see the amount of memory
>> consumed, the number of threads, and others - the MIB is well
>> defined);
>> * a shell script sending the results of a kill -3 on the JVM (a thread
>> dump);
>> * a script which triggers when a java.lang.OutOfMemoryError is thrown.
>>
>> If you are interested in any of it, I can tell you how we do it.
>>
>> And, considering that your server has 12 GB RAM, you could indeed do
>> with a little more heap space!
>>
>
> Thanks. I think I'll accept your offer of details.
>
> I don't think that OOM is a problem, but the first 2 would be a good start.
>
> About SNMP monitoring : I already have a "munin agent" installed on that
> system, and a munin server collecting stuff like overall cpu and memory
> usage.
> To your knowledge, does there exist a way to collect this JVM SNMP
> information via a munin plugin ?
>
I don't use Munin, I use cacti, but if Munin can collect SNMP
information then it's only a matter of having Munin fetch the
appropriate OIDs. More details in a separate mail.
>
> About the script doing a kill -3 : where does the thread dump go, when you
> do that ? and how does the script know which PID is Tomcat ?
>
Mine does since the environment fetches the PID file and sends the
signal to that process. It sends output to either stdout or stderr, I
don't know. Script attached. Read the comments carefully, it is
somewhat a hack (could be improved by detecting, say, if the output
went to stderr).
> On that system, the tomcat processes look like this :
>
> # ps -ef | grep java
> root 20997 1 0 12:18 ? 00:00:00 /usr/bin/jsvc -user tomcat55
> -cp
> /usr/share/java/commons-daemon.jar:/usr/share/tomcat5.5/bin/bootstrap.jar
> -outfile SYSLOG -errfile SYSLOG -pidfile /var/run/tomcat5.5.pid
> -Djava.awt.headless=true -Xms128M -Xmx128M
> -Djava.endorsed.dirs=/usr/share/tomcat5.5/common/endorsed
> -Dcatalina.base=/var/lib/tomcat5.5 -Dcatalina.home=/usr/share/tomcat5.5
> -Djava.io.tmpdir=/var/lib/tomcat5.5/temp
> -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
> -Djava.util.logging.config.file=/var/lib/tomcat5.5/conf/logging.properties
> org.apache.catalina.startup.Bootstrap
> root 20998 20997 0 12:18 ? 00:00:00 /usr/bin/jsvc -user tomcat55
> -cp
> /usr/share/java/commons-daemon.jar:/usr/share/tomcat5.5/bin/bootstrap.jar
> -outfile SYSLOG -errfile SYSLOG -pidfile /var/run/tomcat5.5.pid
> -Djava.awt.headless=true -Xms128M -Xmx128M
> -Djava.endorsed.dirs=/usr/share/tomcat5.5/common/endorsed
> -Dcatalina.base=/var/lib/tomcat5.5 -Dcatalina.home=/usr/share/tomcat5.5
> -Djava.io.tmpdir=/var/lib/tomcat5.5/temp
> -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
> -Djava.util.logging.config.file=/var/lib/tomcat5.5/conf/logging.properties
> org.apache.catalina.startup.Bootstrap
> tomcat55 20999 20997 0 12:18 ? 00:00:04 /usr/bin/jsvc -user tomcat55
> -cp
> /usr/share/java/commons-daemon.jar:/usr/share/tomcat5.5/bin/bootstrap.jar
> -outfile SYSLOG -errfile SYSLOG -pidfile /var/run/tomcat5.5.pid
> -Djava.awt.headless=true -Xms128M -Xmx128M
> -Djava.endorsed.dirs=/usr/share/tomcat5.5/common/endorsed
> -Dcatalina.base=/var/lib/tomcat5.5 -Dcatalina.home=/usr/share/tomcat5.5
> -Djava.io.tmpdir=/var/lib/tomcat5.5/temp
> -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
> -Djava.util.logging.config.file=/var/lib/tomcat5.5/conf/logging.properties
> org.apache.catalina.startup.Bootstrap
> vogon2:~#
>
> So there are 3 Tomcat-related processes, all running under jsvc. I am not
> quite sure which one does what.
>
Wild guess: it's the one in the PID file that matters (-pidfile
/xxxxxxx). It should be a variable in the Tomcat base configuration
file.
--
Francis Galiegue
ONE2TEAM
Ingénieur système
Mob : +33 (0) 683 877 875
Tel : +33 (0) 178 945 552
fge@one2team.com
40 avenue Raymond Poincaré
75116 Paris
Re: Simple Tomcat monitoring ?
Posted by André Warnier <aw...@ice-sa.com>.
Francis GALIEGUE wrote:
> On Tue, Apr 12, 2011 at 12:20, André Warnier <aw...@ice-sa.com> wrote:
> [...]
>> Considering all the above, which would be the easiest/quickest way of
>> starting to figure out what this tomcat is doing, and where the
>> system/application bottleneck might be ?
>>
>
> We have the following tools, all very easy to setup and run:
>
> * SNMP monitoring of the JVM (allows to see the amount of memory
> consumed, the number of threads, and others - the MIB is well
> defined);
> * a shell script sending the results of a kill -3 on the JVM (a thread dump);
> * a script which triggers when a java.lang.OutOfMemoryError is thrown.
>
> If you are interested in any of it, I can tell you how we do it.
>
> And, considering that your server has 12 GB RAM, you could indeed do
> with a little more heap space!
>
Thanks. I think I'll accept your offer of details.
I don't think that OOM is a problem, but the first 2 would be a good start.
About SNMP monitoring : I already have a "munin agent" installed on that system, and a
munin server collecting stuff like overall cpu and memory usage.
To your knowledge, does there exist a way to collect this JVM SNMP information via a munin
plugin ?
About the script doing a kill -3 : where does the thread dump go, when you do that ? and
how does the script know which PID is Tomcat ?
On that system, the tomcat processes look like this :
# ps -ef | grep java
root 20997 1 0 12:18 ? 00:00:00 /usr/bin/jsvc -user tomcat55 -cp
/usr/share/java/commons-daemon.jar:/usr/share/tomcat5.5/bin/bootstrap.jar -outfile SYSLOG
-errfile SYSLOG -pidfile /var/run/tomcat5.5.pid -Djava.awt.headless=true -Xms128M -Xmx128M
-Djava.endorsed.dirs=/usr/share/tomcat5.5/common/endorsed
-Dcatalina.base=/var/lib/tomcat5.5 -Dcatalina.home=/usr/share/tomcat5.5
-Djava.io.tmpdir=/var/lib/tomcat5.5/temp
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
-Djava.util.logging.config.file=/var/lib/tomcat5.5/conf/logging.properties
org.apache.catalina.startup.Bootstrap
root 20998 20997 0 12:18 ? 00:00:00 /usr/bin/jsvc -user tomcat55 -cp
/usr/share/java/commons-daemon.jar:/usr/share/tomcat5.5/bin/bootstrap.jar -outfile SYSLOG
-errfile SYSLOG -pidfile /var/run/tomcat5.5.pid -Djava.awt.headless=true -Xms128M -Xmx128M
-Djava.endorsed.dirs=/usr/share/tomcat5.5/common/endorsed
-Dcatalina.base=/var/lib/tomcat5.5 -Dcatalina.home=/usr/share/tomcat5.5
-Djava.io.tmpdir=/var/lib/tomcat5.5/temp
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
-Djava.util.logging.config.file=/var/lib/tomcat5.5/conf/logging.properties
org.apache.catalina.startup.Bootstrap
tomcat55 20999 20997 0 12:18 ? 00:00:04 /usr/bin/jsvc -user tomcat55 -cp
/usr/share/java/commons-daemon.jar:/usr/share/tomcat5.5/bin/bootstrap.jar -outfile SYSLOG
-errfile SYSLOG -pidfile /var/run/tomcat5.5.pid -Djava.awt.headless=true -Xms128M -Xmx128M
-Djava.endorsed.dirs=/usr/share/tomcat5.5/common/endorsed
-Dcatalina.base=/var/lib/tomcat5.5 -Dcatalina.home=/usr/share/tomcat5.5
-Djava.io.tmpdir=/var/lib/tomcat5.5/temp
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
-Djava.util.logging.config.file=/var/lib/tomcat5.5/conf/logging.properties
org.apache.catalina.startup.Bootstrap
vogon2:~#
So there are 3 Tomcat-related processes, all running under jsvc. I am not quite sure
which one does what.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Simple Tomcat monitoring ?
Posted by Francis GALIEGUE <fg...@one2team.com>.
On Tue, Apr 12, 2011 at 12:20, André Warnier <aw...@ice-sa.com> wrote:
[...]
>
> Considering all the above, which would be the easiest/quickest way of
> starting to figure out what this tomcat is doing, and where the
> system/application bottleneck might be ?
>
We have the following tools, all very easy to setup and run:
* SNMP monitoring of the JVM (allows to see the amount of memory
consumed, the number of threads, and others - the MIB is well
defined);
* a shell script sending the results of a kill -3 on the JVM (a thread dump);
* a script which triggers when a java.lang.OutOfMemoryError is thrown.
If you are interested in any of it, I can tell you how we do it.
And, considering that your server has 12 GB RAM, you could indeed do
with a little more heap space!
--
Francis Galiegue
ONE2TEAM
Ingénieur système
Mob : +33 (0) 683 877 875
Tel : +33 (0) 178 945 552
fge@one2team.com
40 avenue Raymond Poincaré
75116 Paris
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org