You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by André Warnier <aw...@ice-sa.com> on 2008/11/24 20:43:23 UTC
Monitoring Tomcat with Jconsole
Hi.
I lied in the subject.
This is not strictly a question about Tomcat monitoring, although the
ultimate aim is to monitor a specific Tomcat application.
But I'm trying to start small first, and I'm encountering a problem with
Jconsole, for which this is my first-ever try.
I have a java stand-alone application (daemon) running on host "myhost"
under Linux on a Sun JVM 1.5. It's a kind of network server, listening
on port 11100.
I started this application with the following parameters :
./java "-Dcom.sun.management.jmxremote.port=11200
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.password.file=/mydir/
management.jmxremote.password"
-Dpgm=STARXMLServer -jar "/home/star/xml/STARXMLServer.jar"
The file management.jmxremote.password contains this line at the end:
monitorRole mypass
On the other hand, I run Jconsole on a Windows XP station, using a Sun
Java 1.6 JVM as follows :
java version "1.6.0_06"
Java(TM) SE Runtime Environment (build 1.6.0_06-b02)
Java HotSpot(TM) Client VM (build 10.0-b22, mixed mode, sharing).
When I use the "New connection" dialog to specify that I want a remote
connection with
Host:port : myhost:11200
userid : monitorRole
password : mypass
then the dialog disappear for a few seconds, then comes back, with an
error message at the very bottom saying :
Connection failed: Connection refused: connect
(Note : this Jconsole can monitor its own JVM when using the "local
process" choice)
So, what am I doing wrong ?
One think I noticed, is that on the server myhost, despite the daemon
application being loaded without any apparent error messages, and being
responsive to requests on its own port 11100, a
netstat -an
does not show any port 11200 in a LISTEN state.
I do not know at this stage if this is significant.
My understanding is that it is the JVM on the "myhost" system that
should listen for connections on that port 11200, not the application
itself, right ?
Nor can I think of any system-level reason why the system would be
rejecting incoming connections on port 11200. This is a local network,
and there is no firewall or anything between the Windows workstation and
the myhost host.
Thanks in advance for any tips
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Monitoring Tomcat with Jconsole
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: André Warnier [mailto:aw@ice-sa.com]
> Subject: Re: Monitoring Tomcat with Jconsole
>
> I gather that I was wrongly fixated on that number (266M)
> and that it does not really represent any real memory used
> in any permanent way by the process.
Correct; it's analagous to profit claimed by certain financial institutions...
> What really matters is the "Resident" part of it (23M), as
> compared to how much physical RAM the system really has
Yes, but be wary of how big the Java heap can get. One normally runs server programs with -Xms == -Xmx, partly to avoid any chance of the heap growing and running into page thrashing.
> > How big the new + old can get, if needed.
> And can be controlled by the -Xmx<size> switch, yes ?
Correct.
> By running the requests that lead to the snapshot 2 above,
> I believe I have pretty much "stressed" that app as much as
> I can (they were relatively "heavy" requests, involving a
> fair amount of XML parsing).
Depends on whether or not the app is capable of processing multiple requests concurrently (like Tomcat and most servers can).
> My idea in doing this was to reduce the amount of system memory
> this application maybe reserves for no benefit, and thus leave
> more real RAM for others.
It's useful to size the heap appropriately, balancing GC frequency against RAM usage. I wouldn't mess with the max code size attribute - let the JVM use what it needs; the rest won't get any pages allocated to it anyhow.
> Considering the numbers above, in your opinion is there
> anything I could/should still try, while still leaving
> a margin for the application to support maybe a higher
> level of requests ?
I'm not really in a position to say - I have no real feel for what this particular app needs when the going gets tough. Best for you to continue to monitor it, preferably in a production environment if you can (as well as doing your own stress testing, of course).
> Like for instance if you told me to never touch this
> -Xmaxjitcodesize, that it doesn't bring anything and
> could crash the application
It probably provides no benefit, and could cause the app to run slowly if the JIT doesn't have enough room to store all the generated native code, causing recompilation of methods. It's not likely to cause a crash, but I suspect the main purpose of that option is to stress test the JIT.
> and cause all my hair to fall off
If it hasn't done so by now, that's probably not a concern. (I've still got mine.)
> The point is, after that I'm going to go through the same
> kind of exercise with Tomcat, and I want a feel for what
> I should or should not play with.
The main thing to adjust is the max heap size. Fiddling with the other stuff will buy you a lot less than streamlining your webapp code.
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Monitoring Tomcat with Jconsole
Posted by André Warnier <aw...@ice-sa.com>.
Caldarale, Charles R wrote:
[...]
Thanks for the many comments.
>
>> After this processing, the process in a "top" display shows
>> the following :
>> 22175 star 21 0 266m 23m 11m S 1.0 4.6 0:14.61 java
>> (where 266m is the virtual memory, and 11m the resident one).
>
> The top virtual number is just what's in the memory map for the process, so that includes files, code, non-Java stuff, etc., as well as the JVM heap. The resident memory number is the pages that happen to be in RAM at the moment.
>
>> Compared to the numbers I see in the jconsole VM summary below,
>> these memory figures seem to show a gross disproportion, or am I
>> interpreting this wrong ?
>
What I meant above was the apparent disproportion between the "266M"
number and what (I think) I see really used by the application according
to jconsole.
But from you comment above, I gather that I was wrongly fixated on that
number (266M) and that it does not really represent any real memory used
in any permanent way by the process. What really matters is the
"Resident" part of it (23M), as compared to how much physical RAM the
system really has, right ?
>> Snapshot 1
>> Current heap size:
>> 785 kbytes
>
> This is just the combined virtual size of the new and old generations; it does not include PermGen.
>
>> Maximum heap size:
>> 65.088 kbytes
>
> How big the new + old can get, if needed.
And can be controlled by the -Xmx<size> switch, yes ?
>
>> Committed memory:
>> 1.984 kbytes
>
> Includes the JIT's code pool, C heap, Java heap, PermGen, and various other bits and pieces the JVM knows about.
>
>> Committed virtual memory:
>> 270.980 kbytes
>> Total physical memory:
>> 516.920 kbytes
>> Free physical memory:
>> 6.852 kbytes
>> Total swap space:
>> 1.297.564 kbytes
>> Free swap space:
>> 1.234.376 kbytes
>
> All of the above are for the entire system, not just this one process.
>
Understood.
>> Snapshot 2
>> Current heap size:
>> 1.828 kbytes
>
> The heap has expanded a bit.
>
>> Maximum heap size:
>> 65.088 kbytes
>
> This can't change.
>
>> Committed memory:
>> 2.452 kbytes
>
> Increased due to heap expansion plus probable enlarging of the code pool, C heap, PermGen, etc.
>
> Nothing unexpected here.
>
Ok.
Now a final question on this app :
By running the requests that lead to the snapshot 2 above, I believe I
have pretty much "stressed" that app as much as I can (they were
relatively "heavy" requests, involving a fair amount of XML parsing).
After watching jconsole for a while, I noted that the total heap space
used always remained below 8 Mb, and the non-heap space remained below 9
Mb at all times. So, I have incrementally brought down the value of some
switches, to this level :
-server -Xms16M -Xmx32M -Xmaxjitcodesize16M
without apparently having any effect on the application's ability to
process requests. My idea in doing this was to reduce the amount of
system memory this application maybe reserves for no benefit, and thus
leave more real RAM for others.
Considering the numbers above, in your opinion is there anything I
could/should still try, while still leaving a margin for the application
to support maybe a higher level of requests ?
I am not looking for definite answers here, just trying to get a feel
for the stuff, what is important or not.
Like for instance if you told me to never touch this -Xmaxjitcodesize,
that it doesn't bring anything and could crash the application and cause
all my hair to fall off, or anything along those lines.
The point is, after that I'm going to go through the same kind of
exercise with Tomcat, and I want a feel for what I should or should not
play with.
Thanks
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Monitoring Tomcat with Jconsole
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: André Warnier [mailto:aw@ice-sa.com]
> Subject: Re: Monitoring Tomcat with Jconsole
>
> I don't know if the jconsole offers a cleaner way of doing this
I'm not aware of any mechanism in JConsole to send the output anywhere, other than via the clipboard.
> After this processing, the process in a "top" display shows
> the following :
> 22175 star 21 0 266m 23m 11m S 1.0 4.6 0:14.61 java
> (where 266m is the virtual memory, and 11m the resident one).
The top virtual number is just what's in the memory map for the process, so that includes files, code, non-Java stuff, etc., as well as the JVM heap. The resident memory number is the pages that happen to be in RAM at the moment.
> Compared to the numbers I see in the jconsole VM summary below,
> these memory figures seem to show a gross disproportion, or am I
> interpreting this wrong ?
> Snapshot 1
> Current heap size:
> 785 kbytes
This is just the combined virtual size of the new and old generations; it does not include PermGen.
> Maximum heap size:
> 65.088 kbytes
How big the new + old can get, if needed.
> Committed memory:
> 1.984 kbytes
Includes the JIT's code pool, C heap, Java heap, PermGen, and various other bits and pieces the JVM knows about.
> Committed virtual memory:
> 270.980 kbytes
> Total physical memory:
> 516.920 kbytes
> Free physical memory:
> 6.852 kbytes
> Total swap space:
> 1.297.564 kbytes
> Free swap space:
> 1.234.376 kbytes
All of the above are for the entire system, not just this one process.
> Snapshot 2
> Current heap size:
> 1.828 kbytes
The heap has expanded a bit.
> Maximum heap size:
> 65.088 kbytes
This can't change.
> Committed memory:
> 2.452 kbytes
Increased due to heap expansion plus probable enlarging of the code pool, C heap, PermGen, etc.
Nothing unexpected here.
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Monitoring Tomcat with Jconsole
Posted by André Warnier <aw...@ice-sa.com>.
Caldarale, Charles R wrote:
>> From: André Warnier [mailto:aw@ice-sa.com]
>> Subject: Re: Monitoring Tomcat with Jconsole
>>
>> This script by the way (not one of mine) does a "cd" into
>> the java/bin directory and then launches the program from
>> there. Can that have something to do with it ?
>
> The cd won't be a problem, but I'm wondering if the script is discarding or otherwise corrupting parameters it doesn't recognize. Try bypassing the script and running the app directly using java/bin/java with the JMX parameters.
>
Ha ! Why did I not think of that myself ?
That works fine, now we're cooking.
Great thanks, I was a bit puzzled there.
(I'll go and have a closer look at that script, too).
I think I was fooled by the command-line that the script itself prints
to the screen. It's obviously not the one it is really running.
So now, I am enjoying the display, but I am not quite sure how to
interpret what I'm seeing.
I don't know if the jconsole offers a cleaner way of doing this, but
here below I copied and pasted what I could from the "VM summary" tab.
Snapshot 1 is after the app launched, but had not yet done anything.
Snapshot 2 is after processing 2 requests, which involve parsing an XML
request, talking to a database, getting the result back, creating an XML
response from that and sending it back to the client.
After this processing, the process in a "top" display shows the following :
22175 star 21 0 266m 23m 11m S 1.0 4.6 0:14.61 java
(where 266m is the virtual memory, and 11m the resident one).
Compared to the numbers I see in the jconsole VM summary below, these
memory figures seem to show a gross disproportion, or am I interpreting
this wrong ?
Snapshot 1
VM Summary
Montag, 24. November 2008 22.07 Uhr CET
Connection name:
monitorRole@arthur:11200
Virtual Machine:
Java HotSpot(TM) Client VM version 1.5.0_14-b03
Vendor:
Sun Microsystems Inc.
Name:
22175@arthur
Uptime:
2 minutes
Process CPU time:
5,810 seconds
JIT compiler:
HotSpot Client Compiler
Total compile time:
1,168 seconds
Live threads:
14
Peak:
16
Daemon threads:
12
Total threads started:
19
Current classes loaded:
1.140
Total classes loaded:
1.140
Total classes unloaded:
0
Current heap size:
785 kbytes
Maximum heap size:
65.088 kbytes
Committed memory:
1.984 kbytes
Pending finalization:
0 objects
Garbage collector:
Name = 'MarkSweepCompact', Collections = 0, Total time spent = 0,000 seconds
Garbage collector:
Name = 'Copy', Collections = 23, Total time spent = 0,192 seconds
Operating System:
Linux 2.6.18-6-686
Architecture:
i386
Number of processors:
1
Committed virtual memory:
270.980 kbytes
Total physical memory:
516.920 kbytes
Free physical memory:
6.852 kbytes
Total swap space:
1.297.564 kbytes
Free swap space:
1.234.376 kbytes
Snapshot 2
VM Summary
Montag, 24. November 2008 22.11 Uhr CET
Connection name:
monitorRole@arthur:11200
Virtual Machine:
Java HotSpot(TM) Client VM version 1.5.0_14-b03
Vendor:
Sun Microsystems Inc.
Name:
22175@arthur
Uptime:
6 minutes
Process CPU time:
11,930 seconds
JIT compiler:
HotSpot Client Compiler
Total compile time:
2,312 seconds
Live threads:
13
Peak:
19
Daemon threads:
11
Total threads started:
35
Current classes loaded:
1.373
Total classes loaded:
1.416
Total classes unloaded:
43
Current heap size:
1.828 kbytes
Maximum heap size:
65.088 kbytes
Committed memory:
2.452 kbytes
Pending finalization:
0 objects
Garbage collector:
Name = 'MarkSweepCompact', Collections = 0, Total time spent = 0,000 seconds
Garbage collector:
Name = 'Copy', Collections = 60, Total time spent = 0,868 seconds
Operating System:
Linux 2.6.18-6-686
Architecture:
i386
Number of processors:
1
Committed virtual memory:
273.300 kbytes
Total physical memory:
516.920 kbytes
Free physical memory:
18.020 kbytes
Total swap space:
1.297.564 kbytes
Free swap space:
1.234.376 kbytes
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Monitoring Tomcat with Jconsole
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: André Warnier [mailto:aw@ice-sa.com]
> Subject: Re: Monitoring Tomcat with Jconsole
>
> This script by the way (not one of mine) does a "cd" into
> the java/bin directory and then launches the program from
> there. Can that have something to do with it ?
The cd won't be a problem, but I'm wondering if the script is discarding or otherwise corrupting parameters it doesn't recognize. Try bypassing the script and running the app directly using java/bin/java with the JMX parameters.
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Monitoring Tomcat with Jconsole
Posted by André Warnier <aw...@ice-sa.com>.
Caldarale, Charles R wrote:
>> From: André Warnier [mailto:aw@ice-sa.com]
>> Subject: Re: Monitoring Tomcat with Jconsole
>
>> Can I "provoke" some meaningful message by misspelling one of these -D
>> switches, just to check ?
>
> Not by misspelling, but if you change the port number to alphabetics, it will complain.
>
> Are you sure you're using a 1.5 HotSpot JVM? What does "java -version" say?
>
I added "./java -version" in the script that starts this application.
Just for the fun, I also changed the port number to ABCDE.
This is what it now says :
java version "1.5.0_14"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_14-b03)
Java HotSpot(TM) Client VM (build 1.5.0_14-b03, mixed mode, sharing)
./java -Dcom.sun.management.jmxremote.port=ABCDE
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.password.file=/home/star/xml/config/management.jmxremote.password
-Dpgm=STARXMLServer -jar "/home/star/xml/STARXMLServer.jar"
Press Control-C to stop STAR XML Server.
Server Version 4.4.2 Started on Port 11100
So yes, it is 1.5, and it doesn't seem to care a bit about the port number.
??
This script by the way (not one of mine) does a "cd" into the java/bin
directory and then launches the program from there. Can that have
something to do with it ?
Maybe I should have started with Tomcat after all..
(I just didn't because there were other people working with that Tomcat
when I started my study of jconsole)
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Monitoring Tomcat with Jconsole
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: André Warnier [mailto:aw@ice-sa.com]
> Subject: Re: Monitoring Tomcat with Jconsole
> Can I "provoke" some meaningful message by misspelling one of these -D
> switches, just to check ?
Not by misspelling, but if you change the port number to alphabetics, it will complain.
Are you sure you're using a 1.5 HotSpot JVM? What does "java -version" say?
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Monitoring Tomcat with Jconsole
Posted by André Warnier <aw...@ice-sa.com>.
Caldarale, Charles R wrote:
>> From: André Warnier [mailto:aw@ice-sa.com]
>> Subject: Monitoring Tomcat with Jconsole
>
> ./java "-Dcom.sun.management.jmxremote.port=11200
> -Dcom.sun.management.jmxremote.ssl=false
> -Dcom.sun.management.jmxremote.password.file=/mydir/
> management.jmxremote.password"
> -Dpgm=STARXMLServer -jar "/home/star/xml/STARXMLServer.jar"
>
> Remove the quotes; each -D setting must be a separate parameter, but you've lumped them all into one.
>
>> One think I noticed, is that on the server myhost, despite
>> the daemon application being loaded without any apparent
>> error messages, and being responsive to requests on its
>> own port 11100, a netstat -an does not show any port 11200
>> in a LISTEN state.
>>
>> I do not know at this stage if this is significant.
>
> It is. Without that port being in the listening state, you'll never get a JMX connection.
>
I did remove the quotes, so now it is started as :
./java -Dcom.sun.management.jmxremote.port=11200
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.password.file=/home/star/xml/config/management.jmxremote.password
-Dpgm=STARXMLServer -jar "/home/star/xml/STARXMLServer.jar"
But the end result is the same : I can still not connect, and still get
the same error message in jconsole :
Connection failed: Connection refused: connect
and it still does not show any port 11200 in LISTEN state on the server.
No error messages shown on-screen either when starting that server app
in a command-line console.
The application on the other hand works fine, accepts connections on its
own port 11100, and prints status messages on-screen.
The message "connection refused" leads me to believe that the problem is
on the server side, which is confirmed by the port 11200 not even being
listening. But if the JVM on that side "sees" these -D command-line
parameters, should it not be printing error messages somewhere if it
cannot process them ?
Can I "provoke" some meaningful message by misspelling one of these -D
switches, just to check ?
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Monitoring Tomcat with Jconsole
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: André Warnier [mailto:aw@ice-sa.com]
> Subject: Monitoring Tomcat with Jconsole
./java "-Dcom.sun.management.jmxremote.port=11200
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.password.file=/mydir/
management.jmxremote.password"
-Dpgm=STARXMLServer -jar "/home/star/xml/STARXMLServer.jar"
Remove the quotes; each -D setting must be a separate parameter, but you've lumped them all into one.
> One think I noticed, is that on the server myhost, despite
> the daemon application being loaded without any apparent
> error messages, and being responsive to requests on its
> own port 11100, a netstat -an does not show any port 11200
> in a LISTEN state.
>
> I do not know at this stage if this is significant.
It is. Without that port being in the listening state, you'll never get a JMX connection.
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org