You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Christopher Schultz <ch...@christopherschultz.net> on 2012/01/26 19:32:23 UTC
[OT] Inspecting JMX
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
All,
I'm trying to get into the business of inspecting my JVM using the
mbeans that Tomcat (and the JVM) expose, but I'm having great
difficulty. I'm not even sure if I'm doing the right things.
I have a running Tomcat instance (happens to be 7.0.25 running on Sun
JRE 1.6.something on Mac OS X Lion), and I can happily connect to it
using jconsole (using a process id) with no special startup parameters
for Tomcat itself.
If I understand this correctly, it's because when the JVM starts up,
it allows localhost JMX inspection with no authentication using the
"Attach" API [1].
I have enabled neither Tomcat's
mbeans.GlobalResourcesLifecycleListener not the
JmxRemoveLifecycleListener [2].
I can see a wealth of information about the JVM and Tomcat in this
configuration.
Now I'm trying to get similar information using a command-line tool
that is very simple called check_jmx -- it's a plug-in for Nagios. It
appears that this tool does not support the "attach" API and so it
looks like I'll have to enable "remote JMX", so I've followed the
instructions on Tomcat's monitoring page to enable remote JMX [3]:
$ export CATALINA_OPTS='\
- -Dcom.sun.management.jmxremote \
- -Dcom.sun.management.jmxremote.port=1234 \
- -Dcom.sun.management.jmxremote.ssl=false \
- -Dcom.sun.management.jmxremote.authenticate=false'
When running Tomcat, I can see that those parameters are actually part
of the command line. I can also see that the port is bound:
$ netstat -an | grep 1234
tcp46 0 0 *.1234 *.*
LISTEN
tcp4 0 0 127.0.0.1.12345 *.*
LISTEN
tcp6 0 0 ::1.12345 *.*
LISTEN
I can use this tool with a command line such as the following:
$ ./check_jmx -U service:jmx:rmi:///jndi/rmi://localhost:1234/jmxrmi \
-O java.lang:type=Memory -A HeapMemoryUsage -K used \
-I HeapMemoryUsage -J used -vvvv -w 4248302272 -c 5498760192
JMX OK
HeapMemoryUsage.used=35587776{committed=85000192;init=0;max=129957888;used=35587776}
Without going into too much detail, those options allow check_jmx to
probe a certain value, then use the -w and -c options to determine
whether the value obtained from the JMX server is okay, in the warning
zone, or in the critical zone, and then builds a text response based
upon that.
So, now I want to move to my server which happens to be an Amazon EC2
instance with an elastic IP assigned to it so I can get to it from the
outside. Most ports are blocked. I try the same type of thing: set up
the com.sun.management.etc parameters and launch Tomcat. netstat reports:
$ netstat -plan | grep 1234
tcp6 0 0 :::1234 :::*
LISTEN 2819/java
So I should be good to go. I launch the command the same as on my
local machine, but this time it hangs and then I get an exception:
$ ./check_jmx -U service:jmx:rmi:///jndi/rmi://localhost:1234/jmxrmi
- -O java.lang:type=Memory -A HeapMemoryUsage -K used -I HeapMemoryUsage
- -J used -vvvv -w 4248302272 -c 5498760192
JMX CRITICAL Connection refused to host: [public IP]; nested exception
is:
java.net.ConnectException: Connection timed out connecting to
java.lang:type=Memory by URL
service:jmx:rmi:///jndi/rmi://localhost:1234/jmxrmijava.rmi.ConnectException:
Connection refused to host: 107.21.113.203; nested exception is:
java.net.ConnectException: Connection timed out
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:601)
at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:198)
at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:184)
at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:110)
at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown
Source)
at
javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2329)
at
javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:279)
at
javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:248)
at
javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:207)
at org.nagios.JMXQuery.connect(JMXQuery.java:53)
at org.nagios.JMXQuery.main(JMXQuery.java:75)
Caused by: java.net.ConnectException: Connection timed out
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:529)
at java.net.Socket.connect(Socket.java:478)
at java.net.Socket.<init>(Socket.java:375)
at java.net.Socket.<init>(Socket.java:189)
at
sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(RMIDirectSocketFactory.java:22)
at
sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(RMIMasterSocketFactory.java:128)
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:595)
... 10 more
So, [public IP] is actually my public, IPv4 address. I figure the
problem might be IPv4-versus-IPv6, but before I go "preferring" the
IPv4 stack, I check netstat during the execution of check_jmx:
$ netstat -plan | grep 1234
tcp6 0 0 :::1234 :::*
LISTEN 2819/java
tcp6 0 0 127.0.0.1:40292 127.0.0.1:1234
ESTABLISHED 2950/java
tcp6 0 0 127.0.0.1:1234 127.0.0.1:40292
ESTABLISHED 2819/java
(pid 2819 is my Tomcat process, and 2950 is check_jmx). So, what else
is check_jmx doing?
$ netstat -plan | grep 2950
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
tcp6 0 0 127.0.0.1:40292 127.0.0.1:1234
ESTABLISHED 2950/java
tcp6 0 1 10.192.214.127:44419 107.21.113.203:51087
SYN_SENT 2950/java
So, check_jmx is actually successfully establishing a socket with the
JMX port but it looks like it's doing something weird with the
(random) RMI port. First, it's trying to use the public IP address
instead of just localhost (which seems stupid, but that's probably
what the JMX service told it to do). Second, it's calling-out on the
internet IP address, which is probably a result of using a
non-localhost destination IP address.
So, a couple of questions:
1. Is there a way to tell the JVM which interface to use for the
second, random RMI port that gets opened? I think if everything runs
on :: or 127.0.0.1 then all will probably be well.
2. Should I just give up and use JmxRemoteLifecycleListener? Will that
actually buy me anything? I will always be running check_jmx
locally. I'm not s huge fan of having to deploy with an optional
library tossed-into Tomcat's lib direcory (even though it's actually
from the find Apache folks).
3. Should I just give up and use the manager app's jmxproxy? I don't
currently deploy the manager app, and I'd like to avoid doing that
if possible. But, it may be a slightly cleaner solution.
4. Should I hack the code for check_jmx to use the Attach API and
try to avoid all of this stupid port business? Getting the PID
of the Tomcat process shouldn't be hard as long as I use
CATALINA_PID and get the value from there.
Those playing along at home might remember that I recently posted a
question about Kitty, another JMX CLI program. The documentation for
Kitty has the same type of ugly RMI URLs in it, so I suspect I'll have
the same problem with another tool: I think this is squarely a
server-process configuration issue that I have to deal with.
Any suggestions would be most appreciated.
Thanks,
- -chris
[1]
http://docs.oracle.com/javase/6/docs/technotes/guides/management/agent.html
[2]
http://tomcat.apache.org/tomcat-7.0-doc/config/listeners.html#JMX_Remote_Lifecycle_Listener_-_org.apache.catalina.mbeans.JmxRemoteLifecycleListener
[3]
http://tomcat.apache.org/tomcat-7.0-doc/monitoring.html#Enabling_JMX_Remote
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk8hnDcACgkQ9CaO5/Lv0PCFqgCgm0Ai/TITuEPzCIuwZvRF7WGR
SzsAmgKtgVD4Mzv4QBs4Yq4UycXwuRDd
=V/nO
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: [OT] Inspecting JMX
Posted by Mark Eggers <it...@yahoo.com>.
Chris,
(comments inline)
----- Original Message -----
> From: Christopher Schultz <ch...@christopherschultz.net>
> To: Tomcat Users List <us...@tomcat.apache.org>
> Cc:
> Sent: Tuesday, January 31, 2012 12:29 PM
> Subject: Re: [OT] Inspecting JMX
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Rainer,
>
> On 1/28/12 8:51 AM, Rainer Jung wrote:
>> On 26.01.2012 19:32, Christopher Schultz wrote:
>>
>>> Now I'm trying to get similar information using a command-line
>>> tool that is very simple called check_jmx -- it's a plug-in for
>>> Nagios. It appears that this tool does not support the
> "attach"
>>> API and so it looks like I'll have to enable "remote
> JMX", so
>>> I've followed the instructions on Tomcat's monitoring page to
>>> enable remote JMX [3]:
>>
>>
>>> 3. Should I just give up and use the manager app's jmxproxy? I
>>> don't currently deploy the manager app, and I'd like to avoid
>>> doing that if possible. But, it may be a slightly cleaner
>>> solution.
>>>
>>> 4. Should I hack the code for check_jmx to use the Attach API
>>> and try to avoid all of this stupid port business? Getting the
>>> PID of the Tomcat process shouldn't be hard as long as I use
>>> CATALINA_PID and get the value from there.
>>
>> Note that the naive check_jmx attempt will not scale. Monitoring
>> JVMs using JMX by starting a new JVM on the polling server for each
>> poll and each monitored instance will soon killk your monitoring
>> server.
>
> Yeah, we're using check_nrpe to invoke the check_jmx processes on the
> server that is being monitored, but I expect that we will be
> monitoring half a dozen JMX variables on several VMs, and that means
> lots of JVMs being fired up just to connect and ask a simple question.
>
>> You either need to use an agent running in the target JVM and
>> providing access via a simpler non-Java protocol, or you need a
>> long running Java based gateway, which does the JMX communication
>> with the target JVMs and gets itself queried with something
>> simpler.
>
> Are you suggesting a long-running process with a persistent connection
> to the JVM, so that a new JVM for each request can be avoided? Someone
> has to have done that already, right?
You might take a look at collectd (http://collectd.org/ ). There's a Java and JMX plugin available.
You could run that, connect to your local Tomcat, gather statistics, and then send the information back to a central server with the network plugin. It appears that there is an option to encrypt the information as well.
On the central server, you could equip the collectd daemon with the collectd-nagios program to inform Nagios about trouble. Unfortunately, that approach looks a little inflexible at the moment.
One other issue is that collectd just generates information and sends it off (network, RRD, CSV). What you do with it after that is up to you. While there are examples given in the documentation about how to deal with the information (mostly focused on RRD), the actual work is left as an exercise for the implementer.
Finally, the collectd tool is designed to run on UNIX/Linux. While this fits your clients in the cloud, it may or may not fit your server platform. There is a collectd to JMX program written in Java, but I couldn't get it to work with the later versions of collectd. I suspect that someone will have to write a collectd server that implements the bare necessities (network, RRD, CSV) if your collection / server platform is Windows.
>> In that sense the Tomcat Manager can act as an agent via its
>> jmxproxy feature, making JMX data available for each HTTP client
>> that can parse simle text output.
>
> I may consider doing that, though we aren't using the manager app for
> anything else right now. I'll see if I can use it without allowing
> access to any other services (like deploy, etc.).
>
>> Another a bit more sophisticated approach which can be well
>> integrated with Nagios is Jmx4Perl as a client in combination with
>> Jolikia as the agent (all Open Source).
>
> I'll check that out.
>
>> Of course there are many more options available.
>
> I'll document what I find on the wiki. I'm surprised that there's
> little monitoring information on the Tomcat site. :(
>
> - -chris
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAk8oTw4ACgkQ9CaO5/Lv0PAahQCfQ2NreQ55K10o4DVJQzGRxv9R
> ZlEAoJ/mc8sUL+q3CDYxJAV1+7XAUib+
> =FpZG
> -----END PGP SIGNATURE-----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
. . . . just my two cents.
/mde/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: [OT] Inspecting JMX
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Rainer,
On 1/28/12 8:51 AM, Rainer Jung wrote:
> On 26.01.2012 19:32, Christopher Schultz wrote:
>
>> Now I'm trying to get similar information using a command-line
>> tool that is very simple called check_jmx -- it's a plug-in for
>> Nagios. It appears that this tool does not support the "attach"
>> API and so it looks like I'll have to enable "remote JMX", so
>> I've followed the instructions on Tomcat's monitoring page to
>> enable remote JMX [3]:
>
>
>> 3. Should I just give up and use the manager app's jmxproxy? I
>> don't currently deploy the manager app, and I'd like to avoid
>> doing that if possible. But, it may be a slightly cleaner
>> solution.
>>
>> 4. Should I hack the code for check_jmx to use the Attach API
>> and try to avoid all of this stupid port business? Getting the
>> PID of the Tomcat process shouldn't be hard as long as I use
>> CATALINA_PID and get the value from there.
>
> Note that the naive check_jmx attempt will not scale. Monitoring
> JVMs using JMX by starting a new JVM on the polling server for each
> poll and each monitored instance will soon killk your monitoring
> server.
Yeah, we're using check_nrpe to invoke the check_jmx processes on the
server that is being monitored, but I expect that we will be
monitoring half a dozen JMX variables on several VMs, and that means
lots of JVMs being fired up just to connect and ask a simple question.
> You either need to use an agent running in the target JVM and
> providing access via a simpler non-Java protocol, or you need a
> long running Java based gateway, which does the JMX communication
> with the target JVMs and gets itself queried with something
> simpler.
Are you suggesting a long-running process with a persistent connection
to the JVM, so that a new JVM for each request can be avoided? Someone
has to have done that already, right?
> In that sense the Tomcat Manager can act as an agent via its
> jmxproxy feature, making JMX data available for each HTTP client
> that can parse simle text output.
I may consider doing that, though we aren't using the manager app for
anything else right now. I'll see if I can use it without allowing
access to any other services (like deploy, etc.).
> Another a bit more sophisticated approach which can be well
> integrated with Nagios is Jmx4Perl as a client in combination with
> Jolikia as the agent (all Open Source).
I'll check that out.
> Of course there are many more options available.
I'll document what I find on the wiki. I'm surprised that there's
little monitoring information on the Tomcat site. :(
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk8oTw4ACgkQ9CaO5/Lv0PAahQCfQ2NreQ55K10o4DVJQzGRxv9R
ZlEAoJ/mc8sUL+q3CDYxJAV1+7XAUib+
=FpZG
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: [OT] Inspecting JMX
Posted by Rainer Jung <ra...@kippdata.de>.
On 26.01.2012 19:32, Christopher Schultz wrote:
> Now I'm trying to get similar information using a command-line tool
> that is very simple called check_jmx -- it's a plug-in for Nagios. It
> appears that this tool does not support the "attach" API and so it
> looks like I'll have to enable "remote JMX", so I've followed the
> instructions on Tomcat's monitoring page to enable remote JMX [3]:
> 3. Should I just give up and use the manager app's jmxproxy? I don't
> currently deploy the manager app, and I'd like to avoid doing that
> if possible. But, it may be a slightly cleaner solution.
>
> 4. Should I hack the code for check_jmx to use the Attach API and
> try to avoid all of this stupid port business? Getting the PID
> of the Tomcat process shouldn't be hard as long as I use
> CATALINA_PID and get the value from there.
Note that the naive check_jmx attempt will not scale. Monitoring JVMs
using JMX by starting a new JVM on the polling server for each poll and
each monitored instance will soon killk your monitoring server.
You either need to use an agent running in the target JVM and providing
access via a simpler non-Java protocol, or you need a long running Java
based gateway, which does the JMX communication with the target JVMs and
gets itself queried with something simpler.
In that sense the Tomcat Manager can act as an agent via its jmxproxy
feature, making JMX data available for each HTTP client that can parse
simle text output.
Another a bit more sophisticated approach which can be well integrated
with Nagios is Jmx4Perl as a client in combination with Jolikia as the
agent (all Open Source).
Of course there are many more options available.
Regards,
Rainer
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: [OT] Inspecting JMX
Posted by André Warnier <aw...@ice-sa.com>.
Christopher Schultz wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> André,
>
> On 1/26/12 3:30 PM, André Warnier wrote:
>> independently of anything else, you may want to have a look at
>> jmxsh http://code.google.com/p/jmxsh/
>
> Now that I have *something* working, I can experiment to see what
> other things work as well :)
>
>> It is interactive as well as scriptable. (If I, who do not
>> understand much about JMX and mbeans (or even java), managed to use
>> it with Tomcat and other stand-alone java applications, it means
>> that it is really simple).
>
> If you got lost in my original post and you managed to get JMX
> monitoring up and running, then I think you might be on to something,
> here.
>
I needed a simple scriptable tool at some point in the past, to force Tomcat (and other
Java programs) to do a major GC from time to time, to get rid of some dangling native
socket left open in an inaccessible (but not yet collected) java object.
I believe it was Chuck who pointed me to jmxsh originally.
I only used it further by curiosity, without really understanding what I was doing.
But I know one useful feature is that it allows one to browse interactively, to find the
available JMX?/mbeans?.
Example : force (the JVM running) Tomcat to do a GC :
# gc_tomcat.jmxsh
# force the target JVM to do a GC, via the jmxsh shell.
# invoke as :
# java -jar jmxsh.jar gc_tomcat.jmxsh
#
# In the following command, replace the port number (-p)
# by the port which has been specified in the parameter :
# -Dcom.sun.management.jmxremote.port=xxxxx
# of the target Java JVM startup line
# (and so on for -U and -P)
jmx_connect -h 127.0.0.1 -p 8200 -U controlRole -P mypass
jmx_invoke -m java.lang:type=Memory gc
jmx_close
To get the interactive shell, use the command "java -jar jmxsh.jar".
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: [OT] Inspecting JMX
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
André,
On 1/26/12 3:30 PM, André Warnier wrote:
> independently of anything else, you may want to have a look at
> jmxsh http://code.google.com/p/jmxsh/
Now that I have *something* working, I can experiment to see what
other things work as well :)
> It is interactive as well as scriptable. (If I, who do not
> understand much about JMX and mbeans (or even java), managed to use
> it with Tomcat and other stand-alone java applications, it means
> that it is really simple).
If you got lost in my original post and you managed to get JMX
monitoring up and running, then I think you might be on to something,
here.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk8huQsACgkQ9CaO5/Lv0PDGcQCeNke1z3nPSnQJvL4bw2pmYXIb
yZkAoJ8Sk9Y3q+vow6eWeChbVxloSzmW
=fG1y
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: [OT] Inspecting JMX
Posted by André Warnier <aw...@ice-sa.com>.
Chris,
independently of anything else, you may want to have a look at jmxsh
http://code.google.com/p/jmxsh/
It is interactive as well as scriptable.
(If I, who do not understand much about JMX and mbeans (or even java), managed to use it
with Tomcat and other stand-alone java applications, it means that it is really simple).
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: [OT] Inspecting JMX
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Kees Jan,
On 1/26/12 1:52 PM, Kees Jan Koster wrote:
> Dear Chris,
>
>>>> Should I just give up and use JmxRemoteLifecycleListener?
>>>
>>> Yes. You really aren't get to get anywhere on EC2 without it.
>>
>> I figured since I was using localhost everything would stay
>> there anyway. I'll give JmxRemoteLifecycleListener a chance.
>
>
> Or you could write a small piece of code that lives inside your
> app and writes the monitoring data into the Nagios queue directly.
> It is a simple text-based protocol.
I'm monitoring the server remotely, so writing-out isn't currently in
the cards.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk8hrLwACgkQ9CaO5/Lv0PBoTACgqpH7uRGVVldGXPCouTPtzLy7
vqwAn0MQYyoZ+c0+FHYB52h0cQ7YX1Vw
=xNPw
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: [OT] Inspecting JMX
Posted by Kees Jan Koster <kj...@gmail.com>.
Dear Chris,
>>> Should I just give up and use JmxRemoteLifecycleListener?
>>
>> Yes. You really aren't get to get anywhere on EC2 without it.
>
> I figured since I was using localhost everything would stay there
> anyway. I'll give JmxRemoteLifecycleListener a chance.
Or you could write a small piece of code that lives inside your app and writes the monitoring data into the Nagios queue directly. It is a simple text-based protocol.
--
Kees Jan
http://java-monitor.com/
kjkoster@kjkoster.org
+31651838192
Human beings make life so interesting. Do you know that in a universe so full of wonders,
they have managed to invent boredom. Quite astonishing... -- Terry Pratchett
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: [OT] Inspecting JMX
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Chuck,
On 1/26/12 1:41 PM, Caldarale, Charles R wrote:
>> From: Christopher Schultz [mailto:chris@christopherschultz.net]
>> Subject: [OT] Inspecting JMX
>
>> Should I just give up and use JmxRemoteLifecycleListener?
>
> Yes. You really aren't get to get anywhere on EC2 without it.
I figured since I was using localhost everything would stay there
anyway. I'll give JmxRemoteLifecycleListener a chance.
>> Should I hack the code for check_jmx to use the Attach API.
>
> Maybe; might be fun. (Always seemed to be a security hole to me
> :-)
I'm not sure you can disable it at the JVM level, can you? Actually, I
guess I could tell the JMX server to require a valid client cert, and
then just have an empty trust store :)
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk8hn08ACgkQ9CaO5/Lv0PCfoQCfThzcyl5sp9FAJ0f4RWcjTmUg
lSkAnAxZ2X9AowBCfgmkGWFzIgkMTLIh
=xQA4
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: [OT] Inspecting JMX
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Pid,
On 1/26/12 5:35 PM, Pid wrote:
> On 26/01/2012 20:33, Christopher Schultz wrote:
>> Chuck
>>
>> On 1/26/12 1:41 PM, Caldarale, Charles R wrote:
>>>> From: Christopher Schultz
>>>> [mailto:chris@christopherschultz.net] Subject: [OT]
>>>> Inspecting JMX
>>
>>>> Should I just give up and use JmxRemoteLifecycleListener?
>>
>>> Yes. You really aren't get to get anywhere on EC2 without it.
>>
>> Okay, so I enabled the JmxRemoteLifecycleListener like this:
>>
>> <Listener
>> className="org.apache.catalina.mbeans.JmxRemoteLifecycleListener"
>>
>>
rmiRegistryPortPlatform="1234"
>> rmiServerPortPlatform="1235" />
>>
>> ... and had the same problem: the client connects to the registry
>> port and then tries to connect to [public IP]:1235.
>>
>> So, I tried adding useLocalPorts="true" to the above and not I'm
>> able to connect -- using the same URL as before, actually:
>
> In addition to the valve...
>
> See if you can get the unauthenticated version to work first, you
> will probably need to set the hostname, too:
>
> -Djava.rmi.server.hostname=www.externalhostname.com
> -Dcom.sun.management.jmxremote=true
> -Dcom.sun.management.jmxremote.ssl=false
> -Dcom.sun.management.jmxremote.authenticate=false
Hmm... I'll have to look at using the hostname: that might solve the
initial problem. The problem is that the RMI server is using the
public hostname of the server to redirect connections, and I want it
to use localhost because I don't intend to expose JMX services to the
world.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk8oTdgACgkQ9CaO5/Lv0PCStgCeKziudhrsOYhW9/Zed3eQrIsc
V6QAn0qcPWrla2owvcSk6i9qNcz6PU2c
=yWRu
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: [OT] Inspecting JMX
Posted by Pid <pi...@pidster.com>.
On 26/01/2012 20:33, Christopher Schultz wrote:
> Chuck
>
> On 1/26/12 1:41 PM, Caldarale, Charles R wrote:
>>> From: Christopher Schultz [mailto:chris@christopherschultz.net]
>>> Subject: [OT] Inspecting JMX
>
>>> Should I just give up and use JmxRemoteLifecycleListener?
>
>> Yes. You really aren't get to get anywhere on EC2 without it.
>
> Okay, so I enabled the JmxRemoteLifecycleListener like this:
>
> <Listener
> className="org.apache.catalina.mbeans.JmxRemoteLifecycleListener"
> rmiRegistryPortPlatform="1234"
> rmiServerPortPlatform="1235"
> />
>
> ... and had the same problem: the client connects to the registry port
> and then tries to connect to [public IP]:1235.
>
> So, I tried adding useLocalPorts="true" to the above and not I'm able
> to connect -- using the same URL as before, actually:
In addition to the valve...
See if you can get the unauthenticated version to work first, you will
probably need to set the hostname, too:
-Djava.rmi.server.hostname=www.externalhostname.com
-Dcom.sun.management.jmxremote=true
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false
p
> ./check_jmx \
> -U service:jmx:rmi:///jndi/rmi://localhost:1234/jmxrmi \
> -O java.lang:type=Memory -A HeapMemoryUsage -K used \
> -I HeapMemoryUsage -J used -vvvv -w 4248302272 -c 5498760192
>
> JMX OK
> HeapMemoryUsage.used=39749208{committed=59572224;init=67108864;max=59703296;used=39749208}
>
> So, that's good. The JmxRemoteLifecycleListener installs a special
> RMIClientSocketFactory that always binds to "localhost" instead of
> whatever the default is that the JVM uses. I'm surprised there's not a
> way to tell the JVM to do this kind of thing. I wasn't able to find
> any canonical source of information about what system properties
> affect the JMX configuration... just lots of "simple" examples of how
> things should "just work". :(
>
> The nice thing about having the JmxRemoteLifecycleListener installed
> is that the ports are predictable and so I can tunnel them. Now I can
> use jconsole from my computer to observe whats happening on the server.
>
>>> Should I hack the code for check_jmx to use the Attach API.
>
>> Maybe; might be fun. (Always seemed to be a security hole to me
>> :-)
>
> It looks simple to code... I'll take a look at it. I'm sure I'll have
> some similar kind of problem.
>
> Thanks,
> -chris
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
--
[key:62590808]
Re: [OT] Inspecting JMX
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Chuck
On 1/26/12 1:41 PM, Caldarale, Charles R wrote:
>> From: Christopher Schultz [mailto:chris@christopherschultz.net]
>> Subject: [OT] Inspecting JMX
>
>> Should I just give up and use JmxRemoteLifecycleListener?
>
> Yes. You really aren't get to get anywhere on EC2 without it.
Okay, so I enabled the JmxRemoteLifecycleListener like this:
<Listener
className="org.apache.catalina.mbeans.JmxRemoteLifecycleListener"
rmiRegistryPortPlatform="1234"
rmiServerPortPlatform="1235"
/>
... and had the same problem: the client connects to the registry port
and then tries to connect to [public IP]:1235.
So, I tried adding useLocalPorts="true" to the above and not I'm able
to connect -- using the same URL as before, actually:
./check_jmx \
-U service:jmx:rmi:///jndi/rmi://localhost:1234/jmxrmi \
-O java.lang:type=Memory -A HeapMemoryUsage -K used \
-I HeapMemoryUsage -J used -vvvv -w 4248302272 -c 5498760192
JMX OK
HeapMemoryUsage.used=39749208{committed=59572224;init=67108864;max=59703296;used=39749208}
So, that's good. The JmxRemoteLifecycleListener installs a special
RMIClientSocketFactory that always binds to "localhost" instead of
whatever the default is that the JVM uses. I'm surprised there's not a
way to tell the JVM to do this kind of thing. I wasn't able to find
any canonical source of information about what system properties
affect the JMX configuration... just lots of "simple" examples of how
things should "just work". :(
The nice thing about having the JmxRemoteLifecycleListener installed
is that the ports are predictable and so I can tunnel them. Now I can
use jconsole from my computer to observe whats happening on the server.
>> Should I hack the code for check_jmx to use the Attach API.
>
> Maybe; might be fun. (Always seemed to be a security hole to me
> :-)
It looks simple to code... I'll take a look at it. I'm sure I'll have
some similar kind of problem.
Thanks,
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk8huJQACgkQ9CaO5/Lv0PDVggCdE0aT954QjHRRVRNSl6ieY/cA
JGUAn2pD2JbEjwiTl5Gy/t0TyVAmQViI
=3VmZ
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: [OT] Inspecting JMX
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: Christopher Schultz [mailto:chris@christopherschultz.net]
> Subject: [OT] Inspecting JMX
> Should I just give up and use JmxRemoteLifecycleListener?
Yes. You really aren't get to get anywhere on EC2 without it.
> Should I hack the code for check_jmx to use the Attach API.
Maybe; might be fun. (Always seemed to be a security hole to me :-)
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.