You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sis.apache.org by Martin Desruisseaux <ma...@geomatys.fr> on 2012/11/27 03:53:48 UTC
Experimenting the addition of a MBean
Hello all
Yesterday I added I feature that we didn't had before, which can be
removed from compilation if considered undesirable. We did not had
previously any monitoring tools about the state of a running instance of
the library. We found that it was sometime problematic, since it make
very hard to diagnostic problems on a server. I think there is numerous
places where JMX MBeans could be used. For example report how many
objects are cached, provide an option to clear the caches, detect if
there is a problem in the running SIS instance (e.g. a daemon thread
which unexpectedly died), etc.
So I added an experimental "Supervisor" MBean in the internal package.
For now it contains only one method, "warnings()", which report any
problem detected in the running SIS instance. The only problem reported
at this time is unexpected death of the daemon thread consuming the weak
references queue, but future versions could perform more extensive tests
of SIS state. This MBean can been seen in JConsole (i.e. if an
application using SIS is currently running, start "jconsole" in another
window, select the application in the proposed list, click on "MBeans",
select "org.apache.sis". The "warnings" operation can been seen).
The inconvenient is that SIS has to register its MBean to the
PlatformMBeanServer (the same instance than the one managing the
standard ClassLoadingJMXBean, MemoryJMXBean, GarbageCollectionJMXBean,
etc.), and I have no idea if it put a significant burden. In particular
I don't know if it trigs JMX class loading that would normally not
happen. In my few tests, I didn't saw noticeable slowdown at startup time...
Any though?
Martin
Re: Experimenting the addition of a MBean
Posted by Martin Desruisseaux <ma...@geomatys.fr>.
Hello Adam
Le 27/11/12 12:06, Adam Estrada a écrit :
> Its sort of like a service within a service, right?
Kind of. JMX is a standard way to expose attributes and operations to an
external monitoring application like JConsole is a way similar to RMI.
In a web application, we would have an administrator page with buttons
for querying informations or perform maintenance operations. In JMX
there is only the programmatic interfaces and the visual widgets is left
to the external monitoring application.
> I have seen this used in ElasticSearch (I'm not sure what its referred to in there exactly) and it seems to work really well. There is even a config file that allows you to turn the verbosity level up too which is a nice feature.
With JMX there would be no need for a config file, since changes in the
verbosity level could be operations. Actually the standard JDK already
provides a MBean for exactly that:
http://docs.oracle.com/javase/7/docs/api/java/lang/management/PlatformLoggingMXBean.html
> For SIS, I see this as being used in really long running processes like tile cache generation and maybe even image/vector mosaicking for future versions. Where do you see this being used first?
I see it used in Web Map Server, Web Coverage Server or any other kind
of servers. But this is true that mosaic creation and other long running
process are also targets.
> You point out monitoring internal object caches but what about externally facing client caches?
It depends what we call "client". On a webserver - browser architecture,
the browser side is more a JavaScript world in which case SIS would not
affect caching. On a desktop application using SIS locally, client cache
could also be monitored through JMX is desired. The application can
provide its own MBean. Or maybe the application would just rely on SIS
internal caching, in which case those caches would be visible be the SIS
MBeans.
> I've seen other java-based geo-apps totally puke on long running processes and I've had no clue what went wrong. I feel like what you're suggesting could help alleviate some of these pain points within SIS.
Yes this is exactly the point. One of our clients having various
web-applications from various projects, unconditionally reboot their
Tomcat servers every night for avoiding performance degradation and
other problems seen in "long" running processes (while one day is not
supposed to be long). This is the kind of situation that we would like
to avoid, and monitoring (together with logging and other approaches)
are one tools among other which may hopefully help.
Martin
Re: Experimenting the addition of a MBean
Posted by Adam Estrada <es...@gmail.com>.
Martin,
I think that this is a much needed capability for most applications, especially in SIS. Its sort of like a service within a service, right? I have seen this used in ElasticSearch (I'm not sure what its referred to in there exactly) and it seems to work really well. There is even a config file that allows you to turn the verbosity level up too which is a nice feature.
For SIS, I see this as being used in really long running processes like tile cache generation and maybe even image/vector mosaicking for future versions. Where do you see this being used first? You point out monitoring internal object caches but what about externally facing client caches? I've seen other java-based geo-apps totally puke on long running processes and I've had no clue what went wrong. I feel like what you're suggesting could help alleviate some of these pain points within SIS. Sorry for the ramble...
Thanks,
Adam
On Nov 26, 2012, at 9:53 PM, Martin Desruisseaux wrote:
> Hello all
>
> Yesterday I added I feature that we didn't had before, which can be removed from compilation if considered undesirable. We did not had previously any monitoring tools about the state of a running instance of the library. We found that it was sometime problematic, since it make very hard to diagnostic problems on a server. I think there is numerous places where JMX MBeans could be used. For example report how many objects are cached, provide an option to clear the caches, detect if there is a problem in the running SIS instance (e.g. a daemon thread which unexpectedly died), etc.
>
> So I added an experimental "Supervisor" MBean in the internal package. For now it contains only one method, "warnings()", which report any problem detected in the running SIS instance. The only problem reported at this time is unexpected death of the daemon thread consuming the weak references queue, but future versions could perform more extensive tests of SIS state. This MBean can been seen in JConsole (i.e. if an application using SIS is currently running, start "jconsole" in another window, select the application in the proposed list, click on "MBeans", select "org.apache.sis". The "warnings" operation can been seen).
>
> The inconvenient is that SIS has to register its MBean to the PlatformMBeanServer (the same instance than the one managing the standard ClassLoadingJMXBean, MemoryJMXBean, GarbageCollectionJMXBean, etc.), and I have no idea if it put a significant burden. In particular I don't know if it trigs JMX class loading that would normally not happen. In my few tests, I didn't saw noticeable slowdown at startup time...
>
> Any though?
>
> Martin
>
Re: Experimenting the addition of a MBean
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
Hey Martin,
On Nov 27, 2012, at 3:32 AM, Martin Desruisseaux wrote:
> Le 27/11/12 13:46, Mattmann, Chris A (388J) a écrit :
>> I think JMX is fine for experimenting and for testing, so long as the SIS library itself remains compact and doesn't induce or
>> force this dependency on downstream consumers.
>
> Actually it doesn't induce any library dependency since JMX is part of standard JDK since Java 5. Furthermore, the JVM itself uses JMX for its own monitoring, so it is possible that JMX classes are loaded anyway, in which case using JMX on our side doesn't add any significant cost. But I wasn't sure (I guess I need to test with java --verbose), which is why I asked...
>
> But in any case, there is a flag for excluding the MBean.
Looks like we are in great shape then! :)
Cheers,
Chris
Re: Experimenting the addition of a MBean
Posted by Greg Reddin <gr...@gmail.com>.
On Tue, Nov 27, 2012 at 2:32 AM, Martin Desruisseaux <
martin.desruisseaux@geomatys.fr> wrote:
> Actually it doesn't induce any library dependency since JMX is part of
> standard JDK since Java 5. Furthermore, the JVM itself uses JMX for its own
> monitoring, so it is possible that JMX classes are loaded anyway, in which
> case using JMX on our side doesn't add any significant cost.
I was thinking that was the case too. Good deal. Now I can look at your
code and get ideas for how to instrument my code with JMX :-) Been wanting
to do that for a while now.
Greg
Re: Experimenting the addition of a MBean
Posted by Martin Desruisseaux <ma...@geomatys.fr>.
Le 27/11/12 13:46, Mattmann, Chris A (388J) a écrit :
> I think JMX is fine for experimenting and for testing, so long as the SIS library itself remains compact and doesn't induce or
> force this dependency on downstream consumers.
Actually it doesn't induce any library dependency since JMX is part of
standard JDK since Java 5. Furthermore, the JVM itself uses JMX for its
own monitoring, so it is possible that JMX classes are loaded anyway, in
which case using JMX on our side doesn't add any significant cost. But I
wasn't sure (I guess I need to test with java --verbose), which is why I
asked...
But in any case, there is a flag for excluding the MBean.
Martin
Re: Experimenting the addition of a MBean
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
Hey Martin,
I think JMX is fine for experimenting and for testing, so long as the SIS library itself remains compact and doesn't induce or
force this dependency on downstream consumers.
Thoughts?
Cheers,
Chris
On Nov 26, 2012, at 9:53 PM, Martin Desruisseaux wrote:
> Hello all
>
> Yesterday I added I feature that we didn't had before, which can be removed from compilation if considered undesirable. We did not had previously any monitoring tools about the state of a running instance of the library. We found that it was sometime problematic, since it make very hard to diagnostic problems on a server. I think there is numerous places where JMX MBeans could be used. For example report how many objects are cached, provide an option to clear the caches, detect if there is a problem in the running SIS instance (e.g. a daemon thread which unexpectedly died), etc.
>
> So I added an experimental "Supervisor" MBean in the internal package. For now it contains only one method, "warnings()", which report any problem detected in the running SIS instance. The only problem reported at this time is unexpected death of the daemon thread consuming the weak references queue, but future versions could perform more extensive tests of SIS state. This MBean can been seen in JConsole (i.e. if an application using SIS is currently running, start "jconsole" in another window, select the application in the proposed list, click on "MBeans", select "org.apache.sis". The "warnings" operation can been seen).
>
> The inconvenient is that SIS has to register its MBean to the PlatformMBeanServer (the same instance than the one managing the standard ClassLoadingJMXBean, MemoryJMXBean, GarbageCollectionJMXBean, etc.), and I have no idea if it put a significant burden. In particular I don't know if it trigs JMX class loading that would normally not happen. In my few tests, I didn't saw noticeable slowdown at startup time...
>
> Any though?
>
> Martin
>