Server monitoring and management

["Erik B. Craig" <gi...@gmail.com>]

All,

Recently Viet Nguyen and myself have been working on a project that would
ultimately assist a great deal in monitoring and managing services within a
single geronimo instance, as well as in a cluster environment with upwards
of thousands of machines. While it is currently in its infant stages, we
felt it best to get some basic framework down and take it here as soon as
possible to get as much community involvement in the project as possible,
hopefully resulting in a much more polished product at the end.

Currently what we have is a collection agent (MBean) that will run on a
given instance of geronimo (can be anything minimal and up), collecting data
from Tomcat only at present (No worries, we haven't forgotten about jetty
and our other core component friends). This is done at a set interval and
thrown into an XML file residing in the /var directory. This mbean also
serves as a jmx server connection, ultimately providing the gateway to any
number of operations and data requests (deploy/undeploys, star/stops,
attribute modifications) from a 'management node' to the machine running
this mbean in a clustered environment.

In conjuction with this, we have also been doing the basic work on a porlet
designed around the new (improved) much more plugable admin console that
Paul has been doing a lot of work on lately. Currently it is utilizing dojo
chart to generate graphs on request with data that is pulled from the
Management MBean, ultimately coming from the logged snapshots in the xml
file.

It would be greatly appreciated if some of you would take the time to have a
look at what we currently have, maybe give some input... and just maybe if
some of you are interested, hopping on board as well =)

There is a jira up that has the necessary files attached to it
https://issues.apache.org/jira/browse/GERONIMO-3441

There is also a more in-depth explanation of long term goals with this up in
the wiki here:
http://cwiki.apache.org/confluence/display/GMOxDEV/Monitoring+and+Management+Service

In addition to these, here are a couple direct links to some screenshots to
ohh and ahh over the pretty graphs
https://issues.apache.org/jira/secure/attachment/12364451/screenshot1.jpg
https://issues.apache.org/jira/secure/attachment/12364451/screenshot2.jpg


Thanks!
-- 
Erik B. Craig

Re: Server monitoring and management

["Erik B. Craig" <gi...@gmail.com>]

Paul,

I agree that an integration of the JMX view and the work we are doing is
something we should be doing (and are now planning on), as well as moving
towards using JSR77 as Anita has also mentioned. As far as the 'big picture'
with what we're working on goes, I believe the best approach would be
integrate what is currently done with the JMX viewer into the plug-in that
we're developing, as you have mentioned, expanding it's capability to view
any of the machines that the monitoring node would able to view.

As far as the Hyperic plug-in goes... I'd definitely be willing to work
towards that, however with the goal of this project resulting in more of a
'Management plug-in with some monitoring abilities' instead of a 'Monitoring
plug-in with some management abilities', I'm wondering if it would be worth
the focus at present? I suppose it really depends on how closely we can
align the two goals, given the licensing differences between Geronimo and
Hyperic (GPL2 if I'm not mistaken).

What you mentioned about collaborating with the terracotta guys, that would
be incredibly helpful in outlining some basic things to strive for, I would
definitely like to go down this avenue if possible. However, I'm not
entirely sure who I would contact or how I would go about doing it, the same
goes for Hyperic.


Thanks
Erik B. Craig

On 8/24/07, Paul McMahan <pa...@gmail.com> wrote:
>
> Erik,  this is some very interesting work.  Thanks for posting all
> these details.
>
> Like Anita's comment in GERONIMO-3441, I'm wondering whether or not
> it would make sense to implement these new capabilities more tightly
> with the JMX capabilities already in the server.  I don't have her
> depth of knowledge to follow on all the details, but from a
> conceptual point of view I do tend agree with her.  All that console
> refactoring you mentioned is still a work in progress.  But FYI I
> included the JMX viewer in a "debug views" plugin [1] that also
> contains those other dojo based viewers (classloader, dependency,
> JNDI, and LDAP).    Given the amount of additional capabilities that
> we could add around the JMX view (i.e. the work you have described
> here) I think we could actually make it part of a separate management
> plugin instead.
>
> On a related note, for some time I have thought it would be cool to
> have a Hyperic plugin for Geronimo.  It would provide many of the
> capabilities you have described.  From a technical point of view I
> think its possible because its uses j2ee as its runtime framework,
> but there may be non-technical issues someone could point out.  If
> there's interest we could look into collaborating with them on a
> plugin as part of the investigation you have described below.
>
> One more thing -- I don't know if its within the scope of this work,
> but I think it would be very useful we could also think about
> providing some generic way to manage and configure clusters in
> Geronimo.  Clustering technology is coming at us pretty fast right
> now and our users are going to need some way to interact with all
> those knobs and dials without cracking open a bunch of XML files.
> Last I heard the Terracotta guys were working on a portlet for the
> admin console.  Maybe they can suggest some functionality that can/
> should be implemented in a generic way within a management view.
>
> Best wishes,
> Paul
>
> [1] http://svn.apache.org/repos/asf/geronimo/plugins/debugviews
>
>
> On Aug 23, 2007, at 5:56 PM, Erik B. Craig wrote:
>
> > All,
> >
> > Recently Viet Nguyen and myself have been working on a project that
> > would ultimately assist a great deal in monitoring and managing
> > services within a single geronimo instance, as well as in a cluster
> > environment with upwards of thousands of machines. While it is
> > currently in its infant stages, we felt it best to get some basic
> > framework down and take it here as soon as possible to get as much
> > community involvement in the project as possible, hopefully
> > resulting in a much more polished product at the end.
> >
> > Currently what we have is a collection agent (MBean) that will run
> > on a given instance of geronimo (can be anything minimal and up),
> > collecting data from Tomcat only at present (No worries, we haven't
> > forgotten about jetty and our other core component friends). This
> > is done at a set interval and thrown into an XML file residing in
> > the /var directory. This mbean also serves as a jmx server
> > connection, ultimately providing the gateway to any number of
> > operations and data requests (deploy/undeploys, star/stops,
> > attribute modifications) from a 'management node' to the machine
> > running this mbean in a clustered environment.
> >
> > In conjuction with this, we have also been doing the basic work on
> > a porlet designed around the new (improved) much more plugable
> > admin console that Paul has been doing a lot of work on lately.
> > Currently it is utilizing dojo chart to generate graphs on request
> > with data that is pulled from the Management MBean, ultimately
> > coming from the logged snapshots in the xml file.
> >
> > It would be greatly appreciated if some of you would take the time
> > to have a look at what we currently have, maybe give some input...
> > and just maybe if some of you are interested, hopping on board as
> > well =)
> >
> > There is a jira up that has the necessary files attached to it
> > https://issues.apache.org/jira/browse/GERONIMO-3441
> >
> > There is also a more in-depth explanation of long term goals with
> > this up in the wiki here:
> > http://cwiki.apache.org/confluence/display/GMOxDEV/Monitoring+and
> > +Management+Service
> >
> > In addition to these, here are a couple direct links to some
> > screenshots to ohh and ahh over the pretty graphs
> > https://issues.apache.org/jira/secure/attachment/12364451/
> > screenshot1.jpg
> > https://issues.apache.org/jira/secure/attachment/12364451/
> > screenshot2.jpg
> >
> >
> > Thanks!
> > --
> > Erik B. Craig
>
>

Re: Server monitoring and management

[Paul McMahan <pa...@gmail.com>]

Erik,  this is some very interesting work.  Thanks for posting all  
these details.

Like Anita's comment in GERONIMO-3441, I'm wondering whether or not  
it would make sense to implement these new capabilities more tightly  
with the JMX capabilities already in the server.  I don't have her  
depth of knowledge to follow on all the details, but from a  
conceptual point of view I do tend agree with her.  All that console  
refactoring you mentioned is still a work in progress.  But FYI I  
included the JMX viewer in a "debug views" plugin [1] that also  
contains those other dojo based viewers (classloader, dependency,  
JNDI, and LDAP).    Given the amount of additional capabilities that  
we could add around the JMX view (i.e. the work you have described  
here) I think we could actually make it part of a separate management  
plugin instead.

On a related note, for some time I have thought it would be cool to  
have a Hyperic plugin for Geronimo.  It would provide many of the  
capabilities you have described.  From a technical point of view I  
think its possible because its uses j2ee as its runtime framework,  
but there may be non-technical issues someone could point out.  If  
there's interest we could look into collaborating with them on a  
plugin as part of the investigation you have described below.

One more thing -- I don't know if its within the scope of this work,  
but I think it would be very useful we could also think about  
providing some generic way to manage and configure clusters in  
Geronimo.  Clustering technology is coming at us pretty fast right  
now and our users are going to need some way to interact with all  
those knobs and dials without cracking open a bunch of XML files.    
Last I heard the Terracotta guys were working on a portlet for the  
admin console.  Maybe they can suggest some functionality that can/ 
should be implemented in a generic way within a management view.

Best wishes,
Paul

[1] http://svn.apache.org/repos/asf/geronimo/plugins/debugviews


On Aug 23, 2007, at 5:56 PM, Erik B. Craig wrote:

> All,
>
> Recently Viet Nguyen and myself have been working on a project that  
> would ultimately assist a great deal in monitoring and managing  
> services within a single geronimo instance, as well as in a cluster  
> environment with upwards of thousands of machines. While it is  
> currently in its infant stages, we felt it best to get some basic  
> framework down and take it here as soon as possible to get as much  
> community involvement in the project as possible, hopefully  
> resulting in a much more polished product at the end.
>
> Currently what we have is a collection agent (MBean) that will run  
> on a given instance of geronimo (can be anything minimal and up),  
> collecting data from Tomcat only at present (No worries, we haven't  
> forgotten about jetty and our other core component friends). This  
> is done at a set interval and thrown into an XML file residing in  
> the /var directory. This mbean also serves as a jmx server  
> connection, ultimately providing the gateway to any number of  
> operations and data requests (deploy/undeploys, star/stops,  
> attribute modifications) from a 'management node' to the machine  
> running this mbean in a clustered environment.
>
> In conjuction with this, we have also been doing the basic work on  
> a porlet designed around the new (improved) much more plugable  
> admin console that Paul has been doing a lot of work on lately.  
> Currently it is utilizing dojo chart to generate graphs on request  
> with data that is pulled from the Management MBean, ultimately  
> coming from the logged snapshots in the xml file.
>
> It would be greatly appreciated if some of you would take the time  
> to have a look at what we currently have, maybe give some input...  
> and just maybe if some of you are interested, hopping on board as  
> well =)
>
> There is a jira up that has the necessary files attached to it
> https://issues.apache.org/jira/browse/GERONIMO-3441
>
> There is also a more in-depth explanation of long term goals with  
> this up in the wiki here:
> http://cwiki.apache.org/confluence/display/GMOxDEV/Monitoring+and 
> +Management+Service
>
> In addition to these, here are a couple direct links to some  
> screenshots to ohh and ahh over the pretty graphs
> https://issues.apache.org/jira/secure/attachment/12364451/ 
> screenshot1.jpg
> https://issues.apache.org/jira/secure/attachment/12364451/ 
> screenshot2.jpg
>
>
> Thanks!
> -- 
> Erik B. Craig