[Fwd: Re: New AltProfile package]

[Leif Mortenson <le...@silveregg.co.jp>]

Resend this, it didn't show up on the list...

Berin Loritsch wrote:

> The biggest thing that I want to say on this point though is that the
> GUI would only be attached some of the time.  When the GUI is not
> attached anymore, we should not encur any (or much) overhead for
> updating the profile points. 

As things stand right now, the ProfilerManager is not even aware of the
ProfilerFrame (GUI).  Rather the GUI is created and registers itself
as a listener to the points that it is interrested in when one only when the
information is needed for the display.  So that sounds in line with your
thinking.

> Also, there was a slightly different approach that I had for managing
> Profilables:
>
> 1) the Profilable would have an enableProfiling() method invoked so
>    that they can start sending events to the ProfilePoints.
>
> 2) the Profilable would have a disableProfiling() method invoked so
>    that the profilable can not even send events to the ProfilePoints--
>    or maybe even remove references to the ProfilePoints altogether. 

I actually didn't like these two methods and intentionally designed them
out of the system.  Here is my reasoning.  With these methods present,
it becomes the job of the Profilable to keep track of whether profiling
is going on or not.  This will also mean that there would have to be some
kind synchronization going on around any such code to ensure that there
were no problems when the the Profilable was enabling or disabling itself.

My approach comes at the problem from a completely different angle.
I really liked the way the logger stuff was written.  Asside from the
logger.isDebugEnabled() methods (Which have their analogous methods
profilePoint.isActive() ), the component developer can basicly ignore log
levels while coding.  They just add log messages at will throughout the
code assigning the appropriate log levels to each message.  Whether or
not the log messages are actually being used is completely controlled by
the LoggerManager.  This not only makes this very easy for the developer,
but it also makes it much less likely that there will be any problems caused
by misuse of the API.

The Profiler takes a simmilar approach.  A class which implements Profilable
will always create and use all of its ProfilePoints.  They should never 
have to
worry about whether the profile points are actually being used or to have to
keep track of any kind of Profiler state.  The ProfilePoints just are. 
 I added
the isActive() methods so that the developer can avoid running any code on
their end which is needed to prepare data for a ProfilerPoint.  This is 
the same
thinking as went into the Logger APIs.

The nice thing about this is that now, control of the Profile Points is 
100% in
the hands of the ProfilerManager.  If a ProfilerManager never registers with
the ProfilePoints, then they are effectively noops and the classes which 
contain
them continue to function the same.  If the ProfilerManager is present, 
then
it takes care of which ProfilePoints' is used and when.  There is 
nothing stopping
the flow of data from being turned on and off at will and the 
synchronization
around such state changes is all nicely contained in the ProfilerManager.

> 3) The ProfilePoints were lightweight with very simple logic.  They
>    had a getValue() that would be used by the Profiler--but each profile
>    point would be able to use a different setting mechanism.  This would
>    enable average timing to be returned, or something of that nature. 

My thinking here is that deciding whether the average value should be taken
or the max value etc.  Should not be up to the Profilable to decide, but 
rather
up to the users of the ProfilerManager.

If this functionality was needed, then it would be very easy to create a 
subclass
of the ValueProfilePoint to do this without changing any of the APIs. 
 The problem
that I see with doing this is that an AverageValueProfilePoint would 
have to contain
state.  This state would mean that it would need to have some 
synchronization.  In
addition to complicating things, it would also have an adverse effect on the
performance of the code which was using the ProfilePoint.

> I don't know all of the pros and cons of the way Leif implemented it,
> but the fact that we have something that works is really cool.

Thanks.  I am sure it is far from perfect still.  But hopefiully we can 
get it in that
direction soon :-)

> We don't want to profile all the time--but we do want to be able to turn
> profiling off an on multiple times during the life of a system.  As it
> stands now, it is only useful for standalone programs.

Actually everything can be turned on and off at will.  The ProfilePoints 
can be
activated and deactivated at will simply by adding and removing listeners.

The ProfileSamples can be modified slightly to turn on and off at will 
as well.
As it is, they are designed to be used in cases where you really do want 
to be
collecting information all the time while an application is running.  This
information can then be viewed at any time in the future simply by 
requesting
a snapshot of the sample.

One thing that it does not do a good job of right now is adding and removing
ProfilerSamples at run time which reside inside of the ProfilerManager. 
 Outside
is no problem.

The question is how should this API be secured.  Who should have access 
to it
etc.  Currently, which ProfileSamples are to be made available is up to 
the system
administrator.  If anyone can add or remove these ProfileSamples at 
will, it could
break other systems which are using them.  Anything added outside of the
ProfilerManager is controlled by the user code which does the adding.

While some profile data is used for "profiling".  Other profile data is 
needed as a
resource for an application.  For example a system administration 
application
may have an HTTP interface which contains servlets showing the system memory
usage over time as well as the access rates.  This is information which 
is being used
as a resource and must always be collected for the system to behave 
correctly.

> We need a way to
> make it useful for situations where we can't directly control what
> classloader is used to load the system (i.e. Servlet environment, etc.) 

We are currently using it in a Servlet system running under Jo.  We had 
a couple
problems getting the XML serialization stuff working because of 
conflicts with
the jars that came with Jo, but other than that it works great.  Those 
problems
are only coming up because of the ability to save the state of the 
samples in the
ProfilerManager and saving of the desktop of the ProfilerFrame.  The
ProfilerFrame will soon be broken into a different program so the second 
should
go away.

>> ||| I liked the fact that the memory was well managed.  When I tested it
>> ||| with 100,000 doAction() events, I never exceeded 11M of max memory
>> ||| in the JVM.  That is a problem with JMeter's GUI event listeners.
>>
>> Leif is a memory ninja. :^)
>
>
> I gathered that.  He is a definite asset to our team. 

(grin)  Time will tell though...

>> ||| Also, how exactly is the ProfilerManager listening for events?
>>
>> Let's see...ProfilerManager has a bunch of ProfilableProxies, each of
>> which has a bunch of ProfilePointProxies. Each ProfilePointProxy has
>> different types of Samples registered as listeners for ProfilePoint 
>> events (as determined by the configuration file).
>
>
> It's being tied to the config file that I am not a fan of.  If the
> config file is merely a way to set up the last suite of ProfilePoints
> we were monitoring, then excellent.  I just don't want to be tied to
> what the config says I have, if my software is changing. 

Was this made clearer above?  The profilable attribute on each component
is required to allow an administrator to decide whether or not a component
should be visible to the ProfilerManager and its APIs.  It also makes it 
possible
to configure the name which will be used to reference the component in the
profiler space.

The profilable configuration section may be where some issues still need 
to be
worked out.  Currently,  this configuration is only required if you wish 
to assign
descriptions to the Profilables and ProfilePoints.  If you do not do 
this, then their
descriptions will be the same as their names.  Never null.

The profilable section is also where you define any ProfileSamples which 
will be
created and always be available as resources to the system.

But the profilable configuration is not required.

>> Then each of these entities can return descriptors back to their
>> parents, all the way up to the ProfilerManager, which can then be
>> used by GUIs or anything else that want to process/display the 
>> collected data.
>
> Yep.  I do want to be able to serialize the results to a file of
> whatever format I want.  That way I can have another piece of software
> to analyze the results in a more detailed fashion.  Now a particular
> ProfileReport would be set up for each set of ProfilePoints I want
> to monitor in the report.  So if I can have multiple reports occuring
> while I am viewing the screen, that would be stellar. 

I agree that there should be a way to serialize output data in whatever 
format you
want.   But do not confuse that with the profiler.sampledata file which 
is currently
being saved at regular intervals and shutdown by the ProfilerManager. 
 This file
really only has one purpose.  Making it possible to preserve the state of
ProfileSample data across invocations of the JVM.  That means that the 
file must
be read as well as saved.  If this file format can be changed then the 
reading also
becomes very complicated.  I am all in favor of adding a way to make other
snapshots to any file format.  Maybe implement something like the 
targets in the
logger.

Cheers,
Leif




--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>