[RT] Expanding JMeter's Responsibilities

[Berin Loritsch <bl...@apache.org>]

For the uninitiated, RT stands for Random Thought.  Stefano Mazzocchi uses
them to talk about some theorhetical and design discussions with the team,
to get some feedback.  The Avalon team adopted this approach, and now hopefully
I can infect this community as well.

There are a few limitations of JMeter that cannot be easily overcome--some
with a complete redesign, and some with additional capabilities.  First I
want to introduce some ideas that the Avalon team is working with, and how
they can help the JMeter team.

Instrumentation
---------------
The Avalon team (or at least me) is working on instrumenting Avalon's components.
That means that there are specific hooks to enable meaningful samples.  These
can range from maximimum simultaneous active objects from a pool to the number
of times an object is actually returned to the pool.  There are other things
that can be instrumented as well like number of active threads in a pool, etc.

I would love to expand JMeter to be able to handle metering and reporting these
things.  However, there are some design limitations to JMeter that must be overcome
in order to allow for meaningful reports.  The first of which is the concept of
multiple bits of information for each sample.  In other words, one Sample with
Avalon's instrumentation could have as many as 100 sample Points.   These points
represent different aspects of the system being instrumented.  The array size of
points sampled per interval does not change--but the current graphing and reporting
does not support this effectively.  In fact, the most differentiation that JMeter
has is all derived from the same sample point.  Examples include actual response
time, average, and standard deviation.  While this is great for sampling response
time from a web server, it doesn't provide enough for anything more complex.

Event Based Architecture
------------------------
Another concept that we are working on is an Event Based Pipeline approach to
IO and management.  This is much like the way most GUI systems work:  you have
one or more threads listening for events, and the listener fires callbacks.  This
approach is amazingly scalable.  When applied to non-blocking IO (a JDK 1.4 feature),
it will allow JMeter to break the barriers of only a couple hundred simultaneous
threads.  In fact, the only truly limiting issue will be the network card's ability
to output the information.  The Staged Event Driven Architecture (SEDA) has used
this to great extent to outperform Apache at it's own game using a Java based
web server!  The URL is http://www.cs.berkeley.edu/~mdw/proj/seda/.  The papers
presented there make for an interesting read.

Better Separation of Concerns
-----------------------------
One of the issues I have with the JMeter code is that it is difficult to determine
when the actual work code ends and the GUI code starts.  This makes designing load
test servers a difficult task.  The Avalon project (http://jakarta.apache.org/avalon)
provides a framework where we can clearly separate these aspects, and also provide
for a more scalable architecture.

Plan of Attack
--------------
I propose that we start up either a branch in CVS, or a proposal area where the
new code can live while it is being developed.  It could be the best opportunity
for version 2.0--as well as provide a nice framework that would be more attractive
to developers and maintainers in the long run.  What is everyone's thought on this?
I don't want to go and just create a branch without consulting with the development
team first.  It would cause too much upheaval to merely progress with the current
branch in CVS, as it would be a bit before we have some useable code.

-- 

"They that give up essential liberty to obtain a little temporary safety
  deserve neither liberty nor safety."
                 - Benjamin Franklin


--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>