[RT] Scalable Event Based Systems

[Berin Loritsch <bl...@apache.org>]

I would like to propose a new scratchpad component to help with management
and maintenance tasks that should be performed asynchronously from the
main program flow.

While I do want to base a similar approach to SEDA in Avalon, I think it
would be better served by defining the concept of EventQueues in Excalibur,
with full fledged Stages defined in CornerStone.  This will change the
way we think about some problems, but that's a good thing.

The full Staged Event-Driven Architecture assembles a pipeline of Stages.
These Stages contain one or more Sources, and one or more Sinks.  Both
the Sources and Sinks are Queues, and consist of QueueElements.  The Stage
is wrapped by the system to associate an EventHandler and a ThreadManager.

In ASCII art, a single stage is something (conceptually) like this:

+--------------------------------------------+
|              StageWrapper                  |
|                                            |
| +--------+  +-----------+  +------------+  |
| | Source |  |  Stage    |  |EventHandler|  |
| +--------+  |           |  +------------+  |
|             | +-------+ |                  |
| +--------+  | | Sink  | |                  |
| | Source |  | +-------+ |  +-------------+ |
| +--------+  |           |  |ThreadManager| |
|             +-----------+  +-------------+ |
+--------------------------------------------+

The application is built on Stages and associated EventHandlers.  For instance,
the EventHandler would have a reference to the Stage, and forward events to the
actual stage logic.  The Stage's responsibility is to act on those events, and
send new events to the various Sinks in it's responsibility.

With the exception of the getWrapper() method, Matt Welsh did a good job of
conceptualizing SEDA with Inversion of Control principles.

The ThreadManager's responsibility is to both manage the thread creation policies,
but also pull the events from the Source queue, and send them to the EventHandler.
There are two different types of EventHandlers:  regular (ThreadSafe) and SingleThreaded.
We already express these differences with our interfaces, so using the specialized
interface is not needed.

What I want to do for Excalibur is to take the concept of an EventQueue (Source
or Sink is not necessary at this time) and have it controlled by a ThreadManager
going to an EventHandler.

In essence, I am proposing everything inside the StageWrapper for Excalibur
and the Stage and purpose-specific queues for Cornerstone.

For Excalibur, it would provide a nice way of performing asynchronous maintenance
commands on all the Excalibur Components.  In the absence of the event queues,
all the Components would be unchanging, or at the very least blocking for management
events.

This does follow IoC, and makes the perceived performance of the system faster
under high load.  We will have to have a method of adding events to the CommandQueue.

The Queue hierarchy I would like to see in the end:

EventQueue   (Basic mechanics)
CommandQueue (Contains a queue of maintenance commands--some repeatable events, some
               run once)
Source       (In Cornerstone)
Sink         (In CornerStone)

The Source and Sink queues should not contain repeatable events, as they are only needed
for the CommandQueue.

-- 

"They that give up essential liberty to obtain a little temporary safety
  deserve neither liberty nor safety."
                 - Benjamin Franklin


--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>