You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avalon.apache.org by Berin Loritsch <bl...@apache.org> on 2001/12/14 16:32:16 UTC
[Review] Event Queues
I placed the interfaces for the Event Queues in the org.apache.avalon.excalibur.event
package. The more I look at it, the separation of the types of queues makes too many
different dirivations. I think it might be better to merge the Source and Sink types.
At first, I was thinking that blocking concerns can be applied to the Queue as a whole
(i.e. the Queue is Blocking or NonBlocking by implementation). A Blocking Queue would
have a standard timeout set for _all_ dequeue operations. In the end, it still may
be better to do it that way (thus limiting the dirivations of Queues to only three
instead of six). The idea I had was that blocking is a system concern, and not a
stage concern. I added the BlockingSink interface after I thought about it some
more, thinking that maybe the system could identify the blocking time period it was
willing to wait for new events.
In the end, I have come around full circle to my original way of thinking. Blocking
should be an implementation concern with one set time for the entire queue. One
reason is it's intended use. If you recall my original posting when I described what
a Stage was, you have a background thread that dequeues events from one queue, and
gives them to an event handler. The event handler in turn works with the Stage,
which decides what type of events it will pass to the next queue, or which
modifications it will perform to the event. In this use case, blocking queues can
actually be a detriment. It artificially locks a thread for a period of time,
keeping it from processing events from another queue in the mean time.
In that spirit, I will remove the BlockingQueue and BlockingSink interfaces as this
is both an implementation issue, and not the preferred use.
As to Regular Queues vs. Lossy Queues, I had the same dicotomy of thought. Just because
the SEDA architecture provides it does not mean we have to implement it in the same
manner. In fact, the concept of a Lossy Queue should be an implementation detail.
To this end, I added the boolean return value to the enqueue() commands so that the
Stage would still have to handle events that were removed due to the queue being full.
Again, queue length can be finite, or unbounded. I am still open on this issue.
Should the lossiness of a Queue be implementation dependant or specifically mandated
by interface?
Lastly, the the TransactionalSource interface I did in an attempt to separate those
concerns, as not all stages will require transactional queues. However, using the
JDBC Connection object as inspiration, I believe that the Transactional queue interface
should be merged in with the base queue. I implemented transactional queueing more
in line with the JDBC Connection in that the prepareEnqueue() method returns a
PreparedQueue object with the commit() and abort() interfaces. Since transactional
queueing is required to have both the commit and abort methods succeed if a
PreparedQueue object is returned, they do not throw exceptions (thus making transactions
easier to write for the programmer).
I find the Connection approach to transactional queues superior to the SEDA implementation
as it can allow for a cleaner API. For example, compare the two approaches:
SEDA:
Object key = sink.enqueue_prepare( events ); // SEDA queues start with a sink ?!?
if ( should_commit )
{
sink.enqueue_commit( key ); // Throws BadKeyException
}
else
{
sink.enqueue_abort( key ); // Throws BadKeyException
}
Avalon:
PreparedEnqueue transaction = source.prepareEnqueue( events ); // Avalon queues start with a source.
if ( should_commit )
{
transaction.commit(); // Never have a bad key!
}
else
{
transaction.abort(); // Never have a bad key!
}
I would like to have reactions on the Blocking and EventDropping issues though....
--
"They that give up essential liberty to obtain a little temporary safety
deserve neither liberty nor safety."
- Benjamin Franklin
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: [Review] Event Queues
Posted by Berin Loritsch <bl...@apache.org>.
Chad Stansbury wrote:
> Do you intend a given Sink to be utilized by several processes, each of
> which might want to specify a different timeout value? If so, then the
> should be passed as a parameter. Otherwise, a member variable is fine.
> Personally, I would err towards flexibility and overload the dequeue methods
> to allow one to either specify a timeout value, or to utilize the default
> (which is set at creation time).
That's the issue I am wrestling with. In the full version of the SEDA
architecture where you have multiple Stages connected by queues, I don't find
any real value in blocking a thread until an event comes because that thread
can be used for a different stage to process it's events.
However, in the simple CommandQueue that I am going to implement, the control
events will be within one queue, with one thread for that implementation.
In that case, I can see blocking the dequeue operation because the thread is
mostly going to be used strictly in that context. However, if the command
queue is used in the larger system again, blocking is again not good.
It is quite easy to perform the blocking within the CommandQueue's thread,
or simply to tell it to sleep.
>
> Chad Stansbury
>
> ----- Original Message -----
> From: "Berin Loritsch" <bl...@apache.org>
> To: "Avalon Developers List" <av...@jakarta.apache.org>
> Sent: Friday, December 14, 2001 3:09 PM
> Subject: Re: [Review] Event Queues
>
>
>
>>Berin Loritsch wrote:
>>
>>
>>>Peter Donald wrote:
>>>
>>>Remember this is dequeue().
>>>
>>>The Sink side of the equation is active, and therefore a call to pop
>>>
> events
>
>>>off the queue would block if the timeout was set.
>>>
>>>The Source side of the equation is passive, and therefore a call to push
>>>events on the queue never block, but will throw an exception unless
>>>tryEnqueue() is called.
>>>
>>>In essence, enqueue() can assert backpressure if the queue is not
>>>
> emptied
>
>>>or processed quick enough. Dequeue can block until there are actual
>>>events to dequeue ( or there are X events to dequeue in the case of
>>>dequeue( X ) ).
>>>
>>>I will merge the interfaces with some of the ideas talked about today,
>>>
> and
>
>>>let me know if you like it better....
>>>
>>
>>
>>I committed a less cluttered queue interface, which should be cleaner
>>than the SEDA version, and more semantically correct. I also took your
>>advice regarding virtualization.
>>
>>I am particularly interested if you are satisfied with the setTimeOut()
>>method, or if you would rather a different approach.
>>
>>
>>
>>
>>--
>>
>>"They that give up essential liberty to obtain a little temporary safety
>> deserve neither liberty nor safety."
>> - Benjamin Franklin
>>
>>
>>--
>>To unsubscribe, e-mail:
>>
> <ma...@jakarta.apache.org>
>
>>For additional commands, e-mail:
>>
> <ma...@jakarta.apache.org>
>
>>
>
>
> --
> To unsubscribe, e-mail: <ma...@jakarta.apache.org>
> For additional commands, e-mail: <ma...@jakarta.apache.org>
>
> .
>
>
--
"They that give up essential liberty to obtain a little temporary safety
deserve neither liberty nor safety."
- Benjamin Franklin
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: [Review] Event Queues
Posted by Peter Donald <pe...@apache.org>.
On Tue, 18 Dec 2001 12:15, vinay nair wrote:
> I had a few comments and would like to get some views
> on them.
>
> I was thinking that "Profiling" capability and the
> "SEDA" architecture snippets
> that are being included in the Avalon can be adapted
> into
> the Management cloud of Avalon.
> e.g.: calls like getProfilablePoints() , can be
> something that be synonymously tied to
> getMBeanInfo() call, if needed.(enabling us to manage
> the profiling feature of components
> at runtime from the management console exposed through
> jmx adaptors).
thats definetly a good idea. A simplified JMX bean that read configuration
informtation (like the names and descriptions of points etc) and who had
access to the appropriate profilable points would be very useful.
> The runtime sink/source configuration with stages can
> be realized with the entire
> configuration mechanism of stages made available
> through a JMX management interface.
Thats definetly a good aim and something that will be more doable in the near
future (I hope).
--
Cheers,
Pete
"The fact that a believer is happier than a skeptic is no more to the
point than the fact that a drunken man is happier than a sober one.
The happiness of credulity is a cheap and dangerous quality."
-- George Bernard Shaw
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: [Review] Event Queues
Posted by vinay nair <vi...@yahoo.com>.
I had a few comments and would like to get some views
on them.
I was thinking that "Profiling" capability and the
"SEDA" architecture snippets
that are being included in the Avalon can be adapted
into
the Management cloud of Avalon.
e.g.: calls like getProfilablePoints() , can be
something that be synonymously tied to
getMBeanInfo() call, if needed.(enabling us to manage
the profiling feature of components
at runtime from the management console exposed through
jmx adaptors).
The runtime sink/source configuration with stages can
be realized with the entire
configuration mechanism of stages made available
through a JMX management interface.
Although I am not in a good position right now to
gauge the work done with the JMX cloud
that surrounds Avalon, MBeanServer can be realized as
a Service running on Phoenix
with default adaptors tied to MBeanService made
configurable through Avalon configuration.
In fact maybe the components can relay messages to the
MBeanServer through the "jmx connectors"
which are nothing but "event queues" to the
MBeanServer.
and yet another quick question i had was since the
stages query the next stage(& sink) from the
component manager wont it be a good idea to put
reference to that stage into the event queue itself,
(inside the QueueElement),and helping us to get rid of
the switch-case clause in the ThreadManager
which eventually calls the evenhandlers of the
subsequent Stage.(Command pattern).
interface QueueElement
{
setStage(Stage stage);
Stage getStage();
}
ThreadManager can then do a simple ------
{
StageIF stage=queueElement.getStage();
EventHandlerIF[]
eventhandlers=manager.getEventHandlers(stage);
for(...)
eventhandlers[i].handleEvent(queueElement);
}
Are there any flaws in my thoughts here ?? (I hope
there are many :-)
I must admit my ignorance about the entire Avalon
framework (right now),
so tell me if I am wrong in the all of these.
Vinay.
__________________________________________________
Do You Yahoo!?
Check out Yahoo! Shopping and Yahoo! Auctions for all of
your unique holiday gifts! Buy at http://shopping.yahoo.com
or bid at http://auctions.yahoo.com
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: [Review] Event Queues
Posted by Leo Sutic <le...@inspireinfrastructure.com>.
> From: Berin Loritsch [mailto:bloritsch@apache.org]
> We could write the requirement that this happens, i.e.:
>
> The Type returned MUST be unique to a given class so that
>
> a.getType() == b.getType();
>
> if
>
> a.getClass().equals(b.getClass());
>
> However, if the Classes for "a" and "b" are different classes
> (even if one is a subclass of the other), then a.getType != b.getType();
That would be perfect. All I want is to put it in writing that an EventHandler
must be able to process events correctly just by looking at the class
of the event. (We could, of course, require all events to be final classes.)
When I wrote the ReflectionEventHandler I thought about
someone writing something like this:
public class ConnectionEvent implements Signal {
private final long type;
public ConnectionEvent (long type) {
this.type = type;
}
.
.
.
public long getType () {
return type;
}
}
In order to avoid having to declare many classes. Since the ReflectionEventHandler
uses the type of the argument to lookup the handler function, and not the typeid,
this would make it dispatch incorrectly.
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: [Review] Event Queues
Posted by Berin Loritsch <bl...@apache.org>.
Leo Sutic wrote:
> Berin,
>
> for QueueElement, is it always true that:
>
> a.getType() == b.getType()
> <------->
> a.getClass() == b.getClass()
>
> that is, one can always discriminate depending on event class?
It SHOULD be. Although, when you are validating long against
long that is written by a developer, you can never enforce this.
We could write the requirement that this happens, i.e.:
The Type returned MUST be unique to a given class so that
a.getType() == b.getType();
if
a.getClass().equals(b.getClass());
However, if the Classes for "a" and "b" are different classes
(even if one is a subclass of the other), then a.getType != b.getType();
--
"They that give up essential liberty to obtain a little temporary safety
deserve neither liberty nor safety."
- Benjamin Franklin
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: [Review] Event Queues
Posted by Leo Sutic <le...@inspireinfrastructure.com>.
Berin,
for QueueElement, is it always true that:
a.getType() == b.getType()
<------->
a.getClass() == b.getClass()
that is, one can always discriminate depending on event class?
/LS
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: [Review] Event Queues
Posted by Berin Loritsch <bl...@apache.org>.
Leo Sutic wrote:
>>From: Berin Loritsch [mailto:bloritsch@apache.org]
>>
>>It is in CVS, nothing stopping you from doing it yourself....
>>
>>I only have so much time I can squeeze to do this.
>>
>
> Well, if you do not have any objections to the
> proposed structure - then I'll look into it.
> I was just worried that we'd end up with incompatible drafts.
As long as the queue is ok, go ahead and work on the other
parts.
--
"They that give up essential liberty to obtain a little temporary safety
deserve neither liberty nor safety."
- Benjamin Franklin
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: [Review] Event Queues
Posted by Leo Sutic <le...@inspireinfrastructure.com>.
> From: Berin Loritsch [mailto:bloritsch@apache.org]
>
> It is in CVS, nothing stopping you from doing it yourself....
>
> I only have so much time I can squeeze to do this.
Well, if you do not have any objections to the
proposed structure - then I'll look into it.
I was just worried that we'd end up with incompatible drafts.
/LS
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: [Review] Event Queues
Posted by Berin Loritsch <bl...@apache.org>.
Leo Sutic wrote:
>
>>-----Original Message-----
>>From: Berin Loritsch [mailto:bloritsch@apache.org]
>>Sent: den 17 december 2001 20:25
>>To: Avalon Developers List
>>Subject: Re: [Review] Event Queues
>>
>>
>>Leo Sutic wrote:
>>
>>
>>That would be the way to minimize the cost of creating a Method object,
>>
>
> But this would maybe not have to be done more than once:
>
> class MyStage extends AbstractStageWrapper {
>
> public void handleConnectionOpen (ConnectionOpenEvent event) {
> ...
> }
>
> public void handleConnectionClosed (ConnectionClosedEvent event) {
> ...
> }
>
> }
>
> class AbstractStageWrapper implements EventHandler {
>
> private final HashMap handlerMethods = new HashMap ();
>
> public void initialize () throws Exception {
> // Iterate over own handler methods, and build a
> // Class -> Method map.
> }
>
> public void handleEvent (QueueElement event) {
> Method handlerMethod = (Method) handlerMethods.get (event.getClass ());
> handlerMethod.invoke (...);
> }
> }
Go ahead and play with it. I won't mind if you add it to CVS. Right now, I am
concentrating on the queue portion, and have not addressed the Stage portion yet.
>
>
>
>>and invoking it--done several hundred times a second, and now you are
>>thrashing your Garbage Collection.
>>
>
> Like creating event methods 100 times a second?
:). It's a good thing modern JVMs are better at small final classes than
caching them would be. Otherwise this would not be a viable option.
>
>
>>In many cases, the switch/case would not only be sufficient, but
>>preferred.
>>
>
> I can only see it as preferred when the cost for invokation via
> a Method instance is comparable to the cost for running the handler,
> and I believe that to be the exception - the handler will always be
> much more expensive than the invokation.
>
> Looking above, can you fit the AbstractStageWrapper into the your Stage
> interface design? I think it gives all the flexibility needed: Use
> switch() if you want, use reflection if you want. Above all, it would
> allow me to experiment. If reflection is too expensive then the idea can
> die, and we can move on without having lost anything.
It is in CVS, nothing stopping you from doing it yourself....
I only have so much time I can squeeze to do this.
--
"They that give up essential liberty to obtain a little temporary safety
deserve neither liberty nor safety."
- Benjamin Franklin
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: [Review] Event Queues
Posted by Leo Sutic <le...@inspireinfrastructure.com>.
> -----Original Message-----
> From: Berin Loritsch [mailto:bloritsch@apache.org]
> Sent: den 17 december 2001 20:25
> To: Avalon Developers List
> Subject: Re: [Review] Event Queues
>
>
> Leo Sutic wrote:
>
> >
> That would be the way to minimize the cost of creating a Method object,
But this would maybe not have to be done more than once:
class MyStage extends AbstractStageWrapper {
public void handleConnectionOpen (ConnectionOpenEvent event) {
...
}
public void handleConnectionClosed (ConnectionClosedEvent event) {
...
}
}
class AbstractStageWrapper implements EventHandler {
private final HashMap handlerMethods = new HashMap ();
public void initialize () throws Exception {
// Iterate over own handler methods, and build a
// Class -> Method map.
}
public void handleEvent (QueueElement event) {
Method handlerMethod = (Method) handlerMethods.get (event.getClass ());
handlerMethod.invoke (...);
}
}
> and invoking it--done several hundred times a second, and now you are
> thrashing your Garbage Collection.
Like creating event methods 100 times a second?
> In many cases, the switch/case would not only be sufficient, but
> preferred.
I can only see it as preferred when the cost for invokation via
a Method instance is comparable to the cost for running the handler,
and I believe that to be the exception - the handler will always be
much more expensive than the invokation.
Looking above, can you fit the AbstractStageWrapper into the your Stage
interface design? I think it gives all the flexibility needed: Use
switch() if you want, use reflection if you want. Above all, it would
allow me to experiment. If reflection is too expensive then the idea can
die, and we can move on without having lost anything.
/LS
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: [Review] Event Queues
Posted by Berin Loritsch <bl...@apache.org>.
Leo Sutic wrote:
>
>>Ok, now with the actual problem at hand. The typesafe interface
>>would be handy, but I have to wonder *how* to practically use it.
>>The issue is that I want the same feel for both C++
>>as well as Java if that is how you are working with it.
>>
>
>
>>I want to stay away from strings being the signal type, and if
>>possible, the class type
>>being the signal type.
>>
>
> if (a instanceof A) <--------> if (dynamic_cast<A> (a) != NULL)
> Java C++
>
> There are ways to solve this, with a switch/case construct. But I
> want to get rid of the construct.
:/
>
> Ideally, one would introduce a layer in front of the Sink, and one after
> the Source, and by magic be able to write code like this:
>
> class MyStage extends AbstractStageWrapper {
>
> public void handleConnectionOpen (ConnectionOpenEvent event) {
> ...
> }
>
> public void handleConnectionClosed (ConnectionClosedEvent event) {
> ...
> }
>
> }
I see. In Java, this is pretty expensive. I can see how it would be
helpful, but you now would have to approach it like Swing does. There
are specific events a Swing Handler of a certain type can handle. An
extension of an EventHandler would be able to farm out those types of
events and encapsulate the switch/case logic in that EventHandler.
That would be the way to minimize the cost of creating a Method object,
and invoking it--done several hundred times a second, and now you are
thrashing your Garbage Collection.
In many cases, the switch/case would not only be sufficient, but preferred.
In other cases, the separate EventHandler that calls methods on the
Stage object would be better. For example:
class MyStage implements Stage, ConnectionHandler {
public void handleConnectionOpen( Connection conn ) { ... }
public void handleConnectionClosed( Connection conn ) { ... }
public void handleEvent( QueueElement event ) { ... }
}
class ConnectionDistributionHandler
{
private final ConnectionHandler m_target;
public ConnectionDistributionHandler( ConnectionHandler target )
{
m_target = target;
}
public void handleEvents( QueueElement[] events )
{
int len = events.length;
for (int i = 0; i < len; i++)
{
switch ( events[i].getType() )
{
case CONNECTION_OPEN:
m_target.handleConnectionOpen( (Connection) events[i].getAttachment() );
break;
case CONNECTION_CLOSED:
m_target.handleConnectionClosed( (Connection) events[i].getAttachment() );
break;
default:
m_target.handleEvent( events[i] );
}
}
}
}
Using reflection is pretty expensive for this type of thing.
>
> Which, as I think about reflection, one actually can. Does anyone
> know the performance penalty for dynamic invocation of methods,
> as related to the expected time to run a typical SEDA event handler?
>
> I just want to get rid of that cursed switch/nested if!
You would have to abstract it out to specific subsystems....
>
> In C++ I think I could do it with some macros.
>
> Summary: As far as the Queue interfaces go, go for it. I don't
> think you can get them any better without trying to use them.
> They have no apparent faults.
>
> Just trying to wrap my head around the SEDA stuff and how to best
> express the architecture in Java/C++. I want as direct a mapping
> between the language and the concepts as possible, and I do not
> think the current implementation of SEDA is that.
I think now is time to actually create the implementations of the
interfaces.
--
"They that give up essential liberty to obtain a little temporary safety
deserve neither liberty nor safety."
- Benjamin Franklin
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: [Review] Event Queues
Posted by Leo Sutic <le...@inspireinfrastructure.com>.
> -----Original Message-----
> From: Berin Loritsch [mailto:bloritsch@apache.org]
> Sent: den 17 december 2001 19:16
> To: Avalon Developers List
> Subject: Re: [Review] Event Queues
>
>
> Leo Sutic wrote:
> Leo, can you please hand wrap your paragraphs?
Berin,
will do.
> Ok, now with the actual problem at hand. The typesafe interface
> would be handy, but I have to wonder *how* to practically use it.
> The issue is that I want the same feel for both C++
> as well as Java if that is how you are working with it.
> I want to stay away from strings being the signal type, and if
> possible, the class type
> being the signal type.
if (a instanceof A) <--------> if (dynamic_cast<A> (a) != NULL)
Java C++
There are ways to solve this, with a switch/case construct. But I
want to get rid of the construct.
Ideally, one would introduce a layer in front of the Sink, and one after
the Source, and by magic be able to write code like this:
class MyStage extends AbstractStageWrapper {
public void handleConnectionOpen (ConnectionOpenEvent event) {
...
}
public void handleConnectionClosed (ConnectionClosedEvent event) {
...
}
}
Which, as I think about reflection, one actually can. Does anyone
know the performance penalty for dynamic invocation of methods,
as related to the expected time to run a typical SEDA event handler?
I just want to get rid of that cursed switch/nested if!
In C++ I think I could do it with some macros.
Summary: As far as the Queue interfaces go, go for it. I don't
think you can get them any better without trying to use them.
They have no apparent faults.
Just trying to wrap my head around the SEDA stuff and how to best
express the architecture in Java/C++. I want as direct a mapping
between the language and the concepts as possible, and I do not
think the current implementation of SEDA is that.
/LS
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: [Review] Event Queues
Posted by Berin Loritsch <bl...@apache.org>.
Leo Sutic wrote:
>>Hmm. What about Events like QueueClosed, or other control events
>>that go through the same queue?
>>
>
> Berin,
>
> the problem is that I want a typesafe interface to the queue. If a queue accepts events of type A, B and C, then that should be made explicit via the Sink interface from that queue. I do not mind having to cast when removing an item from the queue, but when I have to cast something, I want to be sure the cast is correct without too much trouble.
>
> Looking at Haboob, the event handlers are like:
>
> if (element instanceof A) {
> ...
> } else if (element instanceof B) {
> ...
> } else if (element instanceof C) {
> ...
> }
>
> Which is a regression to the old procedural style programming paradigm where structures are passed around to idempotent functions. Just replace function with stage and structure with queue element.
Leo, can you please hand wrap your paragraphs? My mail client doesn't do it for
responses. It makes it easier for writing the responses.
Ok, now with the actual problem at hand. The typesafe interface would be handy, but I have
to wonder *how* to practically use it. The issue is that I want the same feel for both C++
as well as Java if that is how you are working with it.
There is always the possibility of adding methods such as attach(Object), getAttachment(),
getSignal(). The best one of course being the getSignal(). By attaching an integer to the
signal type, you can use a switch/case construct which is arguably better.
I want to stay away from strings being the signal type, and if possible, the class type
being the signal type.
>
>
>>Not every queue will be passing references to Sockets or File
>>Handles. They will be passing control
>>events as well. I don't think this is one place where templates
>>while a cool feature won't help
>>the Queue implementation--esp. with respects to the type of events.
>>
>
> True. But if a typecheck and a cast is necessary, then the last one to see the "true" type of the object should ideally be the same one casting it back. (http://rampages.onramp.net/~huston/dp/type_restorer.html)
That approach may be good for C++ (although I have to find more info), but how does it translate
to Java?
I think that will help me wrap my head around the issue you are referring to. I don't think
we are on the same page at all.
>>I mean the guy that wrote SEDA not only has a Ph.D., but it doesn't seem to be of the
>>variety where it is Piled Higher and Deeper (assuming you start with B.S.).
>>
>
> I have no doubts about Matt's ability to architect systems. Besides Ph.D and being funded by DARPA (an organization I hold in very high regard), he came up with SEDA and I didn't, and that has to count for something.
:)
>
>
>>I think we should learn from the lessons that Matt Welsh put
>>forth, as they are tested and not purely theorhetical.
>>
>
> True, but I believe that you can improve on his design. He has shown what works with Haboob, and his papers list what does not work. Now, if we stay away from what does not work I see no reason why we should not do our very best to improve on it.
Oh, no doubt. That is what I hope to accomplish. He has the theorhetical know how,
but we have practical API writing experience. While he is addressing performance
and scalability we are addressing practical aspects of the system.
--
"They that give up essential liberty to obtain a little temporary safety
deserve neither liberty nor safety."
- Benjamin Franklin
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: [Review] Event Queues
Posted by Leo Sutic <le...@inspireinfrastructure.com>.
> Hmm. What about Events like QueueClosed, or other control events
> that go through the same queue?
Berin,
the problem is that I want a typesafe interface to the queue. If a queue accepts events of type A, B and C, then that should be made explicit via the Sink interface from that queue. I do not mind having to cast when removing an item from the queue, but when I have to cast something, I want to be sure the cast is correct without too much trouble.
Looking at Haboob, the event handlers are like:
if (element instanceof A) {
...
} else if (element instanceof B) {
...
} else if (element instanceof C) {
...
}
Which is a regression to the old procedural style programming paradigm where structures are passed around to idempotent functions. Just replace function with stage and structure with queue element.
> Not every queue will be passing references to Sockets or File
> Handles. They will be passing control
> events as well. I don't think this is one place where templates
> while a cool feature won't help
> the Queue implementation--esp. with respects to the type of events.
True. But if a typecheck and a cast is necessary, then the last one to see the "true" type of the object should ideally be the same one casting it back. (http://rampages.onramp.net/~huston/dp/type_restorer.html)
> See my replacement for this. [Sink interface] It is soo much better.
No objection.
> I mean the guy that wrote SEDA not only has a Ph.D., but it doesn't seem to be of the
> variety where it is Piled Higher and Deeper (assuming you start with B.S.).
I have no doubts about Matt's ability to architect systems. Besides Ph.D and being funded by DARPA (an organization I hold in very high regard), he came up with SEDA and I didn't, and that has to count for something.
> I think we should learn from the lessons that Matt Welsh put
> forth, as they are tested and not purely theorhetical.
True, but I believe that you can improve on his design. He has shown what works with Haboob, and his papers list what does not work. Now, if we stay away from what does not work I see no reason why we should not do our very best to improve on it.
/LS
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: [Review] Event Queues
Posted by Berin Loritsch <bl...@apache.org>.
Leo Sutic wrote:
> Berin,
>
> nice work as usual.
>
> However, architecturally I have some random thoughts that I'd like to share.
>
> I have looked into SEDA as part of my Avalon-for-C++ project, and regarding queues, I can not really say whether the queue should or should not be exposed to the stages producing events for that stage.
>
Keep in mind that in the SEDA system, Sandstorm Kernel, stages are decoupled from each other
by the queues. The Stage is given a set of Sinks from the Manager. The Sinks are in reality
the beginning of the queue (since in SEDA, the concepts are Stage-centric rather than Queue-
centric). The Stage has no control over what queue is assigned to it. This is not really a
problem.
In essence, a system can be completely redesigned at runtime by the manager. The queues can
be reassigned to connect to other stages, etc.
This is FS (Flexibility Syndrome), but it drives home the point that Stages need to be
decoupled.
There are a couple of ways of dealing with the system, but while Queues do introduce a latency,
they do allow the system to scale evenly. It also allows for multiple threads to adaptively
help queues that reach a certain threshold of backpressure (i.e. the following stage cannot
keep up processing events).
In the end, you will find that queues are in the end a better solution to direct coupling.
> Axiom: Stage isa Component
>
> Scenario:
>
> class StageBeforeMyStage implements Stage { ... }
>
> class MyStage implements Stage { ... }
>
> class MyStageWorkElement implements QueueElement {
>
> public MyStageWorkElement (String workDescription) { ... }
>
> public String getWorkDescription () { ... }
>
> }
>
> StageBeforeMyStage produces MyStageWorkElements and feeds them to MyStage.
>
> Now, I thought like this: The queue should hold event objects. But C++ templates enables you to design the queues with type safety - that is, you do not need to cast the event object after dequeueing it. This is a major advantage, as you do not need to rely on the previous stage producing the correct work elements.
Hmm. What about Events like QueueClosed, or other control events that go through the same queue?
Not every queue will be passing references to Sockets or File Handles. They will be passing control
events as well. I don't think this is one place where templates while a cool feature won't help
the Queue implementation--esp. with respects to the type of events.
>
> I imagine StageBeforeMyStage will use a ComponentManager to lookup(MyStage.ROLE). Now, what does StageBeforeMyStage see of MyStage? Does it see the Sink interface?
>
> I would propose that it did not.
Keep in mind the statements mentioned above regarding scalability. Also, keep in mind
that I requested the review of the Queue interfaces--I haven't even begun to do the
Stage implementation.
We can attempt different approaches at that time, however, I have other uses for the
queue than for just the stages.
>
> I would like to show this interface:
>
> interface TransactionalSink {
> public void beginTransaction ();
> public void commitTransaction ();
> public void abortTransaction ();
> }
See my replacement for this. It is soo much better.
>
> interface MyStageSink extends TransactionalSink {
> public void addWorkDescription (String workDescription);
> }
>
> The usage would then be like this:
>
> MyStageSink sink = (MyStageSink) manager.lookup (MyStageSink.ROLE);
>
> .
> .
> .
>
> try {
> sink.beginTransaction ();
> sink.addWorkDescription ("Do this.");
> sink.addWorkDescription ("Do that.");
> sink.commitTransaction ();
> } catch (Exception e) {
> sink.abortTransaction ();
> }
>
> A stage would then be just like a component with asynchronous methods. The thread pool for each stage would be managed by the component manager the stage belongs to.
-1
You are mixing some key concerns. A ComponentManager != SystemManager. It is
not the ComponentManager's role to maintain thread pools for the system. Also,
you are limiting the Queue to using strings--which is too limiting. There are
a number of different types of events from opening a file, to actually reading
information from it. All of these can come from a pipeline--if the stage doen't
handle it, than just like the nice Swing counterpart, it should pass the event
on....
We have QueueElements to mark the types of objects that can go in the queue,
and that should be enough. If we need to open it some more, we can make them
simple objects. But I don't think that the direct coupling of stages would be
a good pattern.
Stages != Components either. Stages ~ Filters. I think we should strive for
conceptual simplicity while we are implementing this. I mean the guy that wrote
SEDA not only has a Ph.D., but it doesn't seem to be of the variety where it is
Piled Higher and Deeper (assuming you start with B.S.).
I think we should learn from the lessons that Matt Welsh put forth, as they are
tested and not purely theorhetical.
--
"They that give up essential liberty to obtain a little temporary safety
deserve neither liberty nor safety."
- Benjamin Franklin
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: [Review] Event Queues
Posted by Leo Sutic <le...@inspireinfrastructure.com>.
Berin,
nice work as usual.
However, architecturally I have some random thoughts that I'd like to share.
I have looked into SEDA as part of my Avalon-for-C++ project, and regarding queues, I can not really say whether the queue should or should not be exposed to the stages producing events for that stage.
Axiom: Stage isa Component
Scenario:
class StageBeforeMyStage implements Stage { ... }
class MyStage implements Stage { ... }
class MyStageWorkElement implements QueueElement {
public MyStageWorkElement (String workDescription) { ... }
public String getWorkDescription () { ... }
}
StageBeforeMyStage produces MyStageWorkElements and feeds them to MyStage.
Now, I thought like this: The queue should hold event objects. But C++ templates enables you to design the queues with type safety - that is, you do not need to cast the event object after dequeueing it. This is a major advantage, as you do not need to rely on the previous stage producing the correct work elements.
I imagine StageBeforeMyStage will use a ComponentManager to lookup(MyStage.ROLE). Now, what does StageBeforeMyStage see of MyStage? Does it see the Sink interface?
I would propose that it did not.
I would like to show this interface:
interface TransactionalSink {
public void beginTransaction ();
public void commitTransaction ();
public void abortTransaction ();
}
interface MyStageSink extends TransactionalSink {
public void addWorkDescription (String workDescription);
}
The usage would then be like this:
MyStageSink sink = (MyStageSink) manager.lookup (MyStageSink.ROLE);
.
.
.
try {
sink.beginTransaction ();
sink.addWorkDescription ("Do this.");
sink.addWorkDescription ("Do that.");
sink.commitTransaction ();
} catch (Exception e) {
sink.abortTransaction ();
}
A stage would then be just like a component with asynchronous methods. The thread pool for each stage would be managed by the component manager the stage belongs to.
/LS
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: [Review] Event Queues
Posted by Chad Stansbury <st...@earthlink.net>.
Do you intend a given Sink to be utilized by several processes, each of
which might want to specify a different timeout value? If so, then the
should be passed as a parameter. Otherwise, a member variable is fine.
Personally, I would err towards flexibility and overload the dequeue methods
to allow one to either specify a timeout value, or to utilize the default
(which is set at creation time).
Chad Stansbury
----- Original Message -----
From: "Berin Loritsch" <bl...@apache.org>
To: "Avalon Developers List" <av...@jakarta.apache.org>
Sent: Friday, December 14, 2001 3:09 PM
Subject: Re: [Review] Event Queues
> Berin Loritsch wrote:
>
> > Peter Donald wrote:
> >
> > Remember this is dequeue().
> >
> > The Sink side of the equation is active, and therefore a call to pop
events
> > off the queue would block if the timeout was set.
> >
> > The Source side of the equation is passive, and therefore a call to push
> > events on the queue never block, but will throw an exception unless
> > tryEnqueue() is called.
> >
> > In essence, enqueue() can assert backpressure if the queue is not
emptied
> > or processed quick enough. Dequeue can block until there are actual
> > events to dequeue ( or there are X events to dequeue in the case of
> > dequeue( X ) ).
> >
> > I will merge the interfaces with some of the ideas talked about today,
and
> > let me know if you like it better....
>
>
>
> I committed a less cluttered queue interface, which should be cleaner
> than the SEDA version, and more semantically correct. I also took your
> advice regarding virtualization.
>
> I am particularly interested if you are satisfied with the setTimeOut()
> method, or if you would rather a different approach.
>
>
>
>
> --
>
> "They that give up essential liberty to obtain a little temporary safety
> deserve neither liberty nor safety."
> - Benjamin Franklin
>
>
> --
> To unsubscribe, e-mail:
<ma...@jakarta.apache.org>
> For additional commands, e-mail:
<ma...@jakarta.apache.org>
>
>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: [Review] Event Queues
Posted by Berin Loritsch <bl...@apache.org>.
Berin Loritsch wrote:
> Peter Donald wrote:
>
> Remember this is dequeue().
>
> The Sink side of the equation is active, and therefore a call to pop events
> off the queue would block if the timeout was set.
>
> The Source side of the equation is passive, and therefore a call to push
> events on the queue never block, but will throw an exception unless
> tryEnqueue() is called.
>
> In essence, enqueue() can assert backpressure if the queue is not emptied
> or processed quick enough. Dequeue can block until there are actual
> events to dequeue ( or there are X events to dequeue in the case of
> dequeue( X ) ).
>
> I will merge the interfaces with some of the ideas talked about today, and
> let me know if you like it better....
I committed a less cluttered queue interface, which should be cleaner
than the SEDA version, and more semantically correct. I also took your
advice regarding virtualization.
I am particularly interested if you are satisfied with the setTimeOut()
method, or if you would rather a different approach.
--
"They that give up essential liberty to obtain a little temporary safety
deserve neither liberty nor safety."
- Benjamin Franklin
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: [Review] Event Queues
Posted by Berin Loritsch <bl...@apache.org>.
Peter Donald wrote:
> On Sat, 15 Dec 2001 08:43, Berin Loritsch wrote:
>
>>>>I would like to have reactions on the Blocking and EventDropping issues
>>>>though....
>>>>
>>>Event dropping should be an application specific thing. However I am not
>>>sure why Blocking and non-Blocking versions of interfaces need to be
>>>separate. I haven't looked closely at the proposal though - will try to
>>>do so real soon now ;)
>>>
>>Yes, but should they affect the way a system is written? For instance,
>>the basic operation of dequeue() doesn't change. Is it sufficient
>>for the blocking timeout to be set once for the pipeline, or should it be
>>set for *each* dequeue() method? Personally, I lean toward setting it
>>once.
>>
>
> Hmmm I guess so. What I was thinking is that "void enqueue()" should either
> always pass (and thus block) or throw an exception. If you wanted more
> control (ie making sure it doesn't block) then you should use tryEnqueue
Remember this is dequeue().
The Sink side of the equation is active, and therefore a call to pop events
off the queue would block if the timeout was set.
The Source side of the equation is passive, and therefore a call to push
events on the queue never block, but will throw an exception unless
tryEnqueue() is called.
In essence, enqueue() can assert backpressure if the queue is not emptied
or processed quick enough. Dequeue can block until there are actual
events to dequeue ( or there are X events to dequeue in the case of
dequeue( X ) ).
I will merge the interfaces with some of the ideas talked about today, and
let me know if you like it better....
--
"They that give up essential liberty to obtain a little temporary safety
deserve neither liberty nor safety."
- Benjamin Franklin
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: [Review] Event Queues
Posted by Peter Donald <pe...@apache.org>.
On Sat, 15 Dec 2001 08:43, Berin Loritsch wrote:
> >>I would like to have reactions on the Blocking and EventDropping issues
> >>though....
> >
> > Event dropping should be an application specific thing. However I am not
> > sure why Blocking and non-Blocking versions of interfaces need to be
> > separate. I haven't looked closely at the proposal though - will try to
> > do so real soon now ;)
>
> Yes, but should they affect the way a system is written? For instance,
> the basic operation of dequeue() doesn't change. Is it sufficient
> for the blocking timeout to be set once for the pipeline, or should it be
> set for *each* dequeue() method? Personally, I lean toward setting it
> once.
Hmmm I guess so. What I was thinking is that "void enqueue()" should either
always pass (and thus block) or throw an exception. If you wanted more
control (ie making sure it doesn't block) then you should use tryEnqueue
> SEDA:
>
> boolean success = queue.enqueue_lossy( event );
> if ( ! success )
> {
> handleFailure();
> }
>
> Avalon:
>
> boolean success = queue.tryEnqueue( event );
> if ( ! success )
> {
> handleFailure();
> }
+1
--
Cheers,
Pete
---------------------------------------
Be nice to your friends. If it weren't
for them, you'd be a complete stranger.
---------------------------------------
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: [Review] Event Queues
Posted by Berin Loritsch <bl...@apache.org>.
Peter Donald wrote:
> On Sat, 15 Dec 2001 02:32, Berin Loritsch wrote:
>
>>Avalon:
>>
>>PreparedEnqueue transaction = source.prepareEnqueue( events ); // Avalon
>>queues start with a source. if ( should_commit )
>>{
>> transaction.commit(); // Never have a bad key!
>>}
>>else
>>{
>> transaction.abort(); // Never have a bad key!
>>}
>>
>
> +1
I'm glad you like it. :)
>
>
>>I would like to have reactions on the Blocking and EventDropping issues
>>though....
>>
>
> Event dropping should be an application specific thing. However I am not sure
> why Blocking and non-Blocking versions of interfaces need to be separate. I
> haven't looked closely at the proposal though - will try to do so real soon
> now ;)
Yes, but should they affect the way a system is written? For instance,
the basic operation of dequeue() doesn't change. Is it sufficient
for the blocking timeout to be set once for the pipeline, or should it be
set for *each* dequeue() method? Personally, I lean toward setting it once.
The only use I _personally_ see for a blocking queue is something like the
CommandQueue that I will expand this proposal to include. In that case,
there is a reason to block the thread. However, in the full staged system,
blocking would be detrimental.
Also, the EventDropping thing is arguably an application specific thing,
so if that is the consensus, I will add it to the Source interface as
tryEnqueue() which is much better than enqueue_lossy(), and is more semantically
correct.
SEDA:
boolean success = queue.enqueue_lossy( event );
if ( ! success )
{
handleFailure();
}
Avalon:
boolean success = queue.tryEnqueue( event );
if ( ! success )
{
handleFailure();
}
The name tryEnqueue builds on the familiar tryAquire() in Mutex and DijkstraSemaphore.
The name enqueue_lossy() seems to leave too many questions unanswered by it's
name. Restated, I will easily derive the logic of tryEnqueue() without having
to read the documentation (enqueue_lossy raises questions of algorithm and is the
queue going to drop other events to make room for this one?).
--
"They that give up essential liberty to obtain a little temporary safety
deserve neither liberty nor safety."
- Benjamin Franklin
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: [Review] Event Queues
Posted by Peter Donald <pe...@apache.org>.
On Sat, 15 Dec 2001 02:32, Berin Loritsch wrote:
> Avalon:
>
> PreparedEnqueue transaction = source.prepareEnqueue( events ); // Avalon
> queues start with a source. if ( should_commit )
> {
> transaction.commit(); // Never have a bad key!
> }
> else
> {
> transaction.abort(); // Never have a bad key!
> }
+1
> I would like to have reactions on the Blocking and EventDropping issues
> though....
Event dropping should be an application specific thing. However I am not sure
why Blocking and non-Blocking versions of interfaces need to be separate. I
haven't looked closely at the proposal though - will try to do so real soon
now ;)
--
Cheers,
Pete
----------------------------------------
Why does everyone always overgeneralize?
----------------------------------------
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>