You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@felix.apache.org by Joel Schuster <jo...@gmx.com> on 2012/04/04 19:18:30 UTC

EventAdmin Performance

Carsten et al.,

 

I've been running some profiling against some test components using
EventAdmin. I'm seeing some really degenerate behavior that's causing some
real slowdowns for message delivery coming from a few places.

 

Environment: running 400+ messages per second and with 15+ topics with
various levels of hierarchy.

 

Because of the general brute force means by which
org.apache.felix.framework.capabilityset.CapabilitySet.convertArrayToList
works, and how the multiple layers of
org.apache.felix.framework.capabilityset.CapabilitySet.match work when
topics have multiple layers of hierarchy with the names I can double the
throughput of EventAdmin by simply changing my topic names to one layer of
hierarchy. Of course I lose the advantages of the fiters, but if I can
double the throughput then it seems worth it.

 

Even then its still taking more than half a millisecond (optimally,
sometimes many tens of milliseconds) for any particular message to make its
way through the EventAdmin system, being pushed and handled. That seems like
a long time. Part of the overhead comes from the
org.apache.felix.framework.ServiceRegistry.getService and ungetService calls
that happen for every single message push.

 

It seems to me that some sort of caching of previous search results in each
of these domains would be well worth the effort.

 

Carsten, I noticed that you had commented about an improved EventAdmin in
the sandbox. Can you please point that out to me, Id like to try it out.

 

Also, Ive notice that the EventAdmin is not delivering all the events that
are pushed to a topic. Small percentages (1 / 1000) are simply being eaten
up somewhere I cant seem to track down any reason or log for why the
events simply disappeared.

 

 

Thanks all for consideration.

 

 

 



 

- Joel

 

 

> -----Original Message-----

> From: Carsten Ziegeler [mailto:cziegeler@apache.org]

> Sent: Tuesday, January 24, 2012 2:55 AM

> To: users@felix.apache.org

> Subject: Re: EventAdmin Asynchronous/Parallel EventHandler

> 

> Hi,

> 

> we haven't implemented this feature yet but will do once the final

> spec is publically available.

> 

> I've written a new implementation of the event admin which is in the

> felix sandbox atm, this one is significanlty faster than the old

> implementation from trunk.

> 

> Regards

> Carsten

> 

> 2012/1/18 DBinSD <cd...@gmail.com>:

> >

> > Thanks Holger, this is exactly the info I was looking for.  It sounds
like

> > the Async Unordered Events might get me the performance I was looking

> for

> > (though the implications of 'unordered' events might be a problem in
some

> > applications).  I will keep an eye out for an implementation of this

> > feature.

> >

> > Thanks again

> > Chris

> >

> >

> >

> > Holger Hoffstätte-4 wrote:

> >>

> >> On 18.01.2012 03:35, DBinSD wrote:

> >>> Does the felix implementation of the EventAdmin service support the

> >>> delivery

> >>> of events, asynchronously and in parallel to multiple listeners at
once?

> >>> It

> >>

> >> This was addressed as part of RFC 157 ("Updates to EventAdmin")

> sometime

> >> in 2010 and accepted, though I don't remember offhand what the

> >> implementation status was/is. IIRC it was part of 4.3.

> >>

> >> The solution added a set of constants that express basic delivery

> >> behaviour/expectations. Not really enforceable end-to-end QoS

> constraints

> >> (which are impossible in Java anyway) but better than nothing.

> >>

> >> --snip--

> >>

> >> 5.2    Asynchronous Unordered Events

> >>

> >> The following constants are added to EventConstants. This allows an

> Event

> >> Handler to indicate that it is willing to relax the event ordering

> >> requirements so that Event Admin can optimize delivery.

> >>

> >> String EVENT_DELIVERY = "event.delivery"

> >> Service Registration property specifying the delivery qualities
requested

> >> by an Event Handler service.

> >>

> >> Event handlers MAY be registered with this property. Each value of this

> >> property is a string specifying a delivery quality for the Event
handler.

> >>

> >> The value of this property must be of type String, String[], or

> >> Collection<String>.

> >>

> >> Since:

> >>         1.3

> >> See Also:

> >>         DELIVERY_ASYNC_ORDERED

> >>         DELIVERY_ASYNC_UNORDERED

> >>

> >> String DELIVERY_ASYNC_ORDERED = "async.ordered"

> >> Event Handler delivery quality value specifying the Event Handler
requires

> >> asynchronously delivered events be delivered in order. Ordered delivery

> is

> >> the default for asynchronously delivered events.

> >>

> >> This delivery quality value is mutually exclusive with

> >> DELIVERY_ASYNC_UNORDERED. However, if both this value and

> >> DELIVERY_ASYNC_UNORDERED are specifed for an event handler, this

> value

> >> takes precedence.

> >>

> >> Since:

> >>         1.3

> >> See Also:

> >>         EVENT_DELIVERY

> >>

> >> String DELIVERY_ASYNC_UNORDERED = "async.unordered"

> >> Event Handler delivery quality value specifying the Event Handler does

> not

> >> require asynchronously delivered events be delivered in order. This may

> >> allow an Event Admin implementation to optimize asynchronous event

> >> delivery by relaxing ordering requirements.

> >>

> >> This delivery quality value is mutually exclusive with

> >> DELIVERY_ASYNC_ORDERED. However, if both this value and

> >> DELIVERY_ASYNC_ORDERED are specifed for an event handler,

> >> DELIVERY_ASYNC_ORDERED takes precedence.

> >>

> >> Since:

> >>         1.3

> >>

> >> See Also:

> >>         EVENT_DELIVERY

> >>

> >> --snip--

> >>

> >> Also see

> >> http://stackoverflow.com/questions/7018762/when-to-use-osgi-

> eventadmin-and-when-not

> >> for BJ's official answer.

> >>

> >> hope this helps

> >> Holger

> >>

> >> ---------------------------------------------------------------------

> >> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org

> >> For additional commands, e-mail: users-help@felix.apache.org

> >>

> >>

> >>

> >

> > --

> > View this message in context: http://old.nabble.com/EventAdmin-

> Asynchronous-Parallel-EventHandler-tp33158446p33162303.html

> > Sent from the Apache Felix - Users mailing list archive at Nabble.com.

> >

> >

> > ---------------------------------------------------------------------

> > To unsubscribe, e-mail: users-unsubscribe@felix.apache.org

> > For additional commands, e-mail: users-help@felix.apache.org

> >

> 

> 

> 

> --

> Carsten Ziegeler

> cziegeler@apache.org

> 

> ---------------------------------------------------------------------

> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org

> For additional commands, e-mail: users-help@felix.apache.org

RE: EventAdmin Performance

Posted by Joel Schuster <jo...@gmx.com>.

All,

Thanks for the input. I'm trying out the 1.2.15-SNAPSHOT version of event
admin and it's behaving much better. Thanks!

I am still getting dropped events; now to look into that.

- Joel


> -----Original Message-----
> From: Marcel Offermans [mailto:marcel.offermans@luminis.nl]
> Sent: Wednesday, April 04, 2012 12:40 PM
> To: users@felix.apache.org
> Subject: Re: EventAdmin Performance
> 
> How about just putting those bundles in your sandbox? I would be
> interested to have something to measure the performance of EventAdmin.
> It does not necessarily need to be a Pax Exam compatible test.
> 
> Greetings, Marcel
> 
> 
> On Apr 4, 2012, at 19:50 PM, Carsten Ziegeler wrote:
> 
> > I've a performance test (and several others checking for deadlocks,
> > event delivery etc). They all come as bundles which I deploy into a
> > running instance and then check the results. I guess these could be
> > converted to pax exam tests - but right now I don't have time to do
> > that :(
> >
> > Regards
> > Carsten
> >
> > 2012/4/4 Marcel Offermans <ma...@luminis.nl>:
> >> Out of curiosity, did anybody ever write a "test" that we can run to
> benchmark the performance of EventAdmin. I don't mean an ad-hoc test but
> something we can put in the Felix repository (in a sandbox).
> >>
> >> Greetings, Marcel
> >>
> >> On Apr 4, 2012, at 19:30 PM, Carsten Ziegeler wrote:
> >>
> >>> Hi Joel,
> >>>
> >>> yes, we noticed some performance problems as well - apart from the
> problem
> >>> you mentioned in our case it was a very high load on the service
registry
> >>> (which in addition could also cause deadlocks in the framework).
> >>> So I reimplemented the whole thing (FELIX-3321). You can find the code
> now
> >>> in the latest trunk, so you could just build the version from trunk
and see
> >>> how it performs. It basically changes the way how event handlers are
> >>> fetched from the registry and how the event topic is compared -
filters
> are
> >>> now only used if really required, the main checks are done by simple
> string
> >>> comparisons.
> >>>
> >>> For our really simple performance test case with more than 1 million
> >>> events, it took more than 30 seconds with the old implementation, the
> new
> >>> one processes the same amount in less than a second.
> >>>
> >>> Not sure about your problem if events getting eaten up. With the new
> impl
> >>> it should be much easier to track this down (if it still happens)
> >>>
> >>> Regards
> >>> Carsten
> >>>
> >>> 2012/4/4 Joel Schuster <jo...@gmx.com>
> >>>
> >>>> Carsten et al.,****
> >>>>
> >>>> ** **
> >>>>
> >>>> I've been running some profiling against some test components using
> >>>> EventAdmin. I'm seeing some really degenerate behavior that's causing
> some
> >>>> real slowdowns for message delivery coming from a few places.****
> >>>>
> >>>> ** **
> >>>>
> >>>> Environment: running 400+ messages per second and with 15+ topics
> with
> >>>> various levels of hierarchy.****
> >>>>
> >>>> ** **
> >>>>
> >>>> Because of the general brute force means by which
> >>>>
> org.apache.felix.framework.capabilityset.CapabilitySet.convertArrayToList
> >>>> works, and how the multiple layers of
> >>>> org.apache.felix.framework.capabilityset.CapabilitySet.match work
> when
> >>>> topics have multiple layers of hierarchy with the names I can double
> the
> >>>> throughput of EventAdmin by simply changing my topic names to one
> layer of
> >>>> hierarchy. Of course I lose the advantages of the fiters, but if I
can
> >>>> double the throughput then it seems worth it.****
> >>>>
> >>>> ** **
> >>>>
> >>>> Even then its still taking more than half a millisecond (optimally,
> >>>> sometimes many tens of milliseconds) for any particular message to
> make its
> >>>> way through the EventAdmin system, being pushed and handled. That
> seems
> >>>> like a long time. Part of the overhead comes from the
> >>>> org.apache.felix.framework.ServiceRegistry.getService and
> ungetService
> >>>> calls that happen for every single message push.****
> >>>>
> >>>> ** **
> >>>>
> >>>> It seems to me that some sort of caching of previous search results
in
> >>>> each of these domains would be well worth the effort.****
> >>>>
> >>>> ** **
> >>>>
> >>>> Carsten, I noticed that you had commented about an improved
> EventAdmin in
> >>>> the sandbox. Can you please point that out to me, Id like to try it
out.*
> >>>> ***
> >>>>
> >>>> ** **
> >>>>
> >>>> Also, Ive notice that the EventAdmin is not delivering all the
events
> >>>> that are pushed to a topic. Small percentages (1 / 1000) are simply
> being
> >>>> eaten up somewhere I cant seem to track down any reason or log for
> why
> >>>> the events simply disappeared.****
> >>>>
> >>>> ** **
> >>>>
> >>>> ** **
> >>>>
> >>>> Thanks all for consideration.****
> >>>>
> >>>> ** **
> >>>>
> >>>> ** **
> >>>>
> >>>> ** **
> >>>>
> >>>> ****
> >>>>
> >>>> ** **
> >>>>
> >>>> - Joel****
> >>>>
> >>>> ** **
> >>>>
> >>>> ** **
> >>>>
> >>>>> -----Original Message-----
> >>>>
> >>>>> From: Carsten Ziegeler [mailto:cziegeler@apache.org]
> >>>>
> >>>>> Sent: Tuesday, January 24, 2012 2:55 AM
> >>>>
> >>>>> To: users@felix.apache.org
> >>>>
> >>>>> Subject: Re: EventAdmin Asynchronous/Parallel EventHandler
> >>>>
> >>>>>
> >>>>
> >>>>> Hi,
> >>>>
> >>>>>
> >>>>
> >>>>> we haven't implemented this feature yet but will do once the final
> >>>>
> >>>>> spec is publically available.
> >>>>
> >>>>>
> >>>>
> >>>>> I've written a new implementation of the event admin which is in the
> >>>>
> >>>>> felix sandbox atm, this one is significanlty faster than the old
> >>>>
> >>>>> implementation from trunk.
> >>>>
> >>>>>
> >>>>
> >>>>> Regards
> >>>>
> >>>>> Carsten
> >>>>
> >>>>>
> >>>>
> >>>>> 2012/1/18 DBinSD <cd...@gmail.com>:
> >>>>
> >>>>>>
> >>>>
> >>>>>> Thanks Holger, this is exactly the info I was looking for.  It
sounds
> >>>> like
> >>>>
> >>>>>> the Async Unordered Events might get me the performance I was
> looking
> >>>>
> >>>>> for
> >>>>
> >>>>>> (though the implications of 'unordered' events might be a problem
> in
> >>>> some
> >>>>
> >>>>>> applications).  I will keep an eye out for an implementation of
this
> >>>>
> >>>>>> feature.
> >>>>
> >>>>>>
> >>>>
> >>>>>> Thanks again
> >>>>
> >>>>>> Chris
> >>>>
> >>>>>>
> >>>>
> >>>>>>
> >>>>
> >>>>>>
> >>>>
> >>>>>> Holger Hoffstätte-4 wrote:
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> On 18.01.2012 03:35, DBinSD wrote:
> >>>>
> >>>>>>>> Does the felix implementation of the EventAdmin service support
> the
> >>>>
> >>>>>>>> delivery
> >>>>
> >>>>>>>> of events, asynchronously and in parallel to multiple listeners
at
> >>>> once?
> >>>>
> >>>>>>>> It
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> This was addressed as part of RFC 157 ("Updates to EventAdmin")
> >>>>
> >>>>> sometime
> >>>>
> >>>>>>> in 2010 and accepted, though I don't remember offhand what the
> >>>>
> >>>>>>> implementation status was/is. IIRC it was part of 4.3.
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> The solution added a set of constants that express basic delivery
> >>>>
> >>>>>>> behaviour/expectations. Not really enforceable end-to-end QoS
> >>>>
> >>>>> constraints
> >>>>
> >>>>>>> (which are impossible in Java anyway) but better than nothing.
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> --snip--
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> 5.2    Asynchronous Unordered Events
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> The following constants are added to EventConstants. This allows
> an
> >>>>
> >>>>> Event
> >>>>
> >>>>>>> Handler to indicate that it is willing to relax the event ordering
> >>>>
> >>>>>>> requirements so that Event Admin can optimize delivery.
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> String EVENT_DELIVERY = "event.delivery"
> >>>>
> >>>>>>> Service Registration property specifying the delivery qualities
> >>>> requested
> >>>>
> >>>>>>> by an Event Handler service.
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> Event handlers MAY be registered with this property. Each value of
> >>>> this
> >>>>
> >>>>>>> property is a string specifying a delivery quality for the Event
> >>>> handler.
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> The value of this property must be of type String, String[], or
> >>>>
> >>>>>>> Collection<String>.
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> Since:
> >>>>
> >>>>>>>        1.3
> >>>>
> >>>>>>> See Also:
> >>>>
> >>>>>>>        DELIVERY_ASYNC_ORDERED
> >>>>
> >>>>>>>        DELIVERY_ASYNC_UNORDERED
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> String DELIVERY_ASYNC_ORDERED = "async.ordered"
> >>>>
> >>>>>>> Event Handler delivery quality value specifying the Event Handler
> >>>> requires
> >>>>
> >>>>>>> asynchronously delivered events be delivered in order. Ordered
> >>>> delivery
> >>>>
> >>>>> is
> >>>>
> >>>>>>> the default for asynchronously delivered events.
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> This delivery quality value is mutually exclusive with
> >>>>
> >>>>>>> DELIVERY_ASYNC_UNORDERED. However, if both this value and
> >>>>
> >>>>>>> DELIVERY_ASYNC_UNORDERED are specifed for an event handler,
> this
> >>>>
> >>>>> value
> >>>>
> >>>>>>> takes precedence.
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> Since:
> >>>>
> >>>>>>>        1.3
> >>>>
> >>>>>>> See Also:
> >>>>
> >>>>>>>        EVENT_DELIVERY
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> String DELIVERY_ASYNC_UNORDERED = "async.unordered"
> >>>>
> >>>>>>> Event Handler delivery quality value specifying the Event Handler
> does
> >>>>
> >>>>> not
> >>>>
> >>>>>>> require asynchronously delivered events be delivered in order.
This
> >>>> may
> >>>>
> >>>>>>> allow an Event Admin implementation to optimize asynchronous
> event
> >>>>
> >>>>>>> delivery by relaxing ordering requirements.
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> This delivery quality value is mutually exclusive with
> >>>>
> >>>>>>> DELIVERY_ASYNC_ORDERED. However, if both this value and
> >>>>
> >>>>>>> DELIVERY_ASYNC_ORDERED are specifed for an event handler,
> >>>>
> >>>>>>> DELIVERY_ASYNC_ORDERED takes precedence.
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> Since:
> >>>>
> >>>>>>>        1.3
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> See Also:
> >>>>
> >>>>>>>        EVENT_DELIVERY
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> --snip--
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> Also see
> >>>>
> >>>>>>> http://stackoverflow.com/questions/7018762/when-to-use-osgi-
> >>>>
> >>>>> eventadmin-and-when-not
> >>>>
> >>>>>>> for BJ's official answer.
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> hope this helps
> >>>>
> >>>>>>> Holger
> >>>>
> >>>>>>>
> >>>>
> >>>>>>>
---------------------------------------------------------------------
> >>>>
> >>>>>>> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
> >>>>
> >>>>>>> For additional commands, e-mail: users-help@felix.apache.org
> >>>>
> >>>>>>>
> >>>>
> >>>>>>>
> >>>>
> >>>>>>>
> >>>>
> >>>>>>
> >>>>
> >>>>>> --
> >>>>
> >>>>>> View this message in context: http://old.nabble.com/EventAdmin-
> >>>>
> >>>>> Asynchronous-Parallel-EventHandler-tp33158446p33162303.html
> >>>>
> >>>>>> Sent from the Apache Felix - Users mailing list archive at
> Nabble.com.
> >>>>
> >>>>>>
> >>>>
> >>>>>>
> >>>>
> >>>>>>
---------------------------------------------------------------------
> >>>>
> >>>>>> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
> >>>>
> >>>>>> For additional commands, e-mail: users-help@felix.apache.org
> >>>>
> >>>>>>
> >>>>
> >>>>>
> >>>>
> >>>>>
> >>>>
> >>>>>
> >>>>
> >>>>> --
> >>>>
> >>>>> Carsten Ziegeler
> >>>>
> >>>>> cziegeler@apache.org
> >>>>
> >>>>>
> >>>>
> >>>>>
---------------------------------------------------------------------
> >>>>
> >>>>> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
> >>>>
> >>>>> For additional commands, e-mail: users-help@felix.apache.org
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> Carsten Ziegeler
> >>> cziegeler@apache.org
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
> >> For additional commands, e-mail: users-help@felix.apache.org
> >>
> >
> >
> >
> > --
> > Carsten Ziegeler
> > cziegeler@apache.org
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
> > For additional commands, e-mail: users-help@felix.apache.org
> >
> >
> >
> >
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
> For additional commands, e-mail: users-help@felix.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
For additional commands, e-mail: users-help@felix.apache.org