Batched processing of OSGi events happening in the same short period

[Robert Munteanu <ro...@apache.org>]

Hi,

I have an OSGi EventListener which receives notifications related to a
certain content tree. This listener works with a remote system and also
writes to the repository.

When a lot of changes happen in the subtree I'm interested in, I get
flooded with many events, and since I'm working with remote systems I
have implemented some locking within the listener to make sure I send a
coherent state. But this is starting to get hairy once I start writing
to the repo as well.

What I'd like to do is batch and process events only after some quiet
time ( 500 ms for instance ). For each event, I would place it in queue
and wait for 500ms to see if another event comes in. If another event
comes in, the timer is restarted. Once the timer is complete, I'd
process all the events in one go.

I can implement this myself ( and have fun ), but I was wondering
whether there are any building blocks in Sling that I can use for this.

Thanks,

Robert

Re: Batched processing of OSGi events happening in the same short period

[Robert Munteanu <ro...@apache.org>]

On Wed, Jan 21, 2015 at 1:26 PM, Bertrand Delacretaz
<bd...@apache.org> wrote:
> Hi,
>
> On Wed, Jan 21, 2015 at 12:06 PM, Robert Munteanu <ro...@apache.org> wrote:
>> ...What I'd like to do is batch and process events only after some quiet
>> time ( 500 ms for instance ). For each event, I would place it in queue
>> and wait for 500ms to see if another event comes in. If another event
>> comes in, the timer is restarted. Once the timer is complete, I'd
>> process all the events in one go....
>
> Note that you'll need some additional logic to make sure you still
> process events if they keep on coming without quiet time.

Ack

>
>> ...I can implement this myself ( and have fun ), but I was wondering
>> whether there are any building blocks in Sling that I can use for this....
>
> I'm pretty sure I implemented something like this to process JCR
> events in this way, but I cannot find it now :-/
>
> We probably don't have something reusable, if you can create a
> reusable EventClusteringQueue (?) that's useful. Maybe under
> bundles/commons?

OK. I Intentionally simplified my scenario a bit - the batching should
be applied only for distinct parts of the content subtree, e.g.
/content/a and /content/b will not get batched together, but
/content/a and /content/a/1 OR /content/a/1 and /content/a/2 will. If
I can create something generic I'll submit back.

Thanks,

Robert

Re: Batched processing of OSGi events happening in the same short period

[Bertrand Delacretaz <bd...@apache.org>]

Hi,

On Wed, Jan 21, 2015 at 12:06 PM, Robert Munteanu <ro...@apache.org> wrote:
> ...What I'd like to do is batch and process events only after some quiet
> time ( 500 ms for instance ). For each event, I would place it in queue
> and wait for 500ms to see if another event comes in. If another event
> comes in, the timer is restarted. Once the timer is complete, I'd
> process all the events in one go....

Note that you'll need some additional logic to make sure you still
process events if they keep on coming without quiet time.

> ...I can implement this myself ( and have fun ), but I was wondering
> whether there are any building blocks in Sling that I can use for this....

I'm pretty sure I implemented something like this to process JCR
events in this way, but I cannot find it now :-/

We probably don't have something reusable, if you can create a
reusable EventClusteringQueue (?) that's useful. Maybe under
bundles/commons?

-Bertrand