Scalabilty of EventListener

[Dominik Süß <do...@gmail.com>]

In a discussion [0] within the Oak mailinglist it became clear that the way
Sling listens zu JCR Repository Changes and transforms all of them to
events will not scale well in some big scale scenarios that oak is aiming
to enable. Therefore the question was posted if it would be feasible and/or
even necessary to refactor the API and deprecate (or at least discurrage)
registration in a global scope as currently done.

Since I do not want to copy & paste parts of the discussion I do hope that
the participants of the oak discussion add the remaining options with some
more detail about consequences than can be found in the linked discussion.

It would be great to also get some feedbacks of consumers of the existing
API about the usage to identify how finegrained a potential
registrationlogic with paths/properties might need to be.

Best regards
Dominik

[0] http://markmail.org/message/n5vllhjoawypteck

Re: Scalabilty of EventListener

[Alexander Klimetschek <ak...@adobe.com>]

On 26.10.2013, at 09:49, Justin Edelson <ju...@justinedelson.com> wrote:

> Why does the workflow engine have to make that decision itself? Can't
> that decision be expressed declaratively? They are, after all, being
> defined declaratively by the administrator, so it should just be a
> matter of mapping the administrator's selections to a filter
> expression and then letting the EventAdmin implementation handle the
> filter. For example, you could have a workflow launcher expressed as a
> listener with a filter of
> "(&(resourceType=nt:file)(path=/content/dam/*/original))".

Yes, but someone needs to be able to implement that - this would include more, not only a resource type check, but any property, e.g. (someProperty!=somevalue). Whether this is in the workflow or in the event admin doesn't matter for my POV, both are above Oak in the stack. Even firing the event admin in OSGi would be wasted resources if you can know upfront - while you have access to the content being changed - that you don't need to fire the event at all.

> NOTE - I'm not necessarily saying that everything in CQ's workflow
> engine is supported by Sling Resource events *today*; there might be a
> need to add new event properties, but that's far from impossible.
> 
> My point is that it is in large part the lack of good filtering (which
> is available in EventAdmin and JMS) is what leads to having multiple
> dispatching systems in the same applicatio and that good filtering is
> already available in Sling applications *because* of the
> Observation/EventAdmin bridge.

We are talking about performance and scalability, and I think for that it is best handled as early as possible.

Cheers,
Alex

Re: Scalabilty of EventListener

[Justin Edelson <ju...@justinedelson.com>]

Hi,

On Fri, Oct 25, 2013 at 4:54 PM, Alexander Klimetschek
<ak...@adobe.com> wrote:
> On 24.10.2013, at 16:55, Justin Edelson <ju...@justinedelson.com> wrote:
>
>> Given that there's already an expression syntax defined in the
>> EventAdmin specification, wouldn't it make more sense for Oak to just
>> use OSGi Events? That's the only way I personally could forsee us
>> being in a position to deprecate/remote the JCR Observation/OSGi
>> EventAdmin bridge which we have in Sling today.
>
> I don't think Oak wants to be tied to OSGi, but the expression syntax could maybe reused.

Better IMHO would be for Oak to use the event/messaging system
provided by the platform. In an OSGi environment, this would be
EventAdmin. In JavaEE, it would be JMS (for which message selectors
can be used for filtering).

>
>> FWIW, I'm not sure it's a straight comparison between the bridge we
>> have in Sling and the CQ Workflow engine. The CQ Workflow engine
>> *could* be implemented by registering specific OSGi event listeners
>> (one per launcher). It just isn't (or so your email implies :).
>
> Not really, because it has to make the decision on matching events *itself*. Because of that, it is natural to only have one central listener (it quickly forks off into separate jobs once a match for a certain entry has been found).

Why does the workflow engine have to make that decision itself? Can't
that decision be expressed declaratively? They are, after all, being
defined declaratively by the administrator, so it should just be a
matter of mapping the administrator's selections to a filter
expression and then letting the EventAdmin implementation handle the
filter. For example, you could have a workflow launcher expressed as a
listener with a filter of
"(&(resourceType=nt:file)(path=/content/dam/*/original))".

NOTE - I'm not necessarily saying that everything in CQ's workflow
engine is supported by Sling Resource events *today*; there might be a
need to add new event properties, but that's far from impossible.

My point is that it is in large part the lack of good filtering (which
is available in EventAdmin and JMS) is what leads to having multiple
dispatching systems in the same applicatio and that good filtering is
already available in Sling applications *because* of the
Observation/EventAdmin bridge.

Justin

>
> What I am asking for is that Oak provides those options (match on property values etc.) directly and it can handled as early as possible, without Oak having to call an event listener in the first place (and start a new thread etc.) in case there is no match.
>
> Cheers,
> Alex
>

Re: Scalabilty of EventListener

[Alexander Klimetschek <ak...@adobe.com>]

On 24.10.2013, at 16:55, Justin Edelson <ju...@justinedelson.com> wrote:

> Given that there's already an expression syntax defined in the
> EventAdmin specification, wouldn't it make more sense for Oak to just
> use OSGi Events? That's the only way I personally could forsee us
> being in a position to deprecate/remote the JCR Observation/OSGi
> EventAdmin bridge which we have in Sling today.

I don't think Oak wants to be tied to OSGi, but the expression syntax could maybe reused.

> FWIW, I'm not sure it's a straight comparison between the bridge we
> have in Sling and the CQ Workflow engine. The CQ Workflow engine
> *could* be implemented by registering specific OSGi event listeners
> (one per launcher). It just isn't (or so your email implies :).

Not really, because it has to make the decision on matching events *itself*. Because of that, it is natural to only have one central listener (it quickly forks off into separate jobs once a match for a certain entry has been found).

What I am asking for is that Oak provides those options (match on property values etc.) directly and it can handled as early as possible, without Oak having to call an event listener in the first place (and start a new thread etc.) in case there is no match.

Cheers,
Alex

Re: Scalabilty of EventListener

[Justin Edelson <ju...@justinedelson.com>]

Hi Alex,
Given that there's already an expression syntax defined in the
EventAdmin specification, wouldn't it make more sense for Oak to just
use OSGi Events? That's the only way I personally could forsee us
being in a position to deprecate/remote the JCR Observation/OSGi
EventAdmin bridge which we have in Sling today.

FWIW, I'm not sure it's a straight comparison between the bridge we
have in Sling and the CQ Workflow engine. The CQ Workflow engine
*could* be implemented by registering specific OSGi event listeners
(one per launcher). It just isn't (or so your email implies :). The
Observation/EventAdmin bridge, however, is intended to bridge these
two eventing systems and so it pretty much has to listen for all
events and then rebroadcast them (in slightly mutated form). Same
thing is true if you are building an EventAdmin/JMS bridge or a JCR
Observation/JMS bridge or anything like that.

Justin

On Thu, Oct 24, 2013 at 7:14 PM, Alexander Klimetschek
<ak...@adobe.com> wrote:
> I don't think it's so theoretical: a generic observation listener such as the jcr listener in sling, that handles all events all the time with no option to reduce the scope in any way, seems like a barrier to scalability.
>
> The goal should be that any kind of observation should always be as specific as possible (by path, properties, resource type etc.) and that should ideally be handled on the oak level where it can be optimized the most.
>
> The problem basically was that the JCR observation listener API doesn't allow much constraints, only path and node types. But not by property values. Or other things. So "everyone" came up with his own generic dispatching listener: sling with one that checks the resource type, the aforementioned workflow launcher in Adobe CQ which allows any property values and other constraints etc.
>
> There should be one in Oak, with a few common options (property values etc.) that can be heavily optimized and optionally a custom function that runs on the node in question and could handle more complex decisions whether to handle the event or not (at your own risk regarding performance :)).
>
> Cheers,
> Alex
>
> On 24.10.2013, at 00:04, Carsten Ziegeler <cz...@apache.org> wrote:
>
>> As stated, so far the whole discussion is very theoretical - apart from the
>> point that right now the jcr listener does a read for each and every
>> changed node to get the resource type. This could definitely be improved
>> with Oak as Oak is doing a content diff and therefore it should be possible
>> to send this information down to the jcr listener. This would speed up
>> things dramatically.
>> But without concrete numbers and performance information, all of this
>> sounds like premature optimization. Reinventing the wheel just slightly
>> different without any way to proof whether it helps just doesn't sound
>> right to me.
>> We need to have a look at the whole system - it doesn't help if we talk
>> about fixing one part of it for such scenarios while completely neglecting
>> other parts. Therefore, (I know I repeat myself) I'm really looking forward
>> to numbers showing the bottlenecks.
>>
>> Thanks
>> Carsten
>> --
>> Carsten Ziegeler
>> cziegeler@apache.org
>

Re: Scalabilty of EventListener

[Alexander Klimetschek <ak...@adobe.com>]

I don't think it's so theoretical: a generic observation listener such as the jcr listener in sling, that handles all events all the time with no option to reduce the scope in any way, seems like a barrier to scalability.

The goal should be that any kind of observation should always be as specific as possible (by path, properties, resource type etc.) and that should ideally be handled on the oak level where it can be optimized the most.

The problem basically was that the JCR observation listener API doesn't allow much constraints, only path and node types. But not by property values. Or other things. So "everyone" came up with his own generic dispatching listener: sling with one that checks the resource type, the aforementioned workflow launcher in Adobe CQ which allows any property values and other constraints etc.

There should be one in Oak, with a few common options (property values etc.) that can be heavily optimized and optionally a custom function that runs on the node in question and could handle more complex decisions whether to handle the event or not (at your own risk regarding performance :)).

Cheers,
Alex

On 24.10.2013, at 00:04, Carsten Ziegeler <cz...@apache.org> wrote:

> As stated, so far the whole discussion is very theoretical - apart from the
> point that right now the jcr listener does a read for each and every
> changed node to get the resource type. This could definitely be improved
> with Oak as Oak is doing a content diff and therefore it should be possible
> to send this information down to the jcr listener. This would speed up
> things dramatically.
> But without concrete numbers and performance information, all of this
> sounds like premature optimization. Reinventing the wheel just slightly
> different without any way to proof whether it helps just doesn't sound
> right to me.
> We need to have a look at the whole system - it doesn't help if we talk
> about fixing one part of it for such scenarios while completely neglecting
> other parts. Therefore, (I know I repeat myself) I'm really looking forward
> to numbers showing the bottlenecks.
> 
> Thanks
> Carsten
> -- 
> Carsten Ziegeler
> cziegeler@apache.org

Re: Scalabilty of EventListener

[Carsten Ziegeler <cz...@apache.org>]

As stated, so far the whole discussion is very theoretical - apart from the
point that right now the jcr listener does a read for each and every
changed node to get the resource type. This could definitely be improved
with Oak as Oak is doing a content diff and therefore it should be possible
to send this information down to the jcr listener. This would speed up
things dramatically.
But without concrete numbers and performance information, all of this
sounds like premature optimization. Reinventing the wheel just slightly
different without any way to proof whether it helps just doesn't sound
right to me.
We need to have a look at the whole system - it doesn't help if we talk
about fixing one part of it for such scenarios while completely neglecting
other parts. Therefore, (I know I repeat myself) I'm really looking forward
to numbers showing the bottlenecks.

Thanks
Carsten
-- 
Carsten Ziegeler
cziegeler@apache.org

Re: Scalabilty of EventListener

[Bertrand Delacretaz <bd...@apache.org>]

On Wed, Oct 23, 2013 at 4:42 PM, Dominik Süß <do...@gmail.com> wrote:
> ...Regarding the options you were stating:
> - service properties: my initial thought, but static and can therefore not
> adress all scenarios...

Service properties for a given service are static, but a service that
has specific properties can come and go.

IMO we need to work on concrete use cases to decide on the best option.

-Bertrand

Re: Scalabilty of EventListener

[Dominik Süß <do...@gmail.com>]

Hi Bertrand,

I'm not so sure about the seldom changes - it is not about the amound of
changes but about the amount of changes in a specific timeframe. The
initial discussion started with the asumtion to have to deal with tons of
changes within a second, e.g. for huge imports.  In some cases those
imports are real imports (so new creation) so postponing the processing and
delegating to some postprocessing might work - but in case of updates this
is not so easy (and I know such cases from the automotive area where masses
of technical data is synched to all markets).

Regarding the options you were stating:
- service properties: my initial thought, but static and can therefore not
adress all scenarios
- registring for certain types: same problem, a bit more possibilities but
restricted to the capabilities of the registry

Therefore I thought of a filterlike behavior where the listeners can
implement their own logic. Including measuring the time such a call takes
and having a healthcheck announcing such a logic not to be performing well
would be a good way to give developers the tooling to identify where they
lose performance in this area.

---
Dominik


On Wed, Oct 23, 2013 at 4:19 PM, Bertrand Delacretaz <bdelacretaz@apache.org
> wrote:

> Hi,
>
> On Wed, Oct 23, 2013 at 4:01 PM, Dominik Süß <do...@gmail.com>
> wrote:
> > ...This might not apply to the whole
> > tree, but does this really matter when the tree that needs to be watched
> > contains over 90% of the data?...
>
> What's important is the frequency of observation events - if that 90%
> seldom changes, scaling won't be a problem.
>
> > ...If I got the problem right the overhead is created by the fact that an
> > event object is created and sent regardles if a consumer cares about it
> or
> > not. So it might be worth to add something that "asks" the listeners if
> > there is one that would consume this event...
>
> That's what I meant above:
>
> > ...A simple way of making our wide observation more specific is to
> > require users of our rebroadcast OSGi events to indicate more
> > specifically what they're interested in...
>
> There's various ways of doing that: service properties, calling a
> service to register your interest in certain types of events etc.
>
> -Bertrand
>

Re: Scalabilty of EventListener

[Bertrand Delacretaz <bd...@apache.org>]

Hi,

On Wed, Oct 23, 2013 at 4:01 PM, Dominik Süß <do...@gmail.com> wrote:
> ...This might not apply to the whole
> tree, but does this really matter when the tree that needs to be watched
> contains over 90% of the data?...

What's important is the frequency of observation events - if that 90%
seldom changes, scaling won't be a problem.

> ...If I got the problem right the overhead is created by the fact that an
> event object is created and sent regardles if a consumer cares about it or
> not. So it might be worth to add something that "asks" the listeners if
> there is one that would consume this event...

That's what I meant above:

> ...A simple way of making our wide observation more specific is to
> require users of our rebroadcast OSGi events to indicate more
> specifically what they're interested in...

There's various ways of doing that: service properties, calling a
service to register your interest in certain types of events etc.

-Bertrand

Re: Scalabilty of EventListener

[Dominik Süß <do...@gmail.com>]

The analysis looks pretty good to me but does not provide answers of how to
solve this. The first two topics "Cached Content" and "Content Export,
Replication to Remote Systems" are the ones where I don't see an option to
get rid of the content change triggers. This might not apply to the whole
tree, but does this really matter when the tree that needs to be watched
contains over 90% of the data?

If I got the problem right the overhead is created by the fact that an
event object is created and sent regardles if a consumer cares about it or
not. So it might be worth to add something that "asks" the listeners if
there is one that would consume this event and (something like an
"accepts()" Method handing over raw metadata without generating the event
object itself - more like a filter) and send the event then to be consumed
by this and potential other consumers.

Or did i get something completely wrong?

Cheers
Dominik


On Wed, Oct 23, 2013 at 2:30 PM, Bertrand Delacretaz <bdelacretaz@apache.org
> wrote:

> On Wed, Oct 23, 2013 at 2:10 PM, Carsten Ziegeler <cz...@apache.org>
> wrote:
> > ....So far, I heard that the global listener which translates jcr events
> into
> > OSGi events is a problem - however as far as I followed the discussion,
> > this isn't a problem right now but might lead to problems in some time if
> > huge Oak based clustered repositories are used...
>
> Agreed.
>
> > ...In the same category fall the global listeners we have in the resource
> > resolver for the mapping and the i18n one..
> > Is there more?...
>
> I looked at an analysis the we did earlier this year on our
> Sling-based app and I don't have more than that.
>
> -Bertrand
>

Re: Scalabilty of EventListener

[Bertrand Delacretaz <bd...@apache.org>]

On Wed, Oct 23, 2013 at 2:10 PM, Carsten Ziegeler <cz...@apache.org> wrote:
> ....So far, I heard that the global listener which translates jcr events into
> OSGi events is a problem - however as far as I followed the discussion,
> this isn't a problem right now but might lead to problems in some time if
> huge Oak based clustered repositories are used...

Agreed.

> ...In the same category fall the global listeners we have in the resource
> resolver for the mapping and the i18n one..
> Is there more?...

I looked at an analysis the we did earlier this year on our
Sling-based app and I don't have more than that.

-Bertrand

Re: Scalabilty of EventListener

[Carsten Ziegeler <cz...@apache.org>]

Ok, so I suggest we identify these parts then first concretely before we
speculate that there is a problem and replace it with something where we
don't know the real impact.

So far, I heard that the global listener which translates jcr events into
OSGi events is a problem - however as far as I followed the discussion,
this isn't a problem right now but might lead to problems in some time if
huge Oak based clustered repositories are used. But we don't have concrete
numbers yet, so we could only fix this in a theoretical level.
In the same category fall the global listeners we have in the resource
resolver for the mapping and the i18n one.

Is there more?

Carsten
-- 
Carsten Ziegeler
cziegeler@apache.org

Re: Scalabilty of EventListener

[Bertrand Delacretaz <bd...@apache.org>]

On Wed, Oct 23, 2013 at 1:37 PM, Carsten Ziegeler <cz...@apache.org> wrote:
> ...I get this with global observation, but we're now talking about observation
> usage patterns and replacing them with queries. And these usage patterns
> are usually only observing partial parts of the repository....

I'm not saying we need to get rid of all observation, but to make sure
Sling itself does not limit scalability we might need to change parts
of our code to use alternatives, or at least make alternatives
possible.

-Bertrand

Re: Scalabilty of EventListener

[Carsten Ziegeler <cz...@apache.org>]

I get this with global observation, but we're now talking about observation
usage patterns and replacing them with queries. And these usage patterns
are usually only observing partial parts of the repository.

Carsten


2013/10/23 Ian Boston <ie...@tfd.co.uk>

> On 23 October 2013 11:25, Carsten Ziegeler <cz...@apache.org> wrote:
> > So you mean instead of doing observation, doing a query periodically?
> > This would mean that we basically say, one of the main features of JCR,
> > observation, is not usable.
>
> Past experience says that global observation of all changes in a
> cluster is not usable, and is best replaced by application specific
> messaging over a channel designed to scale.
>
> JCR Observation works just fine in the same memory space but beyond
> that it is far too noisy for a repository performing write operations.
>
> Ian
>
>
> >
> > Carsten
> >
> >
> > 2013/10/23 Bertrand Delacretaz <bd...@apache.org>
> >
> >> Hi,
> >>
> >> On Tue, Oct 22, 2013 at 5:48 PM, Carsten Ziegeler <cziegeler@apache.org
> >
> >> wrote:
> >> > ...The mapping handler in the resource resolver is probably the most
> >> > interesting one as it changes for nodes with some well defined
> >> properties,
> >> > basically scanning the whole repository...
> >>
> >> This is one example where latency is not a problem, so periodic
> >> queries could be used instead of observation if that's more scalable.
> >> We're basically interested in whether anything changed in the results
> >> of a query since we last ran it, a pattern which might be optimized in
> >> Oak by taking advantage of the underlying MVCC storage.
> >>
> >> -Bertrand
> >>
> >
> >
> >
> > --
> > Carsten Ziegeler
> > cziegeler@apache.org
>



-- 
Carsten Ziegeler
cziegeler@apache.org

Re: Scalabilty of EventListener

[Ian Boston <ie...@tfd.co.uk>]

On 23 October 2013 11:25, Carsten Ziegeler <cz...@apache.org> wrote:
> So you mean instead of doing observation, doing a query periodically?
> This would mean that we basically say, one of the main features of JCR,
> observation, is not usable.

Past experience says that global observation of all changes in a
cluster is not usable, and is best replaced by application specific
messaging over a channel designed to scale.

JCR Observation works just fine in the same memory space but beyond
that it is far too noisy for a repository performing write operations.

Ian


>
> Carsten
>
>
> 2013/10/23 Bertrand Delacretaz <bd...@apache.org>
>
>> Hi,
>>
>> On Tue, Oct 22, 2013 at 5:48 PM, Carsten Ziegeler <cz...@apache.org>
>> wrote:
>> > ...The mapping handler in the resource resolver is probably the most
>> > interesting one as it changes for nodes with some well defined
>> properties,
>> > basically scanning the whole repository...
>>
>> This is one example where latency is not a problem, so periodic
>> queries could be used instead of observation if that's more scalable.
>> We're basically interested in whether anything changed in the results
>> of a query since we last ran it, a pattern which might be optimized in
>> Oak by taking advantage of the underlying MVCC storage.
>>
>> -Bertrand
>>
>
>
>
> --
> Carsten Ziegeler
> cziegeler@apache.org

Re: Scalabilty of EventListener

[Carsten Ziegeler <cz...@apache.org>]

So you mean instead of doing observation, doing a query periodically?
This would mean that we basically say, one of the main features of JCR,
observation, is not usable.

Carsten


2013/10/23 Bertrand Delacretaz <bd...@apache.org>

> Hi,
>
> On Tue, Oct 22, 2013 at 5:48 PM, Carsten Ziegeler <cz...@apache.org>
> wrote:
> > ...The mapping handler in the resource resolver is probably the most
> > interesting one as it changes for nodes with some well defined
> properties,
> > basically scanning the whole repository...
>
> This is one example where latency is not a problem, so periodic
> queries could be used instead of observation if that's more scalable.
> We're basically interested in whether anything changed in the results
> of a query since we last ran it, a pattern which might be optimized in
> Oak by taking advantage of the underlying MVCC storage.
>
> -Bertrand
>



-- 
Carsten Ziegeler
cziegeler@apache.org

Re: Scalabilty of EventListener

[Bertrand Delacretaz <bd...@apache.org>]

Hi,

On Tue, Oct 22, 2013 at 5:48 PM, Carsten Ziegeler <cz...@apache.org> wrote:
> ...The mapping handler in the resource resolver is probably the most
> interesting one as it changes for nodes with some well defined properties,
> basically scanning the whole repository...

This is one example where latency is not a problem, so periodic
queries could be used instead of observation if that's more scalable.
We're basically interested in whether anything changed in the results
of a query since we last ran it, a pattern which might be optimized in
Oak by taking advantage of the underlying MVCC storage.

-Bertrand

Re: Scalabilty of EventListener

[Bertrand Delacretaz <bd...@apache.org>]

Hi,

On Wed, Oct 23, 2013 at 11:30 AM, Dominik Süß <do...@gmail.com> wrote:
> ...I have at least 2 cases in mind where the paths cannot be
> determined at deployment time...

We studied observation usage patterns [1] earlier this year on a few
Sling-based apps and it turns out that a lot of that can be done
without observation, which might help scalability.

So I'd say let's concentrate on Sling itself, and provide a way to be
more specific in how observation is used, for the cases where that
matters. And/or replace observation with other mechanisms where
appropriate.

A simple way of making our wide observation more specific is to
require users of our rebroadcast OSGi events to indicate more
specifically what they're interested in. This can be enabled by a
configuration switch, so that small apps that don't care require no
changes, and larger systems that require scalability can turn on that
switch and have to do a bit of work to provide more details. Large
scalable systems require work anyway, so this is not too much to ask
for IMO.

-Bertrand

[1] https://cwiki.apache.org/confluence/display/SLING/Observation+usage+patterns

Re: Scalabilty of EventListener

[Dominik Süß <do...@gmail.com>]

I just realized that there might be the need to dynamicaly add paths to the
Listener. I have at least 2 cases in mind where the paths cannot be
determined at deployment time which is the workflow launcher in CQ (where a
user does define a ruleset at runtime) which might be changed to individual
listeners being registerd, and the other case is when you have the
temporary need to watch for changes of specific nodes (e.g. when a cache
needs to be invalidated based on contentchanges - so the listener just
needs to react on changes of resources relevant to the invalidation logic).

During writing that I just came up with the question, what about such
global listeners as the replication agent of CQ which potentially nees to
act on all paths. The restrictions I have in mind is the path /content and
that it would be sufficient to have one aggregated event for a page
(substructure containing all resource to be used for rendering one
request)  which needs to be touched on subcontentchanges anyway
(lastmodified).

--
Dominik


On Tue, Oct 22, 2013 at 5:48 PM, Carsten Ziegeler <cz...@apache.org>wrote:

> From the use cases I know, the listeners register for a specific path and
> are rarely interested in anything else. Some do check the resource type as
> well.
> For example the script engines check for changes under /libs and /apps (the
> resource paths actually) for changes to scripts, the job engine checks for
> new jobs somewhere under /var/jobs and checks the resource type as well
> etc.
> Many use cases check for changes (resource added, resource removed,
> resource changed) and then re-read the sub tree based on the changes.
>
> The mapping handler in the resource resolver is probably the most
> interesting one as it changes for nodes with some well defined properties,
> basically scanning the whole repository. And I think the i18n stuff does
> something similar (but this one might still be using jcr observation).
>
> This list is by no means exhaustive for Sling, but we see that we already
> have two listeners scanning the whole repository and require access to
> properties.
>
> Carsten
>
>
> 2013/10/22 Dominik Süß <do...@gmail.com>
>
> > In a discussion [0] within the Oak mailinglist it became clear that the
> way
> > Sling listens zu JCR Repository Changes and transforms all of them to
> > events will not scale well in some big scale scenarios that oak is aiming
> > to enable. Therefore the question was posted if it would be feasible
> and/or
> > even necessary to refactor the API and deprecate (or at least discurrage)
> > registration in a global scope as currently done.
> >
> > Since I do not want to copy & paste parts of the discussion I do hope
> that
> > the participants of the oak discussion add the remaining options with
> some
> > more detail about consequences than can be found in the linked
> discussion.
> >
> > It would be great to also get some feedbacks of consumers of the existing
> > API about the usage to identify how finegrained a potential
> > registrationlogic with paths/properties might need to be.
> >
> > Best regards
> > Dominik
> >
> > [0] http://markmail.org/message/n5vllhjoawypteck
> >
>
>
>
> --
> Carsten Ziegeler
> cziegeler@apache.org
>

Re: Scalabilty of EventListener

[Carsten Ziegeler <cz...@apache.org>]

>From the use cases I know, the listeners register for a specific path and
are rarely interested in anything else. Some do check the resource type as
well.
For example the script engines check for changes under /libs and /apps (the
resource paths actually) for changes to scripts, the job engine checks for
new jobs somewhere under /var/jobs and checks the resource type as well etc.
Many use cases check for changes (resource added, resource removed,
resource changed) and then re-read the sub tree based on the changes.

The mapping handler in the resource resolver is probably the most
interesting one as it changes for nodes with some well defined properties,
basically scanning the whole repository. And I think the i18n stuff does
something similar (but this one might still be using jcr observation).

This list is by no means exhaustive for Sling, but we see that we already
have two listeners scanning the whole repository and require access to
properties.

Carsten


2013/10/22 Dominik Süß <do...@gmail.com>

> In a discussion [0] within the Oak mailinglist it became clear that the way
> Sling listens zu JCR Repository Changes and transforms all of them to
> events will not scale well in some big scale scenarios that oak is aiming
> to enable. Therefore the question was posted if it would be feasible and/or
> even necessary to refactor the API and deprecate (or at least discurrage)
> registration in a global scope as currently done.
>
> Since I do not want to copy & paste parts of the discussion I do hope that
> the participants of the oak discussion add the remaining options with some
> more detail about consequences than can be found in the linked discussion.
>
> It would be great to also get some feedbacks of consumers of the existing
> API about the usage to identify how finegrained a potential
> registrationlogic with paths/properties might need to be.
>
> Best regards
> Dominik
>
> [0] http://markmail.org/message/n5vllhjoawypteck
>



-- 
Carsten Ziegeler
cziegeler@apache.org