You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sling.apache.org by Roy Teeuwen <ro...@teeuwen.be> on 2018/12/15 17:39:26 UTC
Re: [DISCUSS] Deprecate query part of Sling Event API
Hey guys,
Seeing as you are all talking about deprecating or even removing the entire Sling Job API in the future, what would you propose as alternative / migration path? We have a lot JobConsumers in our projects, what would be a reason to stop using them?
Greets,
Roy
> On 30 Oct 2018, at 11:27, Stefan Egli <st...@apache.org> wrote:
>
> On 30.10.18 10:22, Bertrand Delacretaz wrote:
>> Hi,
>> I agree, and we might use the following trick to avoid that with
>> minimal disruption:
>> -Consider the current interface "Legacy" that needs half of its
>> methods deprecated
>> -Move the non-deprecated methods to a new "Modern" interface
>> -Legacy inherits from Modern
>> Existing clients can then use Legacy with no change.
>> New clients move to Modern, to make it clear that they don't want to
>> use legacy stuff.
>> Mark the whole Legacy interface as deprecated, recommend moving to Modern
>> Existing services are explicitly registered as both Modern and Legacy.
>> No need to redesign things, just move them around.
>> WDYT?
>
> +1, which would bring us to naming.. one suggestion would be
>
> /** @deprecated */
> public interface JobManager extends JobManager2 /* Modern */ {
> ..
> }
>
> Cheers,
> Stefan
Re: [DISCUSS] Deprecate query part of Sling Event API
Posted by Stefan Egli <st...@apache.org>.
On 09.01.19 15:38, Jason E Bailey wrote:
> On Wed, Jan 9, 2019, at 3:05 AM, Stefan Egli wrote:
>> (a) having to support job queries makes messaging-based implementations
>> hard. You typically don't want to do queries against a messaging system.
>> So this is not only about the default resource-based implementation but
>> allowing alternatives better.
>
> I don't support your logic. It's a poor design to perform queries via a messaging system, but you absolutely need to be able to query 'that which is doing the processing' as to the state of the jobs.
Right, some implementation might need that, but why do you absolutely
need to expose that state in the API.
> This is almost moot with a resource based implementation because it's really easy to find that information out by looking at the resource tree. However if the goal is to implement it in another manner then you'll need a consistent interface to access that data.
>
> If you're talking about doing a PDMS and you're looking at the Sling Event API as the model for message generation across distinct micro service implementations. I do agree that the API for the queries should be pulled out to a separate API, that makes sense since your doing a CQRS pattern. But you're replacing the methods not eliminating them.
What would you suggest replacing the methods with?
> The problem is that your using oak queries to process data that is in a specific location. That's not a good practice. Doing a JCR query to find disparate resources is useful when you have a large set. When you already know where the set of data is and your effectively using a query to filter through the nodes. You end up killing yourself.
>
> I can modify the code this week to be as performant as you are currently getting with an index, without using an index, with less memory , eliminating the whole concern of maintaining indexes.
I don't know the full history of why the index was introduced - I'm
guessing it was considered an improvement to traversing the tree
manually for doing rescans..
Cheers,
Stefan
Re: [DISCUSS] Deprecate query part of Sling Event API
Posted by Jason E Bailey <ja...@24601.org>.
On Wed, Jan 9, 2019, at 3:05 AM, Stefan Egli wrote:
> The problem is two fold:
>
> (a) having to support job queries makes messaging-based implementations
> hard. You typically don't want to do queries against a messaging system.
> So this is not only about the default resource-based implementation but
> allowing alternatives better.
I don't support your logic. It's a poor design to perform queries via a messaging system, but you absolutely need to be able to query 'that which is doing the processing' as to the state of the jobs. This is almost moot with a resource based implementation because it's really easy to find that information out by looking at the resource tree. However if the goal is to implement it in another manner then you'll need a consistent interface to access that data.
If you're talking about doing a PDMS and you're looking at the Sling Event API as the model for message generation across distinct micro service implementations. I do agree that the API for the queries should be pulled out to a separate API, that makes sense since your doing a CQRS pattern. But you're replacing the methods not eliminating them.
> (b) in order to support queries in the existing resource-based
> implementation we're using indexes already. So the query itself is not
> expensive - but maintaining the index is a cost which you don't need to
> do if there wouldn't be any query to support in the first place.
The problem is that your using oak queries to process data that is in a specific location. That's not a good practice. Doing a JCR query to find disparate resources is useful when you have a large set. When you already know where the set of data is and your effectively using a query to filter through the nodes. You end up killing yourself.
I can modify the code this week to be as performant as you are currently getting with an index, without using an index, with less memory , eliminating the whole concern of maintaining indexes.
Re: [DISCUSS] Deprecate query part of Sling Event API
Posted by Stefan Egli <st...@apache.org>.
On 09.01.19 16:41, Daniel Klco wrote:
> Because I was looking at the Sling 7 JavaDoc :-) Strangely the Job
> interface doesn't seem to be present in the newer JavaDocs.
That must be a mistake. Perhaps related to separating the API from the
implementation bundle?
Cheers,
Stefan
Re: [DISCUSS] Deprecate query part of Sling Event API
Posted by Daniel Klco <da...@gmail.com>.
Because I was looking at the Sling 7 JavaDoc :-) Strangely the Job
interface doesn't seem to be present in the newer JavaDocs.
On Wed, Jan 9, 2019 at 10:31 AM Stefan Egli <st...@apache.org> wrote:
> On 09.01.19 15:01, Daniel Klco wrote:
> > I would argue though that there is a significant loss of functionality by
> > eliminating the ability to see what jobs exist on an instance.
>
> Agree, that's why I suggested B over A.
>
> > What if
> > rather than eliminating the functionality entirely a new interface was
> > created for a "LocalJobManager" and the query based functionality moved
> > there? This interface would only be supported for the resource-based
> > implementation and would allow for querying only the local instance (e.g.
> > it may not accurately reflect distributed jobs).
>
> If the LocalJobManager interface becomes optional, then what is a user
> of that interface to do when it is not available (eg in an alternative
> event implementation)?
>
> > On another topic, what does everyone think of adding a method into the
> Job
> > interface for getting the entire ValueMap / java.util.Map of properties?
> > Right now it's very cumbersome to work with jobs via Expression Language.
>
> Not sure I understand - why would Job.getProperties() not suffice?
>
> Cheers,
> Stefan
>
> >
> > On Wed, Jan 9, 2019 at 3:05 AM Stefan Egli <st...@apache.org>
> wrote:
> >
> >> The problem is two fold:
> >>
> >> (a) having to support job queries makes messaging-based implementations
> >> hard. You typically don't want to do queries against a messaging system.
> >> So this is not only about the default resource-based implementation but
> >> allowing alternatives better.
> >>
> >> (b) in order to support queries in the existing resource-based
> >> implementation we're using indexes already. So the query itself is not
> >> expensive - but maintaining the index is a cost which you don't need to
> >> do if there wouldn't be any query to support in the first place.
> >>
> >> So it's two different aspects which both would benefit if there were no
> >> queries to support.
> >>
> >> With variant A we'd get rid of them long term, with variant B we're
> >> selectively getting rid of them on a case-by-case basis.
> >>
> >> Cheers,
> >> Stefan
> >>
> >> On 09.01.19 01:32, Daniel Klco wrote:
> >>> Yeah I must admit I'm confused as well, looking through the email
> threads
> >>> the original proposal is to return and a iterator rather than a
> >> collection.
> >>> If the problem is that too many jobs being returned causes heap
> issues, I
> >>> would think this would resolve that problem. Is there some other
> problem
> >>> here? If it is in query performance, could it possibly be solved using
> >> oak
> >>> indexes?
> >>>
> >>> On Tue, Jan 8, 2019, 3:50 PM Jason E Bailey <jason.bailey@24601.org
> >> wrote:
> >>>
> >>>> So I'm missing something here, from what I gather so far, the methods
> >> that
> >>>> are being deprecated are to prevent a performance issue but the node
> >>>> structure that represents the jobs is in one place isn't it? How in
> the
> >>>> world are we getting a performance issue from that?
> >>>>
> >>>> Is there a use case to demonstrate the performance problem?
> >>>>
> >>>> --
> >>>> Jason
> >>>>
> >>>> On Tue, Jan 8, 2019, at 11:12 AM, Carsten Ziegeler wrote:
> >>>>> Hi,
> >>>>>
> >>>>> I agree, variant B sounds like the better option due to the reasons
> you
> >>>>> mention. It also provides a step by step way of moving a single type
> of
> >>>>> jobs to not use search anymore.
> >>>>>
> >>>>> The only downside is that this doesn't force anyone to think about
> her
> >>>>> job usage as everything just runs as before. Only if you want to
> >> improve
> >>>>> you can and adjust the configurations.
> >>>>>
> >>>>>
> >>>>> Regards
> >>>>>
> >>>>> Carsten
> >>>>>
> >>>>>
> >>>>> Stefan Egli wrote
> >>>>>> Hi,
> >>>>>>
> >>>>>> I've given this job query issue some more thought and would like us
> to
> >>>>>> discuss and hopefully soon agree on which way we go:
> >>>>>>
> >>>>>>
> >>>>>> Variant A: deprecate job query methods (as suggested originally):
> >>>>>> * upside: eventually we'll have a cleaner job API
> >>>>>> * downside: users of the job query API need to find alternatives to
> >>>>>> queries, individually
> >>>>>>
> >>>>>>
> >>>>>> Variant B: allow disabling job queries (https://s.apache.org/68ys):
> >>>>>> * upside: allows performance optimizations on a case-by-case basis
> >>>>>> (optimizations include not requiring indexing), thereby promoting
> >>>>>> query-less use cases going forward without API deprecation
> >>>> nevertheless.
> >>>>>> * downside: we stick to the current job API
> >>>>>>
> >>>>>>
> >>>>>> Note that in both cases the 'org.apache.sling.event.jobs' export
> >>>> version
> >>>>>> will have to be updated from 2.0.1 to 2.1.0 - which will affect
> >>>>>> users/customers that implement interfaces from that package (that's
> >>>> not
> >>>>>> something typical though, but there are a few exotic cases where
> >>>> that's
> >>>>>> the case).
> >>>>>>
> >>>>>>
> >>>>>> At this stage I'm in favour of variant B as I don't see a good
> >>>>>> alternative for job queries other than users re-implementing a fair
> >>>>>> chunk of the sling event implementation themselves (which would sort
> >>>> of
> >>>>>> defeat the purpose, besides the fact that it would not be trivial as
> >>>>>> there aren't enough hooks in the API for anyone other than the
> >>>>>> implementor to do this consistently).
> >>>>>>
> >>>>>>
> >>>>>> Cheers,
> >>>>>> Stefan
> >>>>>>
> >>>>>> On 17.12.18 16:08, Stefan Egli wrote:
> >>>>>>> We could introduce a config option which would allow a job queue to
> >>>>>>> opt-out of job queries voluntarily. That would allow the
> >>>>>>> implementation to treat those jobs differently (cheaper, without
> >>>>>>> indexing them).
> >>>>>>>
> >>>>>>> I guess such a new opt-out of queries mechanism could be less
> >>>>>>> controversial and provide at least some short-term gain. It doesn't
> >>>>>>> answer the question how job queries should be replaced though..
> >>>>>>
> >>>>> --
> >>>>> Carsten Ziegeler
> >>>>> Adobe Research Switzerland
> >>>>> cziegeler@apache.org
> >>>>>
> >>>>
> >>>
> >>
> >
>
Re: [DISCUSS] Deprecate query part of Sling Event API
Posted by Stefan Egli <st...@apache.org>.
On 09.01.19 15:01, Daniel Klco wrote:
> I would argue though that there is a significant loss of functionality by
> eliminating the ability to see what jobs exist on an instance.
Agree, that's why I suggested B over A.
> What if
> rather than eliminating the functionality entirely a new interface was
> created for a "LocalJobManager" and the query based functionality moved
> there? This interface would only be supported for the resource-based
> implementation and would allow for querying only the local instance (e.g.
> it may not accurately reflect distributed jobs).
If the LocalJobManager interface becomes optional, then what is a user
of that interface to do when it is not available (eg in an alternative
event implementation)?
> On another topic, what does everyone think of adding a method into the Job
> interface for getting the entire ValueMap / java.util.Map of properties?
> Right now it's very cumbersome to work with jobs via Expression Language.
Not sure I understand - why would Job.getProperties() not suffice?
Cheers,
Stefan
>
> On Wed, Jan 9, 2019 at 3:05 AM Stefan Egli <st...@apache.org> wrote:
>
>> The problem is two fold:
>>
>> (a) having to support job queries makes messaging-based implementations
>> hard. You typically don't want to do queries against a messaging system.
>> So this is not only about the default resource-based implementation but
>> allowing alternatives better.
>>
>> (b) in order to support queries in the existing resource-based
>> implementation we're using indexes already. So the query itself is not
>> expensive - but maintaining the index is a cost which you don't need to
>> do if there wouldn't be any query to support in the first place.
>>
>> So it's two different aspects which both would benefit if there were no
>> queries to support.
>>
>> With variant A we'd get rid of them long term, with variant B we're
>> selectively getting rid of them on a case-by-case basis.
>>
>> Cheers,
>> Stefan
>>
>> On 09.01.19 01:32, Daniel Klco wrote:
>>> Yeah I must admit I'm confused as well, looking through the email threads
>>> the original proposal is to return and a iterator rather than a
>> collection.
>>> If the problem is that too many jobs being returned causes heap issues, I
>>> would think this would resolve that problem. Is there some other problem
>>> here? If it is in query performance, could it possibly be solved using
>> oak
>>> indexes?
>>>
>>> On Tue, Jan 8, 2019, 3:50 PM Jason E Bailey <jason.bailey@24601.org
>> wrote:
>>>
>>>> So I'm missing something here, from what I gather so far, the methods
>> that
>>>> are being deprecated are to prevent a performance issue but the node
>>>> structure that represents the jobs is in one place isn't it? How in the
>>>> world are we getting a performance issue from that?
>>>>
>>>> Is there a use case to demonstrate the performance problem?
>>>>
>>>> --
>>>> Jason
>>>>
>>>> On Tue, Jan 8, 2019, at 11:12 AM, Carsten Ziegeler wrote:
>>>>> Hi,
>>>>>
>>>>> I agree, variant B sounds like the better option due to the reasons you
>>>>> mention. It also provides a step by step way of moving a single type of
>>>>> jobs to not use search anymore.
>>>>>
>>>>> The only downside is that this doesn't force anyone to think about her
>>>>> job usage as everything just runs as before. Only if you want to
>> improve
>>>>> you can and adjust the configurations.
>>>>>
>>>>>
>>>>> Regards
>>>>>
>>>>> Carsten
>>>>>
>>>>>
>>>>> Stefan Egli wrote
>>>>>> Hi,
>>>>>>
>>>>>> I've given this job query issue some more thought and would like us to
>>>>>> discuss and hopefully soon agree on which way we go:
>>>>>>
>>>>>>
>>>>>> Variant A: deprecate job query methods (as suggested originally):
>>>>>> * upside: eventually we'll have a cleaner job API
>>>>>> * downside: users of the job query API need to find alternatives to
>>>>>> queries, individually
>>>>>>
>>>>>>
>>>>>> Variant B: allow disabling job queries (https://s.apache.org/68ys):
>>>>>> * upside: allows performance optimizations on a case-by-case basis
>>>>>> (optimizations include not requiring indexing), thereby promoting
>>>>>> query-less use cases going forward without API deprecation
>>>> nevertheless.
>>>>>> * downside: we stick to the current job API
>>>>>>
>>>>>>
>>>>>> Note that in both cases the 'org.apache.sling.event.jobs' export
>>>> version
>>>>>> will have to be updated from 2.0.1 to 2.1.0 - which will affect
>>>>>> users/customers that implement interfaces from that package (that's
>>>> not
>>>>>> something typical though, but there are a few exotic cases where
>>>> that's
>>>>>> the case).
>>>>>>
>>>>>>
>>>>>> At this stage I'm in favour of variant B as I don't see a good
>>>>>> alternative for job queries other than users re-implementing a fair
>>>>>> chunk of the sling event implementation themselves (which would sort
>>>> of
>>>>>> defeat the purpose, besides the fact that it would not be trivial as
>>>>>> there aren't enough hooks in the API for anyone other than the
>>>>>> implementor to do this consistently).
>>>>>>
>>>>>>
>>>>>> Cheers,
>>>>>> Stefan
>>>>>>
>>>>>> On 17.12.18 16:08, Stefan Egli wrote:
>>>>>>> We could introduce a config option which would allow a job queue to
>>>>>>> opt-out of job queries voluntarily. That would allow the
>>>>>>> implementation to treat those jobs differently (cheaper, without
>>>>>>> indexing them).
>>>>>>>
>>>>>>> I guess such a new opt-out of queries mechanism could be less
>>>>>>> controversial and provide at least some short-term gain. It doesn't
>>>>>>> answer the question how job queries should be replaced though..
>>>>>>
>>>>> --
>>>>> Carsten Ziegeler
>>>>> Adobe Research Switzerland
>>>>> cziegeler@apache.org
>>>>>
>>>>
>>>
>>
>
Re: [DISCUSS] Deprecate query part of Sling Event API
Posted by Daniel Klco <da...@gmail.com>.
I would argue though that there is a significant loss of functionality by
eliminating the ability to see what jobs exist on an instance. What if
rather than eliminating the functionality entirely a new interface was
created for a "LocalJobManager" and the query based functionality moved
there? This interface would only be supported for the resource-based
implementation and would allow for querying only the local instance (e.g.
it may not accurately reflect distributed jobs).
On another topic, what does everyone think of adding a method into the Job
interface for getting the entire ValueMap / java.util.Map of properties?
Right now it's very cumbersome to work with jobs via Expression Language.
On Wed, Jan 9, 2019 at 3:05 AM Stefan Egli <st...@apache.org> wrote:
> The problem is two fold:
>
> (a) having to support job queries makes messaging-based implementations
> hard. You typically don't want to do queries against a messaging system.
> So this is not only about the default resource-based implementation but
> allowing alternatives better.
>
> (b) in order to support queries in the existing resource-based
> implementation we're using indexes already. So the query itself is not
> expensive - but maintaining the index is a cost which you don't need to
> do if there wouldn't be any query to support in the first place.
>
> So it's two different aspects which both would benefit if there were no
> queries to support.
>
> With variant A we'd get rid of them long term, with variant B we're
> selectively getting rid of them on a case-by-case basis.
>
> Cheers,
> Stefan
>
> On 09.01.19 01:32, Daniel Klco wrote:
> > Yeah I must admit I'm confused as well, looking through the email threads
> > the original proposal is to return and a iterator rather than a
> collection.
> > If the problem is that too many jobs being returned causes heap issues, I
> > would think this would resolve that problem. Is there some other problem
> > here? If it is in query performance, could it possibly be solved using
> oak
> > indexes?
> >
> > On Tue, Jan 8, 2019, 3:50 PM Jason E Bailey <jason.bailey@24601.org
> wrote:
> >
> >> So I'm missing something here, from what I gather so far, the methods
> that
> >> are being deprecated are to prevent a performance issue but the node
> >> structure that represents the jobs is in one place isn't it? How in the
> >> world are we getting a performance issue from that?
> >>
> >> Is there a use case to demonstrate the performance problem?
> >>
> >> --
> >> Jason
> >>
> >> On Tue, Jan 8, 2019, at 11:12 AM, Carsten Ziegeler wrote:
> >>> Hi,
> >>>
> >>> I agree, variant B sounds like the better option due to the reasons you
> >>> mention. It also provides a step by step way of moving a single type of
> >>> jobs to not use search anymore.
> >>>
> >>> The only downside is that this doesn't force anyone to think about her
> >>> job usage as everything just runs as before. Only if you want to
> improve
> >>> you can and adjust the configurations.
> >>>
> >>>
> >>> Regards
> >>>
> >>> Carsten
> >>>
> >>>
> >>> Stefan Egli wrote
> >>>> Hi,
> >>>>
> >>>> I've given this job query issue some more thought and would like us to
> >>>> discuss and hopefully soon agree on which way we go:
> >>>>
> >>>>
> >>>> Variant A: deprecate job query methods (as suggested originally):
> >>>> * upside: eventually we'll have a cleaner job API
> >>>> * downside: users of the job query API need to find alternatives to
> >>>> queries, individually
> >>>>
> >>>>
> >>>> Variant B: allow disabling job queries (https://s.apache.org/68ys):
> >>>> * upside: allows performance optimizations on a case-by-case basis
> >>>> (optimizations include not requiring indexing), thereby promoting
> >>>> query-less use cases going forward without API deprecation
> >> nevertheless.
> >>>> * downside: we stick to the current job API
> >>>>
> >>>>
> >>>> Note that in both cases the 'org.apache.sling.event.jobs' export
> >> version
> >>>> will have to be updated from 2.0.1 to 2.1.0 - which will affect
> >>>> users/customers that implement interfaces from that package (that's
> >> not
> >>>> something typical though, but there are a few exotic cases where
> >> that's
> >>>> the case).
> >>>>
> >>>>
> >>>> At this stage I'm in favour of variant B as I don't see a good
> >>>> alternative for job queries other than users re-implementing a fair
> >>>> chunk of the sling event implementation themselves (which would sort
> >> of
> >>>> defeat the purpose, besides the fact that it would not be trivial as
> >>>> there aren't enough hooks in the API for anyone other than the
> >>>> implementor to do this consistently).
> >>>>
> >>>>
> >>>> Cheers,
> >>>> Stefan
> >>>>
> >>>> On 17.12.18 16:08, Stefan Egli wrote:
> >>>>> We could introduce a config option which would allow a job queue to
> >>>>> opt-out of job queries voluntarily. That would allow the
> >>>>> implementation to treat those jobs differently (cheaper, without
> >>>>> indexing them).
> >>>>>
> >>>>> I guess such a new opt-out of queries mechanism could be less
> >>>>> controversial and provide at least some short-term gain. It doesn't
> >>>>> answer the question how job queries should be replaced though..
> >>>>
> >>> --
> >>> Carsten Ziegeler
> >>> Adobe Research Switzerland
> >>> cziegeler@apache.org
> >>>
> >>
> >
>
Re: [DISCUSS] Deprecate query part of Sling Event API
Posted by Stefan Egli <st...@apache.org>.
The problem is two fold:
(a) having to support job queries makes messaging-based implementations
hard. You typically don't want to do queries against a messaging system.
So this is not only about the default resource-based implementation but
allowing alternatives better.
(b) in order to support queries in the existing resource-based
implementation we're using indexes already. So the query itself is not
expensive - but maintaining the index is a cost which you don't need to
do if there wouldn't be any query to support in the first place.
So it's two different aspects which both would benefit if there were no
queries to support.
With variant A we'd get rid of them long term, with variant B we're
selectively getting rid of them on a case-by-case basis.
Cheers,
Stefan
On 09.01.19 01:32, Daniel Klco wrote:
> Yeah I must admit I'm confused as well, looking through the email threads
> the original proposal is to return and a iterator rather than a collection.
> If the problem is that too many jobs being returned causes heap issues, I
> would think this would resolve that problem. Is there some other problem
> here? If it is in query performance, could it possibly be solved using oak
> indexes?
>
> On Tue, Jan 8, 2019, 3:50 PM Jason E Bailey <jason.bailey@24601.org wrote:
>
>> So I'm missing something here, from what I gather so far, the methods that
>> are being deprecated are to prevent a performance issue but the node
>> structure that represents the jobs is in one place isn't it? How in the
>> world are we getting a performance issue from that?
>>
>> Is there a use case to demonstrate the performance problem?
>>
>> --
>> Jason
>>
>> On Tue, Jan 8, 2019, at 11:12 AM, Carsten Ziegeler wrote:
>>> Hi,
>>>
>>> I agree, variant B sounds like the better option due to the reasons you
>>> mention. It also provides a step by step way of moving a single type of
>>> jobs to not use search anymore.
>>>
>>> The only downside is that this doesn't force anyone to think about her
>>> job usage as everything just runs as before. Only if you want to improve
>>> you can and adjust the configurations.
>>>
>>>
>>> Regards
>>>
>>> Carsten
>>>
>>>
>>> Stefan Egli wrote
>>>> Hi,
>>>>
>>>> I've given this job query issue some more thought and would like us to
>>>> discuss and hopefully soon agree on which way we go:
>>>>
>>>>
>>>> Variant A: deprecate job query methods (as suggested originally):
>>>> * upside: eventually we'll have a cleaner job API
>>>> * downside: users of the job query API need to find alternatives to
>>>> queries, individually
>>>>
>>>>
>>>> Variant B: allow disabling job queries (https://s.apache.org/68ys):
>>>> * upside: allows performance optimizations on a case-by-case basis
>>>> (optimizations include not requiring indexing), thereby promoting
>>>> query-less use cases going forward without API deprecation
>> nevertheless.
>>>> * downside: we stick to the current job API
>>>>
>>>>
>>>> Note that in both cases the 'org.apache.sling.event.jobs' export
>> version
>>>> will have to be updated from 2.0.1 to 2.1.0 - which will affect
>>>> users/customers that implement interfaces from that package (that's
>> not
>>>> something typical though, but there are a few exotic cases where
>> that's
>>>> the case).
>>>>
>>>>
>>>> At this stage I'm in favour of variant B as I don't see a good
>>>> alternative for job queries other than users re-implementing a fair
>>>> chunk of the sling event implementation themselves (which would sort
>> of
>>>> defeat the purpose, besides the fact that it would not be trivial as
>>>> there aren't enough hooks in the API for anyone other than the
>>>> implementor to do this consistently).
>>>>
>>>>
>>>> Cheers,
>>>> Stefan
>>>>
>>>> On 17.12.18 16:08, Stefan Egli wrote:
>>>>> We could introduce a config option which would allow a job queue to
>>>>> opt-out of job queries voluntarily. That would allow the
>>>>> implementation to treat those jobs differently (cheaper, without
>>>>> indexing them).
>>>>>
>>>>> I guess such a new opt-out of queries mechanism could be less
>>>>> controversial and provide at least some short-term gain. It doesn't
>>>>> answer the question how job queries should be replaced though..
>>>>
>>> --
>>> Carsten Ziegeler
>>> Adobe Research Switzerland
>>> cziegeler@apache.org
>>>
>>
>
Re: [DISCUSS] Deprecate query part of Sling Event API
Posted by Daniel Klco <da...@gmail.com>.
Yeah I must admit I'm confused as well, looking through the email threads
the original proposal is to return and a iterator rather than a collection.
If the problem is that too many jobs being returned causes heap issues, I
would think this would resolve that problem. Is there some other problem
here? If it is in query performance, could it possibly be solved using oak
indexes?
On Tue, Jan 8, 2019, 3:50 PM Jason E Bailey <jason.bailey@24601.org wrote:
> So I'm missing something here, from what I gather so far, the methods that
> are being deprecated are to prevent a performance issue but the node
> structure that represents the jobs is in one place isn't it? How in the
> world are we getting a performance issue from that?
>
> Is there a use case to demonstrate the performance problem?
>
> --
> Jason
>
> On Tue, Jan 8, 2019, at 11:12 AM, Carsten Ziegeler wrote:
> > Hi,
> >
> > I agree, variant B sounds like the better option due to the reasons you
> > mention. It also provides a step by step way of moving a single type of
> > jobs to not use search anymore.
> >
> > The only downside is that this doesn't force anyone to think about her
> > job usage as everything just runs as before. Only if you want to improve
> > you can and adjust the configurations.
> >
> >
> > Regards
> >
> > Carsten
> >
> >
> > Stefan Egli wrote
> > > Hi,
> > >
> > > I've given this job query issue some more thought and would like us to
> > > discuss and hopefully soon agree on which way we go:
> > >
> > >
> > > Variant A: deprecate job query methods (as suggested originally):
> > > * upside: eventually we'll have a cleaner job API
> > > * downside: users of the job query API need to find alternatives to
> > > queries, individually
> > >
> > >
> > > Variant B: allow disabling job queries (https://s.apache.org/68ys):
> > > * upside: allows performance optimizations on a case-by-case basis
> > > (optimizations include not requiring indexing), thereby promoting
> > > query-less use cases going forward without API deprecation
> nevertheless.
> > > * downside: we stick to the current job API
> > >
> > >
> > > Note that in both cases the 'org.apache.sling.event.jobs' export
> version
> > > will have to be updated from 2.0.1 to 2.1.0 - which will affect
> > > users/customers that implement interfaces from that package (that's
> not
> > > something typical though, but there are a few exotic cases where
> that's
> > > the case).
> > >
> > >
> > > At this stage I'm in favour of variant B as I don't see a good
> > > alternative for job queries other than users re-implementing a fair
> > > chunk of the sling event implementation themselves (which would sort
> of
> > > defeat the purpose, besides the fact that it would not be trivial as
> > > there aren't enough hooks in the API for anyone other than the
> > > implementor to do this consistently).
> > >
> > >
> > > Cheers,
> > > Stefan
> > >
> > > On 17.12.18 16:08, Stefan Egli wrote:
> > >> We could introduce a config option which would allow a job queue to
> > >> opt-out of job queries voluntarily. That would allow the
> > >> implementation to treat those jobs differently (cheaper, without
> > >> indexing them).
> > >>
> > >> I guess such a new opt-out of queries mechanism could be less
> > >> controversial and provide at least some short-term gain. It doesn't
> > >> answer the question how job queries should be replaced though..
> > >
> > --
> > Carsten Ziegeler
> > Adobe Research Switzerland
> > cziegeler@apache.org
> >
>
Re: [DISCUSS] Deprecate query part of Sling Event API
Posted by Jason E Bailey <ja...@24601.org>.
So I'm missing something here, from what I gather so far, the methods that are being deprecated are to prevent a performance issue but the node structure that represents the jobs is in one place isn't it? How in the world are we getting a performance issue from that?
Is there a use case to demonstrate the performance problem?
--
Jason
On Tue, Jan 8, 2019, at 11:12 AM, Carsten Ziegeler wrote:
> Hi,
>
> I agree, variant B sounds like the better option due to the reasons you
> mention. It also provides a step by step way of moving a single type of
> jobs to not use search anymore.
>
> The only downside is that this doesn't force anyone to think about her
> job usage as everything just runs as before. Only if you want to improve
> you can and adjust the configurations.
>
>
> Regards
>
> Carsten
>
>
> Stefan Egli wrote
> > Hi,
> >
> > I've given this job query issue some more thought and would like us to
> > discuss and hopefully soon agree on which way we go:
> >
> >
> > Variant A: deprecate job query methods (as suggested originally):
> > * upside: eventually we'll have a cleaner job API
> > * downside: users of the job query API need to find alternatives to
> > queries, individually
> >
> >
> > Variant B: allow disabling job queries (https://s.apache.org/68ys):
> > * upside: allows performance optimizations on a case-by-case basis
> > (optimizations include not requiring indexing), thereby promoting
> > query-less use cases going forward without API deprecation nevertheless.
> > * downside: we stick to the current job API
> >
> >
> > Note that in both cases the 'org.apache.sling.event.jobs' export version
> > will have to be updated from 2.0.1 to 2.1.0 - which will affect
> > users/customers that implement interfaces from that package (that's not
> > something typical though, but there are a few exotic cases where that's
> > the case).
> >
> >
> > At this stage I'm in favour of variant B as I don't see a good
> > alternative for job queries other than users re-implementing a fair
> > chunk of the sling event implementation themselves (which would sort of
> > defeat the purpose, besides the fact that it would not be trivial as
> > there aren't enough hooks in the API for anyone other than the
> > implementor to do this consistently).
> >
> >
> > Cheers,
> > Stefan
> >
> > On 17.12.18 16:08, Stefan Egli wrote:
> >> We could introduce a config option which would allow a job queue to
> >> opt-out of job queries voluntarily. That would allow the
> >> implementation to treat those jobs differently (cheaper, without
> >> indexing them).
> >>
> >> I guess such a new opt-out of queries mechanism could be less
> >> controversial and provide at least some short-term gain. It doesn't
> >> answer the question how job queries should be replaced though..
> >
> --
> Carsten Ziegeler
> Adobe Research Switzerland
> cziegeler@apache.org
>
Re: [DISCUSS] Deprecate query part of Sling Event API
Posted by Carsten Ziegeler <cz...@apache.org>.
Hi,
I agree, variant B sounds like the better option due to the reasons you
mention. It also provides a step by step way of moving a single type of
jobs to not use search anymore.
The only downside is that this doesn't force anyone to think about her
job usage as everything just runs as before. Only if you want to improve
you can and adjust the configurations.
Regards
Carsten
Stefan Egli wrote
> Hi,
>
> I've given this job query issue some more thought and would like us to
> discuss and hopefully soon agree on which way we go:
>
>
> Variant A: deprecate job query methods (as suggested originally):
> * upside: eventually we'll have a cleaner job API
> * downside: users of the job query API need to find alternatives to
> queries, individually
>
>
> Variant B: allow disabling job queries (https://s.apache.org/68ys):
> * upside: allows performance optimizations on a case-by-case basis
> (optimizations include not requiring indexing), thereby promoting
> query-less use cases going forward without API deprecation nevertheless.
> * downside: we stick to the current job API
>
>
> Note that in both cases the 'org.apache.sling.event.jobs' export version
> will have to be updated from 2.0.1 to 2.1.0 - which will affect
> users/customers that implement interfaces from that package (that's not
> something typical though, but there are a few exotic cases where that's
> the case).
>
>
> At this stage I'm in favour of variant B as I don't see a good
> alternative for job queries other than users re-implementing a fair
> chunk of the sling event implementation themselves (which would sort of
> defeat the purpose, besides the fact that it would not be trivial as
> there aren't enough hooks in the API for anyone other than the
> implementor to do this consistently).
>
>
> Cheers,
> Stefan
>
> On 17.12.18 16:08, Stefan Egli wrote:
>> We could introduce a config option which would allow a job queue to
>> opt-out of job queries voluntarily. That would allow the
>> implementation to treat those jobs differently (cheaper, without
>> indexing them).
>>
>> I guess such a new opt-out of queries mechanism could be less
>> controversial and provide at least some short-term gain. It doesn't
>> answer the question how job queries should be replaced though..
>
--
Carsten Ziegeler
Adobe Research Switzerland
cziegeler@apache.org
Re: [DISCUSS] Deprecate query part of Sling Event API
Posted by Stefan Egli <st...@apache.org>.
On 08.01.19 17:12, Carsten Ziegeler wrote:
> I agree, variant B sounds like the better option ...
>
> The only downside is that this doesn't force anyone to think about her
> job usage as everything just runs as before. Only if you want to improve
> you can and adjust the configurations.
That's a problem yes. Only A would allow to use some force.
Perhaps we can add something to the system console to visualize which
job queues have queries enabled and which of those have queries actually
executed (since last restart). Together with a note that job queries are
now considered bad/suboptimal practice...
Cheers,
Stefan
Re: [DISCUSS] Deprecate query part of Sling Event API
Posted by Stefan Egli <st...@apache.org>.
Hi,
I've given this job query issue some more thought and would like us to
discuss and hopefully soon agree on which way we go:
Variant A: deprecate job query methods (as suggested originally):
* upside: eventually we'll have a cleaner job API
* downside: users of the job query API need to find alternatives to
queries, individually
Variant B: allow disabling job queries (https://s.apache.org/68ys):
* upside: allows performance optimizations on a case-by-case basis
(optimizations include not requiring indexing), thereby promoting
query-less use cases going forward without API deprecation nevertheless.
* downside: we stick to the current job API
Note that in both cases the 'org.apache.sling.event.jobs' export version
will have to be updated from 2.0.1 to 2.1.0 - which will affect
users/customers that implement interfaces from that package (that's not
something typical though, but there are a few exotic cases where that's
the case).
At this stage I'm in favour of variant B as I don't see a good
alternative for job queries other than users re-implementing a fair
chunk of the sling event implementation themselves (which would sort of
defeat the purpose, besides the fact that it would not be trivial as
there aren't enough hooks in the API for anyone other than the
implementor to do this consistently).
Cheers,
Stefan
On 17.12.18 16:08, Stefan Egli wrote:
> We could introduce a config option which would allow a job queue to
> opt-out of job queries voluntarily. That would allow the implementation
> to treat those jobs differently (cheaper, without indexing them).
>
> I guess such a new opt-out of queries mechanism could be less
> controversial and provide at least some short-term gain. It doesn't
> answer the question how job queries should be replaced though..
Re: [DISCUSS] Deprecate query part of Sling Event API
Posted by Stefan Egli <st...@apache.org>.
One more thought, which has been discussed offline earlier:
While it might take a while to agree on the deprecation and on how we
recommend them to be replaced, there is a short-term option that would
already help lower load on systems:
We could introduce a config option which would allow a job queue to
opt-out of job queries voluntarily. That would allow the implementation
to treat those jobs differently (cheaper, without indexing them).
I guess such a new opt-out of queries mechanism could be less
controversial and provide at least some short-term gain. It doesn't
answer the question how job queries should be replaced though..
Cheers,
Stefan
On 17.12.18 14:02, Stefan Egli wrote:
> Hi Roy,
>
> I think removal of the job API hasn't been discussed much yet, so
> that'll probably first have to happen separately.
>
> Re deprecating the job queries: I think we should better understand the
> use cases for doing these queries. Why is it not possible to fire a job
> and forget, and why are queries needed/used. Ideally there would be an
> alternative data structure (eg the data the job operates on) which would
> be queried instead of the job itself.
>
> Cheers,
> Stefan
>
> On 15.12.18 18:39, Roy Teeuwen wrote:
>> Hey guys,
>>
>> Seeing as you are all talking about deprecating or even removing the
>> entire Sling Job API in the future, what would you propose as
>> alternative / migration path? We have a lot JobConsumers in our
>> projects, what would be a reason to stop using them?
>>
>> Greets,
>> Roy
>>
>>> On 30 Oct 2018, at 11:27, Stefan Egli <st...@apache.org> wrote:
>>>
>>> On 30.10.18 10:22, Bertrand Delacretaz wrote:
>>>> Hi,
>>>> I agree, and we might use the following trick to avoid that with
>>>> minimal disruption:
>>>> -Consider the current interface "Legacy" that needs half of its
>>>> methods deprecated
>>>> -Move the non-deprecated methods to a new "Modern" interface
>>>> -Legacy inherits from Modern
>>>> Existing clients can then use Legacy with no change.
>>>> New clients move to Modern, to make it clear that they don't want to
>>>> use legacy stuff.
>>>> Mark the whole Legacy interface as deprecated, recommend moving to
>>>> Modern
>>>> Existing services are explicitly registered as both Modern and Legacy.
>>>> No need to redesign things, just move them around.
>>>> WDYT?
>>>
>>> +1, which would bring us to naming.. one suggestion would be
>>>
>>> /** @deprecated */
>>> public interface JobManager extends JobManager2 /* Modern */ {
>>> ..
>>> }
>>>
>>> Cheers,
>>> Stefan
>>
Re: [DISCUSS] Deprecate query part of Sling Event API
Posted by Stefan Egli <st...@apache.org>.
Hi Roy,
I think removal of the job API hasn't been discussed much yet, so
that'll probably first have to happen separately.
Re deprecating the job queries: I think we should better understand the
use cases for doing these queries. Why is it not possible to fire a job
and forget, and why are queries needed/used. Ideally there would be an
alternative data structure (eg the data the job operates on) which would
be queried instead of the job itself.
Cheers,
Stefan
On 15.12.18 18:39, Roy Teeuwen wrote:
> Hey guys,
>
> Seeing as you are all talking about deprecating or even removing the entire Sling Job API in the future, what would you propose as alternative / migration path? We have a lot JobConsumers in our projects, what would be a reason to stop using them?
>
> Greets,
> Roy
>
>> On 30 Oct 2018, at 11:27, Stefan Egli <st...@apache.org> wrote:
>>
>> On 30.10.18 10:22, Bertrand Delacretaz wrote:
>>> Hi,
>>> I agree, and we might use the following trick to avoid that with
>>> minimal disruption:
>>> -Consider the current interface "Legacy" that needs half of its
>>> methods deprecated
>>> -Move the non-deprecated methods to a new "Modern" interface
>>> -Legacy inherits from Modern
>>> Existing clients can then use Legacy with no change.
>>> New clients move to Modern, to make it clear that they don't want to
>>> use legacy stuff.
>>> Mark the whole Legacy interface as deprecated, recommend moving to Modern
>>> Existing services are explicitly registered as both Modern and Legacy.
>>> No need to redesign things, just move them around.
>>> WDYT?
>>
>> +1, which would bring us to naming.. one suggestion would be
>>
>> /** @deprecated */
>> public interface JobManager extends JobManager2 /* Modern */ {
>> ..
>> }
>>
>> Cheers,
>> Stefan
>