You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sling.apache.org by Bertrand Delacretaz <bd...@apache.org> on 2012/11/27 14:33:45 UTC
SLING-490, content upgrade use case
Hi,
We've just restarted the SLING-490 (Sling readyness check at startup)
discussion in the issue, and Felix+Carsten are asking about this
requirement:
c) Provide a simple way to hook tasks into the startup sequence, so
that they run at configurable start levels.
For example "start threads A and B once start level reaches 12, and
wait for them to exit before allowing the system to go higher than
start level 12"
The typical use case for this is upgrading content at startup, for example:
1. Sling was running with V1.0 of bundle A
2. Version 2.0 of bundle A starts, and notices that content structures
needs to be upgraded to work with V2.0
3. The content upgrade takes a few minutes, and the system should be
made inaccessible during that time to avoid making a mess or
presenting a half-working system to users
4. All this can happen during the first startup with a new version of
a Sling-based application, where several services might behave like
this bundle A and require somewhat long-running upgrades, which
ideally should run concurrently to minimize upgrading time - that's
where this requirement c) comes in.
My idea with this requirement c) is to allow such upgrades to run a
specific start levels during this upgrade startup, and make sure the
system doesn't show a ready state before the upgrades are finished.
Such upgrades could also happen while Sling is already running, but in
this case best practices + examples could be sufficient.
I'm not sure if this needs to be part of SLING-490, but we need a
clean and documented solution for such cases - thoughts?
-Bertrand
Re: SLING-490, content upgrade use case
Posted by Bertrand Delacretaz <bd...@apache.org>.
On Wed, Nov 28, 2012 at 12:34 PM, Carsten Ziegeler <cz...@apache.org> wrote:
> 2012/11/28 Bertrand Delacretaz <bd...@apache.org>:
>>... The problem is when upgrade code needs SCR services...
> ...Well we're talking about bundle A being the upgrade bundle and it uses
> SCR to register services which in turn is used by the upgrade code?
> Why not create an instance of S1 in the activator and go with it?...
Because S1 is an SCR service, and forcing it out of that intrudes into
user's code.
Anyway - it seems like we're unable to come to consensus here, let's
not waste more time on this. I'll come up with concrete examples if
needed.
-Bertrand
Re: SLING-490, content upgrade use case
Posted by Carsten Ziegeler <cz...@apache.org>.
2012/11/28 Bertrand Delacretaz <bd...@apache.org>:
> On Wed, Nov 28, 2012 at 11:44 AM, Carsten Ziegeler <cz...@apache.org> wrote:
>> ...Each bundle which needs to do an upgrade
>> can be run by itself at the correct start level...
>
> The problem is when upgrade code needs SCR services (*) - if you run
> bundle A's upgrade code from bundle A's Activator, and it needs
> service S1 from bundle A and service S2 from bundle B, you have two
> problems:
>
> -S1 will not be available
Well we're talking about bundle A being the upgrade bundle and it uses
SCR to register services which in turn is used by the upgrade code?
Why not create an instance of S1 in the activator and go with it?
> -You're assuming S2 is available when A's Activator runs, which is not
> a good idea
Why not? If B is started in a lower run mode by SCR?
>
> My proposal solves both issues, the only assumption is that all the
> required services are available when the upgrade.runner's
> Activator.start() runs, which is easy to get by setting that bundle's
> start level.
So you're assuming S2 is available as well :)
Carsten
--
Carsten Ziegeler
cziegeler@apache.org
Re: SLING-490, content upgrade use case
Posted by Bertrand Delacretaz <bd...@apache.org>.
On Wed, Nov 28, 2012 at 11:44 AM, Carsten Ziegeler <cz...@apache.org> wrote:
> ...Each bundle which needs to do an upgrade
> can be run by itself at the correct start level...
The problem is when upgrade code needs SCR services (*) - if you run
bundle A's upgrade code from bundle A's Activator, and it needs
service S1 from bundle A and service S2 from bundle B, you have two
problems:
-S1 will not be available
-You're assuming S2 is available when A's Activator runs, which is not
a good idea
My proposal solves both issues, the only assumption is that all the
required services are available when the upgrade.runner's
Activator.start() runs, which is easy to get by setting that bundle's
start level.
-Bertrand
(*) This is based on a real use case of mine where workflow content
upgrades need workflow services to do their job, or would be very
hacky if they bypass it.
Re: SLING-490, content upgrade use case
Posted by Carsten Ziegeler <cz...@apache.org>.
2012/11/28 Bertrand Delacretaz <bd...@apache.org>:
> Hi,
>
> On Wed, Nov 28, 2012 at 9:58 AM, Felix Meschberger <fm...@adobe.com> wrote:
>> Am 28.11.2012 um 09:53 schrieb Felix Meschberger:
>>> Am 27.11.2012 um 16:28 schrieb Bertrand Delacretaz:
>>>> ...if that doesn't cause trouble at the OSGi framework level I'm
>>>> fine with having that stuff run from BundleActivators.
>>>
>>> As long as you don't block waiting for a service to arrive, this should be fine.
>>
>> Actually, even more on the edge, but probably appropriate here: You might do this content update upon
>> receiving the REGISTERED event for the [Sling]Repository service. Service events are delivered
>> synchronously, so you can update the content while bein sure the rest of the system is waiting to continue....
>
> Quite hacky...I have a different proposal, much cleaner IMO:
>
> 1. A new upgrade.runner bundle is configured with a start level just
> below the "system ready" start level, so that it runs when
> everything's ready to be upgraded
>
> 2. That bundle's Activator.start() looks for Callable services
> ("upgrade Callables") registered with a specific
> "sling.upgrade.callable.ID" property.
>
> 3. Activator.start() starts a separate Thread to run all such
> services, waits for it to exit and blocks HTTP requests with an
> informative 503 response while it's running
>
> 4. Upgrade Callables must be idempotent: calling them several times
> must do no harm, and all calls except the first one that actually does
> something must be fast. This allows the upgrade.runner to blindly call
> them at every startup.
>
> The result is that:
> a) Startup waits for all such Callable to be executed
>
> b) Upgrade Callables can use SCR services (or is anything important
> locked during Activator.start()?)
>
> c) Users can easily inject their own Upgrade Callables, and they all
> run at a specific time during the startup sequence.
>
> Not much to implement, simple, modular...WDYT?
>
Why all these indirections? Each bundle which needs to do an upgrade
can be run by itself at the correct start level. So no need for a
collector bundle, service interface etc.
Blocking http requests is already done in other services we have, so I
don't see a reason why we should do this here as well.
Carsten
> -Bertrand
--
Carsten Ziegeler
cziegeler@apache.org
Re: SLING-490, content upgrade use case
Posted by Felix Meschberger <fm...@adobe.com>.
Hi,
Am 28.11.2012 um 11:39 schrieb Bertrand Delacretaz:
> Hi,
>
> On Wed, Nov 28, 2012 at 9:58 AM, Felix Meschberger <fm...@adobe.com> wrote:
>> Am 28.11.2012 um 09:53 schrieb Felix Meschberger:
>>> Am 27.11.2012 um 16:28 schrieb Bertrand Delacretaz:
>>>> ...if that doesn't cause trouble at the OSGi framework level I'm
>>>> fine with having that stuff run from BundleActivators.
>>>
>>> As long as you don't block waiting for a service to arrive, this should be fine.
>>
>> Actually, even more on the edge, but probably appropriate here: You might do this content update upon
>> receiving the REGISTERED event for the [Sling]Repository service. Service events are delivered
>> synchronously, so you can update the content while bein sure the rest of the system is waiting to continue....
>
> Quite hacky...
Seriously ? Using plain old OSGi API ?
> I have a different proposal, much cleaner IMO:
>
> 1. A new upgrade.runner bundle is configured with a start level just
> below the "system ready" start level, so that it runs when
> everything's ready to be upgraded
>
> 2. That bundle's Activator.start() looks for Callable services
> ("upgrade Callables") registered with a specific
> "sling.upgrade.callable.ID" property.
>
> 3. Activator.start() starts a separate Thread to run all such
> services, waits for it to exit and blocks HTTP requests with an
> informative 503 response while it's running
>
> 4. Upgrade Callables must be idempotent: calling them several times
> must do no harm, and all calls except the first one that actually does
> something must be fast. This allows the upgrade.runner to blindly call
> them at every startup.
>
> The result is that:
> a) Startup waits for all such Callable to be executed
>
> b) Upgrade Callables can use SCR services (or is anything important
> locked during Activator.start()?)
>
> c) Users can easily inject their own Upgrade Callables, and they all
> run at a specific time during the startup sequence.
>
> Not much to implement, simple, modular...WDYT?
Really ?
I think this is overengineered, complicated and brittle (expecting services in the activator).
I don't think this is really worth it.
Regards
Felix
Re: SLING-490, content upgrade use case
Posted by Bertrand Delacretaz <bd...@apache.org>.
Hi,
On Wed, Nov 28, 2012 at 9:58 AM, Felix Meschberger <fm...@adobe.com> wrote:
> Am 28.11.2012 um 09:53 schrieb Felix Meschberger:
>> Am 27.11.2012 um 16:28 schrieb Bertrand Delacretaz:
>>> ...if that doesn't cause trouble at the OSGi framework level I'm
>>> fine with having that stuff run from BundleActivators.
>>
>> As long as you don't block waiting for a service to arrive, this should be fine.
>
> Actually, even more on the edge, but probably appropriate here: You might do this content update upon
> receiving the REGISTERED event for the [Sling]Repository service. Service events are delivered
> synchronously, so you can update the content while bein sure the rest of the system is waiting to continue....
Quite hacky...I have a different proposal, much cleaner IMO:
1. A new upgrade.runner bundle is configured with a start level just
below the "system ready" start level, so that it runs when
everything's ready to be upgraded
2. That bundle's Activator.start() looks for Callable services
("upgrade Callables") registered with a specific
"sling.upgrade.callable.ID" property.
3. Activator.start() starts a separate Thread to run all such
services, waits for it to exit and blocks HTTP requests with an
informative 503 response while it's running
4. Upgrade Callables must be idempotent: calling them several times
must do no harm, and all calls except the first one that actually does
something must be fast. This allows the upgrade.runner to blindly call
them at every startup.
The result is that:
a) Startup waits for all such Callable to be executed
b) Upgrade Callables can use SCR services (or is anything important
locked during Activator.start()?)
c) Users can easily inject their own Upgrade Callables, and they all
run at a specific time during the startup sequence.
Not much to implement, simple, modular...WDYT?
-Bertrand
Re: SLING-490, content upgrade use case
Posted by Felix Meschberger <fm...@adobe.com>.
Hi,
Am 28.11.2012 um 09:53 schrieb Felix Meschberger:
> Hi,
>
> Am 27.11.2012 um 16:28 schrieb Bertrand Delacretaz:
>
>> On Tue, Nov 27, 2012 at 4:12 PM, Carsten Ziegeler <cz...@apache.org> wrote:
>>> 2012/11/27 Bertrand Delacretaz <bd...@apache.org>:
>>>> ...Except that BundleActivator.start(...) should return "in a timely
>>>> manner"...
>>
>>> Now, this statement is clearly there to have a quick/smooth startup -
>>> but we're talking about an upgrade where we explicitely want a slow
>>> startup (or have everything done during startup). So having a long
>>> running start in the upgrade case is fine and regardless what time it
>>> takes, we don't violate the spec....
>>
>> Ok, so if that doesn't cause trouble at the OSGi framework level I'm
>> fine with having that stuff run from BundleActivators.
>
> As long as you don't block waiting for a service to arrive, this should be fine.
Actually, even more on the edge, but probably appropriate here: You might do this content update upon receiving the REGISTERED event for the [Sling]Repository service. Service events are delivered synchronously, so you can update the content while bein sure the rest of the system is waiting to continue.
For sure, this is an exceptional case; but it should also be an exceptional situation to have to do this, right ?
Regards
Felix
Re: SLING-490, content upgrade use case
Posted by Felix Meschberger <fm...@adobe.com>.
Hi,
Am 27.11.2012 um 16:28 schrieb Bertrand Delacretaz:
> On Tue, Nov 27, 2012 at 4:12 PM, Carsten Ziegeler <cz...@apache.org> wrote:
>> 2012/11/27 Bertrand Delacretaz <bd...@apache.org>:
>>> ...Except that BundleActivator.start(...) should return "in a timely
>>> manner"...
>
>> Now, this statement is clearly there to have a quick/smooth startup -
>> but we're talking about an upgrade where we explicitely want a slow
>> startup (or have everything done during startup). So having a long
>> running start in the upgrade case is fine and regardless what time it
>> takes, we don't violate the spec....
>
> Ok, so if that doesn't cause trouble at the OSGi framework level I'm
> fine with having that stuff run from BundleActivators.
As long as you don't block waiting for a service to arrive, this should be fine.
Regards
Felix
Re: SLING-490, content upgrade use case
Posted by Bertrand Delacretaz <bd...@apache.org>.
On Tue, Nov 27, 2012 at 4:12 PM, Carsten Ziegeler <cz...@apache.org> wrote:
> 2012/11/27 Bertrand Delacretaz <bd...@apache.org>:
>> ...Except that BundleActivator.start(...) should return "in a timely
>> manner"...
> Now, this statement is clearly there to have a quick/smooth startup -
> but we're talking about an upgrade where we explicitely want a slow
> startup (or have everything done during startup). So having a long
> running start in the upgrade case is fine and regardless what time it
> takes, we don't violate the spec....
Ok, so if that doesn't cause trouble at the OSGi framework level I'm
fine with having that stuff run from BundleActivators.
-Bertrand
Re: SLING-490, content upgrade use case
Posted by Carsten Ziegeler <cz...@apache.org>.
2012/11/27 Bertrand Delacretaz <bd...@apache.org>:
> On Tue, Nov 27, 2012 at 3:25 PM, Carsten Ziegeler <cz...@apache.org> wrote:
>> ...as Felix pointed out, a
>> bundle activator is the best place to do this, as this works ootb and
>> doesn't require any additional complicated stuff...
>
> Except that BundleActivator.start(...) should return "in a timely
> manner" as stated in this thread, so it's not the place for upgrades
> that might last half an hour, or am I missing something?
Now, this statement is clearly there to have a quick/smooth startup -
but we're talking about an upgrade where we explicitely want a slow
startup (or have everything done during startup). So having a long
running start in the upgrade case is fine and regardless what time it
takes, we don't violate the spec.
Carsten
--
Carsten Ziegeler
cziegeler@apache.org
Re: SLING-490, content upgrade use case
Posted by Bertrand Delacretaz <bd...@apache.org>.
On Tue, Nov 27, 2012 at 3:25 PM, Carsten Ziegeler <cz...@apache.org> wrote:
> ...as Felix pointed out, a
> bundle activator is the best place to do this, as this works ootb and
> doesn't require any additional complicated stuff...
Except that BundleActivator.start(...) should return "in a timely
manner" as stated in this thread, so it's not the place for upgrades
that might last half an hour, or am I missing something?
> ...And as I noted in
> the bug, there is another possibility, however that one needs to be
> handled with care and an activator is usually sufficient....
in SLING-490? I don't see what you mean.
-Bertrand
Re: SLING-490, content upgrade use case
Posted by Carsten Ziegeler <cz...@apache.org>.
2012/11/27 Felix Meschberger <fm...@adobe.com>:
> Hi,
>
> Am 27.11.2012 um 14:33 schrieb Bertrand Delacretaz:
>
>> Hi,
>>
>> We've just restarted the SLING-490 (Sling readyness check at startup)
>> discussion in the issue, and Felix+Carsten are asking about this
>> requirement:
I wasn't asking for a requirement :) But as Felix pointed out, a
bundle activator is the best place to do this, as this works ootb and
doesn't require any additional complicated stuff. And as I noted in
the bug, there is another possibility, however that one needs to be
handled with care and an activator is usually sufficient.
Carsten
>>
>> c) Provide a simple way to hook tasks into the startup sequence, so
>> that they run at configurable start levels.
>> For example "start threads A and B once start level reaches 12, and
>> wait for them to exit before allowing the system to go higher than
>> start level 12"
>>
>> The typical use case for this is upgrading content at startup, for example:
>>
>> 1. Sling was running with V1.0 of bundle A
>>
>> 2. Version 2.0 of bundle A starts, and notices that content structures
>> needs to be upgraded to work with V2.0
>>
>> 3. The content upgrade takes a few minutes, and the system should be
>> made inaccessible during that time to avoid making a mess or
>> presenting a half-working system to users
>>
>> 4. All this can happen during the first startup with a new version of
>> a Sling-based application, where several services might behave like
>> this bundle A and require somewhat long-running upgrades, which
>> ideally should run concurrently to minimize upgrading time - that's
>> where this requirement c) comes in.
>
> I do not get this requirement: If this is handled during startup, the BundleActivator could handle that and thus block startup while operating.
>
> This is essentially the same as your requirement (c) except that it is simpler to do and implement.
>
> Regards
> Felix
>
>>
>> My idea with this requirement c) is to allow such upgrades to run a
>> specific start levels during this upgrade startup, and make sure the
>> system doesn't show a ready state before the upgrades are finished.
>>
>> Such upgrades could also happen while Sling is already running, but in
>> this case best practices + examples could be sufficient.
>>
>> I'm not sure if this needs to be part of SLING-490, but we need a
>> clean and documented solution for such cases - thoughts?
>>
>> -Bertrand
>
--
Carsten Ziegeler
cziegeler@apache.org
Re: SLING-490, content upgrade use case
Posted by Bertrand Delacretaz <bd...@apache.org>.
On Tue, Nov 27, 2012 at 2:44 PM, Felix Meschberger <fm...@adobe.com> wrote:
> Am 27.11.2012 um 14:33 schrieb Bertrand Delacretaz:
>>... 4. All this can happen during the first startup with a new version of
>> a Sling-based application, where several services might behave like
>> this bundle A and require somewhat long-running upgrades, which
>> ideally should run concurrently to minimize upgrading time - that's
>> where this requirement c) comes in.
>
> I do not get this requirement: If this is handled during startup, the BundleActivator could handle that and thus block startup while operating.
For BundleActivator.start(...), [1] says "This method must complete
and return to its caller in a timely manner" so IMO such things must
run asynchronously. The content upgrades that I'm talking about could
take half an hour for example, I don't think we want to block the
framework for that long.
I see two options:
a) Run the upgrade code asynchronously and let the start levels evolve
independently.
This requires a way to signal to the outside world that the system is
not ready, even though the Sling startup listener thinks it is, in
terms of start levels.
b) Do as I suggest in SLING-490 and run asynchronously, outside of the
BundleActivator.start() method, and have the Sling startup code wait
for some conditions that signal that it's ok to to up to the next
start level.
-Bertrand
[1] http://www.osgi.org/javadoc/r4v43/core/org/osgi/framework/BundleActivator.html
Re: SLING-490, content upgrade use case
Posted by Felix Meschberger <fm...@adobe.com>.
Hi,
Am 27.11.2012 um 14:33 schrieb Bertrand Delacretaz:
> Hi,
>
> We've just restarted the SLING-490 (Sling readyness check at startup)
> discussion in the issue, and Felix+Carsten are asking about this
> requirement:
>
> c) Provide a simple way to hook tasks into the startup sequence, so
> that they run at configurable start levels.
> For example "start threads A and B once start level reaches 12, and
> wait for them to exit before allowing the system to go higher than
> start level 12"
>
> The typical use case for this is upgrading content at startup, for example:
>
> 1. Sling was running with V1.0 of bundle A
>
> 2. Version 2.0 of bundle A starts, and notices that content structures
> needs to be upgraded to work with V2.0
>
> 3. The content upgrade takes a few minutes, and the system should be
> made inaccessible during that time to avoid making a mess or
> presenting a half-working system to users
>
> 4. All this can happen during the first startup with a new version of
> a Sling-based application, where several services might behave like
> this bundle A and require somewhat long-running upgrades, which
> ideally should run concurrently to minimize upgrading time - that's
> where this requirement c) comes in.
I do not get this requirement: If this is handled during startup, the BundleActivator could handle that and thus block startup while operating.
This is essentially the same as your requirement (c) except that it is simpler to do and implement.
Regards
Felix
>
> My idea with this requirement c) is to allow such upgrades to run a
> specific start levels during this upgrade startup, and make sure the
> system doesn't show a ready state before the upgrades are finished.
>
> Such upgrades could also happen while Sling is already running, but in
> this case best practices + examples could be sufficient.
>
> I'm not sure if this needs to be part of SLING-490, but we need a
> clean and documented solution for such cases - thoughts?
>
> -Bertrand