Shutting down many bundles takes a lot of time

[AntonR <an...@volvo.com>]

Hi, 

I have a setup where I run ActiveMQ and Camel together and hot deploy
blueprint xml routes. In some environments there are a lot of routes and as
such there are a lot of corresponding bundles installed (since I can not hot
deploy into a running camelContext/bundle). This all works well as far as I
can tell, except for when I need to shut down the application. 

I have written a small script that shuts down all camelContexts at once
(since before it happened sequentially and took additional time due to
routes having to shut down in series). But even with this fix the bundles
themselves take quite some time to stop. I imagine all bundles shut down in
sequential order as well and every single shutdown has a number of tasks it
needs to run through. Is there some fix available for this? Can I have them
shut down in parallel or skip some checks or something? 

Br, 
Anton



--
Sent from: http://karaf.922171.n3.nabble.com/Karaf-User-f930749.html

Re: Shutting down many bundles takes a lot of time

[AntonR <an...@volvo.com>]

I don't think that is the case for me...

After I run my shutdown script all routes take some 1-5 seconds to shut
down, after which all route related bundles are in the "Resolved" state.
What then follows is line after line of:
"Destroying container for blueprint bundle XYZ"
in the log until all of them are stopped. Then it only takes a few seconds
for the remaining bundles to stop (such as activemq broker).

I should add that i have _a lot_ of bundles running, so no individual one
takes a lot of time, probably arond a second, but since I have so many it
can still take some 10-15 minutes.

Br,
Anton



--
Sent from: http://karaf.922171.n3.nabble.com/Karaf-User-f930749.html

Re: Shutting down many bundles takes a lot of time

[Tim Ward <ti...@paremus.com>]

Hi,

When an OSGi bundle is stopped it will run whatever shutdown code it contains. This will unregister any services provided by that bundle. If there are other bundles depending on those services then they will react to the unregistration of that service by doing something, typically disabling/shutting down their functionality. This process can then trigger more services being unregistered, causing further reactions. 

If at any point a service being unregistered can be replaced by a different service then the bundles using the service being unregistered will do work to switch to using the other service. This can still take time as changes ripple through the system. What you will therefore find is that parallel stop doesn’t necessarily help in the way you think it might (it just creates more churn all at the same time). The quickest stops are normally achieved by deactivating things in reverse order (i.e. stop the “top level services” first and then repeat for the next layers).

Your other option will be to profile shutdown and to see which stops are taking a long time (either because they are slow, or because they get bounced by changing dependencies multiple times). You can then focus on making sure the small number of dominators stop more quickly.

I hope this helps,

Tim

> On 4 May 2020, at 13:00, AntonR <an...@volvo.com> wrote:
> 
> Hi, 
> 
> I have a setup where I run ActiveMQ and Camel together and hot deploy
> blueprint xml routes. In some environments there are a lot of routes and as
> such there are a lot of corresponding bundles installed (since I can not hot
> deploy into a running camelContext/bundle). This all works well as far as I
> can tell, except for when I need to shut down the application. 
> 
> I have written a small script that shuts down all camelContexts at once
> (since before it happened sequentially and took additional time due to
> routes having to shut down in series). But even with this fix the bundles
> themselves take quite some time to stop. I imagine all bundles shut down in
> sequential order as well and every single shutdown has a number of tasks it
> needs to run through. Is there some fix available for this? Can I have them
> shut down in parallel or skip some checks or something? 
> 
> Br, 
> Anton
> 
> 
> 
> --
> Sent from: http://karaf.922171.n3.nabble.com/Karaf-User-f930749.html