DependencyManager Parallelism and injection callbacks

[Jeroen Daanen <j....@beinformed.com>]

Hello all,
I am doing some experiments enabling parallelism in our application which has a lot of services. 
Now I sometimes encounter a situation where a service that is being removed via the ‘removed' injection callback is no longer a complete service (the service being removed is missing a required dependency): 
Service A gets remove callback for M; Service M requires service X; Service X is no longer present in M. All services have been started in parallel using the ComponentExecutorFactory with a thread pool.
Is my assumption correct that when enabling parallelism this is likely to happen as the order is not fixed what happens earlier: the removal of X from M or the the removal of M from A?
Thanks,
Jeroen

Re: DependencyManager Parallelism and injection callbacks

[Pierre De Rop <pi...@gmail.com>]

i made an error in my previous mail: so when A depends on M, and M depends
on X, then when you unregister X, the ordering should be the following:

A.remove(M) should first be called while M and X are still active (not
stopped); then M.remove(X) should be called while X is not yet stopped, and
finally X should be stopped.

cheers;
/Pierre


On Fri, Dec 30, 2016 at 4:03 PM, Pierre De Rop <pi...@gmail.com>
wrote:

> Hello Jeroen;
>
> I think you came across a bug, and I committed a testcase for it as well
> as a patch in trunk (I'm sorry about that).
> see FELIX-5471
>
> So, let me clarify what is assumed to happen when you are using DM with a
> threadpool:
>
> So, when using a threadpool, all DM components are getting activated and
> assembled concurrently, but each distinct component is invoked in its
> callbacks serially, like if it was a kind of an actor. May be you know
> hawtdispatch ? DM is using a similar way as hawtdispatch where a dispatch
> queue is internally used to handle external events for a given component.
> All components have its own queue,  and multiple components queues are
> executed in a shared thread pool.
>
> Now, even when components are enabled concurrently, a correct ordering is
> guaranteed, as it happens in normal mode without a threadpool.
> For example, if A depends on C, C.start() is called first, then C is
> registered and injected into A, but asynchronously. So  A.add(C) /
> A.start() callbacks are then called.
> However, if B also depends on C, then A.add(C) and B.add(C) can be called
> in parallel, as well as A.start() // B.start().
>
> But when you unregister a component, that is when using
> "dm.remove(component)"), then I think you reproduced an issue: so, using
> your example: when A depends on M and M depends on X, then the ordering
> should be the following: M.remove(X) should first be called while X is not
> yet stopped, then A.remove(M) should be called while M is not stopped, and
> finally X should be stopped (it should be the same ordering as when no
> threadpool is used).
>
> Can you please update with the latest dm from trunk and confirm that you
> observe now a correct ordering when components are removed ?
> if you do not observe the correct ordering then please update the
> FELIX-5471 issue and I will then re-investigate in case the patch is not
> satisfying.
>
> thanks a lot;
>
> cheers;
> /Pierre
>
>
> On Fri, Dec 30, 2016 at 11:29 AM, Jeroen Daanen <j....@beinformed.com>
> wrote:
>
>> Hello all,
>> I am doing some experiments enabling parallelism in our application which
>> has a lot of services.
>> Now I sometimes encounter a situation where a service that is being
>> removed via the ‘removed' injection callback is no longer a complete
>> service (the service being removed is missing a required dependency):
>> Service A gets remove callback for M; Service M requires service X;
>> Service X is no longer present in M. All services have been started in
>> parallel using the ComponentExecutorFactory with a thread pool.
>> Is my assumption correct that when enabling parallelism this is likely to
>> happen as the order is not fixed what happens earlier: the removal of X
>> from M or the the removal of M from A?
>> Thanks,
>> Jeroen
>
>
>

Re: DependencyManager Parallelism and injection callbacks

[Pierre De Rop <pi...@gmail.com>]

Hello Jeroen;

I think you came across a bug, and I committed a testcase for it as well as
a patch in trunk (I'm sorry about that).
see FELIX-5471

So, let me clarify what is assumed to happen when you are using DM with a
threadpool:

So, when using a threadpool, all DM components are getting activated and
assembled concurrently, but each distinct component is invoked in its
callbacks serially, like if it was a kind of an actor. May be you know
hawtdispatch ? DM is using a similar way as hawtdispatch where a dispatch
queue is internally used to handle external events for a given component.
All components have its own queue,  and multiple components queues are
executed in a shared thread pool.

Now, even when components are enabled concurrently, a correct ordering is
guaranteed, as it happens in normal mode without a threadpool.
For example, if A depends on C, C.start() is called first, then C is
registered and injected into A, but asynchronously. So  A.add(C) /
A.start() callbacks are then called.
However, if B also depends on C, then A.add(C) and B.add(C) can be called
in parallel, as well as A.start() // B.start().

But when you unregister a component, that is when using
"dm.remove(component)"), then I think you reproduced an issue: so, using
your example: when A depends on M and M depends on X, then the ordering
should be the following: M.remove(X) should first be called while X is not
yet stopped, then A.remove(M) should be called while M is not stopped, and
finally X should be stopped (it should be the same ordering as when no
threadpool is used).

Can you please update with the latest dm from trunk and confirm that you
observe now a correct ordering when components are removed ?
if you do not observe the correct ordering then please update the
FELIX-5471 issue and I will then re-investigate in case the patch is not
satisfying.

thanks a lot;

cheers;
/Pierre


On Fri, Dec 30, 2016 at 11:29 AM, Jeroen Daanen <j....@beinformed.com>
wrote:

> Hello all,
> I am doing some experiments enabling parallelism in our application which
> has a lot of services.
> Now I sometimes encounter a situation where a service that is being
> removed via the ‘removed' injection callback is no longer a complete
> service (the service being removed is missing a required dependency):
> Service A gets remove callback for M; Service M requires service X;
> Service X is no longer present in M. All services have been started in
> parallel using the ComponentExecutorFactory with a thread pool.
> Is my assumption correct that when enabling parallelism this is likely to
> happen as the order is not fixed what happens earlier: the removal of X
> from M or the the removal of M from A?
> Thanks,
> Jeroen