You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@camel.apache.org by Luca Burgazzoli <lb...@gmail.com> on 2017/05/29 14:44:00 UTC
Master/Leader RoutePolicy deprecation/fix
Hello,
I did a small review of the RoutePolicy we have for having one route
master and one slave and all have an issue as they are invoked after
the route is started so the consumer may have the time to consume some
data before the policy kicks in [1].
There is now a zookeeper-master component and some work is in in
progress [2] in such area so wondering if we should deprecate such
policies once [2] is done with or without fixing them.
If we decide to fix them we could make the route to not auto start on
policy initialization so then the policy could take care to start/stop
the routes it is supposed to manage, an example of such behaviour can
be see in my experimental branch [3] and in CuratorLeaderPolicy [4].
Thoughts ?
[1] https://github.com/apache/camel/blob/master/camel-core/src/main/java/org/apache/camel/impl/RouteService.java#L213-L232
[2] https://issues.apache.org/jira/browse/CAMEL-10320
[3] https://github.com/lburgazzoli/apache-camel/blob/route-policy/components/camel-infinispan/src/main/java/org/apache/camel/component/infinispan/policy/InfinispanRoutePolicy.java
[4] https://github.com/apache/camel/blob/master/components/camel-zookeeper/src/main/java/org/apache/camel/component/zookeeper/policy/CuratorLeaderRoutePolicy.java
---
Luca Burgazzoli
Re: Master/Leader RoutePolicy deprecation/fix
Posted by Luca Burgazzoli <lb...@gmail.com>.
Make sense, will raise the JIRAs soon to create the
LeaderElectionService which can the be leveraged by Dhiraj on
CAMEL-10320 and route policies.
---
Luca Burgazzoli
On Mon, May 29, 2017 at 5:48 PM, Nicola Ferraro <ni...@gmail.com> wrote:
> To me it seems that the route policy has everything needed to create
> master/slave routes. The autostartup=false flag can be forced on the
> "onInit" callback method as done in [3]. I think it's ok doing so, because
> it's responsibility of the policy to change standard behavior of the route.
>
> I've seen some leader election policies implemented in Camel and I think
> they are pretty similar. So I was thinking why we don't define just a
> single LeaderElectionRoutePolicy in Camel core. It will manage the burden
> of starting/stopping consumers (that is a delicate task and needs to take
> care of many details) and will be notified by a specific
> "LeaderElectionService" when gaining or losing the leadership (a callback
> method with a boolean argument). This way, each leader election "algorithm"
> needs just to implement the "LeaderElectionService" interface and just
> decide when he is the leader (and when he's no more), nothing else.
>
> This will also allow us to re-use leader election easily outside the
> purpose of starting/stopping camel routes. And it is a good thing for us
> developing Camel, but we can also expose leader election services directly
> to users, because creating a cluster-wide singleton service is not a
> trivial task, but Camel can to it easily.
>
> In conclusion, for me:
> - RoutePolicy wins over master
> - Better having just 1 generic LeaderRoutePolicy and a specific
> LeaderElectionService for each "algorithm"
>
>
> Nicola
>
> On Mon, May 29, 2017 at 4:57 PM, Claus Ibsen <cl...@gmail.com> wrote:
>
>> Hi
>>
>> Yeah the set auto startup=false seems like a good idea.
>>
>> On Mon, May 29, 2017 at 4:44 PM, Luca Burgazzoli <lb...@gmail.com>
>> wrote:
>> > Hello,
>> >
>> > I did a small review of the RoutePolicy we have for having one route
>> > master and one slave and all have an issue as they are invoked after
>> > the route is started so the consumer may have the time to consume some
>> > data before the policy kicks in [1].
>> >
>> > There is now a zookeeper-master component and some work is in in
>> > progress [2] in such area so wondering if we should deprecate such
>> > policies once [2] is done with or without fixing them.
>> >
>> > If we decide to fix them we could make the route to not auto start on
>> > policy initialization so then the policy could take care to start/stop
>> > the routes it is supposed to manage, an example of such behaviour can
>> > be see in my experimental branch [3] and in CuratorLeaderPolicy [4].
>> >
>> > Thoughts ?
>> >
>> > [1] https://github.com/apache/camel/blob/master/camel-core/
>> src/main/java/org/apache/camel/impl/RouteService.java#L213-L232
>> > [2] https://issues.apache.org/jira/browse/CAMEL-10320
>> > [3] https://github.com/lburgazzoli/apache-camel/blob/
>> route-policy/components/camel-infinispan/src/main/java/org/
>> apache/camel/component/infinispan/policy/InfinispanRoutePolicy.java
>> > [4] https://github.com/apache/camel/blob/master/components/
>> camel-zookeeper/src/main/java/org/apache/camel/component/zookeeper/policy/
>> CuratorLeaderRoutePolicy.java
>> >
>> > ---
>> > Luca Burgazzoli
>>
>>
>>
>> --
>> Claus Ibsen
>> -----------------
>> http://davsclaus.com @davsclaus
>> Camel in Action 2: https://www.manning.com/ibsen2
>>
Re: Master/Leader RoutePolicy deprecation/fix
Posted by Nicola Ferraro <ni...@gmail.com>.
To me it seems that the route policy has everything needed to create
master/slave routes. The autostartup=false flag can be forced on the
"onInit" callback method as done in [3]. I think it's ok doing so, because
it's responsibility of the policy to change standard behavior of the route.
I've seen some leader election policies implemented in Camel and I think
they are pretty similar. So I was thinking why we don't define just a
single LeaderElectionRoutePolicy in Camel core. It will manage the burden
of starting/stopping consumers (that is a delicate task and needs to take
care of many details) and will be notified by a specific
"LeaderElectionService" when gaining or losing the leadership (a callback
method with a boolean argument). This way, each leader election "algorithm"
needs just to implement the "LeaderElectionService" interface and just
decide when he is the leader (and when he's no more), nothing else.
This will also allow us to re-use leader election easily outside the
purpose of starting/stopping camel routes. And it is a good thing for us
developing Camel, but we can also expose leader election services directly
to users, because creating a cluster-wide singleton service is not a
trivial task, but Camel can to it easily.
In conclusion, for me:
- RoutePolicy wins over master
- Better having just 1 generic LeaderRoutePolicy and a specific
LeaderElectionService for each "algorithm"
Nicola
On Mon, May 29, 2017 at 4:57 PM, Claus Ibsen <cl...@gmail.com> wrote:
> Hi
>
> Yeah the set auto startup=false seems like a good idea.
>
> On Mon, May 29, 2017 at 4:44 PM, Luca Burgazzoli <lb...@gmail.com>
> wrote:
> > Hello,
> >
> > I did a small review of the RoutePolicy we have for having one route
> > master and one slave and all have an issue as they are invoked after
> > the route is started so the consumer may have the time to consume some
> > data before the policy kicks in [1].
> >
> > There is now a zookeeper-master component and some work is in in
> > progress [2] in such area so wondering if we should deprecate such
> > policies once [2] is done with or without fixing them.
> >
> > If we decide to fix them we could make the route to not auto start on
> > policy initialization so then the policy could take care to start/stop
> > the routes it is supposed to manage, an example of such behaviour can
> > be see in my experimental branch [3] and in CuratorLeaderPolicy [4].
> >
> > Thoughts ?
> >
> > [1] https://github.com/apache/camel/blob/master/camel-core/
> src/main/java/org/apache/camel/impl/RouteService.java#L213-L232
> > [2] https://issues.apache.org/jira/browse/CAMEL-10320
> > [3] https://github.com/lburgazzoli/apache-camel/blob/
> route-policy/components/camel-infinispan/src/main/java/org/
> apache/camel/component/infinispan/policy/InfinispanRoutePolicy.java
> > [4] https://github.com/apache/camel/blob/master/components/
> camel-zookeeper/src/main/java/org/apache/camel/component/zookeeper/policy/
> CuratorLeaderRoutePolicy.java
> >
> > ---
> > Luca Burgazzoli
>
>
>
> --
> Claus Ibsen
> -----------------
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2
>
Re: Master/Leader RoutePolicy deprecation/fix
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
Yeah the set auto startup=false seems like a good idea.
On Mon, May 29, 2017 at 4:44 PM, Luca Burgazzoli <lb...@gmail.com> wrote:
> Hello,
>
> I did a small review of the RoutePolicy we have for having one route
> master and one slave and all have an issue as they are invoked after
> the route is started so the consumer may have the time to consume some
> data before the policy kicks in [1].
>
> There is now a zookeeper-master component and some work is in in
> progress [2] in such area so wondering if we should deprecate such
> policies once [2] is done with or without fixing them.
>
> If we decide to fix them we could make the route to not auto start on
> policy initialization so then the policy could take care to start/stop
> the routes it is supposed to manage, an example of such behaviour can
> be see in my experimental branch [3] and in CuratorLeaderPolicy [4].
>
> Thoughts ?
>
> [1] https://github.com/apache/camel/blob/master/camel-core/src/main/java/org/apache/camel/impl/RouteService.java#L213-L232
> [2] https://issues.apache.org/jira/browse/CAMEL-10320
> [3] https://github.com/lburgazzoli/apache-camel/blob/route-policy/components/camel-infinispan/src/main/java/org/apache/camel/component/infinispan/policy/InfinispanRoutePolicy.java
> [4] https://github.com/apache/camel/blob/master/components/camel-zookeeper/src/main/java/org/apache/camel/component/zookeeper/policy/CuratorLeaderRoutePolicy.java
>
> ---
> Luca Burgazzoli
--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2