Re: Clustered routes in master/slave mode

[Matt Pavlovich <ma...@gmail.com>]

+1

Claus-

I could see a lot of value in having this feature for file://, ftp:// 
and several other components as well.

Are you thinking Context or Route level or both?

-Matt


On 5/28/16 2:52 AM, Claus Ibsen wrote:
> Hi
>
> We have a few components that offer capability to run a Camel routes
> in master/slave mode, where one node is elected as master and only
> runs the route at that node, and the other nodes are in slave mode,
> and then will failover if needed.
>
> For example camel-zookeeper, and camel-quartz can do that. And now
> also camel-consul. And possible others I have forgot.
>
> I wonder if we should introduce some API those components
> implementations can implement so we would better be able to manage and
> know about this. Then we can have JMX information about routes being
> in master/slave mode, and you can see for example in a 4 node cluster
> that its node-b that is the master.
>
> node-a route-foo: slave
> node-b route-foo: master
> node-c route-foo: slave
> node-d route-foo: slave
>
> We have a bit old JIRA ticket about this:
> https://issues.apache.org/jira/browse/CAMEL-4454
>
>
>
>

Re: Clustered routes in master/slave mode

[Claus Ibsen <cl...@gmail.com>]

Logged a ticket about the health check API
https://issues.apache.org/jira/browse/CAMEL-10026

On Mon, Jun 6, 2016 at 12:35 PM, Claus Ibsen <cl...@gmail.com> wrote:
> On Wed, Jun 1, 2016 at 7:34 PM, Matt Pavlovich <ma...@gmail.com> wrote:
>> +1
>>
>> Claus-
>>
>> I could see a lot of value in having this feature for file://, ftp:// and
>> several other components as well.
>>
>> Are you thinking Context or Route level or both?
>>
>
> At first it should be route level as that is the most common use case.
> Also its easier to implement as there is one route in the question.
>
> If its context level, then you need to keep track of more, such as all
> routes and figure out when to fail if any of those routes starts to
> have problems. But down the road we should allow both levels.
>
> To aid both we should also look at introducing some health API in
> camel-core, so that makes this easier as well. So you can check for
> health status on context / routes level. And custom components can
> implement custom logic to check their health, such as connecting to a
> FTP server and do a FTP list, or do a HTTP remote call etc.
>
>
>
>> -Matt
>>
>>
>>
>> On 5/28/16 2:52 AM, Claus Ibsen wrote:
>>>
>>> Hi
>>>
>>> We have a few components that offer capability to run a Camel routes
>>> in master/slave mode, where one node is elected as master and only
>>> runs the route at that node, and the other nodes are in slave mode,
>>> and then will failover if needed.
>>>
>>> For example camel-zookeeper, and camel-quartz can do that. And now
>>> also camel-consul. And possible others I have forgot.
>>>
>>> I wonder if we should introduce some API those components
>>> implementations can implement so we would better be able to manage and
>>> know about this. Then we can have JMX information about routes being
>>> in master/slave mode, and you can see for example in a 4 node cluster
>>> that its node-b that is the master.
>>>
>>> node-a route-foo: slave
>>> node-b route-foo: master
>>> node-c route-foo: slave
>>> node-d route-foo: slave
>>>
>>> We have a bit old JIRA ticket about this:
>>> https://issues.apache.org/jira/browse/CAMEL-4454
>>>
>>>
>>>
>>>
>>
>
>
>
> --
> Claus Ibsen
> -----------------
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2



-- 
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2

Re: Clustered routes in master/slave mode

[Arnaud Deprez <ar...@gmail.com>]

+1, I've had this idea too but not the time to get in deeper :-)

On Mon, Jun 6, 2016 at 3:51 PM Bilgin Ibryam <bi...@gmail.com> wrote:

> >
> > To aid both we should also look at introducing some health API in
> > camel-core, so that makes this easier as well. So you can check for
> > health status on context / routes level. And custom components can
> > implement custom logic to check their health, such as connecting to a
> > FTP server and do a FTP list, or do a HTTP remote call etc.
>
> +1
> You are reading my mind.
>
> Also, not sure if there is anything we can do to make startup/shutdown
> faster. I know there area already lazy converter loader.
>
> >
> >
> >
> >> -Matt
> >>
> >>
> >>
> >> On 5/28/16 2:52 AM, Claus Ibsen wrote:
> >>>
> >>> Hi
> >>>
> >>> We have a few components that offer capability to run a Camel routes
> >>> in master/slave mode, where one node is elected as master and only
> >>> runs the route at that node, and the other nodes are in slave mode,
> >>> and then will failover if needed.
> >>>
> >>> For example camel-zookeeper, and camel-quartz can do that. And now
> >>> also camel-consul. And possible others I have forgot.
> >>>
> >>> I wonder if we should introduce some API those components
> >>> implementations can implement so we would better be able to manage and
> >>> know about this. Then we can have JMX information about routes being
> >>> in master/slave mode, and you can see for example in a 4 node cluster
> >>> that its node-b that is the master.
> >>>
> >>> node-a route-foo: slave
> >>> node-b route-foo: master
> >>> node-c route-foo: slave
> >>> node-d route-foo: slave
> >>>
> >>> We have a bit old JIRA ticket about this:
> >>> https://issues.apache.org/jira/browse/CAMEL-4454
> >>>
> >>>
> >>>
> >>>
> >>
> >
> >
> >
> > --
> > Claus Ibsen
> > -----------------
> > http://davsclaus.com @davsclaus
> > Camel in Action 2: https://www.manning.com/ibsen2
>
>
>
> --
> Bilgin Ibryam
> Camel Committer at ASF & Integration Architect at Red Hat
> Blog: http://ofbizian.com | Twitter: @bibryam
>
> Camel Design Patterns https://leanpub.com/camel-design-patterns
> Instant Apache Camel Message Routing http://www.amazon.com/dp/1783283475
>
-- 
Arnaud Deprez
Software Engineer
Phone: +32 497 23 30 44
Linked'In: https://www.linkedin.com/in/deprezarnaud
Github: https://github.com/arnaud-deprez

Re: Clustered routes in master/slave mode

[Bilgin Ibryam <bi...@gmail.com>]

>
> To aid both we should also look at introducing some health API in
> camel-core, so that makes this easier as well. So you can check for
> health status on context / routes level. And custom components can
> implement custom logic to check their health, such as connecting to a
> FTP server and do a FTP list, or do a HTTP remote call etc.

+1
You are reading my mind.

Also, not sure if there is anything we can do to make startup/shutdown
faster. I know there area already lazy converter loader.

>
>
>
>> -Matt
>>
>>
>>
>> On 5/28/16 2:52 AM, Claus Ibsen wrote:
>>>
>>> Hi
>>>
>>> We have a few components that offer capability to run a Camel routes
>>> in master/slave mode, where one node is elected as master and only
>>> runs the route at that node, and the other nodes are in slave mode,
>>> and then will failover if needed.
>>>
>>> For example camel-zookeeper, and camel-quartz can do that. And now
>>> also camel-consul. And possible others I have forgot.
>>>
>>> I wonder if we should introduce some API those components
>>> implementations can implement so we would better be able to manage and
>>> know about this. Then we can have JMX information about routes being
>>> in master/slave mode, and you can see for example in a 4 node cluster
>>> that its node-b that is the master.
>>>
>>> node-a route-foo: slave
>>> node-b route-foo: master
>>> node-c route-foo: slave
>>> node-d route-foo: slave
>>>
>>> We have a bit old JIRA ticket about this:
>>> https://issues.apache.org/jira/browse/CAMEL-4454
>>>
>>>
>>>
>>>
>>
>
>
>
> --
> Claus Ibsen
> -----------------
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2



-- 
Bilgin Ibryam
Camel Committer at ASF & Integration Architect at Red Hat
Blog: http://ofbizian.com | Twitter: @bibryam

Camel Design Patterns https://leanpub.com/camel-design-patterns
Instant Apache Camel Message Routing http://www.amazon.com/dp/1783283475

Re: Clustered routes in master/slave mode

[Claus Ibsen <cl...@gmail.com>]

On Wed, Jun 1, 2016 at 7:34 PM, Matt Pavlovich <ma...@gmail.com> wrote:
> +1
>
> Claus-
>
> I could see a lot of value in having this feature for file://, ftp:// and
> several other components as well.
>
> Are you thinking Context or Route level or both?
>

At first it should be route level as that is the most common use case.
Also its easier to implement as there is one route in the question.

If its context level, then you need to keep track of more, such as all
routes and figure out when to fail if any of those routes starts to
have problems. But down the road we should allow both levels.

To aid both we should also look at introducing some health API in
camel-core, so that makes this easier as well. So you can check for
health status on context / routes level. And custom components can
implement custom logic to check their health, such as connecting to a
FTP server and do a FTP list, or do a HTTP remote call etc.



> -Matt
>
>
>
> On 5/28/16 2:52 AM, Claus Ibsen wrote:
>>
>> Hi
>>
>> We have a few components that offer capability to run a Camel routes
>> in master/slave mode, where one node is elected as master and only
>> runs the route at that node, and the other nodes are in slave mode,
>> and then will failover if needed.
>>
>> For example camel-zookeeper, and camel-quartz can do that. And now
>> also camel-consul. And possible others I have forgot.
>>
>> I wonder if we should introduce some API those components
>> implementations can implement so we would better be able to manage and
>> know about this. Then we can have JMX information about routes being
>> in master/slave mode, and you can see for example in a 4 node cluster
>> that its node-b that is the master.
>>
>> node-a route-foo: slave
>> node-b route-foo: master
>> node-c route-foo: slave
>> node-d route-foo: slave
>>
>> We have a bit old JIRA ticket about this:
>> https://issues.apache.org/jira/browse/CAMEL-4454
>>
>>
>>
>>
>



-- 
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2