[jira] [Comment Edited] (CAMEL-5719) ZooKeeper route policy should initialize using onInit to elect master

["Ioannis Alexandrakis (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CAMEL-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14233674#comment-14233674 ] 

Ioannis Alexandrakis edited comment on CAMEL-5719 at 12/4/14 9:01 AM:
----------------------------------------------------------------------

There is another issue I wanted to address, which applies here (and in the original component, too). Perhaps it deserves its own ticket, perhaps not, perhaps it does not deserve a ticket at all.

Following the above example, if someone has, for instance, activemq components, just stopping the consumers (edit: the endpoint's consumers, not the activemq consumers) is not 'safe' enough. I noticed that (prefetch size * consumer count) messages (I had 2 consumers with prefetch 1, so, all in all, 2 messages) are neither being processed, not forwarded further. They stay in 'inflight' mode.

My setup is a store and forward network of activemq brokers. So, a queue on node A, if it does not have any (activemq) consumers, it forwards to a node B which has. In my case, the elected as master node would have to get the activemq messages, even though I sent jms messages to node A, which was not elected from zookeeper. Instead, it skipped (prefetch size * consumer count) messages before the rest got forwarded (the skipped ones stayed in node B as 'inflight', apparently it awaits for the endpoint consumers to be started). If, instead of stopping the consumers, I was to stop the route itself, then the component did not get those messages as inflight and all messages got forwarded immediately. I have a patch available on top of the above one (it just stops the route where it stops the consumers of the endpoint, and starts the route wherever it starts the consumers. The idea is the same, but instead of consumers, the whole route gets started/stoped). I can attach it, if you like.

This is a matter of opinion, really. If someone sets the prefetch size = 0, then there is no problem. However, it would drop the performance significantly. In my opinion, however (in any case), I think the zookeeper policy should 'stop' the route entirely. I do not know if such a behaviour exists in other components (like jms/activemq's prefetch size and the inflight exchanges problem). So, perhaps this is a really unique/specific case.


was (Author: ialex):
There is another issue I wanted to address, which applies here (and in the original component, too). Perhaps it deserves its own ticket, perhaps not, perhaps it does not deserve a ticket at all.

Following the above example, if someone has, for instance, activemq components, just stopping the consumers is not 'safe' enough. I noticed that (prefetch size * consumer count) messages (I had 2 consumers with prefetch 1, so, all in all, 2 messages) are neither being processed, not forwarded further. They stay in 'inflight' mode.

My setup is a store and forward network of activemq brokers. So, a queue on node A, if it does not have any consumers, it forwards to a node B which has. In my case, the elected as master node would have to get the activemq messages, even though I sent jms messages to node A, which was not elected from zookeeper. Instead, it skipped (prefetch size * consumer count) messages before the rest got forwarded (the skipped ones stayed in node B as 'inflight'). If, instead of stopping the consumers, I was to stop the route itself, then the component did not get those messages as inflight and all messages got forwarded immediately. I have a patch available on top of the above one (it just stops the route where it stops the consumers of the endpoint, and starts the route wherever it starts the consumers. The idea is the same, but instead of consumers, the whole route gets started/stoped). I can attach it, if you like.

This is a matter of opinion, really. If someone sets the prefetch size = 0, then there is no problem. However, it would drop the performance significantly. In my opinion, however (in any case), I think the zookeeper policy should 'stop' the route entirely. I do not know if such a behaviour exists in other components (like jms/activemq's prefetch size and the inflight exchanges problem). So, perhaps this is a really unique/specific case.

> ZooKeeper route policy should initialize using onInit to elect master
> ---------------------------------------------------------------------
>
>                 Key: CAMEL-5719
>                 URL: https://issues.apache.org/jira/browse/CAMEL-5719
>             Project: Camel
>          Issue Type: Improvement
>          Components: camel-zookeeper
>    Affects Versions: 2.10.0
>            Reporter: Claus Ibsen
>             Fix For: Future
>
>         Attachments: zookeeper-election.patch
>
>
> See nabble
> http://camel.465427.n5.nabble.com/Single-instance-of-running-route-in-the-cluster-tp5720846.html



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)