Zookeeper Route Policy not respected on route with sftp consumer?

[ablevine1 <ab...@gmail.com>]

I have an application that runs on several servers but I want certain routes
to run on only one server in the group.  To achieve this I implemented a
Zookeeper route policy.  It seems to work for most of my routes.  When they
attempt to start up on any node except the one with the zookeeper lock, they
throw an exception similar to the following:

Job Camel_camelContextMain.myNonFTPRouteID threw a JobExecutionException: :
org.quartz.JobExecutionException: java.lang.IllegalStateException: Zookeeper
based route policy prohibits processing exchanges, stopping route and
failing the exchange [See nested exception: java.lang.IllegalStateException:
Zookeeper based route policy prohibits processing exchanges, stopping route
and failing the exchange]

That is the desired behavior.  They then startup on one other server when
the master is shut down, as expected.

However, this is not the case for my ftp route. It seems to start up and run
on all machines.  

I define the route policy as follows:

<bean id="masterOnlyRoutePolicy"
class="org.apache.camel.component.zookeeper.policy.ZooKeeperRoutePolicy">
        <constructor-arg
value="zookeeper:#{'${core.clusterMembers.zookeeper-servers}'.replaceAll('
', ',')}:2181/locks/master1"/>
        <constructor-arg value="1"/>
</bean>

I define the ftp route as follows:

<route id="myFTPRoute" routePolicyRef="masterOnlyRoutePolicy">
            
            <from uri="sftp://myftp:22/files?
                                username=username&amp;
                                password=password&amp;
                                delete=true&amp;
                                disconnect=true&amp;
                                filter=#myFilter&amp;
                                scheduler=quartz2&amp;
                                scheduler.cron=0+0/30+*+*+*+%3F
                                "/>
            <unmarshal ref="gzip"/>
            <transacted ref="PROPAGATION_REQUIRES_NEW"/>
            <log message="downloaded ${header.CamelFileName}"
loggingLevel="DEBUG" logName="mylog"/>
        </route>


Am I doing something wrong? I can't have the files download on all machines.
Is there a better way to set up the route to avoid this?



--
View this message in context: http://camel.465427.n5.nabble.com/Zookeeper-Route-Policy-not-respected-on-route-with-sftp-consumer-tp5771610.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Zookeeper Route Policy not respected on route with sftp consumer?

[Claus Ibsen <cl...@gmail.com>]

Hi

Ah good to hear, though maybe the route policy could do this logic a
bit earlier on route startup or something - though I havent looked if
that would be possible. You are welcome to dive into the code.

On Fri, Sep 18, 2015 at 10:59 PM, ablevine1 <ab...@gmail.com> wrote:
> After further findings, it turns out that the route policy is respected.  The
> reason I thought it wasn't is because the route is not stopped on the
> non-master servers until the first time the ftp consumer actually finds a
> qualifying file to download. I was seeing logging from my filter code
> running on all servers which led me to believe that the routes were actually
> running on all servers.  Once there is a qualifying file present, the route
> actually runs and fails on all servers except the one to get the zookeeper
> lock. The route is then stopped on all servers except the master and after
> that, the filter logging only appears on the master.
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Zookeeper-Route-Policy-not-respected-on-route-with-sftp-consumer-tp5771610p5771675.html
> Sent from the Camel - Users mailing list archive at Nabble.com.



-- 
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2nd edition:
https://www.manning.com/books/camel-in-action-second-edition

Re: Zookeeper Route Policy not respected on route with sftp consumer?

[ablevine1 <ab...@gmail.com>]

After further findings, it turns out that the route policy is respected.  The
reason I thought it wasn't is because the route is not stopped on the
non-master servers until the first time the ftp consumer actually finds a
qualifying file to download. I was seeing logging from my filter code
running on all servers which led me to believe that the routes were actually
running on all servers.  Once there is a qualifying file present, the route
actually runs and fails on all servers except the one to get the zookeeper
lock. The route is then stopped on all servers except the master and after
that, the filter logging only appears on the master.



--
View this message in context: http://camel.465427.n5.nabble.com/Zookeeper-Route-Policy-not-respected-on-route-with-sftp-consumer-tp5771610p5771675.html
Sent from the Camel - Users mailing list archive at Nabble.com.