You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@camel.apache.org by gudiseashok <gu...@gmail.com> on 2013/09/10 19:21:24 UTC

Second router in Pipeline not being reached in repeated interval scheldule

I am new to Apache Camel, but I am really trying hard to learn this. 

I working on below scenario 

1) Read from Remote server, to local directory
2) Use above local folder with Apache Lucene to index those files
3) Inform listerner to update this step

 ***Repeating those above steps for every 30 minutes with timer (code is
below), 
and flexibility to add more routiens (steps 1 to 3 as a routine) at run
time, 
To achieve step by step process, I have used <pipeline> below, 

but my problem is it is not going to next router , it is always repeating
first Router (FTP file transfer).

please suggest incase of any other efficient way to achieve above
requirement incase I am going in wrong way. 

Please find my config below,

I wrote this way,

    
    <camel:camelContext utoStartup="true">
         <camel:propertyPlaceholder id="sftpproperties"
location="camel-ftp-config.properties"/>
        <camel:route id="log-sequence-scheduler"> 
        <camel:from
uri="timer://myTimer?fixedRate="true"&amp;period="1200000"/>
         <camel:pipeline>
             <camel:to uri="direct:log-sftp-import"/> 
            <camel:to uri="direct:apache-lucene-indexing"/>
         </camel:pipeline>
</camel:route> 

<camel:route>
    <camel:from uri="direct:log-sftp-import"/>
          <camel:from
uri="sftp://{{sftp.completeurl}}&amp;binary=true&amp;timeout="900000"&amp;delay="600000"/>
         <camel:to uri="bean:fileCopier"/>
       <camel:to uri="file://{{sftp.destination.directory}}"/>
</camel:route>

    <camel:route>
        <camel:from uri="direct:apache-lucene-indexing"/>
        <camel:to uri="bean:luceneIndexer?method=luceneIndexing"/>
       </camel:route>
  </camel:camelContext>

Please help...!




--
View this message in context: http://camel.465427.n5.nabble.com/Second-router-in-Pipeline-not-being-reached-in-repeated-interval-scheldule-tp5739050.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Second router in Pipeline not being reached in repeated interval scheldule

Posted by Claus Ibsen <cl...@gmail.com>.

Then you can use a filter to only call the bean when the batch is complete.

<from ...>
 ...
<filter>
  <when><header> CamelBatchComplete</header></when>
  <to bean:lucene>
</filter>


On Tue, Sep 10, 2013 at 8:24 PM, gudiseashok <gu...@gmail.com> wrote:
> Hi Ibsen,
>
> Thanks allot for responding to my problem, but I have tried like that
> initially doing everything in one route.
>
> But problem with that was, if my "remote folder" has 10 fiels then it has to
> do indexing after completion on all those 10 files, instread that it is
> reacting on each file download. My requirement is I have to do indexing
> after complete transfer done.
>
> Also you have mentioned, I should not use timer, could you please mention
> where should I use scheduler ( I mean is that on router i.e. by using
> SimpleScheduledRoutePolicy or on sftp command? I would be greatful if you
> can specify where scheduling has to go and how to do one after one (if no
> need of pipeline)
>
>
> <from ftp>
> <to file>
> <to bean:lucene>
>
> And you might have already been informed (nothing related to my issue), when
> I post (new or reply) I am getting servlet exception in forum)
>
> I appreciate your help.
>
>
> Regards
> Ashok Gudise
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Second-router-in-Pipeline-not-being-reached-in-repeated-interval-scheldule-tp5739050p5739056.html
> Sent from the Camel - Users mailing list archive at Nabble.com.



-- 
Claus Ibsen
-----------------
Red Hat, Inc.
Email: cibsen@redhat.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen

Re: Second router in Pipeline not being reached in repeated interval scheldule

Posted by gudiseashok <gu...@gmail.com>.

Hi Ibsen,

Thanks allot for responding to my problem, but I have tried like that
initially doing everything in one route.

But problem with that was, if my "remote folder" has 10 fiels then it has to
do indexing after completion on all those 10 files, instread that it is
reacting on each file download. My requirement is I have to do indexing
after complete transfer done.

Also you have mentioned, I should not use timer, could you please mention
where should I use scheduler ( I mean is that on router i.e. by using
SimpleScheduledRoutePolicy or on sftp command? I would be greatful if you
can specify where scheduling has to go and how to do one after one (if no
need of pipeline)


<from ftp> 
<to file> 
<to bean:lucene> 

And you might have already been informed (nothing related to my issue), when
I post (new or reply) I am getting servlet exception in forum)

I appreciate your help.


Regards
Ashok Gudise



--
View this message in context: http://camel.465427.n5.nabble.com/Second-router-in-Pipeline-not-being-reached-in-repeated-interval-scheldule-tp5739050p5739056.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Second router in Pipeline not being reached in repeated interval scheldule

Posted by Claus Ibsen <cl...@gmail.com>.

Hi

You should not use the timer to trigger this, but let the ftp consumer
do that instead. It reacts when there is new files on the ftp server
to download. You can configure its scheduling also how frequent to
check for new files etc.

So something like this will download the files, and call the bean, all
in a single route

<from ftp>
<to file>
<to bean:lucene>



On Tue, Sep 10, 2013 at 7:21 PM, gudiseashok <gu...@gmail.com> wrote:
> I am new to Apache Camel, but I am really trying hard to learn this.
>
> I working on below scenario
>
> 1) Read from Remote server, to local directory
> 2) Use above local folder with Apache Lucene to index those files
> 3) Inform listerner to update this step
>
>  ***Repeating those above steps for every 30 minutes with timer (code is
> below),
> and flexibility to add more routiens (steps 1 to 3 as a routine) at run
> time,
> To achieve step by step process, I have used <pipeline> below,
>
> but my problem is it is not going to next router , it is always repeating
> first Router (FTP file transfer).
>
> please suggest incase of any other efficient way to achieve above
> requirement incase I am going in wrong way.
>
> Please find my config below,
>
> I wrote this way,
>
>
>     <camel:camelContext utoStartup="true">
>          <camel:propertyPlaceholder id="sftpproperties"
> location="camel-ftp-config.properties"/>
>         <camel:route id="log-sequence-scheduler">
>         <camel:from
> uri="timer://myTimer?fixedRate="true"&amp;period="1200000"/>
>          <camel:pipeline>
>              <camel:to uri="direct:log-sftp-import"/>
>             <camel:to uri="direct:apache-lucene-indexing"/>
>          </camel:pipeline>
> </camel:route>
>
> <camel:route>
>     <camel:from uri="direct:log-sftp-import"/>
>           <camel:from
> uri="sftp://{{sftp.completeurl}}&amp;binary=true&amp;timeout="900000"&amp;delay="600000"/>
>          <camel:to uri="bean:fileCopier"/>
>        <camel:to uri="file://{{sftp.destination.directory}}"/>
> </camel:route>
>
>     <camel:route>
>         <camel:from uri="direct:apache-lucene-indexing"/>
>         <camel:to uri="bean:luceneIndexer?method=luceneIndexing"/>
>        </camel:route>
>   </camel:camelContext>
>
> Please help...!
>
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Second-router-in-Pipeline-not-being-reached-in-repeated-interval-scheldule-tp5739050.html
> Sent from the Camel - Users mailing list archive at Nabble.com.



-- 
Claus Ibsen
-----------------
Red Hat, Inc.
Email: cibsen@redhat.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen