You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by "Andreas A." <an...@gmail.com> on 2010/09/14 11:34:20 UTC
How do YOU handle scheduling?
Hi
I keep running into the same problem: The use case of polling an FTP on a
given time of day (via cron-expression) often pops up. I want to do
something like this:
Connect to the FTP at a time given by cron-expression
Poll for files
Download files
Shutdown route until next poll
[another route will then pick up the files as they are written to a local
directory]
I have previously solved it like this:
http://camel.465427.n5.nabble.com/Scheduled-Polling-Consumer-on-FTP-endpoint-td472602.html#a472602
Which is by using a quartz controlled route to start a route that has a
delay which of for instance 12 hours.
This is however a very messy solution and involves a lot of small routes
that starts/stops other routes.
I was thinking about starting a route with a Spring cron trigger, and then
have the route stop itself, but this cannot work as the route will not stop
itself when there are no files on the FTP and will keep polling until there
is a file.
Maybe this route shouldn't even be a route until cron-scheduling is built
directly into the endpoints?
If anyone has solved a use case like this I'd like to hear what you have
done.
--
View this message in context: http://camel.465427.n5.nabble.com/How-do-YOU-handle-scheduling-tp2838886p2838886.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: How do YOU handle scheduling?
Posted by Claus Ibsen <cl...@gmail.com>.
On Tue, Sep 14, 2010 at 3:45 PM, Andreas A. <an...@gmail.com> wrote:
>
> Hi
>
> Ok I have corrected my flawed logic to something that works now (I never
> reached the consumer.stop() line):
>
> public void poll() throws Exception {
> String ftpsUri =
> "ftp:localhost:1981/inbox?username=camel&password=camel123&move=.done";
> String fileUri = "file:{{path.out}}";
> while(true) {
> Exchange ex = consumer.receive(ftpsUri, 3000);
> if(ex != null) {
> producer.send(fileUri, ex);
> }
> else if (ex == null) {
> consumer.stop();
> break;
> }
> }
> }
>
> I'm curious as to when you want to use the receiveNoWait method?
Maybe when you want to poll a message asap, and NOT wait at all. For
example from a consumer which has an queue of received messages. (eg
SEDA etc.)
> --
> View this message in context: http://camel.465427.n5.nabble.com/How-do-YOU-handle-scheduling-tp2838886p2839126.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: How do YOU handle scheduling?
Posted by "Andreas A." <an...@gmail.com>.
Hi
Ok I have corrected my flawed logic to something that works now (I never
reached the consumer.stop() line):
public void poll() throws Exception {
String ftpsUri =
"ftp:localhost:1981/inbox?username=camel&password=camel123&move=.done";
String fileUri = "file:{{path.out}}";
while(true) {
Exchange ex = consumer.receive(ftpsUri, 3000);
if(ex != null) {
producer.send(fileUri, ex);
}
else if (ex == null) {
consumer.stop();
break;
}
}
}
I'm curious as to when you want to use the receiveNoWait method?
--
View this message in context: http://camel.465427.n5.nabble.com/How-do-YOU-handle-scheduling-tp2838886p2839126.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: How do YOU handle scheduling?
Posted by Claus Ibsen <cl...@gmail.com>.
You need to stop the consumer because the FTP consumer is schedule
based and will poll using a timer.
So you can stop the consumer template after usage, which should stop
the consumers.
You may also be able to set the cache size of the consumer template to
0 (i dont know if that makes it auto stop the consumer after use, but
I would assume it could do that)
On Tue, Sep 14, 2010 at 3:04 PM, Andreas A. <an...@gmail.com> wrote:
>
> The same thing happens in another method I use to enrich a message with a
> single specific file:
>
> String ftpsEndpoint =
> "ftps:{{ftp.address}}{{ftp.path.out}}?delete=true&username={{ftp.username}}&password={{ftp.password}}&fileName=myFileName"
> Exchange str = consumer.receive(ftpsEndpoint);
>
> Works fine, except FTP keeps getting polled (nothing gets downloaded) it
> just does LIST forever.
> --
> View this message in context: http://camel.465427.n5.nabble.com/How-do-YOU-handle-scheduling-tp2838886p2839093.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: How do YOU handle scheduling?
Posted by "Andreas A." <an...@gmail.com>.
The same thing happens in another method I use to enrich a message with a
single specific file:
String ftpsEndpoint =
"ftps:{{ftp.address}}{{ftp.path.out}}?delete=true&username={{ftp.username}}&password={{ftp.password}}&fileName=myFileName"
Exchange str = consumer.receive(ftpsEndpoint);
Works fine, except FTP keeps getting polled (nothing gets downloaded) it
just does LIST forever.
--
View this message in context: http://camel.465427.n5.nabble.com/How-do-YOU-handle-scheduling-tp2838886p2839093.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: How do YOU handle scheduling?
Posted by "Andreas A." <an...@gmail.com>.
I tried basing the class on both the example in the wiki and the example in
the book. It works, except the consumer never stops polling when I use the
FTP endpoint.
I don't know what you are trying to hint at. If I use receive or
receiveNoWait does not make any difference. The FTP endpoint keeps on doing
LIST on the FTP.
--
View this message in context: http://camel.465427.n5.nabble.com/How-do-YOU-handle-scheduling-tp2838886p2839085.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: How do YOU handle scheduling?
Posted by Claus Ibsen <cl...@gmail.com>.
Read the java doc and documentation for the ConsumerTemplate.
Or check chapter 3 or appendix C in the Camel book.
On Tue, Sep 14, 2010 at 2:30 PM, Andreas A. <an...@gmail.com> wrote:
>
> Hi
>
> I'm now trying to just use a Spring trigger and a simple class using
> ConsumerTemplate and ProducerTemplate to fetch the files. However, I can't
> get the ConsumerTemplate to stop polling when I first start it, what's going
> on?
>
> public class FtpsPoller {
> @Autowired
> private ConsumerTemplate consumer;
>
> @Autowired
> private ProducerTemplate producer;
>
> public void poll() throws Exception {
> String ftpsUri =
> "ftps:{{ftp.address}}{{ftp.path.out}}?username={{ftp.username}}&password={{ftp.password}}
> String fileUri = "file:{{path.out}}";
> Exchange exchange = null;
> do {
> exchange = consumer.receive(ftpsUri);
> producer.send(fileUri, exchange);
> } while(exchange != null);
> consumer.stop(); //Doesn't stop anything
> }
> }
> --
> View this message in context: http://camel.465427.n5.nabble.com/How-do-YOU-handle-scheduling-tp2838886p2839061.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: How do YOU handle scheduling?
Posted by "Andreas A." <an...@gmail.com>.
Hi
I'm now trying to just use a Spring trigger and a simple class using
ConsumerTemplate and ProducerTemplate to fetch the files. However, I can't
get the ConsumerTemplate to stop polling when I first start it, what's going
on?
public class FtpsPoller {
@Autowired
private ConsumerTemplate consumer;
@Autowired
private ProducerTemplate producer;
public void poll() throws Exception {
String ftpsUri =
"ftps:{{ftp.address}}{{ftp.path.out}}?username={{ftp.username}}&password={{ftp.password}}
String fileUri = "file:{{path.out}}";
Exchange exchange = null;
do {
exchange = consumer.receive(ftpsUri);
producer.send(fileUri, exchange);
} while(exchange != null);
consumer.stop(); //Doesn't stop anything
}
}
--
View this message in context: http://camel.465427.n5.nabble.com/How-do-YOU-handle-scheduling-tp2838886p2839061.html
Sent from the Camel - Users mailing list archive at Nabble.com.