You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Ashwin Pejavar <as...@gmail.com> on 2012/08/25 19:13:24 UTC
Periodic route activation use case
I have a use-case where I need to consume files at a fixed cron schedule.
For example, I need my input route to resume everyday at 6AM, process all
files collected in the input directory and go into suspension till 6AM the
following day.
I was hoping the CronScheduledRoutePolicy would be useful, but that doesn't
quite do what I need. I can use it to start the route at 6AM, but there
seems to be no way of automatically suspending it when all the files are
processed.
Is there a way of doing this in Camel?
--
View this message in context: http://camel.465427.n5.nabble.com/Periodic-route-activation-use-case-tp5718059.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Periodic route activation use case
Posted by Claus Ibsen <cl...@gmail.com>.
On Sat, Aug 25, 2012 at 7:13 PM, Ashwin Pejavar
<as...@gmail.com> wrote:
> I have a use-case where I need to consume files at a fixed cron schedule.
>
> For example, I need my input route to resume everyday at 6AM, process all
> files collected in the input directory and go into suspension till 6AM the
> following day.
>
> I was hoping the CronScheduledRoutePolicy would be useful, but that doesn't
> quite do what I need. I can use it to start the route at 6AM, but there
> seems to be no way of automatically suspending it when all the files are
> processed.
>
> Is there a way of doing this in Camel?
>
See this FAQ
http://camel.apache.org/how-can-i-stop-a-route-from-a-route.html
You could possible either extend the cron scheduled route policy, and
add logic in the onExchangeDone and figure out if its the last
exchange from the file batch. And then if so stop/suspend the
consumer. The file consumer is cheap to suspend as its an internal
flag that indicate its suspended (AFAIR). And therefore you most
likely wont need to use a separate thread as the FAQ tells to do.
The file consumer will set a property on the exchange when there is no
more files (eg the batch is complete)
exchange.setProperty(Exchange.BATCH_COMPLETE, true);
So you can check for that property, and know when you are done, and
can suspend the consumer.
Something a like
onExchangeDone(Route route, Exchange exchange) {
// check for batch complete
if (...) {
// stop will favor suspend over stopping, so its fine to invoke
this method
stopConsumer(route.getConsumer());
}
}
And on the file component you would need to enable the
sendEmptyMessageWhenIdle=true option to send an empty message if there
is no files. So the routing will kick in. You would need to cater for
an empty message in your route logic though.
A total different approach would be to use a custom
PollingConsumerPollStrategy on the file endpoint.
http://camel.apache.org/polling-consumer.html
It has callbacks when the polling starts / commits. And the number of
polled messages (eg number of files found).
You can then use that to implement logic to suspend the consumer in
the commit method.
That may be an easier task to try out at first. Extending the
DefaultPollingConsumerPollStrategy and override the commit method.
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Periodic-route-activation-use-case-tp5718059.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
Re: Periodic route activation use case
Posted by Ashwin Pejavar <as...@gmail.com>.
Thank you for the prompt response guys.
I think I have at least three interesting leads, and I am pretty sure at
least one will work
1) Use sendEmptyMessageWhenIdle in the route and pause the route when an
empty message is detected
2) Extend the DefaultPollingConsumerPollStrategy and override the commit
method
3) Extend the CronScheduledRoutePolicy and override the onExchangeDone
method.
--
View this message in context: http://camel.465427.n5.nabble.com/Periodic-route-activation-use-case-tp5718059p5718105.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Periodic route activation use case
Posted by Pontus Ullgren <ul...@gmail.com>.
Hello,
I do not agree with you. Declarative route definitions is available in
pretty much all integration platforms/frameworks. The real good thing
I find with Camel is the possibility to start/stop routes, build
dynamic routes and even add new routes in runtime using different DSL.
> In the case of the problem, discussed in this thread, maybe it
> would be better approach to have static route consuming files from the
> folder B, and second route, initiated by a timer, that moves files at
> 6AM from the folder A (inbox) to the folder B?
>
I do not see how this approach would be simpler. You would have the
same problem with the second, timer initiated, route as you have with
the route in the initial question. The reason is that the file
component only picks up one file, and not a complete batch, when used
as content enricher.
If it didn't or if you only need to fetch one file then the timer
initiated followed by a pollEnricher("file:...") would be a much
better approach.
> Another possible
> approach could be to have an aggregator in the route with completion
> expression specified accordingly (maybe file references instead of
> streams should be passed in such case)?
>
IMHO using a aggregator for this would be more complicated than
suspending/resuming the route.
Also using this solution would probably not move the files at 6 am but
at the time the next file after 6 am is available. And that could be
much later. Unless you add a second trigger based route that sends a
dummy message into the aggregator.
That said, when faced with the very common requirement "The files must
be moved at 6 am". I always ask if this is the real requirement of if
what they really ask for is "The files must be moved before 6 am".
In the later case you can have a simple file route that moves the
files as they come into the inbox.
But this is just my $0.02
Best regards
Pontus Ullgren
On Sun, Aug 26, 2012 at 12:51 PM, Karolis Petrauskas
<k....@gmail.com> wrote:
> Hi,
>
> My question is maybe not related to the topic directly, but I
> would like to get your opinions. Is it good approach to design
> applications in a way that the routes are dynamically started and
> stopped? As I understand, one of the good points about Camel routes is
> that they are declarative. That makes the system simpler, there is
> less room for bugs, etc. By managing routes dynamically, one broke
> that simplicity.
>
> In the case of the problem, discussed in this thread, maybe it
> would be better approach to have static route consuming files from the
> folder B, and second route, initiated by a timer, that moves files at
> 6AM from the folder A (inbox) to the folder B? Another possible
> approach could be to have an aggregator in the route with completion
> expression specified accordingly (maybe file references instead of
> streams should be passed in such case)? Other solutions could be
> enumerated here also, but the main point here is that the routes
> should be static.
>
> Karolis
>
> On Sun, Aug 26, 2012 at 10:19 AM, Claus Ibsen <cl...@gmail.com> wrote:
>> On Sun, Aug 26, 2012 at 9:14 AM, Pontus Ullgren <ul...@gmail.com> wrote:
>>> Hello,
>>>
>>> You can combine the CronScheduledRoutePolicy with the use of the
>>> sendEmptyMessageWhenIdle option of the file component [1].
>>> You will then get a empty message when there are no more files left
>>> and you can then suspend or stop [2] the route.
>>>
>>
>> Yeah that is also a good solution. As the empty message will only be
>> sent when there is no more files in a separate poll.
>> The only trick parts is that your routing logic would need to cater
>> for an "empty message" being routed.
>>
>>> [1] http://camel.apache.org/file2.html
>>> [2] http://camel.apache.org/how-can-i-stop-a-route-from-a-route.html
>>>
>>> Best regards
>>> Pontus Ullgren
>>>
>>> On Sat, Aug 25, 2012 at 7:13 PM, Ashwin Pejavar
>>> <as...@gmail.com> wrote:
>>>> I have a use-case where I need to consume files at a fixed cron schedule.
>>>>
>>>> For example, I need my input route to resume everyday at 6AM, process all
>>>> files collected in the input directory and go into suspension till 6AM the
>>>> following day.
>>>>
>>>> I was hoping the CronScheduledRoutePolicy would be useful, but that doesn't
>>>> quite do what I need. I can use it to start the route at 6AM, but there
>>>> seems to be no way of automatically suspending it when all the files are
>>>> processed.
>>>>
>>>> Is there a way of doing this in Camel?
>>>>
>>>>
>>>>
>>>> --
>>>> View this message in context: http://camel.465427.n5.nabble.com/Periodic-route-activation-use-case-tp5718059.html
>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>
>>
>>
>> --
>> Claus Ibsen
>> -----------------
>> FuseSource
>> Email: cibsen@fusesource.com
>> Web: http://fusesource.com
>> Twitter: davsclaus, fusenews
>> Blog: http://davsclaus.com
>> Author of Camel in Action: http://www.manning.com/ibsen
Re: Periodic route activation use case
Posted by Karolis Petrauskas <k....@gmail.com>.
Hi,
My question is maybe not related to the topic directly, but I
would like to get your opinions. Is it good approach to design
applications in a way that the routes are dynamically started and
stopped? As I understand, one of the good points about Camel routes is
that they are declarative. That makes the system simpler, there is
less room for bugs, etc. By managing routes dynamically, one broke
that simplicity.
In the case of the problem, discussed in this thread, maybe it
would be better approach to have static route consuming files from the
folder B, and second route, initiated by a timer, that moves files at
6AM from the folder A (inbox) to the folder B? Another possible
approach could be to have an aggregator in the route with completion
expression specified accordingly (maybe file references instead of
streams should be passed in such case)? Other solutions could be
enumerated here also, but the main point here is that the routes
should be static.
Karolis
On Sun, Aug 26, 2012 at 10:19 AM, Claus Ibsen <cl...@gmail.com> wrote:
> On Sun, Aug 26, 2012 at 9:14 AM, Pontus Ullgren <ul...@gmail.com> wrote:
>> Hello,
>>
>> You can combine the CronScheduledRoutePolicy with the use of the
>> sendEmptyMessageWhenIdle option of the file component [1].
>> You will then get a empty message when there are no more files left
>> and you can then suspend or stop [2] the route.
>>
>
> Yeah that is also a good solution. As the empty message will only be
> sent when there is no more files in a separate poll.
> The only trick parts is that your routing logic would need to cater
> for an "empty message" being routed.
>
>> [1] http://camel.apache.org/file2.html
>> [2] http://camel.apache.org/how-can-i-stop-a-route-from-a-route.html
>>
>> Best regards
>> Pontus Ullgren
>>
>> On Sat, Aug 25, 2012 at 7:13 PM, Ashwin Pejavar
>> <as...@gmail.com> wrote:
>>> I have a use-case where I need to consume files at a fixed cron schedule.
>>>
>>> For example, I need my input route to resume everyday at 6AM, process all
>>> files collected in the input directory and go into suspension till 6AM the
>>> following day.
>>>
>>> I was hoping the CronScheduledRoutePolicy would be useful, but that doesn't
>>> quite do what I need. I can use it to start the route at 6AM, but there
>>> seems to be no way of automatically suspending it when all the files are
>>> processed.
>>>
>>> Is there a way of doing this in Camel?
>>>
>>>
>>>
>>> --
>>> View this message in context: http://camel.465427.n5.nabble.com/Periodic-route-activation-use-case-tp5718059.html
>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
>
> --
> Claus Ibsen
> -----------------
> FuseSource
> Email: cibsen@fusesource.com
> Web: http://fusesource.com
> Twitter: davsclaus, fusenews
> Blog: http://davsclaus.com
> Author of Camel in Action: http://www.manning.com/ibsen
Re: Periodic route activation use case
Posted by Claus Ibsen <cl...@gmail.com>.
On Sun, Aug 26, 2012 at 9:14 AM, Pontus Ullgren <ul...@gmail.com> wrote:
> Hello,
>
> You can combine the CronScheduledRoutePolicy with the use of the
> sendEmptyMessageWhenIdle option of the file component [1].
> You will then get a empty message when there are no more files left
> and you can then suspend or stop [2] the route.
>
Yeah that is also a good solution. As the empty message will only be
sent when there is no more files in a separate poll.
The only trick parts is that your routing logic would need to cater
for an "empty message" being routed.
> [1] http://camel.apache.org/file2.html
> [2] http://camel.apache.org/how-can-i-stop-a-route-from-a-route.html
>
> Best regards
> Pontus Ullgren
>
> On Sat, Aug 25, 2012 at 7:13 PM, Ashwin Pejavar
> <as...@gmail.com> wrote:
>> I have a use-case where I need to consume files at a fixed cron schedule.
>>
>> For example, I need my input route to resume everyday at 6AM, process all
>> files collected in the input directory and go into suspension till 6AM the
>> following day.
>>
>> I was hoping the CronScheduledRoutePolicy would be useful, but that doesn't
>> quite do what I need. I can use it to start the route at 6AM, but there
>> seems to be no way of automatically suspending it when all the files are
>> processed.
>>
>> Is there a way of doing this in Camel?
>>
>>
>>
>> --
>> View this message in context: http://camel.465427.n5.nabble.com/Periodic-route-activation-use-case-tp5718059.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
Re: Periodic route activation use case
Posted by Pontus Ullgren <ul...@gmail.com>.
Hello,
You can combine the CronScheduledRoutePolicy with the use of the
sendEmptyMessageWhenIdle option of the file component [1].
You will then get a empty message when there are no more files left
and you can then suspend or stop [2] the route.
[1] http://camel.apache.org/file2.html
[2] http://camel.apache.org/how-can-i-stop-a-route-from-a-route.html
Best regards
Pontus Ullgren
On Sat, Aug 25, 2012 at 7:13 PM, Ashwin Pejavar
<as...@gmail.com> wrote:
> I have a use-case where I need to consume files at a fixed cron schedule.
>
> For example, I need my input route to resume everyday at 6AM, process all
> files collected in the input directory and go into suspension till 6AM the
> following day.
>
> I was hoping the CronScheduledRoutePolicy would be useful, but that doesn't
> quite do what I need. I can use it to start the route at 6AM, but there
> seems to be no way of automatically suspending it when all the files are
> processed.
>
> Is there a way of doing this in Camel?
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Periodic-route-activation-use-case-tp5718059.html
> Sent from the Camel - Users mailing list archive at Nabble.com.