You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Chris Wolf <cw...@gmail.com> on 2013/03/02 00:15:59 UTC
How to schedule FTP with quartz?
I have a requirement to download files via FTP during a certain time
window and according to a schedule. e.g. Only on trading days between
6:30AM and 7:00AM. The FTP component, alone, seems to just do
indefinite polling according to delay/initialDelay.
>From the "Camel In Action" book, chapter 7, I see some examples of
sending a text message with the Timer and Quartz components, but I
can't quite see how to put that together to implement "kicking off
routes at specified intervals", mentioned in the best practices list
at the end of that chapter. How would I use quartz to stop/start the
FTP component, or the route that it's in?
Thanks,
Chris
Re: How to schedule FTP with quartz?
Posted by Chris Wolf <cw...@gmail.com>.
Pontus,
Ok, I got rid of my RoutePolicy and put back the
CronScheduledRoutePolicy and provided both start and resume
cron schedules as you suggested, but CronScheduledRoutePolicy has the
same issue as my RoutePolicy - suspending
the FTPConsumer does NOT stop it from continuing to poll!
56:30,001 CronScheduledRoutePolicy DEBUG Suspended consumer
FtpConsumer[ftp://localhost/download?filter=%23cpmdFileFilter&noop=true&password=******&username=adpt5]
56:35,100 FtpConsumer WARN Cannot connect/login
to: ftp://adpt5@localhost:21. Will skip this poll.
56:35,100 FtpConsumer DEBUG Skipping poll as pre
poll check returned false
-Chris
On Tue, Mar 26, 2013 at 4:12 PM, Pontus Ullgren <ul...@gmail.com> wrote:
> Chris,
>
> Is there no way for you to calculate a cron expression for when the
> suspend should occure ?
>
> Say that you want the route to start every 10 minutes (starting at 0)
> and run for 5 minutes then suspend.
> This would mean that you should be able to define the start/resume
> cron as "0 0,10,20,30,40,50 * * * * ?".
> And the suspend cron expression as "0 5,15,25,35,45,55 * * * * ?".
>
> The CronScheduledRoutePolicy will trigger a start/resum at 00:00:00
> and then a suspend at 00:05:00, then a new start/resume at 00:10:00
> and a new suspends at 00:15:00 and so on.
>
> // Pontus
>
> On Tue, Mar 26, 2013 at 7:39 PM, Chris Wolf <cw...@gmail.com> wrote:
>> Pontus,
>>
>> I actually gave up on CronScheduledRoutePolicy because I don't want to
>> have to calculate an absolute suspend time based on the start/resume
>> time. What I need is a cron-based policy that specifies the initial
>> start time, which there-after becomes the resume time - this is a cron
>> expression - then I just want a relative run duration, after which,
>> the route is suspended until the next cron-specified resume time.
>>
>> So, after a few days, I finally have that and it works in the unit
>> test - even the initial one-off route start will transparently switch
>> to a resume schedule. The only problem is that I really need this
>> policy to control a route with an FTP consumer. What I'm seeing is
>> that
>> even though the route is suspended, the FTP consumer continues to poll
>> - this partially answers my question about the coding of
>> ScheduleRoutePolicy - which only suspends the consumer - not the
>> route, itself. While my policy suspends the route.
>>
>> I suspect that any route which has a component using
>> PollingConsumerPollStrategy will not behave as I though - which is -
>> you only need to suspend the route and all it's components will be
>> suspended. My suspicion is that components whose Consumers are under
>> the influence of PollingConsumerPollStrategy may not suspend by only
>> suspended the route.
>>
>> -Chris
>>
>>
>> On Sat, Mar 23, 2013 at 3:15 AM, Pontus Ullgren <ul...@gmail.com> wrote:
>>> This is probably because your route is autoStart=false. So you the first
>>> time you need to start it. In my example you see I set both the start and
>>> resume schedule to the same cron expression. So the policy will trigger
>>> both a start and a resume action.
>>>
>>> And you will get a WARN log since the first time it can not resume (but it
>>> will start) and after that it can not start but it will resume.
>>>
>>> Perhaps if you share some code it would be easier to help you.
>>> //Pontus
>>> On 22 Mar 2013 22:06, "Chris Wolf" <cw...@gmail.com> wrote:
>>>
>>>> I found the issue with my custom CronScheduledRoutePolicy - initially
>>>> the startTime/resumeTime are only scheduled in
>>>> onInit() - so to re-resume (re-start), you need to call
>>>> scheduleRoute(Action.RESUME, route); in onStart()
>>>>
>>>> ...but now I'm getting:
>>>>
>>>> quartz.ScheduledRoutePolicy WARN Route is not in a started state and
>>>> cannot be resumed. The current route state is Suspended
>>>>
>>>> What is the deal? I thought resumeRoute was the inverse of
>>>> suspenRoute, but this log message seems to indicate that
>>>> calling CamelContext.suspendRoute(routeId) will put the route into a
>>>> state that cannot be resumed.
>>>>
>>>> Thanks,
>>>>
>>>>
>>>> Chris
>>>>
>>>> On Fri, Mar 22, 2013 at 4:04 PM, Chris Wolf <cw...@gmail.com> wrote:
>>>> > Pontus,
>>>> >
>>>> > Thanks for that. Since I already has started implementing a class
>>>> > derived from CronScheduledRoutePolicy, I just finished it.
>>>> > It works by starting a Timer thread in onStart/onResume at the end of
>>>> > the time period, the route is suspended, but then upon
>>>> > the next schedule cron start time, I don't see it being resumed - I
>>>> > wonder if the RoutePolicy itself is being suspend too?
>>>> >
>>>> > Well, I try it your way also.
>>>> >
>>>> >
>>>> > Thanks,
>>>> >
>>>> >
>>>> > Chris
>>>> >
>>>> > On Wed, Mar 20, 2013 at 4:34 AM, Pontus Ullgren <ul...@gmail.com>
>>>> wrote:
>>>> >> Hello,
>>>> >>
>>>> >> On Tue, Mar 19, 2013 at 11:22 PM, Chris Wolf <cw...@gmail.com>
>>>> wrote:
>>>> >>> On Mon, Mar 18, 2013 at 4:57 PM, Pontus Ullgren <ul...@gmail.com>
>>>> wrote:
>>>> >>>> Hello Chris,
>>>> >>>>
>>>> >>>> On Mon, Mar 18, 2013 at 8:54 PM, Chris Wolf <cw...@gmail.com>
>>>> wrote:
>>>> >>>>> Claus,
>>>> >>>>>
>>>> >>>>> I have a few further questions about CronScheduledRoutePolicy. I
>>>> >>>>> noticed that it has setters such as setRouteStartTime,
>>>> >>>>> setRouteStopTime, each which takes a cron expression string. What
>>>> I'm
>>>> >>>>> looking for is to be able to use a cron expression for the start, but
>>>> >>>>> a relative time length for stop. Otherwise, I need to write code to
>>>> >>>>> parse the start time expression, then calculate a stop time cron
>>>> >>>>> expression. Any ideas?
>>>> >>>>>
>>>> >>>> Depending on your needs you could enable "sendEmptyMessageWhenIdle" on
>>>> >>>> the endpoint and then suspend the route when you receive a empty
>>>> >>>> message. Which means that there is no more files to poll at the
>>>> >>>> moment.
>>>> >>>> You can use the content based route EIP for this.
>>>> >>>
>>>> >>> That is interesting to know, thanks. In my case, the files at the
>>>> >>> remote end are themselves deposited at an irregular rate, but within a
>>>> >>> defined time window, so during that time window, there will be
>>>> >>> intermittent idleness...
>>>> >>>>
>>>> >>>> Another solution would be to write your own RoutePolicy to take care
>>>> >>>> of your needs.
>>>> >>>
>>>> >>> Yes, this sounds like the best approach...
>>>> >>>
>>>> >>>
>>>> >>>>
>>>> >>>> I just started to wonder if it might be possible to combine the
>>>> >>>> CronScheduledRoutePolicy with a SimpleScheduledRoutePolicy.
>>>> >>>> I have _not_ tested this so I'm not sure if it works. It might be that
>>>> >>>> there is a collision in the way they work with Quartz.
>>>> >>>>
>>>> >>>>> Also I see that CronScheduledRoutePolicy has setRouteResumeTime,
>>>> >>>>> setRouteSuspendTime such that for my FTP poll window, I could either
>>>> >>>>> do start/stop or resume/suspend - which is recommended?
>>>> >>>>>
>>>> >>>> I would highly recommend resume/suspend.
>>>> >>>> I've had some thread leak problem with the file component when it was
>>>> >>>> repetitively started/stopped.
>>>> >>>
>>>> >>> Ok, but I guess the first policy callback with be onStart since the
>>>> >>> route will be
>>>> >>> configured with noAutoStartup(), so upon that first onStart, I'll
>>>> >>> suspend then toggle
>>>> >>> between onSuspend/onResume...
>>>> >>>
>>>> >> Yes this is what I do. We have a route that should start 06:30 each
>>>> >> day and then poll all files that are in the folder at that time.
>>>> >> After that it should suspend.
>>>> >>
>>>> >> Here is some pseudo code.
>>>> >> SuspendRouteProcessor is a processor that suspends the route based on
>>>> route id.
>>>> >> ---
>>>> >> String cronStr = "* 30 6 * * * ?";
>>>> >> String input = "ftp://user@remotehost
>>>> /inbox?sendEmptyMessageWhenIdle=true&password=secret";
>>>> >> CronScheduledRoutePolicy scheduledRP = new CronScheduledRoutePolicy();
>>>> >> scheduledRoutePolicy.setRouteStartTime(cronStr);
>>>> >> scheduledRoutePolicy.setRouteResumeTime(cronStr);
>>>> >>
>>>> >> from(input)
>>>> >> .routeId("input1")
>>>> >> .routePolicy(versionPolicy, scheduledRoutePolicy)
>>>> >> .noAutoStartup()
>>>> >> .choice()
>>>> >> .when(body().isNotNull())
>>>> >> .to("direct:processFiles")
>>>> >> .end()
>>>> >> .log(LoggingLevel.DEBUG, "All files processed, suspend
>>>> route")
>>>> >> .process(new SuspendRouteProcessor("input1"))
>>>> >> ;
>>>> >> --
>>>> >>
>>>> >> The only downside with this is that after the initial start we get a
>>>> >> WARN log message that the route can not be started since it is in
>>>> >> suspend state.
>>>> >> But as long as you can live with the WARN log it works.
>>>> >>
>>>> >>>>
>>>> >>>> // Pontus
>>>> >>>>
>>>> >>>>> Thanks,
>>>> >>>>>
>>>> >>>>> Chris
>>>> >>>>>
>>>> >>>>> On Sat, Mar 2, 2013 at 1:43 AM, Claus Ibsen <cl...@gmail.com>
>>>> wrote:
>>>> >>>>>> Hi
>>>> >>>>>>
>>>> >>>>>> See about route policy
>>>> >>>>>> http://camel.apache.org/routepolicy
>>>> >>>>>>
>>>> >>>>>> And the scheduled route policy
>>>> >>>>>> http://camel.apache.org/scheduledroutepolicy.html
>>>> >>>>>>
>>>> >>>>>> On Sat, Mar 2, 2013 at 12:15 AM, Chris Wolf <cw...@gmail.com>
>>>> wrote:
>>>> >>>>>>> I have a requirement to download files via FTP during a certain
>>>> time
>>>> >>>>>>> window and according to a schedule. e.g. Only on trading days
>>>> between
>>>> >>>>>>> 6:30AM and 7:00AM. The FTP component, alone, seems to just do
>>>> >>>>>>> indefinite polling according to delay/initialDelay.
>>>> >>>>>>>
>>>> >>>>>>> From the "Camel In Action" book, chapter 7, I see some examples of
>>>> >>>>>>> sending a text message with the Timer and Quartz components, but I
>>>> >>>>>>> can't quite see how to put that together to implement "kicking off
>>>> >>>>>>> routes at specified intervals", mentioned in the best practices
>>>> list
>>>> >>>>>>> at the end of that chapter. How would I use quartz to stop/start
>>>> the
>>>> >>>>>>> FTP component, or the route that it's in?
>>>> >>>>>>>
>>>> >>>>>>> Thanks,
>>>> >>>>>>>
>>>> >>>>>>> Chris
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>> --
>>>> >>>>>> Claus Ibsen
>>>> >>>>>> -----------------
>>>> >>>>>> Red Hat, Inc.
>>>> >>>>>> FuseSource is now part of Red Hat
>>>> >>>>>> Email: cibsen@redhat.com
>>>> >>>>>> Web: http://fusesource.com
>>>> >>>>>> Twitter: davsclaus
>>>> >>>>>> Blog: http://davsclaus.com
>>>> >>>>>> Author of Camel in Action: http://www.manning.com/ibsen
>>>>
Re: How to schedule FTP with quartz?
Posted by Pontus Ullgren <ul...@gmail.com>.
Glad to hear that you solved it.
If possible, I would not mind having a peek at the code for your new policy.
//Pontus
On 27 Mar 2013 04:21, "Chris Wolf" <cw...@gmail.com> wrote:
> Yes, and after Raul's clarification in his response this evening
> (GMT-4), I changed my RoutePolicy implementation
> and now it works!
>
> For example, to kick off a cron job at 0615 daily, with a run duration
> of 30 minutes, you would do:
>
> CronRoutePolicy ftpPolicy = new CronRoutePolicy(TimeUnit.MINUTES);
> ftpPolicy.setRouteStartTime("0 15 6 * * ?");
> ftpPolicy.setPollWindowTime(30);
>
> This implementation uses the same Quartz scheduler instance as
> ScheduledRoutePolicy, but with
> a different job/trigger naming convention, so there's no duplication
> and (hopefully) no conflicts.
>
> I appreciate your empty message solution, however, in my case, the
> remote files will intermittently appear
> during the 30 minute window time, so we have to keep trying, even
> after getting some files.
>
> On Tue, Mar 26, 2013 at 4:43 PM, Pontus Ullgren <ul...@gmail.com> wrote:
> > Sorry read your message a bit to quick.
> >
> > If you do not want to calculate the suspend cron expression I guess
> > you need to mimic the behaviour of
> > org.apache.camel.routepolicy.quartz.ScheduledRoutePolicy where (as you
> > point out in another email) the consumer is suspended and not the
> > route.
> >
> > I'm sorry can not explain why it's implemented this way, I just know
> > that in the case of the existing scheduled route policies it seem to
> > work as expected. :-)
> > Hopefully somebody with more knowledge of the in and outs of Camel can
> > explain this to us both.
> >
> > // Pontus
> >
> >
> >
> > On Tue, Mar 26, 2013 at 9:12 PM, Pontus Ullgren <ul...@gmail.com>
> wrote:
> >> Chris,
> >>
> >> Is there no way for you to calculate a cron expression for when the
> >> suspend should occure ?
> >>
> >> Say that you want the route to start every 10 minutes (starting at 0)
> >> and run for 5 minutes then suspend.
> >> This would mean that you should be able to define the start/resume
> >> cron as "0 0,10,20,30,40,50 * * * * ?".
> >> And the suspend cron expression as "0 5,15,25,35,45,55 * * * * ?".
> >>
> >> The CronScheduledRoutePolicy will trigger a start/resum at 00:00:00
> >> and then a suspend at 00:05:00, then a new start/resume at 00:10:00
> >> and a new suspends at 00:15:00 and so on.
> >>
> >> // Pontus
> >>
> >> On Tue, Mar 26, 2013 at 7:39 PM, Chris Wolf <cw...@gmail.com>
> wrote:
> >>> Pontus,
> >>>
> >>> I actually gave up on CronScheduledRoutePolicy because I don't want to
> >>> have to calculate an absolute suspend time based on the start/resume
> >>> time. What I need is a cron-based policy that specifies the initial
> >>> start time, which there-after becomes the resume time - this is a cron
> >>> expression - then I just want a relative run duration, after which,
> >>> the route is suspended until the next cron-specified resume time.
> >>>
> >>> So, after a few days, I finally have that and it works in the unit
> >>> test - even the initial one-off route start will transparently switch
> >>> to a resume schedule. The only problem is that I really need this
> >>> policy to control a route with an FTP consumer. What I'm seeing is
> >>> that
> >>> even though the route is suspended, the FTP consumer continues to poll
> >>> - this partially answers my question about the coding of
> >>> ScheduleRoutePolicy - which only suspends the consumer - not the
> >>> route, itself. While my policy suspends the route.
> >>>
> >>> I suspect that any route which has a component using
> >>> PollingConsumerPollStrategy will not behave as I though - which is -
> >>> you only need to suspend the route and all it's components will be
> >>> suspended. My suspicion is that components whose Consumers are under
> >>> the influence of PollingConsumerPollStrategy may not suspend by only
> >>> suspended the route.
> >>>
> >>> -Chris
> >>>
> >>>
> >>> On Sat, Mar 23, 2013 at 3:15 AM, Pontus Ullgren <ul...@gmail.com>
> wrote:
> >>>> This is probably because your route is autoStart=false. So you the
> first
> >>>> time you need to start it. In my example you see I set both the start
> and
> >>>> resume schedule to the same cron expression. So the policy will
> trigger
> >>>> both a start and a resume action.
> >>>>
> >>>> And you will get a WARN log since the first time it can not resume
> (but it
> >>>> will start) and after that it can not start but it will resume.
> >>>>
> >>>> Perhaps if you share some code it would be easier to help you.
> >>>> //Pontus
> >>>> On 22 Mar 2013 22:06, "Chris Wolf" <cw...@gmail.com> wrote:
> >>>>
> >>>>> I found the issue with my custom CronScheduledRoutePolicy - initially
> >>>>> the startTime/resumeTime are only scheduled in
> >>>>> onInit() - so to re-resume (re-start), you need to call
> >>>>> scheduleRoute(Action.RESUME, route); in onStart()
> >>>>>
> >>>>> ...but now I'm getting:
> >>>>>
> >>>>> quartz.ScheduledRoutePolicy WARN Route is not in a started state and
> >>>>> cannot be resumed. The current route state is Suspended
> >>>>>
> >>>>> What is the deal? I thought resumeRoute was the inverse of
> >>>>> suspenRoute, but this log message seems to indicate that
> >>>>> calling CamelContext.suspendRoute(routeId) will put the route into a
> >>>>> state that cannot be resumed.
> >>>>>
> >>>>> Thanks,
> >>>>>
> >>>>>
> >>>>> Chris
> >>>>>
> >>>>> On Fri, Mar 22, 2013 at 4:04 PM, Chris Wolf <cw...@gmail.com>
> wrote:
> >>>>> > Pontus,
> >>>>> >
> >>>>> > Thanks for that. Since I already has started implementing a class
> >>>>> > derived from CronScheduledRoutePolicy, I just finished it.
> >>>>> > It works by starting a Timer thread in onStart/onResume at the end
> of
> >>>>> > the time period, the route is suspended, but then upon
> >>>>> > the next schedule cron start time, I don't see it being resumed - I
> >>>>> > wonder if the RoutePolicy itself is being suspend too?
> >>>>> >
> >>>>> > Well, I try it your way also.
> >>>>> >
> >>>>> >
> >>>>> > Thanks,
> >>>>> >
> >>>>> >
> >>>>> > Chris
> >>>>> >
> >>>>> > On Wed, Mar 20, 2013 at 4:34 AM, Pontus Ullgren <ullgren@gmail.com
> >
> >>>>> wrote:
> >>>>> >> Hello,
> >>>>> >>
> >>>>> >> On Tue, Mar 19, 2013 at 11:22 PM, Chris Wolf <
> cwolf.algo@gmail.com>
> >>>>> wrote:
> >>>>> >>> On Mon, Mar 18, 2013 at 4:57 PM, Pontus Ullgren <
> ullgren@gmail.com>
> >>>>> wrote:
> >>>>> >>>> Hello Chris,
> >>>>> >>>>
> >>>>> >>>> On Mon, Mar 18, 2013 at 8:54 PM, Chris Wolf <
> cwolf.algo@gmail.com>
> >>>>> wrote:
> >>>>> >>>>> Claus,
> >>>>> >>>>>
> >>>>> >>>>> I have a few further questions about CronScheduledRoutePolicy.
> I
> >>>>> >>>>> noticed that it has setters such as setRouteStartTime,
> >>>>> >>>>> setRouteStopTime, each which takes a cron expression string.
> What
> >>>>> I'm
> >>>>> >>>>> looking for is to be able to use a cron expression for the
> start, but
> >>>>> >>>>> a relative time length for stop. Otherwise, I need to write
> code to
> >>>>> >>>>> parse the start time expression, then calculate a stop time
> cron
> >>>>> >>>>> expression. Any ideas?
> >>>>> >>>>>
> >>>>> >>>> Depending on your needs you could enable
> "sendEmptyMessageWhenIdle" on
> >>>>> >>>> the endpoint and then suspend the route when you receive a empty
> >>>>> >>>> message. Which means that there is no more files to poll at the
> >>>>> >>>> moment.
> >>>>> >>>> You can use the content based route EIP for this.
> >>>>> >>>
> >>>>> >>> That is interesting to know, thanks. In my case, the files at
> the
> >>>>> >>> remote end are themselves deposited at an irregular rate, but
> within a
> >>>>> >>> defined time window, so during that time window, there will be
> >>>>> >>> intermittent idleness...
> >>>>> >>>>
> >>>>> >>>> Another solution would be to write your own RoutePolicy to take
> care
> >>>>> >>>> of your needs.
> >>>>> >>>
> >>>>> >>> Yes, this sounds like the best approach...
> >>>>> >>>
> >>>>> >>>
> >>>>> >>>>
> >>>>> >>>> I just started to wonder if it might be possible to combine the
> >>>>> >>>> CronScheduledRoutePolicy with a SimpleScheduledRoutePolicy.
> >>>>> >>>> I have _not_ tested this so I'm not sure if it works. It might
> be that
> >>>>> >>>> there is a collision in the way they work with Quartz.
> >>>>> >>>>
> >>>>> >>>>> Also I see that CronScheduledRoutePolicy has
> setRouteResumeTime,
> >>>>> >>>>> setRouteSuspendTime such that for my FTP poll window, I could
> either
> >>>>> >>>>> do start/stop or resume/suspend - which is recommended?
> >>>>> >>>>>
> >>>>> >>>> I would highly recommend resume/suspend.
> >>>>> >>>> I've had some thread leak problem with the file component when
> it was
> >>>>> >>>> repetitively started/stopped.
> >>>>> >>>
> >>>>> >>> Ok, but I guess the first policy callback with be onStart since
> the
> >>>>> >>> route will be
> >>>>> >>> configured with noAutoStartup(), so upon that first onStart, I'll
> >>>>> >>> suspend then toggle
> >>>>> >>> between onSuspend/onResume...
> >>>>> >>>
> >>>>> >> Yes this is what I do. We have a route that should start 06:30
> each
> >>>>> >> day and then poll all files that are in the folder at that time.
> >>>>> >> After that it should suspend.
> >>>>> >>
> >>>>> >> Here is some pseudo code.
> >>>>> >> SuspendRouteProcessor is a processor that suspends the route
> based on
> >>>>> route id.
> >>>>> >> ---
> >>>>> >> String cronStr = "* 30 6 * * * ?";
> >>>>> >> String input = "ftp://user@remotehost
> >>>>> /inbox?sendEmptyMessageWhenIdle=true&password=secret";
> >>>>> >> CronScheduledRoutePolicy scheduledRP = new
> CronScheduledRoutePolicy();
> >>>>> >> scheduledRoutePolicy.setRouteStartTime(cronStr);
> >>>>> >> scheduledRoutePolicy.setRouteResumeTime(cronStr);
> >>>>> >>
> >>>>> >> from(input)
> >>>>> >> .routeId("input1")
> >>>>> >> .routePolicy(versionPolicy, scheduledRoutePolicy)
> >>>>> >> .noAutoStartup()
> >>>>> >> .choice()
> >>>>> >> .when(body().isNotNull())
> >>>>> >> .to("direct:processFiles")
> >>>>> >> .end()
> >>>>> >> .log(LoggingLevel.DEBUG, "All files processed,
> suspend
> >>>>> route")
> >>>>> >> .process(new SuspendRouteProcessor("input1"))
> >>>>> >> ;
> >>>>> >> --
> >>>>> >>
> >>>>> >> The only downside with this is that after the initial start we
> get a
> >>>>> >> WARN log message that the route can not be started since it is in
> >>>>> >> suspend state.
> >>>>> >> But as long as you can live with the WARN log it works.
> >>>>> >>
> >>>>> >>>>
> >>>>> >>>> // Pontus
> >>>>> >>>>
> >>>>> >>>>> Thanks,
> >>>>> >>>>>
> >>>>> >>>>> Chris
> >>>>> >>>>>
> >>>>> >>>>> On Sat, Mar 2, 2013 at 1:43 AM, Claus Ibsen <
> claus.ibsen@gmail.com>
> >>>>> wrote:
> >>>>> >>>>>> Hi
> >>>>> >>>>>>
> >>>>> >>>>>> See about route policy
> >>>>> >>>>>> http://camel.apache.org/routepolicy
> >>>>> >>>>>>
> >>>>> >>>>>> And the scheduled route policy
> >>>>> >>>>>> http://camel.apache.org/scheduledroutepolicy.html
> >>>>> >>>>>>
> >>>>> >>>>>> On Sat, Mar 2, 2013 at 12:15 AM, Chris Wolf <
> cwolf.algo@gmail.com>
> >>>>> wrote:
> >>>>> >>>>>>> I have a requirement to download files via FTP during a
> certain
> >>>>> time
> >>>>> >>>>>>> window and according to a schedule. e.g. Only on trading days
> >>>>> between
> >>>>> >>>>>>> 6:30AM and 7:00AM. The FTP component, alone, seems to just
> do
> >>>>> >>>>>>> indefinite polling according to delay/initialDelay.
> >>>>> >>>>>>>
> >>>>> >>>>>>> From the "Camel In Action" book, chapter 7, I see some
> examples of
> >>>>> >>>>>>> sending a text message with the Timer and Quartz components,
> but I
> >>>>> >>>>>>> can't quite see how to put that together to implement
> "kicking off
> >>>>> >>>>>>> routes at specified intervals", mentioned in the best
> practices
> >>>>> list
> >>>>> >>>>>>> at the end of that chapter. How would I use quartz to
> stop/start
> >>>>> the
> >>>>> >>>>>>> FTP component, or the route that it's in?
> >>>>> >>>>>>>
> >>>>> >>>>>>> Thanks,
> >>>>> >>>>>>>
> >>>>> >>>>>>> Chris
> >>>>> >>>>>>
> >>>>> >>>>>>
> >>>>> >>>>>>
> >>>>> >>>>>> --
> >>>>> >>>>>> Claus Ibsen
> >>>>> >>>>>> -----------------
> >>>>> >>>>>> Red Hat, Inc.
> >>>>> >>>>>> FuseSource is now part of Red Hat
> >>>>> >>>>>> Email: cibsen@redhat.com
> >>>>> >>>>>> Web: http://fusesource.com
> >>>>> >>>>>> Twitter: davsclaus
> >>>>> >>>>>> Blog: http://davsclaus.com
> >>>>> >>>>>> Author of Camel in Action: http://www.manning.com/ibsen
> >>>>>
>
Re: How to schedule FTP with quartz?
Posted by Chris Wolf <cw...@gmail.com>.
Yes, and after Raul's clarification in his response this evening
(GMT-4), I changed my RoutePolicy implementation
and now it works!
For example, to kick off a cron job at 0615 daily, with a run duration
of 30 minutes, you would do:
CronRoutePolicy ftpPolicy = new CronRoutePolicy(TimeUnit.MINUTES);
ftpPolicy.setRouteStartTime("0 15 6 * * ?");
ftpPolicy.setPollWindowTime(30);
This implementation uses the same Quartz scheduler instance as
ScheduledRoutePolicy, but with
a different job/trigger naming convention, so there's no duplication
and (hopefully) no conflicts.
I appreciate your empty message solution, however, in my case, the
remote files will intermittently appear
during the 30 minute window time, so we have to keep trying, even
after getting some files.
On Tue, Mar 26, 2013 at 4:43 PM, Pontus Ullgren <ul...@gmail.com> wrote:
> Sorry read your message a bit to quick.
>
> If you do not want to calculate the suspend cron expression I guess
> you need to mimic the behaviour of
> org.apache.camel.routepolicy.quartz.ScheduledRoutePolicy where (as you
> point out in another email) the consumer is suspended and not the
> route.
>
> I'm sorry can not explain why it's implemented this way, I just know
> that in the case of the existing scheduled route policies it seem to
> work as expected. :-)
> Hopefully somebody with more knowledge of the in and outs of Camel can
> explain this to us both.
>
> // Pontus
>
>
>
> On Tue, Mar 26, 2013 at 9:12 PM, Pontus Ullgren <ul...@gmail.com> wrote:
>> Chris,
>>
>> Is there no way for you to calculate a cron expression for when the
>> suspend should occure ?
>>
>> Say that you want the route to start every 10 minutes (starting at 0)
>> and run for 5 minutes then suspend.
>> This would mean that you should be able to define the start/resume
>> cron as "0 0,10,20,30,40,50 * * * * ?".
>> And the suspend cron expression as "0 5,15,25,35,45,55 * * * * ?".
>>
>> The CronScheduledRoutePolicy will trigger a start/resum at 00:00:00
>> and then a suspend at 00:05:00, then a new start/resume at 00:10:00
>> and a new suspends at 00:15:00 and so on.
>>
>> // Pontus
>>
>> On Tue, Mar 26, 2013 at 7:39 PM, Chris Wolf <cw...@gmail.com> wrote:
>>> Pontus,
>>>
>>> I actually gave up on CronScheduledRoutePolicy because I don't want to
>>> have to calculate an absolute suspend time based on the start/resume
>>> time. What I need is a cron-based policy that specifies the initial
>>> start time, which there-after becomes the resume time - this is a cron
>>> expression - then I just want a relative run duration, after which,
>>> the route is suspended until the next cron-specified resume time.
>>>
>>> So, after a few days, I finally have that and it works in the unit
>>> test - even the initial one-off route start will transparently switch
>>> to a resume schedule. The only problem is that I really need this
>>> policy to control a route with an FTP consumer. What I'm seeing is
>>> that
>>> even though the route is suspended, the FTP consumer continues to poll
>>> - this partially answers my question about the coding of
>>> ScheduleRoutePolicy - which only suspends the consumer - not the
>>> route, itself. While my policy suspends the route.
>>>
>>> I suspect that any route which has a component using
>>> PollingConsumerPollStrategy will not behave as I though - which is -
>>> you only need to suspend the route and all it's components will be
>>> suspended. My suspicion is that components whose Consumers are under
>>> the influence of PollingConsumerPollStrategy may not suspend by only
>>> suspended the route.
>>>
>>> -Chris
>>>
>>>
>>> On Sat, Mar 23, 2013 at 3:15 AM, Pontus Ullgren <ul...@gmail.com> wrote:
>>>> This is probably because your route is autoStart=false. So you the first
>>>> time you need to start it. In my example you see I set both the start and
>>>> resume schedule to the same cron expression. So the policy will trigger
>>>> both a start and a resume action.
>>>>
>>>> And you will get a WARN log since the first time it can not resume (but it
>>>> will start) and after that it can not start but it will resume.
>>>>
>>>> Perhaps if you share some code it would be easier to help you.
>>>> //Pontus
>>>> On 22 Mar 2013 22:06, "Chris Wolf" <cw...@gmail.com> wrote:
>>>>
>>>>> I found the issue with my custom CronScheduledRoutePolicy - initially
>>>>> the startTime/resumeTime are only scheduled in
>>>>> onInit() - so to re-resume (re-start), you need to call
>>>>> scheduleRoute(Action.RESUME, route); in onStart()
>>>>>
>>>>> ...but now I'm getting:
>>>>>
>>>>> quartz.ScheduledRoutePolicy WARN Route is not in a started state and
>>>>> cannot be resumed. The current route state is Suspended
>>>>>
>>>>> What is the deal? I thought resumeRoute was the inverse of
>>>>> suspenRoute, but this log message seems to indicate that
>>>>> calling CamelContext.suspendRoute(routeId) will put the route into a
>>>>> state that cannot be resumed.
>>>>>
>>>>> Thanks,
>>>>>
>>>>>
>>>>> Chris
>>>>>
>>>>> On Fri, Mar 22, 2013 at 4:04 PM, Chris Wolf <cw...@gmail.com> wrote:
>>>>> > Pontus,
>>>>> >
>>>>> > Thanks for that. Since I already has started implementing a class
>>>>> > derived from CronScheduledRoutePolicy, I just finished it.
>>>>> > It works by starting a Timer thread in onStart/onResume at the end of
>>>>> > the time period, the route is suspended, but then upon
>>>>> > the next schedule cron start time, I don't see it being resumed - I
>>>>> > wonder if the RoutePolicy itself is being suspend too?
>>>>> >
>>>>> > Well, I try it your way also.
>>>>> >
>>>>> >
>>>>> > Thanks,
>>>>> >
>>>>> >
>>>>> > Chris
>>>>> >
>>>>> > On Wed, Mar 20, 2013 at 4:34 AM, Pontus Ullgren <ul...@gmail.com>
>>>>> wrote:
>>>>> >> Hello,
>>>>> >>
>>>>> >> On Tue, Mar 19, 2013 at 11:22 PM, Chris Wolf <cw...@gmail.com>
>>>>> wrote:
>>>>> >>> On Mon, Mar 18, 2013 at 4:57 PM, Pontus Ullgren <ul...@gmail.com>
>>>>> wrote:
>>>>> >>>> Hello Chris,
>>>>> >>>>
>>>>> >>>> On Mon, Mar 18, 2013 at 8:54 PM, Chris Wolf <cw...@gmail.com>
>>>>> wrote:
>>>>> >>>>> Claus,
>>>>> >>>>>
>>>>> >>>>> I have a few further questions about CronScheduledRoutePolicy. I
>>>>> >>>>> noticed that it has setters such as setRouteStartTime,
>>>>> >>>>> setRouteStopTime, each which takes a cron expression string. What
>>>>> I'm
>>>>> >>>>> looking for is to be able to use a cron expression for the start, but
>>>>> >>>>> a relative time length for stop. Otherwise, I need to write code to
>>>>> >>>>> parse the start time expression, then calculate a stop time cron
>>>>> >>>>> expression. Any ideas?
>>>>> >>>>>
>>>>> >>>> Depending on your needs you could enable "sendEmptyMessageWhenIdle" on
>>>>> >>>> the endpoint and then suspend the route when you receive a empty
>>>>> >>>> message. Which means that there is no more files to poll at the
>>>>> >>>> moment.
>>>>> >>>> You can use the content based route EIP for this.
>>>>> >>>
>>>>> >>> That is interesting to know, thanks. In my case, the files at the
>>>>> >>> remote end are themselves deposited at an irregular rate, but within a
>>>>> >>> defined time window, so during that time window, there will be
>>>>> >>> intermittent idleness...
>>>>> >>>>
>>>>> >>>> Another solution would be to write your own RoutePolicy to take care
>>>>> >>>> of your needs.
>>>>> >>>
>>>>> >>> Yes, this sounds like the best approach...
>>>>> >>>
>>>>> >>>
>>>>> >>>>
>>>>> >>>> I just started to wonder if it might be possible to combine the
>>>>> >>>> CronScheduledRoutePolicy with a SimpleScheduledRoutePolicy.
>>>>> >>>> I have _not_ tested this so I'm not sure if it works. It might be that
>>>>> >>>> there is a collision in the way they work with Quartz.
>>>>> >>>>
>>>>> >>>>> Also I see that CronScheduledRoutePolicy has setRouteResumeTime,
>>>>> >>>>> setRouteSuspendTime such that for my FTP poll window, I could either
>>>>> >>>>> do start/stop or resume/suspend - which is recommended?
>>>>> >>>>>
>>>>> >>>> I would highly recommend resume/suspend.
>>>>> >>>> I've had some thread leak problem with the file component when it was
>>>>> >>>> repetitively started/stopped.
>>>>> >>>
>>>>> >>> Ok, but I guess the first policy callback with be onStart since the
>>>>> >>> route will be
>>>>> >>> configured with noAutoStartup(), so upon that first onStart, I'll
>>>>> >>> suspend then toggle
>>>>> >>> between onSuspend/onResume...
>>>>> >>>
>>>>> >> Yes this is what I do. We have a route that should start 06:30 each
>>>>> >> day and then poll all files that are in the folder at that time.
>>>>> >> After that it should suspend.
>>>>> >>
>>>>> >> Here is some pseudo code.
>>>>> >> SuspendRouteProcessor is a processor that suspends the route based on
>>>>> route id.
>>>>> >> ---
>>>>> >> String cronStr = "* 30 6 * * * ?";
>>>>> >> String input = "ftp://user@remotehost
>>>>> /inbox?sendEmptyMessageWhenIdle=true&password=secret";
>>>>> >> CronScheduledRoutePolicy scheduledRP = new CronScheduledRoutePolicy();
>>>>> >> scheduledRoutePolicy.setRouteStartTime(cronStr);
>>>>> >> scheduledRoutePolicy.setRouteResumeTime(cronStr);
>>>>> >>
>>>>> >> from(input)
>>>>> >> .routeId("input1")
>>>>> >> .routePolicy(versionPolicy, scheduledRoutePolicy)
>>>>> >> .noAutoStartup()
>>>>> >> .choice()
>>>>> >> .when(body().isNotNull())
>>>>> >> .to("direct:processFiles")
>>>>> >> .end()
>>>>> >> .log(LoggingLevel.DEBUG, "All files processed, suspend
>>>>> route")
>>>>> >> .process(new SuspendRouteProcessor("input1"))
>>>>> >> ;
>>>>> >> --
>>>>> >>
>>>>> >> The only downside with this is that after the initial start we get a
>>>>> >> WARN log message that the route can not be started since it is in
>>>>> >> suspend state.
>>>>> >> But as long as you can live with the WARN log it works.
>>>>> >>
>>>>> >>>>
>>>>> >>>> // Pontus
>>>>> >>>>
>>>>> >>>>> Thanks,
>>>>> >>>>>
>>>>> >>>>> Chris
>>>>> >>>>>
>>>>> >>>>> On Sat, Mar 2, 2013 at 1:43 AM, Claus Ibsen <cl...@gmail.com>
>>>>> wrote:
>>>>> >>>>>> Hi
>>>>> >>>>>>
>>>>> >>>>>> See about route policy
>>>>> >>>>>> http://camel.apache.org/routepolicy
>>>>> >>>>>>
>>>>> >>>>>> And the scheduled route policy
>>>>> >>>>>> http://camel.apache.org/scheduledroutepolicy.html
>>>>> >>>>>>
>>>>> >>>>>> On Sat, Mar 2, 2013 at 12:15 AM, Chris Wolf <cw...@gmail.com>
>>>>> wrote:
>>>>> >>>>>>> I have a requirement to download files via FTP during a certain
>>>>> time
>>>>> >>>>>>> window and according to a schedule. e.g. Only on trading days
>>>>> between
>>>>> >>>>>>> 6:30AM and 7:00AM. The FTP component, alone, seems to just do
>>>>> >>>>>>> indefinite polling according to delay/initialDelay.
>>>>> >>>>>>>
>>>>> >>>>>>> From the "Camel In Action" book, chapter 7, I see some examples of
>>>>> >>>>>>> sending a text message with the Timer and Quartz components, but I
>>>>> >>>>>>> can't quite see how to put that together to implement "kicking off
>>>>> >>>>>>> routes at specified intervals", mentioned in the best practices
>>>>> list
>>>>> >>>>>>> at the end of that chapter. How would I use quartz to stop/start
>>>>> the
>>>>> >>>>>>> FTP component, or the route that it's in?
>>>>> >>>>>>>
>>>>> >>>>>>> Thanks,
>>>>> >>>>>>>
>>>>> >>>>>>> Chris
>>>>> >>>>>>
>>>>> >>>>>>
>>>>> >>>>>>
>>>>> >>>>>> --
>>>>> >>>>>> Claus Ibsen
>>>>> >>>>>> -----------------
>>>>> >>>>>> Red Hat, Inc.
>>>>> >>>>>> FuseSource is now part of Red Hat
>>>>> >>>>>> Email: cibsen@redhat.com
>>>>> >>>>>> Web: http://fusesource.com
>>>>> >>>>>> Twitter: davsclaus
>>>>> >>>>>> Blog: http://davsclaus.com
>>>>> >>>>>> Author of Camel in Action: http://www.manning.com/ibsen
>>>>>
Re: How to schedule FTP with quartz?
Posted by Pontus Ullgren <ul...@gmail.com>.
Sorry read your message a bit to quick.
If you do not want to calculate the suspend cron expression I guess
you need to mimic the behaviour of
org.apache.camel.routepolicy.quartz.ScheduledRoutePolicy where (as you
point out in another email) the consumer is suspended and not the
route.
I'm sorry can not explain why it's implemented this way, I just know
that in the case of the existing scheduled route policies it seem to
work as expected. :-)
Hopefully somebody with more knowledge of the in and outs of Camel can
explain this to us both.
// Pontus
On Tue, Mar 26, 2013 at 9:12 PM, Pontus Ullgren <ul...@gmail.com> wrote:
> Chris,
>
> Is there no way for you to calculate a cron expression for when the
> suspend should occure ?
>
> Say that you want the route to start every 10 minutes (starting at 0)
> and run for 5 minutes then suspend.
> This would mean that you should be able to define the start/resume
> cron as "0 0,10,20,30,40,50 * * * * ?".
> And the suspend cron expression as "0 5,15,25,35,45,55 * * * * ?".
>
> The CronScheduledRoutePolicy will trigger a start/resum at 00:00:00
> and then a suspend at 00:05:00, then a new start/resume at 00:10:00
> and a new suspends at 00:15:00 and so on.
>
> // Pontus
>
> On Tue, Mar 26, 2013 at 7:39 PM, Chris Wolf <cw...@gmail.com> wrote:
>> Pontus,
>>
>> I actually gave up on CronScheduledRoutePolicy because I don't want to
>> have to calculate an absolute suspend time based on the start/resume
>> time. What I need is a cron-based policy that specifies the initial
>> start time, which there-after becomes the resume time - this is a cron
>> expression - then I just want a relative run duration, after which,
>> the route is suspended until the next cron-specified resume time.
>>
>> So, after a few days, I finally have that and it works in the unit
>> test - even the initial one-off route start will transparently switch
>> to a resume schedule. The only problem is that I really need this
>> policy to control a route with an FTP consumer. What I'm seeing is
>> that
>> even though the route is suspended, the FTP consumer continues to poll
>> - this partially answers my question about the coding of
>> ScheduleRoutePolicy - which only suspends the consumer - not the
>> route, itself. While my policy suspends the route.
>>
>> I suspect that any route which has a component using
>> PollingConsumerPollStrategy will not behave as I though - which is -
>> you only need to suspend the route and all it's components will be
>> suspended. My suspicion is that components whose Consumers are under
>> the influence of PollingConsumerPollStrategy may not suspend by only
>> suspended the route.
>>
>> -Chris
>>
>>
>> On Sat, Mar 23, 2013 at 3:15 AM, Pontus Ullgren <ul...@gmail.com> wrote:
>>> This is probably because your route is autoStart=false. So you the first
>>> time you need to start it. In my example you see I set both the start and
>>> resume schedule to the same cron expression. So the policy will trigger
>>> both a start and a resume action.
>>>
>>> And you will get a WARN log since the first time it can not resume (but it
>>> will start) and after that it can not start but it will resume.
>>>
>>> Perhaps if you share some code it would be easier to help you.
>>> //Pontus
>>> On 22 Mar 2013 22:06, "Chris Wolf" <cw...@gmail.com> wrote:
>>>
>>>> I found the issue with my custom CronScheduledRoutePolicy - initially
>>>> the startTime/resumeTime are only scheduled in
>>>> onInit() - so to re-resume (re-start), you need to call
>>>> scheduleRoute(Action.RESUME, route); in onStart()
>>>>
>>>> ...but now I'm getting:
>>>>
>>>> quartz.ScheduledRoutePolicy WARN Route is not in a started state and
>>>> cannot be resumed. The current route state is Suspended
>>>>
>>>> What is the deal? I thought resumeRoute was the inverse of
>>>> suspenRoute, but this log message seems to indicate that
>>>> calling CamelContext.suspendRoute(routeId) will put the route into a
>>>> state that cannot be resumed.
>>>>
>>>> Thanks,
>>>>
>>>>
>>>> Chris
>>>>
>>>> On Fri, Mar 22, 2013 at 4:04 PM, Chris Wolf <cw...@gmail.com> wrote:
>>>> > Pontus,
>>>> >
>>>> > Thanks for that. Since I already has started implementing a class
>>>> > derived from CronScheduledRoutePolicy, I just finished it.
>>>> > It works by starting a Timer thread in onStart/onResume at the end of
>>>> > the time period, the route is suspended, but then upon
>>>> > the next schedule cron start time, I don't see it being resumed - I
>>>> > wonder if the RoutePolicy itself is being suspend too?
>>>> >
>>>> > Well, I try it your way also.
>>>> >
>>>> >
>>>> > Thanks,
>>>> >
>>>> >
>>>> > Chris
>>>> >
>>>> > On Wed, Mar 20, 2013 at 4:34 AM, Pontus Ullgren <ul...@gmail.com>
>>>> wrote:
>>>> >> Hello,
>>>> >>
>>>> >> On Tue, Mar 19, 2013 at 11:22 PM, Chris Wolf <cw...@gmail.com>
>>>> wrote:
>>>> >>> On Mon, Mar 18, 2013 at 4:57 PM, Pontus Ullgren <ul...@gmail.com>
>>>> wrote:
>>>> >>>> Hello Chris,
>>>> >>>>
>>>> >>>> On Mon, Mar 18, 2013 at 8:54 PM, Chris Wolf <cw...@gmail.com>
>>>> wrote:
>>>> >>>>> Claus,
>>>> >>>>>
>>>> >>>>> I have a few further questions about CronScheduledRoutePolicy. I
>>>> >>>>> noticed that it has setters such as setRouteStartTime,
>>>> >>>>> setRouteStopTime, each which takes a cron expression string. What
>>>> I'm
>>>> >>>>> looking for is to be able to use a cron expression for the start, but
>>>> >>>>> a relative time length for stop. Otherwise, I need to write code to
>>>> >>>>> parse the start time expression, then calculate a stop time cron
>>>> >>>>> expression. Any ideas?
>>>> >>>>>
>>>> >>>> Depending on your needs you could enable "sendEmptyMessageWhenIdle" on
>>>> >>>> the endpoint and then suspend the route when you receive a empty
>>>> >>>> message. Which means that there is no more files to poll at the
>>>> >>>> moment.
>>>> >>>> You can use the content based route EIP for this.
>>>> >>>
>>>> >>> That is interesting to know, thanks. In my case, the files at the
>>>> >>> remote end are themselves deposited at an irregular rate, but within a
>>>> >>> defined time window, so during that time window, there will be
>>>> >>> intermittent idleness...
>>>> >>>>
>>>> >>>> Another solution would be to write your own RoutePolicy to take care
>>>> >>>> of your needs.
>>>> >>>
>>>> >>> Yes, this sounds like the best approach...
>>>> >>>
>>>> >>>
>>>> >>>>
>>>> >>>> I just started to wonder if it might be possible to combine the
>>>> >>>> CronScheduledRoutePolicy with a SimpleScheduledRoutePolicy.
>>>> >>>> I have _not_ tested this so I'm not sure if it works. It might be that
>>>> >>>> there is a collision in the way they work with Quartz.
>>>> >>>>
>>>> >>>>> Also I see that CronScheduledRoutePolicy has setRouteResumeTime,
>>>> >>>>> setRouteSuspendTime such that for my FTP poll window, I could either
>>>> >>>>> do start/stop or resume/suspend - which is recommended?
>>>> >>>>>
>>>> >>>> I would highly recommend resume/suspend.
>>>> >>>> I've had some thread leak problem with the file component when it was
>>>> >>>> repetitively started/stopped.
>>>> >>>
>>>> >>> Ok, but I guess the first policy callback with be onStart since the
>>>> >>> route will be
>>>> >>> configured with noAutoStartup(), so upon that first onStart, I'll
>>>> >>> suspend then toggle
>>>> >>> between onSuspend/onResume...
>>>> >>>
>>>> >> Yes this is what I do. We have a route that should start 06:30 each
>>>> >> day and then poll all files that are in the folder at that time.
>>>> >> After that it should suspend.
>>>> >>
>>>> >> Here is some pseudo code.
>>>> >> SuspendRouteProcessor is a processor that suspends the route based on
>>>> route id.
>>>> >> ---
>>>> >> String cronStr = "* 30 6 * * * ?";
>>>> >> String input = "ftp://user@remotehost
>>>> /inbox?sendEmptyMessageWhenIdle=true&password=secret";
>>>> >> CronScheduledRoutePolicy scheduledRP = new CronScheduledRoutePolicy();
>>>> >> scheduledRoutePolicy.setRouteStartTime(cronStr);
>>>> >> scheduledRoutePolicy.setRouteResumeTime(cronStr);
>>>> >>
>>>> >> from(input)
>>>> >> .routeId("input1")
>>>> >> .routePolicy(versionPolicy, scheduledRoutePolicy)
>>>> >> .noAutoStartup()
>>>> >> .choice()
>>>> >> .when(body().isNotNull())
>>>> >> .to("direct:processFiles")
>>>> >> .end()
>>>> >> .log(LoggingLevel.DEBUG, "All files processed, suspend
>>>> route")
>>>> >> .process(new SuspendRouteProcessor("input1"))
>>>> >> ;
>>>> >> --
>>>> >>
>>>> >> The only downside with this is that after the initial start we get a
>>>> >> WARN log message that the route can not be started since it is in
>>>> >> suspend state.
>>>> >> But as long as you can live with the WARN log it works.
>>>> >>
>>>> >>>>
>>>> >>>> // Pontus
>>>> >>>>
>>>> >>>>> Thanks,
>>>> >>>>>
>>>> >>>>> Chris
>>>> >>>>>
>>>> >>>>> On Sat, Mar 2, 2013 at 1:43 AM, Claus Ibsen <cl...@gmail.com>
>>>> wrote:
>>>> >>>>>> Hi
>>>> >>>>>>
>>>> >>>>>> See about route policy
>>>> >>>>>> http://camel.apache.org/routepolicy
>>>> >>>>>>
>>>> >>>>>> And the scheduled route policy
>>>> >>>>>> http://camel.apache.org/scheduledroutepolicy.html
>>>> >>>>>>
>>>> >>>>>> On Sat, Mar 2, 2013 at 12:15 AM, Chris Wolf <cw...@gmail.com>
>>>> wrote:
>>>> >>>>>>> I have a requirement to download files via FTP during a certain
>>>> time
>>>> >>>>>>> window and according to a schedule. e.g. Only on trading days
>>>> between
>>>> >>>>>>> 6:30AM and 7:00AM. The FTP component, alone, seems to just do
>>>> >>>>>>> indefinite polling according to delay/initialDelay.
>>>> >>>>>>>
>>>> >>>>>>> From the "Camel In Action" book, chapter 7, I see some examples of
>>>> >>>>>>> sending a text message with the Timer and Quartz components, but I
>>>> >>>>>>> can't quite see how to put that together to implement "kicking off
>>>> >>>>>>> routes at specified intervals", mentioned in the best practices
>>>> list
>>>> >>>>>>> at the end of that chapter. How would I use quartz to stop/start
>>>> the
>>>> >>>>>>> FTP component, or the route that it's in?
>>>> >>>>>>>
>>>> >>>>>>> Thanks,
>>>> >>>>>>>
>>>> >>>>>>> Chris
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>> --
>>>> >>>>>> Claus Ibsen
>>>> >>>>>> -----------------
>>>> >>>>>> Red Hat, Inc.
>>>> >>>>>> FuseSource is now part of Red Hat
>>>> >>>>>> Email: cibsen@redhat.com
>>>> >>>>>> Web: http://fusesource.com
>>>> >>>>>> Twitter: davsclaus
>>>> >>>>>> Blog: http://davsclaus.com
>>>> >>>>>> Author of Camel in Action: http://www.manning.com/ibsen
>>>>
Re: How to schedule FTP with quartz?
Posted by Pontus Ullgren <ul...@gmail.com>.
Chris,
Is there no way for you to calculate a cron expression for when the
suspend should occure ?
Say that you want the route to start every 10 minutes (starting at 0)
and run for 5 minutes then suspend.
This would mean that you should be able to define the start/resume
cron as "0 0,10,20,30,40,50 * * * * ?".
And the suspend cron expression as "0 5,15,25,35,45,55 * * * * ?".
The CronScheduledRoutePolicy will trigger a start/resum at 00:00:00
and then a suspend at 00:05:00, then a new start/resume at 00:10:00
and a new suspends at 00:15:00 and so on.
// Pontus
On Tue, Mar 26, 2013 at 7:39 PM, Chris Wolf <cw...@gmail.com> wrote:
> Pontus,
>
> I actually gave up on CronScheduledRoutePolicy because I don't want to
> have to calculate an absolute suspend time based on the start/resume
> time. What I need is a cron-based policy that specifies the initial
> start time, which there-after becomes the resume time - this is a cron
> expression - then I just want a relative run duration, after which,
> the route is suspended until the next cron-specified resume time.
>
> So, after a few days, I finally have that and it works in the unit
> test - even the initial one-off route start will transparently switch
> to a resume schedule. The only problem is that I really need this
> policy to control a route with an FTP consumer. What I'm seeing is
> that
> even though the route is suspended, the FTP consumer continues to poll
> - this partially answers my question about the coding of
> ScheduleRoutePolicy - which only suspends the consumer - not the
> route, itself. While my policy suspends the route.
>
> I suspect that any route which has a component using
> PollingConsumerPollStrategy will not behave as I though - which is -
> you only need to suspend the route and all it's components will be
> suspended. My suspicion is that components whose Consumers are under
> the influence of PollingConsumerPollStrategy may not suspend by only
> suspended the route.
>
> -Chris
>
>
> On Sat, Mar 23, 2013 at 3:15 AM, Pontus Ullgren <ul...@gmail.com> wrote:
>> This is probably because your route is autoStart=false. So you the first
>> time you need to start it. In my example you see I set both the start and
>> resume schedule to the same cron expression. So the policy will trigger
>> both a start and a resume action.
>>
>> And you will get a WARN log since the first time it can not resume (but it
>> will start) and after that it can not start but it will resume.
>>
>> Perhaps if you share some code it would be easier to help you.
>> //Pontus
>> On 22 Mar 2013 22:06, "Chris Wolf" <cw...@gmail.com> wrote:
>>
>>> I found the issue with my custom CronScheduledRoutePolicy - initially
>>> the startTime/resumeTime are only scheduled in
>>> onInit() - so to re-resume (re-start), you need to call
>>> scheduleRoute(Action.RESUME, route); in onStart()
>>>
>>> ...but now I'm getting:
>>>
>>> quartz.ScheduledRoutePolicy WARN Route is not in a started state and
>>> cannot be resumed. The current route state is Suspended
>>>
>>> What is the deal? I thought resumeRoute was the inverse of
>>> suspenRoute, but this log message seems to indicate that
>>> calling CamelContext.suspendRoute(routeId) will put the route into a
>>> state that cannot be resumed.
>>>
>>> Thanks,
>>>
>>>
>>> Chris
>>>
>>> On Fri, Mar 22, 2013 at 4:04 PM, Chris Wolf <cw...@gmail.com> wrote:
>>> > Pontus,
>>> >
>>> > Thanks for that. Since I already has started implementing a class
>>> > derived from CronScheduledRoutePolicy, I just finished it.
>>> > It works by starting a Timer thread in onStart/onResume at the end of
>>> > the time period, the route is suspended, but then upon
>>> > the next schedule cron start time, I don't see it being resumed - I
>>> > wonder if the RoutePolicy itself is being suspend too?
>>> >
>>> > Well, I try it your way also.
>>> >
>>> >
>>> > Thanks,
>>> >
>>> >
>>> > Chris
>>> >
>>> > On Wed, Mar 20, 2013 at 4:34 AM, Pontus Ullgren <ul...@gmail.com>
>>> wrote:
>>> >> Hello,
>>> >>
>>> >> On Tue, Mar 19, 2013 at 11:22 PM, Chris Wolf <cw...@gmail.com>
>>> wrote:
>>> >>> On Mon, Mar 18, 2013 at 4:57 PM, Pontus Ullgren <ul...@gmail.com>
>>> wrote:
>>> >>>> Hello Chris,
>>> >>>>
>>> >>>> On Mon, Mar 18, 2013 at 8:54 PM, Chris Wolf <cw...@gmail.com>
>>> wrote:
>>> >>>>> Claus,
>>> >>>>>
>>> >>>>> I have a few further questions about CronScheduledRoutePolicy. I
>>> >>>>> noticed that it has setters such as setRouteStartTime,
>>> >>>>> setRouteStopTime, each which takes a cron expression string. What
>>> I'm
>>> >>>>> looking for is to be able to use a cron expression for the start, but
>>> >>>>> a relative time length for stop. Otherwise, I need to write code to
>>> >>>>> parse the start time expression, then calculate a stop time cron
>>> >>>>> expression. Any ideas?
>>> >>>>>
>>> >>>> Depending on your needs you could enable "sendEmptyMessageWhenIdle" on
>>> >>>> the endpoint and then suspend the route when you receive a empty
>>> >>>> message. Which means that there is no more files to poll at the
>>> >>>> moment.
>>> >>>> You can use the content based route EIP for this.
>>> >>>
>>> >>> That is interesting to know, thanks. In my case, the files at the
>>> >>> remote end are themselves deposited at an irregular rate, but within a
>>> >>> defined time window, so during that time window, there will be
>>> >>> intermittent idleness...
>>> >>>>
>>> >>>> Another solution would be to write your own RoutePolicy to take care
>>> >>>> of your needs.
>>> >>>
>>> >>> Yes, this sounds like the best approach...
>>> >>>
>>> >>>
>>> >>>>
>>> >>>> I just started to wonder if it might be possible to combine the
>>> >>>> CronScheduledRoutePolicy with a SimpleScheduledRoutePolicy.
>>> >>>> I have _not_ tested this so I'm not sure if it works. It might be that
>>> >>>> there is a collision in the way they work with Quartz.
>>> >>>>
>>> >>>>> Also I see that CronScheduledRoutePolicy has setRouteResumeTime,
>>> >>>>> setRouteSuspendTime such that for my FTP poll window, I could either
>>> >>>>> do start/stop or resume/suspend - which is recommended?
>>> >>>>>
>>> >>>> I would highly recommend resume/suspend.
>>> >>>> I've had some thread leak problem with the file component when it was
>>> >>>> repetitively started/stopped.
>>> >>>
>>> >>> Ok, but I guess the first policy callback with be onStart since the
>>> >>> route will be
>>> >>> configured with noAutoStartup(), so upon that first onStart, I'll
>>> >>> suspend then toggle
>>> >>> between onSuspend/onResume...
>>> >>>
>>> >> Yes this is what I do. We have a route that should start 06:30 each
>>> >> day and then poll all files that are in the folder at that time.
>>> >> After that it should suspend.
>>> >>
>>> >> Here is some pseudo code.
>>> >> SuspendRouteProcessor is a processor that suspends the route based on
>>> route id.
>>> >> ---
>>> >> String cronStr = "* 30 6 * * * ?";
>>> >> String input = "ftp://user@remotehost
>>> /inbox?sendEmptyMessageWhenIdle=true&password=secret";
>>> >> CronScheduledRoutePolicy scheduledRP = new CronScheduledRoutePolicy();
>>> >> scheduledRoutePolicy.setRouteStartTime(cronStr);
>>> >> scheduledRoutePolicy.setRouteResumeTime(cronStr);
>>> >>
>>> >> from(input)
>>> >> .routeId("input1")
>>> >> .routePolicy(versionPolicy, scheduledRoutePolicy)
>>> >> .noAutoStartup()
>>> >> .choice()
>>> >> .when(body().isNotNull())
>>> >> .to("direct:processFiles")
>>> >> .end()
>>> >> .log(LoggingLevel.DEBUG, "All files processed, suspend
>>> route")
>>> >> .process(new SuspendRouteProcessor("input1"))
>>> >> ;
>>> >> --
>>> >>
>>> >> The only downside with this is that after the initial start we get a
>>> >> WARN log message that the route can not be started since it is in
>>> >> suspend state.
>>> >> But as long as you can live with the WARN log it works.
>>> >>
>>> >>>>
>>> >>>> // Pontus
>>> >>>>
>>> >>>>> Thanks,
>>> >>>>>
>>> >>>>> Chris
>>> >>>>>
>>> >>>>> On Sat, Mar 2, 2013 at 1:43 AM, Claus Ibsen <cl...@gmail.com>
>>> wrote:
>>> >>>>>> Hi
>>> >>>>>>
>>> >>>>>> See about route policy
>>> >>>>>> http://camel.apache.org/routepolicy
>>> >>>>>>
>>> >>>>>> And the scheduled route policy
>>> >>>>>> http://camel.apache.org/scheduledroutepolicy.html
>>> >>>>>>
>>> >>>>>> On Sat, Mar 2, 2013 at 12:15 AM, Chris Wolf <cw...@gmail.com>
>>> wrote:
>>> >>>>>>> I have a requirement to download files via FTP during a certain
>>> time
>>> >>>>>>> window and according to a schedule. e.g. Only on trading days
>>> between
>>> >>>>>>> 6:30AM and 7:00AM. The FTP component, alone, seems to just do
>>> >>>>>>> indefinite polling according to delay/initialDelay.
>>> >>>>>>>
>>> >>>>>>> From the "Camel In Action" book, chapter 7, I see some examples of
>>> >>>>>>> sending a text message with the Timer and Quartz components, but I
>>> >>>>>>> can't quite see how to put that together to implement "kicking off
>>> >>>>>>> routes at specified intervals", mentioned in the best practices
>>> list
>>> >>>>>>> at the end of that chapter. How would I use quartz to stop/start
>>> the
>>> >>>>>>> FTP component, or the route that it's in?
>>> >>>>>>>
>>> >>>>>>> Thanks,
>>> >>>>>>>
>>> >>>>>>> Chris
>>> >>>>>>
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> --
>>> >>>>>> Claus Ibsen
>>> >>>>>> -----------------
>>> >>>>>> Red Hat, Inc.
>>> >>>>>> FuseSource is now part of Red Hat
>>> >>>>>> Email: cibsen@redhat.com
>>> >>>>>> Web: http://fusesource.com
>>> >>>>>> Twitter: davsclaus
>>> >>>>>> Blog: http://davsclaus.com
>>> >>>>>> Author of Camel in Action: http://www.manning.com/ibsen
>>>
Re: How to schedule FTP with quartz?
Posted by Chris Wolf <cw...@gmail.com>.
Pontus,
I actually gave up on CronScheduledRoutePolicy because I don't want to
have to calculate an absolute suspend time based on the start/resume
time. What I need is a cron-based policy that specifies the initial
start time, which there-after becomes the resume time - this is a cron
expression - then I just want a relative run duration, after which,
the route is suspended until the next cron-specified resume time.
So, after a few days, I finally have that and it works in the unit
test - even the initial one-off route start will transparently switch
to a resume schedule. The only problem is that I really need this
policy to control a route with an FTP consumer. What I'm seeing is
that
even though the route is suspended, the FTP consumer continues to poll
- this partially answers my question about the coding of
ScheduleRoutePolicy - which only suspends the consumer - not the
route, itself. While my policy suspends the route.
I suspect that any route which has a component using
PollingConsumerPollStrategy will not behave as I though - which is -
you only need to suspend the route and all it's components will be
suspended. My suspicion is that components whose Consumers are under
the influence of PollingConsumerPollStrategy may not suspend by only
suspended the route.
-Chris
On Sat, Mar 23, 2013 at 3:15 AM, Pontus Ullgren <ul...@gmail.com> wrote:
> This is probably because your route is autoStart=false. So you the first
> time you need to start it. In my example you see I set both the start and
> resume schedule to the same cron expression. So the policy will trigger
> both a start and a resume action.
>
> And you will get a WARN log since the first time it can not resume (but it
> will start) and after that it can not start but it will resume.
>
> Perhaps if you share some code it would be easier to help you.
> //Pontus
> On 22 Mar 2013 22:06, "Chris Wolf" <cw...@gmail.com> wrote:
>
>> I found the issue with my custom CronScheduledRoutePolicy - initially
>> the startTime/resumeTime are only scheduled in
>> onInit() - so to re-resume (re-start), you need to call
>> scheduleRoute(Action.RESUME, route); in onStart()
>>
>> ...but now I'm getting:
>>
>> quartz.ScheduledRoutePolicy WARN Route is not in a started state and
>> cannot be resumed. The current route state is Suspended
>>
>> What is the deal? I thought resumeRoute was the inverse of
>> suspenRoute, but this log message seems to indicate that
>> calling CamelContext.suspendRoute(routeId) will put the route into a
>> state that cannot be resumed.
>>
>> Thanks,
>>
>>
>> Chris
>>
>> On Fri, Mar 22, 2013 at 4:04 PM, Chris Wolf <cw...@gmail.com> wrote:
>> > Pontus,
>> >
>> > Thanks for that. Since I already has started implementing a class
>> > derived from CronScheduledRoutePolicy, I just finished it.
>> > It works by starting a Timer thread in onStart/onResume at the end of
>> > the time period, the route is suspended, but then upon
>> > the next schedule cron start time, I don't see it being resumed - I
>> > wonder if the RoutePolicy itself is being suspend too?
>> >
>> > Well, I try it your way also.
>> >
>> >
>> > Thanks,
>> >
>> >
>> > Chris
>> >
>> > On Wed, Mar 20, 2013 at 4:34 AM, Pontus Ullgren <ul...@gmail.com>
>> wrote:
>> >> Hello,
>> >>
>> >> On Tue, Mar 19, 2013 at 11:22 PM, Chris Wolf <cw...@gmail.com>
>> wrote:
>> >>> On Mon, Mar 18, 2013 at 4:57 PM, Pontus Ullgren <ul...@gmail.com>
>> wrote:
>> >>>> Hello Chris,
>> >>>>
>> >>>> On Mon, Mar 18, 2013 at 8:54 PM, Chris Wolf <cw...@gmail.com>
>> wrote:
>> >>>>> Claus,
>> >>>>>
>> >>>>> I have a few further questions about CronScheduledRoutePolicy. I
>> >>>>> noticed that it has setters such as setRouteStartTime,
>> >>>>> setRouteStopTime, each which takes a cron expression string. What
>> I'm
>> >>>>> looking for is to be able to use a cron expression for the start, but
>> >>>>> a relative time length for stop. Otherwise, I need to write code to
>> >>>>> parse the start time expression, then calculate a stop time cron
>> >>>>> expression. Any ideas?
>> >>>>>
>> >>>> Depending on your needs you could enable "sendEmptyMessageWhenIdle" on
>> >>>> the endpoint and then suspend the route when you receive a empty
>> >>>> message. Which means that there is no more files to poll at the
>> >>>> moment.
>> >>>> You can use the content based route EIP for this.
>> >>>
>> >>> That is interesting to know, thanks. In my case, the files at the
>> >>> remote end are themselves deposited at an irregular rate, but within a
>> >>> defined time window, so during that time window, there will be
>> >>> intermittent idleness...
>> >>>>
>> >>>> Another solution would be to write your own RoutePolicy to take care
>> >>>> of your needs.
>> >>>
>> >>> Yes, this sounds like the best approach...
>> >>>
>> >>>
>> >>>>
>> >>>> I just started to wonder if it might be possible to combine the
>> >>>> CronScheduledRoutePolicy with a SimpleScheduledRoutePolicy.
>> >>>> I have _not_ tested this so I'm not sure if it works. It might be that
>> >>>> there is a collision in the way they work with Quartz.
>> >>>>
>> >>>>> Also I see that CronScheduledRoutePolicy has setRouteResumeTime,
>> >>>>> setRouteSuspendTime such that for my FTP poll window, I could either
>> >>>>> do start/stop or resume/suspend - which is recommended?
>> >>>>>
>> >>>> I would highly recommend resume/suspend.
>> >>>> I've had some thread leak problem with the file component when it was
>> >>>> repetitively started/stopped.
>> >>>
>> >>> Ok, but I guess the first policy callback with be onStart since the
>> >>> route will be
>> >>> configured with noAutoStartup(), so upon that first onStart, I'll
>> >>> suspend then toggle
>> >>> between onSuspend/onResume...
>> >>>
>> >> Yes this is what I do. We have a route that should start 06:30 each
>> >> day and then poll all files that are in the folder at that time.
>> >> After that it should suspend.
>> >>
>> >> Here is some pseudo code.
>> >> SuspendRouteProcessor is a processor that suspends the route based on
>> route id.
>> >> ---
>> >> String cronStr = "* 30 6 * * * ?";
>> >> String input = "ftp://user@remotehost
>> /inbox?sendEmptyMessageWhenIdle=true&password=secret";
>> >> CronScheduledRoutePolicy scheduledRP = new CronScheduledRoutePolicy();
>> >> scheduledRoutePolicy.setRouteStartTime(cronStr);
>> >> scheduledRoutePolicy.setRouteResumeTime(cronStr);
>> >>
>> >> from(input)
>> >> .routeId("input1")
>> >> .routePolicy(versionPolicy, scheduledRoutePolicy)
>> >> .noAutoStartup()
>> >> .choice()
>> >> .when(body().isNotNull())
>> >> .to("direct:processFiles")
>> >> .end()
>> >> .log(LoggingLevel.DEBUG, "All files processed, suspend
>> route")
>> >> .process(new SuspendRouteProcessor("input1"))
>> >> ;
>> >> --
>> >>
>> >> The only downside with this is that after the initial start we get a
>> >> WARN log message that the route can not be started since it is in
>> >> suspend state.
>> >> But as long as you can live with the WARN log it works.
>> >>
>> >>>>
>> >>>> // Pontus
>> >>>>
>> >>>>> Thanks,
>> >>>>>
>> >>>>> Chris
>> >>>>>
>> >>>>> On Sat, Mar 2, 2013 at 1:43 AM, Claus Ibsen <cl...@gmail.com>
>> wrote:
>> >>>>>> Hi
>> >>>>>>
>> >>>>>> See about route policy
>> >>>>>> http://camel.apache.org/routepolicy
>> >>>>>>
>> >>>>>> And the scheduled route policy
>> >>>>>> http://camel.apache.org/scheduledroutepolicy.html
>> >>>>>>
>> >>>>>> On Sat, Mar 2, 2013 at 12:15 AM, Chris Wolf <cw...@gmail.com>
>> wrote:
>> >>>>>>> I have a requirement to download files via FTP during a certain
>> time
>> >>>>>>> window and according to a schedule. e.g. Only on trading days
>> between
>> >>>>>>> 6:30AM and 7:00AM. The FTP component, alone, seems to just do
>> >>>>>>> indefinite polling according to delay/initialDelay.
>> >>>>>>>
>> >>>>>>> From the "Camel In Action" book, chapter 7, I see some examples of
>> >>>>>>> sending a text message with the Timer and Quartz components, but I
>> >>>>>>> can't quite see how to put that together to implement "kicking off
>> >>>>>>> routes at specified intervals", mentioned in the best practices
>> list
>> >>>>>>> at the end of that chapter. How would I use quartz to stop/start
>> the
>> >>>>>>> FTP component, or the route that it's in?
>> >>>>>>>
>> >>>>>>> Thanks,
>> >>>>>>>
>> >>>>>>> Chris
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> --
>> >>>>>> Claus Ibsen
>> >>>>>> -----------------
>> >>>>>> Red Hat, Inc.
>> >>>>>> FuseSource is now part of Red Hat
>> >>>>>> Email: cibsen@redhat.com
>> >>>>>> Web: http://fusesource.com
>> >>>>>> Twitter: davsclaus
>> >>>>>> Blog: http://davsclaus.com
>> >>>>>> Author of Camel in Action: http://www.manning.com/ibsen
>>
Re: How to schedule FTP with quartz?
Posted by Pontus Ullgren <ul...@gmail.com>.
This is probably because your route is autoStart=false. So you the first
time you need to start it. In my example you see I set both the start and
resume schedule to the same cron expression. So the policy will trigger
both a start and a resume action.
And you will get a WARN log since the first time it can not resume (but it
will start) and after that it can not start but it will resume.
Perhaps if you share some code it would be easier to help you.
//Pontus
On 22 Mar 2013 22:06, "Chris Wolf" <cw...@gmail.com> wrote:
> I found the issue with my custom CronScheduledRoutePolicy - initially
> the startTime/resumeTime are only scheduled in
> onInit() - so to re-resume (re-start), you need to call
> scheduleRoute(Action.RESUME, route); in onStart()
>
> ...but now I'm getting:
>
> quartz.ScheduledRoutePolicy WARN Route is not in a started state and
> cannot be resumed. The current route state is Suspended
>
> What is the deal? I thought resumeRoute was the inverse of
> suspenRoute, but this log message seems to indicate that
> calling CamelContext.suspendRoute(routeId) will put the route into a
> state that cannot be resumed.
>
> Thanks,
>
>
> Chris
>
> On Fri, Mar 22, 2013 at 4:04 PM, Chris Wolf <cw...@gmail.com> wrote:
> > Pontus,
> >
> > Thanks for that. Since I already has started implementing a class
> > derived from CronScheduledRoutePolicy, I just finished it.
> > It works by starting a Timer thread in onStart/onResume at the end of
> > the time period, the route is suspended, but then upon
> > the next schedule cron start time, I don't see it being resumed - I
> > wonder if the RoutePolicy itself is being suspend too?
> >
> > Well, I try it your way also.
> >
> >
> > Thanks,
> >
> >
> > Chris
> >
> > On Wed, Mar 20, 2013 at 4:34 AM, Pontus Ullgren <ul...@gmail.com>
> wrote:
> >> Hello,
> >>
> >> On Tue, Mar 19, 2013 at 11:22 PM, Chris Wolf <cw...@gmail.com>
> wrote:
> >>> On Mon, Mar 18, 2013 at 4:57 PM, Pontus Ullgren <ul...@gmail.com>
> wrote:
> >>>> Hello Chris,
> >>>>
> >>>> On Mon, Mar 18, 2013 at 8:54 PM, Chris Wolf <cw...@gmail.com>
> wrote:
> >>>>> Claus,
> >>>>>
> >>>>> I have a few further questions about CronScheduledRoutePolicy. I
> >>>>> noticed that it has setters such as setRouteStartTime,
> >>>>> setRouteStopTime, each which takes a cron expression string. What
> I'm
> >>>>> looking for is to be able to use a cron expression for the start, but
> >>>>> a relative time length for stop. Otherwise, I need to write code to
> >>>>> parse the start time expression, then calculate a stop time cron
> >>>>> expression. Any ideas?
> >>>>>
> >>>> Depending on your needs you could enable "sendEmptyMessageWhenIdle" on
> >>>> the endpoint and then suspend the route when you receive a empty
> >>>> message. Which means that there is no more files to poll at the
> >>>> moment.
> >>>> You can use the content based route EIP for this.
> >>>
> >>> That is interesting to know, thanks. In my case, the files at the
> >>> remote end are themselves deposited at an irregular rate, but within a
> >>> defined time window, so during that time window, there will be
> >>> intermittent idleness...
> >>>>
> >>>> Another solution would be to write your own RoutePolicy to take care
> >>>> of your needs.
> >>>
> >>> Yes, this sounds like the best approach...
> >>>
> >>>
> >>>>
> >>>> I just started to wonder if it might be possible to combine the
> >>>> CronScheduledRoutePolicy with a SimpleScheduledRoutePolicy.
> >>>> I have _not_ tested this so I'm not sure if it works. It might be that
> >>>> there is a collision in the way they work with Quartz.
> >>>>
> >>>>> Also I see that CronScheduledRoutePolicy has setRouteResumeTime,
> >>>>> setRouteSuspendTime such that for my FTP poll window, I could either
> >>>>> do start/stop or resume/suspend - which is recommended?
> >>>>>
> >>>> I would highly recommend resume/suspend.
> >>>> I've had some thread leak problem with the file component when it was
> >>>> repetitively started/stopped.
> >>>
> >>> Ok, but I guess the first policy callback with be onStart since the
> >>> route will be
> >>> configured with noAutoStartup(), so upon that first onStart, I'll
> >>> suspend then toggle
> >>> between onSuspend/onResume...
> >>>
> >> Yes this is what I do. We have a route that should start 06:30 each
> >> day and then poll all files that are in the folder at that time.
> >> After that it should suspend.
> >>
> >> Here is some pseudo code.
> >> SuspendRouteProcessor is a processor that suspends the route based on
> route id.
> >> ---
> >> String cronStr = "* 30 6 * * * ?";
> >> String input = "ftp://user@remotehost
> /inbox?sendEmptyMessageWhenIdle=true&password=secret";
> >> CronScheduledRoutePolicy scheduledRP = new CronScheduledRoutePolicy();
> >> scheduledRoutePolicy.setRouteStartTime(cronStr);
> >> scheduledRoutePolicy.setRouteResumeTime(cronStr);
> >>
> >> from(input)
> >> .routeId("input1")
> >> .routePolicy(versionPolicy, scheduledRoutePolicy)
> >> .noAutoStartup()
> >> .choice()
> >> .when(body().isNotNull())
> >> .to("direct:processFiles")
> >> .end()
> >> .log(LoggingLevel.DEBUG, "All files processed, suspend
> route")
> >> .process(new SuspendRouteProcessor("input1"))
> >> ;
> >> --
> >>
> >> The only downside with this is that after the initial start we get a
> >> WARN log message that the route can not be started since it is in
> >> suspend state.
> >> But as long as you can live with the WARN log it works.
> >>
> >>>>
> >>>> // Pontus
> >>>>
> >>>>> Thanks,
> >>>>>
> >>>>> Chris
> >>>>>
> >>>>> On Sat, Mar 2, 2013 at 1:43 AM, Claus Ibsen <cl...@gmail.com>
> wrote:
> >>>>>> Hi
> >>>>>>
> >>>>>> See about route policy
> >>>>>> http://camel.apache.org/routepolicy
> >>>>>>
> >>>>>> And the scheduled route policy
> >>>>>> http://camel.apache.org/scheduledroutepolicy.html
> >>>>>>
> >>>>>> On Sat, Mar 2, 2013 at 12:15 AM, Chris Wolf <cw...@gmail.com>
> wrote:
> >>>>>>> I have a requirement to download files via FTP during a certain
> time
> >>>>>>> window and according to a schedule. e.g. Only on trading days
> between
> >>>>>>> 6:30AM and 7:00AM. The FTP component, alone, seems to just do
> >>>>>>> indefinite polling according to delay/initialDelay.
> >>>>>>>
> >>>>>>> From the "Camel In Action" book, chapter 7, I see some examples of
> >>>>>>> sending a text message with the Timer and Quartz components, but I
> >>>>>>> can't quite see how to put that together to implement "kicking off
> >>>>>>> routes at specified intervals", mentioned in the best practices
> list
> >>>>>>> at the end of that chapter. How would I use quartz to stop/start
> the
> >>>>>>> FTP component, or the route that it's in?
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>>
> >>>>>>> Chris
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Claus Ibsen
> >>>>>> -----------------
> >>>>>> Red Hat, Inc.
> >>>>>> FuseSource is now part of Red Hat
> >>>>>> Email: cibsen@redhat.com
> >>>>>> Web: http://fusesource.com
> >>>>>> Twitter: davsclaus
> >>>>>> Blog: http://davsclaus.com
> >>>>>> Author of Camel in Action: http://www.manning.com/ibsen
>
Re: How to schedule FTP with quartz?
Posted by Chris Wolf <cw...@gmail.com>.
I found the issue with my custom CronScheduledRoutePolicy - initially
the startTime/resumeTime are only scheduled in
onInit() - so to re-resume (re-start), you need to call
scheduleRoute(Action.RESUME, route); in onStart()
...but now I'm getting:
quartz.ScheduledRoutePolicy WARN Route is not in a started state and
cannot be resumed. The current route state is Suspended
What is the deal? I thought resumeRoute was the inverse of
suspenRoute, but this log message seems to indicate that
calling CamelContext.suspendRoute(routeId) will put the route into a
state that cannot be resumed.
Thanks,
Chris
On Fri, Mar 22, 2013 at 4:04 PM, Chris Wolf <cw...@gmail.com> wrote:
> Pontus,
>
> Thanks for that. Since I already has started implementing a class
> derived from CronScheduledRoutePolicy, I just finished it.
> It works by starting a Timer thread in onStart/onResume at the end of
> the time period, the route is suspended, but then upon
> the next schedule cron start time, I don't see it being resumed - I
> wonder if the RoutePolicy itself is being suspend too?
>
> Well, I try it your way also.
>
>
> Thanks,
>
>
> Chris
>
> On Wed, Mar 20, 2013 at 4:34 AM, Pontus Ullgren <ul...@gmail.com> wrote:
>> Hello,
>>
>> On Tue, Mar 19, 2013 at 11:22 PM, Chris Wolf <cw...@gmail.com> wrote:
>>> On Mon, Mar 18, 2013 at 4:57 PM, Pontus Ullgren <ul...@gmail.com> wrote:
>>>> Hello Chris,
>>>>
>>>> On Mon, Mar 18, 2013 at 8:54 PM, Chris Wolf <cw...@gmail.com> wrote:
>>>>> Claus,
>>>>>
>>>>> I have a few further questions about CronScheduledRoutePolicy. I
>>>>> noticed that it has setters such as setRouteStartTime,
>>>>> setRouteStopTime, each which takes a cron expression string. What I'm
>>>>> looking for is to be able to use a cron expression for the start, but
>>>>> a relative time length for stop. Otherwise, I need to write code to
>>>>> parse the start time expression, then calculate a stop time cron
>>>>> expression. Any ideas?
>>>>>
>>>> Depending on your needs you could enable "sendEmptyMessageWhenIdle" on
>>>> the endpoint and then suspend the route when you receive a empty
>>>> message. Which means that there is no more files to poll at the
>>>> moment.
>>>> You can use the content based route EIP for this.
>>>
>>> That is interesting to know, thanks. In my case, the files at the
>>> remote end are themselves deposited at an irregular rate, but within a
>>> defined time window, so during that time window, there will be
>>> intermittent idleness...
>>>>
>>>> Another solution would be to write your own RoutePolicy to take care
>>>> of your needs.
>>>
>>> Yes, this sounds like the best approach...
>>>
>>>
>>>>
>>>> I just started to wonder if it might be possible to combine the
>>>> CronScheduledRoutePolicy with a SimpleScheduledRoutePolicy.
>>>> I have _not_ tested this so I'm not sure if it works. It might be that
>>>> there is a collision in the way they work with Quartz.
>>>>
>>>>> Also I see that CronScheduledRoutePolicy has setRouteResumeTime,
>>>>> setRouteSuspendTime such that for my FTP poll window, I could either
>>>>> do start/stop or resume/suspend - which is recommended?
>>>>>
>>>> I would highly recommend resume/suspend.
>>>> I've had some thread leak problem with the file component when it was
>>>> repetitively started/stopped.
>>>
>>> Ok, but I guess the first policy callback with be onStart since the
>>> route will be
>>> configured with noAutoStartup(), so upon that first onStart, I'll
>>> suspend then toggle
>>> between onSuspend/onResume...
>>>
>> Yes this is what I do. We have a route that should start 06:30 each
>> day and then poll all files that are in the folder at that time.
>> After that it should suspend.
>>
>> Here is some pseudo code.
>> SuspendRouteProcessor is a processor that suspends the route based on route id.
>> ---
>> String cronStr = "* 30 6 * * * ?";
>> String input = "ftp://user@remotehost/inbox?sendEmptyMessageWhenIdle=true&password=secret";
>> CronScheduledRoutePolicy scheduledRP = new CronScheduledRoutePolicy();
>> scheduledRoutePolicy.setRouteStartTime(cronStr);
>> scheduledRoutePolicy.setRouteResumeTime(cronStr);
>>
>> from(input)
>> .routeId("input1")
>> .routePolicy(versionPolicy, scheduledRoutePolicy)
>> .noAutoStartup()
>> .choice()
>> .when(body().isNotNull())
>> .to("direct:processFiles")
>> .end()
>> .log(LoggingLevel.DEBUG, "All files processed, suspend route")
>> .process(new SuspendRouteProcessor("input1"))
>> ;
>> --
>>
>> The only downside with this is that after the initial start we get a
>> WARN log message that the route can not be started since it is in
>> suspend state.
>> But as long as you can live with the WARN log it works.
>>
>>>>
>>>> // Pontus
>>>>
>>>>> Thanks,
>>>>>
>>>>> Chris
>>>>>
>>>>> On Sat, Mar 2, 2013 at 1:43 AM, Claus Ibsen <cl...@gmail.com> wrote:
>>>>>> Hi
>>>>>>
>>>>>> See about route policy
>>>>>> http://camel.apache.org/routepolicy
>>>>>>
>>>>>> And the scheduled route policy
>>>>>> http://camel.apache.org/scheduledroutepolicy.html
>>>>>>
>>>>>> On Sat, Mar 2, 2013 at 12:15 AM, Chris Wolf <cw...@gmail.com> wrote:
>>>>>>> I have a requirement to download files via FTP during a certain time
>>>>>>> window and according to a schedule. e.g. Only on trading days between
>>>>>>> 6:30AM and 7:00AM. The FTP component, alone, seems to just do
>>>>>>> indefinite polling according to delay/initialDelay.
>>>>>>>
>>>>>>> From the "Camel In Action" book, chapter 7, I see some examples of
>>>>>>> sending a text message with the Timer and Quartz components, but I
>>>>>>> can't quite see how to put that together to implement "kicking off
>>>>>>> routes at specified intervals", mentioned in the best practices list
>>>>>>> at the end of that chapter. How would I use quartz to stop/start the
>>>>>>> FTP component, or the route that it's in?
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Chris
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Claus Ibsen
>>>>>> -----------------
>>>>>> Red Hat, Inc.
>>>>>> FuseSource is now part of Red Hat
>>>>>> Email: cibsen@redhat.com
>>>>>> Web: http://fusesource.com
>>>>>> Twitter: davsclaus
>>>>>> Blog: http://davsclaus.com
>>>>>> Author of Camel in Action: http://www.manning.com/ibsen
Re: How to schedule FTP with quartz?
Posted by Chris Wolf <cw...@gmail.com>.
Pontus,
Thanks for that. Since I already has started implementing a class
derived from CronScheduledRoutePolicy, I just finished it.
It works by starting a Timer thread in onStart/onResume at the end of
the time period, the route is suspended, but then upon
the next schedule cron start time, I don't see it being resumed - I
wonder if the RoutePolicy itself is being suspend too?
Well, I try it your way also.
Thanks,
Chris
On Wed, Mar 20, 2013 at 4:34 AM, Pontus Ullgren <ul...@gmail.com> wrote:
> Hello,
>
> On Tue, Mar 19, 2013 at 11:22 PM, Chris Wolf <cw...@gmail.com> wrote:
>> On Mon, Mar 18, 2013 at 4:57 PM, Pontus Ullgren <ul...@gmail.com> wrote:
>>> Hello Chris,
>>>
>>> On Mon, Mar 18, 2013 at 8:54 PM, Chris Wolf <cw...@gmail.com> wrote:
>>>> Claus,
>>>>
>>>> I have a few further questions about CronScheduledRoutePolicy. I
>>>> noticed that it has setters such as setRouteStartTime,
>>>> setRouteStopTime, each which takes a cron expression string. What I'm
>>>> looking for is to be able to use a cron expression for the start, but
>>>> a relative time length for stop. Otherwise, I need to write code to
>>>> parse the start time expression, then calculate a stop time cron
>>>> expression. Any ideas?
>>>>
>>> Depending on your needs you could enable "sendEmptyMessageWhenIdle" on
>>> the endpoint and then suspend the route when you receive a empty
>>> message. Which means that there is no more files to poll at the
>>> moment.
>>> You can use the content based route EIP for this.
>>
>> That is interesting to know, thanks. In my case, the files at the
>> remote end are themselves deposited at an irregular rate, but within a
>> defined time window, so during that time window, there will be
>> intermittent idleness...
>>>
>>> Another solution would be to write your own RoutePolicy to take care
>>> of your needs.
>>
>> Yes, this sounds like the best approach...
>>
>>
>>>
>>> I just started to wonder if it might be possible to combine the
>>> CronScheduledRoutePolicy with a SimpleScheduledRoutePolicy.
>>> I have _not_ tested this so I'm not sure if it works. It might be that
>>> there is a collision in the way they work with Quartz.
>>>
>>>> Also I see that CronScheduledRoutePolicy has setRouteResumeTime,
>>>> setRouteSuspendTime such that for my FTP poll window, I could either
>>>> do start/stop or resume/suspend - which is recommended?
>>>>
>>> I would highly recommend resume/suspend.
>>> I've had some thread leak problem with the file component when it was
>>> repetitively started/stopped.
>>
>> Ok, but I guess the first policy callback with be onStart since the
>> route will be
>> configured with noAutoStartup(), so upon that first onStart, I'll
>> suspend then toggle
>> between onSuspend/onResume...
>>
> Yes this is what I do. We have a route that should start 06:30 each
> day and then poll all files that are in the folder at that time.
> After that it should suspend.
>
> Here is some pseudo code.
> SuspendRouteProcessor is a processor that suspends the route based on route id.
> ---
> String cronStr = "* 30 6 * * * ?";
> String input = "ftp://user@remotehost/inbox?sendEmptyMessageWhenIdle=true&password=secret";
> CronScheduledRoutePolicy scheduledRP = new CronScheduledRoutePolicy();
> scheduledRoutePolicy.setRouteStartTime(cronStr);
> scheduledRoutePolicy.setRouteResumeTime(cronStr);
>
> from(input)
> .routeId("input1")
> .routePolicy(versionPolicy, scheduledRoutePolicy)
> .noAutoStartup()
> .choice()
> .when(body().isNotNull())
> .to("direct:processFiles")
> .end()
> .log(LoggingLevel.DEBUG, "All files processed, suspend route")
> .process(new SuspendRouteProcessor("input1"))
> ;
> --
>
> The only downside with this is that after the initial start we get a
> WARN log message that the route can not be started since it is in
> suspend state.
> But as long as you can live with the WARN log it works.
>
>>>
>>> // Pontus
>>>
>>>> Thanks,
>>>>
>>>> Chris
>>>>
>>>> On Sat, Mar 2, 2013 at 1:43 AM, Claus Ibsen <cl...@gmail.com> wrote:
>>>>> Hi
>>>>>
>>>>> See about route policy
>>>>> http://camel.apache.org/routepolicy
>>>>>
>>>>> And the scheduled route policy
>>>>> http://camel.apache.org/scheduledroutepolicy.html
>>>>>
>>>>> On Sat, Mar 2, 2013 at 12:15 AM, Chris Wolf <cw...@gmail.com> wrote:
>>>>>> I have a requirement to download files via FTP during a certain time
>>>>>> window and according to a schedule. e.g. Only on trading days between
>>>>>> 6:30AM and 7:00AM. The FTP component, alone, seems to just do
>>>>>> indefinite polling according to delay/initialDelay.
>>>>>>
>>>>>> From the "Camel In Action" book, chapter 7, I see some examples of
>>>>>> sending a text message with the Timer and Quartz components, but I
>>>>>> can't quite see how to put that together to implement "kicking off
>>>>>> routes at specified intervals", mentioned in the best practices list
>>>>>> at the end of that chapter. How would I use quartz to stop/start the
>>>>>> FTP component, or the route that it's in?
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Chris
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Claus Ibsen
>>>>> -----------------
>>>>> Red Hat, Inc.
>>>>> FuseSource is now part of Red Hat
>>>>> Email: cibsen@redhat.com
>>>>> Web: http://fusesource.com
>>>>> Twitter: davsclaus
>>>>> Blog: http://davsclaus.com
>>>>> Author of Camel in Action: http://www.manning.com/ibsen
Re: How to schedule FTP with quartz?
Posted by Pontus Ullgren <ul...@gmail.com>.
Hello,
On Tue, Mar 19, 2013 at 11:22 PM, Chris Wolf <cw...@gmail.com> wrote:
> On Mon, Mar 18, 2013 at 4:57 PM, Pontus Ullgren <ul...@gmail.com> wrote:
>> Hello Chris,
>>
>> On Mon, Mar 18, 2013 at 8:54 PM, Chris Wolf <cw...@gmail.com> wrote:
>>> Claus,
>>>
>>> I have a few further questions about CronScheduledRoutePolicy. I
>>> noticed that it has setters such as setRouteStartTime,
>>> setRouteStopTime, each which takes a cron expression string. What I'm
>>> looking for is to be able to use a cron expression for the start, but
>>> a relative time length for stop. Otherwise, I need to write code to
>>> parse the start time expression, then calculate a stop time cron
>>> expression. Any ideas?
>>>
>> Depending on your needs you could enable "sendEmptyMessageWhenIdle" on
>> the endpoint and then suspend the route when you receive a empty
>> message. Which means that there is no more files to poll at the
>> moment.
>> You can use the content based route EIP for this.
>
> That is interesting to know, thanks. In my case, the files at the
> remote end are themselves deposited at an irregular rate, but within a
> defined time window, so during that time window, there will be
> intermittent idleness...
>>
>> Another solution would be to write your own RoutePolicy to take care
>> of your needs.
>
> Yes, this sounds like the best approach...
>
>
>>
>> I just started to wonder if it might be possible to combine the
>> CronScheduledRoutePolicy with a SimpleScheduledRoutePolicy.
>> I have _not_ tested this so I'm not sure if it works. It might be that
>> there is a collision in the way they work with Quartz.
>>
>>> Also I see that CronScheduledRoutePolicy has setRouteResumeTime,
>>> setRouteSuspendTime such that for my FTP poll window, I could either
>>> do start/stop or resume/suspend - which is recommended?
>>>
>> I would highly recommend resume/suspend.
>> I've had some thread leak problem with the file component when it was
>> repetitively started/stopped.
>
> Ok, but I guess the first policy callback with be onStart since the
> route will be
> configured with noAutoStartup(), so upon that first onStart, I'll
> suspend then toggle
> between onSuspend/onResume...
>
Yes this is what I do. We have a route that should start 06:30 each
day and then poll all files that are in the folder at that time.
After that it should suspend.
Here is some pseudo code.
SuspendRouteProcessor is a processor that suspends the route based on route id.
---
String cronStr = "* 30 6 * * * ?";
String input = "ftp://user@remotehost/inbox?sendEmptyMessageWhenIdle=true&password=secret";
CronScheduledRoutePolicy scheduledRP = new CronScheduledRoutePolicy();
scheduledRoutePolicy.setRouteStartTime(cronStr);
scheduledRoutePolicy.setRouteResumeTime(cronStr);
from(input)
.routeId("input1")
.routePolicy(versionPolicy, scheduledRoutePolicy)
.noAutoStartup()
.choice()
.when(body().isNotNull())
.to("direct:processFiles")
.end()
.log(LoggingLevel.DEBUG, "All files processed, suspend route")
.process(new SuspendRouteProcessor("input1"))
;
--
The only downside with this is that after the initial start we get a
WARN log message that the route can not be started since it is in
suspend state.
But as long as you can live with the WARN log it works.
>>
>> // Pontus
>>
>>> Thanks,
>>>
>>> Chris
>>>
>>> On Sat, Mar 2, 2013 at 1:43 AM, Claus Ibsen <cl...@gmail.com> wrote:
>>>> Hi
>>>>
>>>> See about route policy
>>>> http://camel.apache.org/routepolicy
>>>>
>>>> And the scheduled route policy
>>>> http://camel.apache.org/scheduledroutepolicy.html
>>>>
>>>> On Sat, Mar 2, 2013 at 12:15 AM, Chris Wolf <cw...@gmail.com> wrote:
>>>>> I have a requirement to download files via FTP during a certain time
>>>>> window and according to a schedule. e.g. Only on trading days between
>>>>> 6:30AM and 7:00AM. The FTP component, alone, seems to just do
>>>>> indefinite polling according to delay/initialDelay.
>>>>>
>>>>> From the "Camel In Action" book, chapter 7, I see some examples of
>>>>> sending a text message with the Timer and Quartz components, but I
>>>>> can't quite see how to put that together to implement "kicking off
>>>>> routes at specified intervals", mentioned in the best practices list
>>>>> at the end of that chapter. How would I use quartz to stop/start the
>>>>> FTP component, or the route that it's in?
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Chris
>>>>
>>>>
>>>>
>>>> --
>>>> Claus Ibsen
>>>> -----------------
>>>> Red Hat, Inc.
>>>> FuseSource is now part of Red Hat
>>>> Email: cibsen@redhat.com
>>>> Web: http://fusesource.com
>>>> Twitter: davsclaus
>>>> Blog: http://davsclaus.com
>>>> Author of Camel in Action: http://www.manning.com/ibsen
Re: How to schedule FTP with quartz?
Posted by Chris Wolf <cw...@gmail.com>.
On Mon, Mar 18, 2013 at 4:57 PM, Pontus Ullgren <ul...@gmail.com> wrote:
> Hello Chris,
>
> On Mon, Mar 18, 2013 at 8:54 PM, Chris Wolf <cw...@gmail.com> wrote:
>> Claus,
>>
>> I have a few further questions about CronScheduledRoutePolicy. I
>> noticed that it has setters such as setRouteStartTime,
>> setRouteStopTime, each which takes a cron expression string. What I'm
>> looking for is to be able to use a cron expression for the start, but
>> a relative time length for stop. Otherwise, I need to write code to
>> parse the start time expression, then calculate a stop time cron
>> expression. Any ideas?
>>
> Depending on your needs you could enable "sendEmptyMessageWhenIdle" on
> the endpoint and then suspend the route when you receive a empty
> message. Which means that there is no more files to poll at the
> moment.
> You can use the content based route EIP for this.
That is interesting to know, thanks. In my case, the files at the
remote end are themselves deposited at an irregular rate, but within a
defined time window, so during that time window, there will be
intermittent idleness...
>
> Another solution would be to write your own RoutePolicy to take care
> of your needs.
Yes, this sounds like the best approach...
>
> I just started to wonder if it might be possible to combine the
> CronScheduledRoutePolicy with a SimpleScheduledRoutePolicy.
> I have _not_ tested this so I'm not sure if it works. It might be that
> there is a collision in the way they work with Quartz.
>
>> Also I see that CronScheduledRoutePolicy has setRouteResumeTime,
>> setRouteSuspendTime such that for my FTP poll window, I could either
>> do start/stop or resume/suspend - which is recommended?
>>
> I would highly recommend resume/suspend.
> I've had some thread leak problem with the file component when it was
> repetitively started/stopped.
Ok, but I guess the first policy callback with be onStart since the
route will be
configured with noAutoStartup(), so upon that first onStart, I'll
suspend then toggle
between onSuspend/onResume...
>
> // Pontus
>
>> Thanks,
>>
>> Chris
>>
>> On Sat, Mar 2, 2013 at 1:43 AM, Claus Ibsen <cl...@gmail.com> wrote:
>>> Hi
>>>
>>> See about route policy
>>> http://camel.apache.org/routepolicy
>>>
>>> And the scheduled route policy
>>> http://camel.apache.org/scheduledroutepolicy.html
>>>
>>> On Sat, Mar 2, 2013 at 12:15 AM, Chris Wolf <cw...@gmail.com> wrote:
>>>> I have a requirement to download files via FTP during a certain time
>>>> window and according to a schedule. e.g. Only on trading days between
>>>> 6:30AM and 7:00AM. The FTP component, alone, seems to just do
>>>> indefinite polling according to delay/initialDelay.
>>>>
>>>> From the "Camel In Action" book, chapter 7, I see some examples of
>>>> sending a text message with the Timer and Quartz components, but I
>>>> can't quite see how to put that together to implement "kicking off
>>>> routes at specified intervals", mentioned in the best practices list
>>>> at the end of that chapter. How would I use quartz to stop/start the
>>>> FTP component, or the route that it's in?
>>>>
>>>> Thanks,
>>>>
>>>> Chris
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> -----------------
>>> Red Hat, Inc.
>>> FuseSource is now part of Red Hat
>>> Email: cibsen@redhat.com
>>> Web: http://fusesource.com
>>> Twitter: davsclaus
>>> Blog: http://davsclaus.com
>>> Author of Camel in Action: http://www.manning.com/ibsen
Re: How to schedule FTP with quartz?
Posted by Pontus Ullgren <ul...@gmail.com>.
Hello Chris,
On Mon, Mar 18, 2013 at 8:54 PM, Chris Wolf <cw...@gmail.com> wrote:
> Claus,
>
> I have a few further questions about CronScheduledRoutePolicy. I
> noticed that it has setters such as setRouteStartTime,
> setRouteStopTime, each which takes a cron expression string. What I'm
> looking for is to be able to use a cron expression for the start, but
> a relative time length for stop. Otherwise, I need to write code to
> parse the start time expression, then calculate a stop time cron
> expression. Any ideas?
>
Depending on your needs you could enable "sendEmptyMessageWhenIdle" on
the endpoint and then suspend the route when you receive a empty
message. Which means that there is no more files to poll at the
moment.
You can use the content based route EIP for this.
Another solution would be to write your own RoutePolicy to take care
of your needs.
I just started to wonder if it might be possible to combine the
CronScheduledRoutePolicy with a SimpleScheduledRoutePolicy.
I have _not_ tested this so I'm not sure if it works. It might be that
there is a collision in the way they work with Quartz.
> Also I see that CronScheduledRoutePolicy has setRouteResumeTime,
> setRouteSuspendTime such that for my FTP poll window, I could either
> do start/stop or resume/suspend - which is recommended?
>
I would highly recommend resume/suspend.
I've had some thread leak problem with the file component when it was
repetitively started/stopped.
// Pontus
> Thanks,
>
> Chris
>
> On Sat, Mar 2, 2013 at 1:43 AM, Claus Ibsen <cl...@gmail.com> wrote:
>> Hi
>>
>> See about route policy
>> http://camel.apache.org/routepolicy
>>
>> And the scheduled route policy
>> http://camel.apache.org/scheduledroutepolicy.html
>>
>> On Sat, Mar 2, 2013 at 12:15 AM, Chris Wolf <cw...@gmail.com> wrote:
>>> I have a requirement to download files via FTP during a certain time
>>> window and according to a schedule. e.g. Only on trading days between
>>> 6:30AM and 7:00AM. The FTP component, alone, seems to just do
>>> indefinite polling according to delay/initialDelay.
>>>
>>> From the "Camel In Action" book, chapter 7, I see some examples of
>>> sending a text message with the Timer and Quartz components, but I
>>> can't quite see how to put that together to implement "kicking off
>>> routes at specified intervals", mentioned in the best practices list
>>> at the end of that chapter. How would I use quartz to stop/start the
>>> FTP component, or the route that it's in?
>>>
>>> Thanks,
>>>
>>> Chris
>>
>>
>>
>> --
>> Claus Ibsen
>> -----------------
>> Red Hat, Inc.
>> FuseSource is now part of Red Hat
>> Email: cibsen@redhat.com
>> Web: http://fusesource.com
>> Twitter: davsclaus
>> Blog: http://davsclaus.com
>> Author of Camel in Action: http://www.manning.com/ibsen
Re: How to schedule FTP with quartz?
Posted by Chris Wolf <cw...@gmail.com>.
Claus,
I have a few further questions about CronScheduledRoutePolicy. I
noticed that it has setters such as setRouteStartTime,
setRouteStopTime, each which takes a cron expression string. What I'm
looking for is to be able to use a cron expression for the start, but
a relative time length for stop. Otherwise, I need to write code to
parse the start time expression, then calculate a stop time cron
expression. Any ideas?
Also I see that CronScheduledRoutePolicy has setRouteResumeTime,
setRouteSuspendTime such that for my FTP poll window, I could either
do start/stop or resume/suspend - which is recommended?
Thanks,
Chris
On Sat, Mar 2, 2013 at 1:43 AM, Claus Ibsen <cl...@gmail.com> wrote:
> Hi
>
> See about route policy
> http://camel.apache.org/routepolicy
>
> And the scheduled route policy
> http://camel.apache.org/scheduledroutepolicy.html
>
> On Sat, Mar 2, 2013 at 12:15 AM, Chris Wolf <cw...@gmail.com> wrote:
>> I have a requirement to download files via FTP during a certain time
>> window and according to a schedule. e.g. Only on trading days between
>> 6:30AM and 7:00AM. The FTP component, alone, seems to just do
>> indefinite polling according to delay/initialDelay.
>>
>> From the "Camel In Action" book, chapter 7, I see some examples of
>> sending a text message with the Timer and Quartz components, but I
>> can't quite see how to put that together to implement "kicking off
>> routes at specified intervals", mentioned in the best practices list
>> at the end of that chapter. How would I use quartz to stop/start the
>> FTP component, or the route that it's in?
>>
>> Thanks,
>>
>> Chris
>
>
>
> --
> Claus Ibsen
> -----------------
> Red Hat, Inc.
> FuseSource is now part of Red Hat
> Email: cibsen@redhat.com
> Web: http://fusesource.com
> Twitter: davsclaus
> Blog: http://davsclaus.com
> Author of Camel in Action: http://www.manning.com/ibsen
Re: How to schedule FTP with quartz?
Posted by Chris Wolf <cw...@gmail.com>.
Ok, that's what I was looking for - specifically the Cron version:
http://camel.apache.org/cronscheduledroutepolicy.html
Thanks for the help,
Chris
On Sat, Mar 2, 2013 at 1:43 AM, Claus Ibsen <cl...@gmail.com> wrote:
> Hi
>
> See about route policy
> http://camel.apache.org/routepolicy
>
> And the scheduled route policy
> http://camel.apache.org/scheduledroutepolicy.html
>
> On Sat, Mar 2, 2013 at 12:15 AM, Chris Wolf <cw...@gmail.com> wrote:
>> I have a requirement to download files via FTP during a certain time
>> window and according to a schedule. e.g. Only on trading days between
>> 6:30AM and 7:00AM. The FTP component, alone, seems to just do
>> indefinite polling according to delay/initialDelay.
>>
>> From the "Camel In Action" book, chapter 7, I see some examples of
>> sending a text message with the Timer and Quartz components, but I
>> can't quite see how to put that together to implement "kicking off
>> routes at specified intervals", mentioned in the best practices list
>> at the end of that chapter. How would I use quartz to stop/start the
>> FTP component, or the route that it's in?
>>
>> Thanks,
>>
>> Chris
>
>
>
> --
> Claus Ibsen
> -----------------
> Red Hat, Inc.
> FuseSource is now part of Red Hat
> Email: cibsen@redhat.com
> Web: http://fusesource.com
> Twitter: davsclaus
> Blog: http://davsclaus.com
> Author of Camel in Action: http://www.manning.com/ibsen
Re: How to schedule FTP with quartz?
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
See about route policy
http://camel.apache.org/routepolicy
And the scheduled route policy
http://camel.apache.org/scheduledroutepolicy.html
On Sat, Mar 2, 2013 at 12:15 AM, Chris Wolf <cw...@gmail.com> wrote:
> I have a requirement to download files via FTP during a certain time
> window and according to a schedule. e.g. Only on trading days between
> 6:30AM and 7:00AM. The FTP component, alone, seems to just do
> indefinite polling according to delay/initialDelay.
>
> From the "Camel In Action" book, chapter 7, I see some examples of
> sending a text message with the Timer and Quartz components, but I
> can't quite see how to put that together to implement "kicking off
> routes at specified intervals", mentioned in the best practices list
> at the end of that chapter. How would I use quartz to stop/start the
> FTP component, or the route that it's in?
>
> Thanks,
>
> Chris
--
Claus Ibsen
-----------------
Red Hat, Inc.
FuseSource is now part of Red Hat
Email: cibsen@redhat.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen