You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by klauss42 <kl...@ethalon.de> on 2013/01/15 17:50:02 UTC
Concat routes
Hello Camel experts
I have multiple routes that perform some background processing, in detail
the routes just trigger some Spring-Batch jobs. I want to be able to start
each single route alone, therefore I am using the Servlet component to be
able to trigger the routes by URL.
Now I wanted to concatenate the routes in a new route. The routes should be
perfromed in sequential order. I tried multicast with
"parallelProcessing=false" and that works fine if the new route
"invoiceProcessingRoute" gets triggerred by the Servlet component. The
routes get started in proper sequence. Here is an excerpt of my Camel
config:
<route id="invoiceProcessingServlet">
<from uri="servlet:///invoiceProcessing" />
<to uri="vm:invoiceProcessing?timeout=0" />
<transform>
<constant>Invoice Processing job finished</constant>
</transform>
</route>
<route id="invoiceProcessingRoute">
<from uri="vm:invoiceProcessing" />
<camel:multicast parallelProcessing="false" stopOnException="false">
<to uri="vm:invoiceMapping?timeout=0" />
<to uri="vm:invoiceControl?timeout=0" />
<to uri="vm:invoiceAccounting?timeout=0" />
</camel:multicast>
</route>
But if I use the Quartz component to trigger the above route by a cron, the
individual routes get started in parallel, which is not the indented
behavior. Here is the scheduled route:
<route id="invoiceProcessingScheduled"
autoStartup="{{il.cron.invoiceProcessing.startup}}">
<from
uri="quartz://invoiceProcessing/?stateful=true&cron={{il.cron.invoiceProcessing}}"
/>
<to uri="vm:invoiceProcessing" />
</route>
The problem is the parallel execution. Do I misunderstand something? Is it a
bug? What is the recommended way to start routes sequentially?
Remark: In these routes I do not use the content of any messages, I just use
Camel to start my Spring-batch jobs.
Thanks for any help
Klaus
--
View this message in context: http://camel.465427.n5.nabble.com/Concat-routes-tp5725581.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Concat routes
Posted by klauss42 <kl...@ethalon.de>.
Hi Christian
> I guess the behavior is different because the quartz component use the
> inOnly MEP and the servlet component use the inOut MEP. I cannot check
> this at present...
>
That makes sense. As documented in http://camel.apache.org/seda.html the
"waitForTaskToComplete" option has as default "IfReplyExpected". So if I
"call" a vm: route from a servlet, then the caller will wait until the route
is finished, because servlet: uses inOut, as you said. If I call vm: route
from quartz, then it is called asynchroneously, because the inOnly behavior.
Therefore I have to use "waitForTaskToComplete=Always".
Thanks for your reply
Klaus
--
View this message in context: http://camel.465427.n5.nabble.com/Concat-routes-tp5725581p5725678.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Concat routes
Posted by Christian Müller <ch...@gmail.com>.
I guess the behavior is different because the quartz component use the
inOnly MEP and the servlet component use the inOut MEP. I cannot check
this at present...
Best,
Christian
Sent from a mobile device
Am 16.01.2013 14:00 schrieb "klauss42" <kl...@ethalon.de>:
> Hi Claus
>
> > What do you mean by parallel?
>
> The 3 routes (vm:invoiceMapping, vm:invoiceControl + vm:invoiceAccounting)
> get started in parallel when the cron starts the vm:invoiceProcessing
> route.
> Here is some log output of a test that hopefully explains the behavior:
>
> Routes "called" sequentially when launched via servlet:
> 2013-01-16 13:50:29,701 [INFO ][invoiceMapping] - Job 'invoiceMapping'
> started
> 2013-01-16 13:50:29,769 [INFO ][invoiceMapping] - Executing step:
> [invoiceMappingStep]
> 2013-01-16 13:50:29,781 [INFO ][invoiceMapping] - invoiceMapping started
> 2013-01-16 13:50:30,781 [INFO ][invoiceMapping] - working 0 sec
> 2013-01-16 13:50:31,781 [INFO ][invoiceMapping] - working 1 sec
> ..
> 2013-01-16 13:50:34,781 [INFO ][invoiceMapping] - invoiceMapping ended
> 2013-01-16 13:50:34,809 [INFO ][invoiceMapping] - Job 'invoiceMapping'
> completed
> 2013-01-16 13:50:34,851 [INFO ][invoiceControl] - Job 'invoiceControl'
> started
> 2013-01-16 13:50:34,870 [INFO ][invoiceControl] - Executing step:
> [invoiceControlStep]
> 2013-01-16 13:50:34,878 [INFO ][invoiceControl] - invoiceControl started
> 2013-01-16 13:50:35,879 [INFO ][invoiceControl] - working 0 sec
> 2013-01-16 13:50:36,879 [INFO ][invoiceControl] - working 1 sec
> ...
> 2013-01-16 13:50:44,881 [INFO ][invoiceControl] - invoiceControl ended
> 2013-01-16 13:50:44,919 [INFO ][invoiceControl] - Job 'invoiceControl'
> completed
> 2013-01-16 13:50:44,958 [INFO ][invoiceAccounting] - Job
> 'invoiceAccounting'
> started
> 2013-01-16 13:50:44,979 [INFO ][invoiceAccounting] - Executing step:
> [invoiceAccountingStep]
> 2013-01-16 13:50:44,994 [INFO ][invoiceAccounting] - invoiceAccounting
> started
> 2013-01-16 13:50:45,995 [INFO ][invoiceAccounting] - working 0 sec
> 2013-01-16 13:50:46,995 [INFO ][invoiceAccounting] - working 1 sec
> ...
> 2013-01-16 13:50:54,997 [INFO ][invoiceAccounting] - invoiceAccounting
> ended
> 2013-01-16 13:50:55,032 [INFO ][invoiceAccounting] - Job
> 'invoiceAccounting'
> completed
>
>
> Routes "called" in parallel execution when launched via quartz:
> 2013-01-16 11:57:33,132 [INFO ][invoiceControl] - Job 'invoiceControl'
> started
> 2013-01-16 11:57:33,132 [INFO ][invoiceAccounting] - Job
> 'invoiceAccounting'
> started
> 2013-01-16 11:57:33,132 [INFO ][invoiceMapping] - Job 'invoiceMapping'
> started
> 2013-01-16 11:57:33,185 [INFO ][invoiceAccounting] - Executing step:
> [invoiceAccountingStep]
> 2013-01-16 11:57:33,221 [INFO ][invoiceAccounting] - invoiceAccounting
> started
> 2013-01-16 11:57:33,221 [INFO ][invoiceControl] - invoiceControl started
> 2013-01-16 11:57:33,221 [INFO ][invoiceMapping] - invoiceMapping started
> 2013-01-16 11:57:34,222 [INFO ][invoiceAccounting] - working 0 sec
> 2013-01-16 11:57:34,223 [INFO ][invoiceControl] - working 0 sec
> 2013-01-16 11:57:34,226 [INFO ][invoiceMapping] - working 0 sec
> 2013-01-16 11:57:35,222 [INFO ][invoiceAccounting] - working 1 sec
> 2013-01-16 11:57:35,224 [INFO ][invoiceControl] - working 1 sec
> 2013-01-16 11:57:35,227 [INFO ][invoiceMapping] - working 1 sec
> 2013-01-16 11:57:36,222 [INFO ][invoiceAccounting] - working 2 sec
> 2013-01-16 11:57:36,224 [INFO ][invoiceControl] - working 2 sec
> 2013-01-16 11:57:36,228 [INFO ][invoiceMapping] - working 2 sec
> 2013-01-16 11:57:37,222 [INFO ][invoiceAccounting] - working 3 sec
> 2013-01-16 11:57:37,224 [INFO ][invoiceControl] - working 3 sec
> ...
>
> In this log output you can see that all 3 routes get fired at the same time
> instead of being executed one after the other. I do not understand why
> Camel
> behaves different if triggered by Quartz. The intended behavior is the
> sequential execution of the routes.
>
>
> > How frequent have you set the cron job to trigger?
>
> The cron is set to run once per night only.
>
> >
> > I think you can set quartz stateful=true, so it wont trigger new jobs
> > while previous jobs are still active.
>
> No, I am already using stateful=true and that only prevents multiple
> parallel executions of the same scheduled route.
>
> > Maybe that is what you mean by parallel?
>
>
>
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/Concat-routes-tp5725581p5725641.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
Re: Concat routes
Posted by klauss42 <kl...@ethalon.de>.
Hi again
finally I found it on my own: Seems that I need to use the
"waitForTaskToComplete=Always" option on the vm: component to enforce
sequential processing. If I configure my master route like this:
<camel:multicast parallelProcessing="false" stopOnException="false">
<to uri="vm:invoiceMapping?timeout=0&waitForTaskToComplete=Always"
/>
<to uri="vm:invoiceControl?timeout=0&waitForTaskToComplete=Always"
/>
<to
uri="vm:invoiceAccounting?timeout=0&waitForTaskToComplete=Always" />
</camel:multicast>
it works fine from both servlet and quartz trigger.
Thanks
Klaus
--
View this message in context: http://camel.465427.n5.nabble.com/Concat-routes-tp5725581p5725642.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Concat routes
Posted by klauss42 <kl...@ethalon.de>.
Hi Claus
> What do you mean by parallel?
The 3 routes (vm:invoiceMapping, vm:invoiceControl + vm:invoiceAccounting)
get started in parallel when the cron starts the vm:invoiceProcessing route.
Here is some log output of a test that hopefully explains the behavior:
Routes "called" sequentially when launched via servlet:
2013-01-16 13:50:29,701 [INFO ][invoiceMapping] - Job 'invoiceMapping'
started
2013-01-16 13:50:29,769 [INFO ][invoiceMapping] - Executing step:
[invoiceMappingStep]
2013-01-16 13:50:29,781 [INFO ][invoiceMapping] - invoiceMapping started
2013-01-16 13:50:30,781 [INFO ][invoiceMapping] - working 0 sec
2013-01-16 13:50:31,781 [INFO ][invoiceMapping] - working 1 sec
..
2013-01-16 13:50:34,781 [INFO ][invoiceMapping] - invoiceMapping ended
2013-01-16 13:50:34,809 [INFO ][invoiceMapping] - Job 'invoiceMapping'
completed
2013-01-16 13:50:34,851 [INFO ][invoiceControl] - Job 'invoiceControl'
started
2013-01-16 13:50:34,870 [INFO ][invoiceControl] - Executing step:
[invoiceControlStep]
2013-01-16 13:50:34,878 [INFO ][invoiceControl] - invoiceControl started
2013-01-16 13:50:35,879 [INFO ][invoiceControl] - working 0 sec
2013-01-16 13:50:36,879 [INFO ][invoiceControl] - working 1 sec
...
2013-01-16 13:50:44,881 [INFO ][invoiceControl] - invoiceControl ended
2013-01-16 13:50:44,919 [INFO ][invoiceControl] - Job 'invoiceControl'
completed
2013-01-16 13:50:44,958 [INFO ][invoiceAccounting] - Job 'invoiceAccounting'
started
2013-01-16 13:50:44,979 [INFO ][invoiceAccounting] - Executing step:
[invoiceAccountingStep]
2013-01-16 13:50:44,994 [INFO ][invoiceAccounting] - invoiceAccounting
started
2013-01-16 13:50:45,995 [INFO ][invoiceAccounting] - working 0 sec
2013-01-16 13:50:46,995 [INFO ][invoiceAccounting] - working 1 sec
...
2013-01-16 13:50:54,997 [INFO ][invoiceAccounting] - invoiceAccounting ended
2013-01-16 13:50:55,032 [INFO ][invoiceAccounting] - Job 'invoiceAccounting'
completed
Routes "called" in parallel execution when launched via quartz:
2013-01-16 11:57:33,132 [INFO ][invoiceControl] - Job 'invoiceControl'
started
2013-01-16 11:57:33,132 [INFO ][invoiceAccounting] - Job 'invoiceAccounting'
started
2013-01-16 11:57:33,132 [INFO ][invoiceMapping] - Job 'invoiceMapping'
started
2013-01-16 11:57:33,185 [INFO ][invoiceAccounting] - Executing step:
[invoiceAccountingStep]
2013-01-16 11:57:33,221 [INFO ][invoiceAccounting] - invoiceAccounting
started
2013-01-16 11:57:33,221 [INFO ][invoiceControl] - invoiceControl started
2013-01-16 11:57:33,221 [INFO ][invoiceMapping] - invoiceMapping started
2013-01-16 11:57:34,222 [INFO ][invoiceAccounting] - working 0 sec
2013-01-16 11:57:34,223 [INFO ][invoiceControl] - working 0 sec
2013-01-16 11:57:34,226 [INFO ][invoiceMapping] - working 0 sec
2013-01-16 11:57:35,222 [INFO ][invoiceAccounting] - working 1 sec
2013-01-16 11:57:35,224 [INFO ][invoiceControl] - working 1 sec
2013-01-16 11:57:35,227 [INFO ][invoiceMapping] - working 1 sec
2013-01-16 11:57:36,222 [INFO ][invoiceAccounting] - working 2 sec
2013-01-16 11:57:36,224 [INFO ][invoiceControl] - working 2 sec
2013-01-16 11:57:36,228 [INFO ][invoiceMapping] - working 2 sec
2013-01-16 11:57:37,222 [INFO ][invoiceAccounting] - working 3 sec
2013-01-16 11:57:37,224 [INFO ][invoiceControl] - working 3 sec
...
In this log output you can see that all 3 routes get fired at the same time
instead of being executed one after the other. I do not understand why Camel
behaves different if triggered by Quartz. The intended behavior is the
sequential execution of the routes.
> How frequent have you set the cron job to trigger?
The cron is set to run once per night only.
>
> I think you can set quartz stateful=true, so it wont trigger new jobs
> while previous jobs are still active.
No, I am already using stateful=true and that only prevents multiple
parallel executions of the same scheduled route.
> Maybe that is what you mean by parallel?
--
View this message in context: http://camel.465427.n5.nabble.com/Concat-routes-tp5725581p5725641.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Concat routes
Posted by Claus Ibsen <cl...@gmail.com>.
On Tue, Jan 15, 2013 at 5:50 PM, klauss42 <kl...@ethalon.de> wrote:
> Hello Camel experts
> I have multiple routes that perform some background processing, in detail
> the routes just trigger some Spring-Batch jobs. I want to be able to start
> each single route alone, therefore I am using the Servlet component to be
> able to trigger the routes by URL.
>
> Now I wanted to concatenate the routes in a new route. The routes should be
> perfromed in sequential order. I tried multicast with
> "parallelProcessing=false" and that works fine if the new route
> "invoiceProcessingRoute" gets triggerred by the Servlet component. The
> routes get started in proper sequence. Here is an excerpt of my Camel
> config:
>
> <route id="invoiceProcessingServlet">
> <from uri="servlet:///invoiceProcessing" />
> <to uri="vm:invoiceProcessing?timeout=0" />
> <transform>
> <constant>Invoice Processing job finished</constant>
> </transform>
> </route>
>
> <route id="invoiceProcessingRoute">
> <from uri="vm:invoiceProcessing" />
> <camel:multicast parallelProcessing="false" stopOnException="false">
> <to uri="vm:invoiceMapping?timeout=0" />
> <to uri="vm:invoiceControl?timeout=0" />
> <to uri="vm:invoiceAccounting?timeout=0" />
> </camel:multicast>
> </route>
>
> But if I use the Quartz component to trigger the above route by a cron, the
> individual routes get started in parallel, which is not the indented
> behavior. Here is the scheduled route:
>
What do you mean by parallel? How frequent have you set the cron job to trigger?
I think you can set quartz stateful=true, so it wont trigger new jobs
while previous jobs are still active.
Maybe that is what you mean by parallel?
> <route id="invoiceProcessingScheduled"
> autoStartup="{{il.cron.invoiceProcessing.startup}}">
> <from
> uri="quartz://invoiceProcessing/?stateful=true&cron={{il.cron.invoiceProcessing}}"
> />
> <to uri="vm:invoiceProcessing" />
> </route>
>
> The problem is the parallel execution. Do I misunderstand something? Is it a
> bug? What is the recommended way to start routes sequentially?
>
> Remark: In these routes I do not use the content of any messages, I just use
> Camel to start my Spring-batch jobs.
>
> Thanks for any help
> Klaus
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Concat-routes-tp5725581.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
Red Hat, Inc.
FuseSource is now part of Red Hat
Email: cibsen@redhat.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen