You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Okello Nelson <cn...@gmail.com> on 2013/04/12 15:20:46 UTC
How to implement concurrent route processing
Hi Guys,
I'm currently working on a project in which I have several routes. To
improve throughput, I'd like to have all the routes executing concurrently.
What I've observed so far is that when one route is executing, the others
do not.
Each route is composed of File, JMS, and JPA endpoints, with possible
additional endpoints later.
Any ideas on how I can implement this will be appreciated very much.
Kind Regards,
Okello Nelson.
Re: How to implement concurrent route processing
Posted by Okello Nelson <cn...@gmail.com>.
Hi Claus,
The blog entry was extremely useful. I'm using Camel 2.10.4, and I'm
assuming "async" was renamed to "threads" in this version.
Thanks a lot.
Kind Regards,
Okello Nelson.
On Sat, Apr 13, 2013 at 11:26 AM, Okello Nelson <cn...@gmail.com> wrote:
> Thanks loads, Claus.
>
> I'm checking it out. I'll let you know how it goes.
>
> Kind Regards,
> Okello it Nelson.
> On Apr 13, 2013 11:09 AM, "Claus Ibsen" <cl...@gmail.com> wrote:
>
>> Yes this is possible
>>
>> There is a threads in the DSL you can use for that.
>>
>> I wrote a blog entry many years ago - note back then threads was named
>> async
>> http://www.davsclaus.com/2009/05/on-road-to-camel-20-concurrency-with.html
>>
>>
>> On Sat, Apr 13, 2013 at 6:41 AM, Okello Nelson <cn...@gmail.com>
>> wrote:
>> > Hi Christian,
>> >
>> > I think the problem was with my file sizes. When i started splitting the
>> > files into smaller chunks before feeding to the queues, i started seeing
>> > the concurrent execution.
>> >
>> > This started me thinking: is it possible for the file component to start
>> > processing several files at the same time. I think that, in my setup, it
>> > processes one file then goes on to the next one. This is quite a hold up
>> > when the files are big.
>> >
>> > Kind Regards
>> > On Apr 13, 2013 2:00 AM, "Christian Müller" <
>> christian.mueller@gmail.com>
>> > wrote:
>> >
>> >> The camel routes are executed concurrently.
>> >>
>> >> Best,
>> >> Christian
>> >>
>> >> Sent from a mobile device
>> >> Am 12.04.2013 15:21 schrieb "Okello Nelson" <cn...@gmail.com>:
>> >>
>> >> > Hi Guys,
>> >> >
>> >> > I'm currently working on a project in which I have several routes. To
>> >> > improve throughput, I'd like to have all the routes executing
>> >> concurrently.
>> >> > What I've observed so far is that when one route is executing, the
>> others
>> >> > do not.
>> >> >
>> >> > Each route is composed of File, JMS, and JPA endpoints, with possible
>> >> > additional endpoints later.
>> >> >
>> >> > Any ideas on how I can implement this will be appreciated very much.
>> >> >
>> >> > Kind Regards,
>> >> > Okello Nelson.
>> >> >
>> >>
>>
>>
>>
>> --
>> Claus Ibsen
>> -----------------
>> Red Hat, Inc.
>> FuseSource is now part of Red Hat
>> Email: cibsen@redhat.com
>> Web: http://fusesource.com
>> Twitter: davsclaus
>> Blog: http://davsclaus.com
>> Author of Camel in Action: http://www.manning.com/ibsen
>>
>
--
Kind Regards,
Okello Nelson
+254 722 137 826
cn.okello@gmail.com
Re: How to implement concurrent route processing
Posted by Okello Nelson <cn...@gmail.com>.
Thanks loads, Claus.
I'm checking it out. I'll let you know how it goes.
Kind Regards,
Okello it Nelson.
On Apr 13, 2013 11:09 AM, "Claus Ibsen" <cl...@gmail.com> wrote:
> Yes this is possible
>
> There is a threads in the DSL you can use for that.
>
> I wrote a blog entry many years ago - note back then threads was named
> async
> http://www.davsclaus.com/2009/05/on-road-to-camel-20-concurrency-with.html
>
>
> On Sat, Apr 13, 2013 at 6:41 AM, Okello Nelson <cn...@gmail.com>
> wrote:
> > Hi Christian,
> >
> > I think the problem was with my file sizes. When i started splitting the
> > files into smaller chunks before feeding to the queues, i started seeing
> > the concurrent execution.
> >
> > This started me thinking: is it possible for the file component to start
> > processing several files at the same time. I think that, in my setup, it
> > processes one file then goes on to the next one. This is quite a hold up
> > when the files are big.
> >
> > Kind Regards
> > On Apr 13, 2013 2:00 AM, "Christian Müller" <
> christian.mueller@gmail.com>
> > wrote:
> >
> >> The camel routes are executed concurrently.
> >>
> >> Best,
> >> Christian
> >>
> >> Sent from a mobile device
> >> Am 12.04.2013 15:21 schrieb "Okello Nelson" <cn...@gmail.com>:
> >>
> >> > Hi Guys,
> >> >
> >> > I'm currently working on a project in which I have several routes. To
> >> > improve throughput, I'd like to have all the routes executing
> >> concurrently.
> >> > What I've observed so far is that when one route is executing, the
> others
> >> > do not.
> >> >
> >> > Each route is composed of File, JMS, and JPA endpoints, with possible
> >> > additional endpoints later.
> >> >
> >> > Any ideas on how I can implement this will be appreciated very much.
> >> >
> >> > Kind Regards,
> >> > Okello Nelson.
> >> >
> >>
>
>
>
> --
> Claus Ibsen
> -----------------
> Red Hat, Inc.
> FuseSource is now part of Red Hat
> Email: cibsen@redhat.com
> Web: http://fusesource.com
> Twitter: davsclaus
> Blog: http://davsclaus.com
> Author of Camel in Action: http://www.manning.com/ibsen
>
Re: How to implement concurrent route processing
Posted by Claus Ibsen <cl...@gmail.com>.
Yes this is possible
There is a threads in the DSL you can use for that.
I wrote a blog entry many years ago - note back then threads was named async
http://www.davsclaus.com/2009/05/on-road-to-camel-20-concurrency-with.html
On Sat, Apr 13, 2013 at 6:41 AM, Okello Nelson <cn...@gmail.com> wrote:
> Hi Christian,
>
> I think the problem was with my file sizes. When i started splitting the
> files into smaller chunks before feeding to the queues, i started seeing
> the concurrent execution.
>
> This started me thinking: is it possible for the file component to start
> processing several files at the same time. I think that, in my setup, it
> processes one file then goes on to the next one. This is quite a hold up
> when the files are big.
>
> Kind Regards
> On Apr 13, 2013 2:00 AM, "Christian Müller" <ch...@gmail.com>
> wrote:
>
>> The camel routes are executed concurrently.
>>
>> Best,
>> Christian
>>
>> Sent from a mobile device
>> Am 12.04.2013 15:21 schrieb "Okello Nelson" <cn...@gmail.com>:
>>
>> > Hi Guys,
>> >
>> > I'm currently working on a project in which I have several routes. To
>> > improve throughput, I'd like to have all the routes executing
>> concurrently.
>> > What I've observed so far is that when one route is executing, the others
>> > do not.
>> >
>> > Each route is composed of File, JMS, and JPA endpoints, with possible
>> > additional endpoints later.
>> >
>> > Any ideas on how I can implement this will be appreciated very much.
>> >
>> > Kind Regards,
>> > Okello Nelson.
>> >
>>
--
Claus Ibsen
-----------------
Red Hat, Inc.
FuseSource is now part of Red Hat
Email: cibsen@redhat.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
Re: How to implement concurrent route processing
Posted by Okello Nelson <cn...@gmail.com>.
Hi Christian,
I think the problem was with my file sizes. When i started splitting the
files into smaller chunks before feeding to the queues, i started seeing
the concurrent execution.
This started me thinking: is it possible for the file component to start
processing several files at the same time. I think that, in my setup, it
processes one file then goes on to the next one. This is quite a hold up
when the files are big.
Kind Regards
On Apr 13, 2013 2:00 AM, "Christian Müller" <ch...@gmail.com>
wrote:
> The camel routes are executed concurrently.
>
> Best,
> Christian
>
> Sent from a mobile device
> Am 12.04.2013 15:21 schrieb "Okello Nelson" <cn...@gmail.com>:
>
> > Hi Guys,
> >
> > I'm currently working on a project in which I have several routes. To
> > improve throughput, I'd like to have all the routes executing
> concurrently.
> > What I've observed so far is that when one route is executing, the others
> > do not.
> >
> > Each route is composed of File, JMS, and JPA endpoints, with possible
> > additional endpoints later.
> >
> > Any ideas on how I can implement this will be appreciated very much.
> >
> > Kind Regards,
> > Okello Nelson.
> >
>
Re: How to implement concurrent route processing
Posted by Christian Müller <ch...@gmail.com>.
The camel routes are executed concurrently.
Best,
Christian
Sent from a mobile device
Am 12.04.2013 15:21 schrieb "Okello Nelson" <cn...@gmail.com>:
> Hi Guys,
>
> I'm currently working on a project in which I have several routes. To
> improve throughput, I'd like to have all the routes executing concurrently.
> What I've observed so far is that when one route is executing, the others
> do not.
>
> Each route is composed of File, JMS, and JPA endpoints, with possible
> additional endpoints later.
>
> Any ideas on how I can implement this will be appreciated very much.
>
> Kind Regards,
> Okello Nelson.
>