You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@camel.apache.org by Hema T <he...@ge.com> on 2012/10/22 13:23:01 UTC

Camel Routes - Multi threading nor running parallel

Hi, 

I have defined a route to take messages from activemq and do some processing
and send it to other end point. I have configured thread pool so that many
threads pick up messages from the activemq and process it in parallel. But
this is not happening. 

>From the logs I could figure out that the messages are not picked up in
parallel. Only after the first message if processed (by thread 1) and
delivered to the end system, the next message is picked up (by thread 2).
Here is the code snippet. 

*from(ACTIVEMQ_QUEUE)
                .threads().executorServiceRef(bigpool)
*                .log(LoggingLevel.INFO, 'Thread started')
                *.processRef('Processor')*
                .log(LoggingLevel.INFO, 'Sending to MPI')
               * .to(TCP_ENDPOINT)
                .processRef('Processor2')
                .to(TCP_ENDPOINT2)*
                .log(LoggingLevel.INFO, 'completed')
               * .end()*


Threadpool is defined in the camel context file as:
<camel:threadPoolProfile id="myDefaultProfile"
                           defaultProfile="true"
                           maxPoolSize="50"/>
        <camel:threadPoolProfile id="bigPool" maxPoolSize="5"/>

Ideally I could seee various threads are picking up the messages (like
thread #8, thread #20, ....) but they are not picked up in parallel instead
they are picking up message in sequence (after processing the first message,
second is picked up from the queue). 

Please help me to make the threads pick up messages and process them in
parallel.  

Thanks in advance,
Hema 



--
View this message in context: http://camel.465427.n5.nabble.com/Camel-Routes-Multi-threading-nor-running-parallel-tp5721385.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Camel Routes - Multi threading nor running parallel

Posted by Stephan Burkard <sb...@gmail.com>.

You have to use concurrentConsumer settings on the JMS or ActiveMQ
component as Claus suggested: http://camel.apache.org/jms

The JMS component has per default just 1 consumer and therefore the
messages are consumed synchronous no matter what you do after the
from(ACTIVEMQ_QUEUE) statement in your route.

Stephan

On Wed, Mar 5, 2014 at 9:38 PM, ame <ah...@noaa.gov> wrote:

> Hello Hema,
>
> Have you had a luck to resolve this...? I am having similar issue...
>
>
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/Camel-Routes-Multi-threading-nor-running-parallel-tp5721385p5748359.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>

Re: Camel Routes - Multi threading nor running parallel

Posted by ame <ah...@noaa.gov>.

Hello Hema,

Have you had a luck to resolve this...? I am having similar issue...



--
View this message in context: http://camel.465427.n5.nabble.com/Camel-Routes-Multi-threading-nor-running-parallel-tp5721385p5748359.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Camel Routes - Multi threading nor running parallel

Posted by Claus Ibsen <cl...@gmail.com>.

Hi

The JMS component (consumer) has built-in concurrency. So use that instead.

Its the concurrentConsumers and maxConcurrentConsumers options
http://camel.apache.org/jms

And you may also take a look at the asyncConsumer option as well.


On Mon, Oct 22, 2012 at 1:23 PM, Hema T <he...@ge.com> wrote:
> Hi,
>
> I have defined a route to take messages from activemq and do some processing
> and send it to other end point. I have configured thread pool so that many
> threads pick up messages from the activemq and process it in parallel. But
> this is not happening.
>
> From the logs I could figure out that the messages are not picked up in
> parallel. Only after the first message if processed (by thread 1) and
> delivered to the end system, the next message is picked up (by thread 2).
> Here is the code snippet.
>
> *from(ACTIVEMQ_QUEUE)
>                 .threads().executorServiceRef(bigpool)
> *                .log(LoggingLevel.INFO, 'Thread started')
>                 *.processRef('Processor')*
>                 .log(LoggingLevel.INFO, 'Sending to MPI')
>                * .to(TCP_ENDPOINT)
>                 .processRef('Processor2')
>                 .to(TCP_ENDPOINT2)*
>                 .log(LoggingLevel.INFO, 'completed')
>                * .end()*
>
>
> Threadpool is defined in the camel context file as:
> <camel:threadPoolProfile id="myDefaultProfile"
>                            defaultProfile="true"
>                            maxPoolSize="50"/>
>         <camel:threadPoolProfile id="bigPool" maxPoolSize="5"/>
>
> Ideally I could seee various threads are picking up the messages (like
> thread #8, thread #20, ....) but they are not picked up in parallel instead
> they are picking up message in sequence (after processing the first message,
> second is picked up from the queue).
>
> Please help me to make the threads pick up messages and process them in
> parallel.
>
> Thanks in advance,
> Hema
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Camel-Routes-Multi-threading-nor-running-parallel-tp5721385.html
> Sent from the Camel - Users mailing list archive at Nabble.com.



-- 
Claus Ibsen
-----------------
Red Hat, Inc.
FuseSource is now part of Red Hat
Email: cibsen@redhat.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen