Re: Question regarding SEDA & threadpool

[Claus Ibsen <cl...@gmail.com>]

On Wed, Jul 25, 2012 at 12:40 AM, wantastic <Wa...@barclays.com> wrote:
> Hello all,
>
> I am a beginner of Camel and trying to figure out how to create route
> properly.
> What I'm trying to do is pretty simple. I have a JMS queue listener which
> takes message one by one. Then this message will be logged by WireTap and
> gets thrown into a threadpool(a pool of worker threads) to be processed. My
> first approach was like this:
> <camelContext -->
> <route>
>         <threadPool id="myThreadPool" threadName="MyThread" poolSize="10"
> maxPoolSize="20" maxQueueSize="1000" />
>         <threadPool id="dataLoggerPool" threadName="DataLogger" poolSize="1"
> maxPoolSize="1" maxQueueSize="1000" />
>
>         <from uri="pns-jms:queue:blahblah" />
>         <wireTap uri="dataLogger" executorServiceRef="dataLoggerPool"/>
>         <threads executorServiceRef="myThreadPool">
>                 <to uri="myValidationProcess" />
>                 <to uri="myConvertProcess" />
>         </threads>
> </camelContext>
>
> However, this turned out to be a wrong implementation since I realized that
> messages are not being distributed to multiple threads concurrently. For
> example, if I'm doing "Thread.currentThread.sleep(3000)" inside
> myConvertProcess, I see threads are being invoked one at a time, not 10
> threads at the same time. In order to solve this problem, I had to use SEDA
> to send message to the threadpool asynchronously. So my second
> implementation came out like this:
>

The JMS consumer in the <from> is synchronized by default. eg it waits
until the message is done processed before it consume the next message
from the queue. This ensures the messages is processed sequentielly.

You can use the asynConsumer=true option, to allow it to consume the
next message eagerly.




>         <route>
>                 <from uri="pns-jms:queue:blahblah" />
>                 <to uri="seda:processMsg?size=1000;blockWhenFull=true" />
>     </route>
>         <route>
>                 <from uri="seda:processMsg" />
>                 <threads executorServiceRef="myThreadPool">
>                         <to uri="myValidationProcess" />
>                         <to uri="myConvertProcess" />
>                 </threads>
>         </route>
>
> This seems to be working, however, I get two blocking queues, SEDA and
> threadpool, which is not so desirable. Therefore, I decided to go with
> SEDA's concurrentConsumers.
>
>         <route>
>                 <from uri="pns-jms:queue:blahblah" />
>                 <to uri="seda:processMsg?size=1000;blockWhenFull=true" />
>     </route>
>         <route>
>                 <from uri="seda:processMsg?concurrentConsumers=10" />
>                 <to uri="myValidationProcess" />
>                 <to uri="myConvertProcess" />
>         </route>
>
> This seems to be working okay but I came to another question: What if the
> seda queue is full? Will SEDA block from queue to get message when its queue
> is full?
> Does anyone have a better suggestion for implementation? It is a simple
> model: get message, throw it into a threadpool to be picked up by a worker
> thread to process it.
>
> If anyone can give feedback on this, I will be very grateful.
>
> Thank you,
> Wan
>
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Question-regarding-SEDA-threadpool-tp5716431.html
> Sent from the Camel - Users mailing list archive at Nabble.com.



-- 
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen