You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by diwakar <di...@nsn.com> on 2011/08/22 07:14:48 UTC
Thread pool for components in Camel
Hi,
In Servicemix we can configure thread pools for components:
servicemix.corePoolSize = 32
servicemix.maximumPoolSize = 128
In camel, how can I configure thread pool for components? From
documentation parallelProcessing, executorServiceRef options can be set to
configure thread pools. But these options do not exist for all components.
<route>
<from uri="file:input"/>
<log message="Copying ${file:name} to the output directory"/>
<to uri="file:output"/>
</route>
In the above example, how can I configure thread pool for file
component to have multiple threads to poll the "input" folder?
Please let me know your comment.
With Best Regards,
Diwakar
--
View this message in context: http://camel.465427.n5.nabble.com/Thread-pool-for-components-in-Camel-tp4722190p4722190.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Thread pool for components in Camel
Posted by Claus Ibsen <cl...@gmail.com>.
On Fri, Nov 25, 2011 at 10:40 AM, diwakar <di...@nsn.com> wrote:
> Hi,
>
> In Servicemix/JBI thread pools by are there by default (at
> component level).
> Why isn't the same approach taken for Camel? In Camel thread
> (/async) has to be configured explicitly in the route.
> Please let me know your comment.
>
SMX != Camel.
SMX is an ESB and thus its a server and therefore SMX can expose
container level configuration etc.
Camel is a framework which is deployed/embedded into any kind of
runtime. Could indeed by SMX as well. But can run standalone,
WebSphere, Tomcat etc.
Check out this page about Camel thread configuration
http://camel.apache.org/threading-model.html
You can configure thread pool profiles on a per CamelContext level.
> With Best Regards,
> Diwakar
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Thread-pool-for-components-in-Camel-tp4722190p5022361.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: Thread pool for components in Camel
Posted by diwakar <di...@nsn.com>.
Hi,
In Servicemix/JBI thread pools by are there by default (at
component level).
Why isn't the same approach taken for Camel? In Camel thread
(/async) has to be configured explicitly in the route.
Please let me know your comment.
With Best Regards,
Diwakar
--
View this message in context: http://camel.465427.n5.nabble.com/Thread-pool-for-components-in-Camel-tp4722190p5022361.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Thread pool for components in Camel
Posted by diwakar <di...@nsn.com>.
Hi,
>> Yes the file is deleted at the end, when the exchange is
done.
Just to understand better.
Tpoller thread will read the file1, Tlogger1 will process and fwd to
Twriter1 thread.
Tpoller thread will read the file2, Tlogger2 will process and fwd to
Twriter2 thread.
.
.
Tpoller thread will read the file10, Tlogger10 will process and fwd to
Twriter10 thread.
When the file is written the done message is sent to poller and then the
poller will delete the file.
Even though there is only one thread for poller, it reads the
file one after the other and deletesthe file when done message arrives.
Is this correct? Please let me know.
With Best Regards,
Diwakar
--
View this message in context: http://camel.465427.n5.nabble.com/Thread-pool-for-components-in-Camel-tp4722190p4903407.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Thread pool for components in Camel
Posted by Claus Ibsen <cl...@gmail.com>.
On Mon, Oct 10, 2011 at 10:36 AM, diwakar <di...@nsn.com> wrote:
> Hi,
>
> >> The file is a single threaded scheduled consumer
> Thanks for the reply. This is one difference compared to
> servicemix JBI. In servicemix multiple threads can poll the input folder.
> If we use multiple thread only for processing, when is the file
> deleted? Is it deleted immediately after delivering the message to the next
> endpoint? Is there someway to wait for the complete flow to finish and then
> delete the file? (If the flow fails, then dont delete the). How can this be
> done if processing is done by multiple threads, as there is no way to track
> which file processing succeded and which failed?
>
> Please let me know your comment.
>
Yes the file is deleted at the end, when the exchange is done.
> With Best Regards,
> Diwakar
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Thread-pool-for-components-in-Camel-tp4722190p4887428.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: Thread pool for components in Camel
Posted by diwakar <di...@nsn.com>.
Hi,
>> The file is a single threaded scheduled consumer
Thanks for the reply. This is one difference compared to
servicemix JBI. In servicemix multiple threads can poll the input folder.
If we use multiple thread only for processing, when is the file
deleted? Is it deleted immediately after delivering the message to the next
endpoint? Is there someway to wait for the complete flow to finish and then
delete the file? (If the flow fails, then dont delete the). How can this be
done if processing is done by multiple threads, as there is no way to track
which file processing succeded and which failed?
Please let me know your comment.
With Best Regards,
Diwakar
--
View this message in context: http://camel.465427.n5.nabble.com/Thread-pool-for-components-in-Camel-tp4722190p4887428.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Thread pool for components in Camel
Posted by Claus Ibsen <cl...@gmail.com>.
On Sun, Oct 9, 2011 at 6:14 PM, diwakar <di...@nsn.com> wrote:
> Hi,
>
> <from uri="file:input"/>
> <threads corePoolSize="10">
> <log message="Copying ${file:name} to the output directory"/>
> <to uri="file:output"/>
> </threads>
>
> With this we can have 10 threads for the logging and the file
> writing indpoints. But how to configure 10 threads for the poller
> (file:input)?
> Please let me know your comment.
>
The file is a single threaded scheduled consumer, so you cannot
configure it to use more threads.
Using the <threads> is the way to increase concurrency on file processing.
> With Bets Regards,
> Diwakar
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Thread-pool-for-components-in-Camel-tp4722190p4885604.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: Thread pool for components in Camel
Posted by diwakar <di...@nsn.com>.
Hi,
<from uri="file:input"/>
<threads corePoolSize="10">
<log message="Copying ${file:name} to the output directory"/>
<to uri="file:output"/>
</threads>
With this we can have 10 threads for the logging and the file
writing indpoints. But how to configure 10 threads for the poller
(file:input)?
Please let me know your comment.
With Bets Regards,
Diwakar
--
View this message in context: http://camel.465427.n5.nabble.com/Thread-pool-for-components-in-Camel-tp4722190p4885604.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Thread pool for components in Camel
Posted by diwakar <di...@nsn.com>.
Hi,
>> unless you are already use JBI a lot, and are happy with it as
is.
Yes, we are using JBI a lot.
>> No JBI is dying/dead
Yes, this is what we are concerned about.
Is staying with Servicemix 3.x an option? Or should we migrate to
Camel? Is camel as goos as ESB from performance/functionality pov. What are
other users of JBI in Servicemix 3.x doing.
Please let me know your comment.
With Best Regards,
Diwakar
--
View this message in context: http://camel.465427.n5.nabble.com/Thread-pool-for-components-in-Camel-tp4722190p4861409.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Thread pool for components in Camel
Posted by Claus Ibsen <cl...@gmail.com>.
On Fri, Sep 30, 2011 at 8:31 PM, diwakar <di...@nsn.com> wrote:
> Hi,
>
> Thanks for the reply. With async and defaultThreadPoolProfile, is
> Camel support for parallel processing as good as Servicemix 3.x. Are there
> any known limitations in Camel over JBI(Servicemix3.x). Is there any good
> reason to use Servicemix JBI instead of Camel.
> Please let me know your comment.
>
No JBI is dying/dead.
http://gnodet.blogspot.com/2010/12/thoughts-about-servicemix.html
For new projects you should consider *not* to use JBI, unless you are
already use JBI a lot,
and are happy with it as is.
We recommend to use Camel as replacement for JBI in SMX.
> With Best Regards,
> Diwakar
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Thread-pool-for-components-in-Camel-tp4722190p4858019.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: Thread pool for components in Camel
Posted by diwakar <di...@nsn.com>.
Hi,
Thanks for the reply. With async and defaultThreadPoolProfile, is
Camel support for parallel processing as good as Servicemix 3.x. Are there
any known limitations in Camel over JBI(Servicemix3.x). Is there any good
reason to use Servicemix JBI instead of Camel.
Please let me know your comment.
With Best Regards,
Diwakar
--
View this message in context: http://camel.465427.n5.nabble.com/Thread-pool-for-components-in-Camel-tp4722190p4858019.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Thread pool for components in Camel
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
See more details here
http://camel.apache.org/threading-model.html
On Mon, Aug 22, 2011 at 7:14 AM, diwakar <di...@nsn.com> wrote:
> Hi,
>
> In Servicemix we can configure thread pools for components:
> servicemix.corePoolSize = 32
> servicemix.maximumPoolSize = 128
> In camel, how can I configure thread pool for components? From
> documentation parallelProcessing, executorServiceRef options can be set to
> configure thread pools. But these options do not exist for all components.
> <route>
> <from uri="file:input"/>
> <log message="Copying ${file:name} to the output directory"/>
> <to uri="file:output"/>
> </route>
> In the above example, how can I configure thread pool for file
> component to have multiple threads to poll the "input" folder?
> Please let me know your comment.
>
The file consumer is using a scheduled thread pool to trigger to run
every Xth period (you can configure the period).
Its a single threaded task. You cannot run that in parallel.
For that you can enable the threads DSL in the route.
<from uri="file:input"/>
<threads corePoolSize="10">
<log message="Copying ${file:name} to the output directory"/>
<to uri="file:output"/>
</threads>
See this blog entry
http://davsclaus.blogspot.com/2009/05/on-road-to-camel-20-concurrency-with.html
And this page as well
http://camel.apache.org/async
> With Best Regards,
> Diwakar
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Thread-pool-for-components-in-Camel-tp4722190p4722190.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/