You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Charles Moulliard <cm...@gmail.com> on 2010/08/27 15:20:56 UTC

ThreadPool limit !!

Hi,

To be able to allow 5000 concurentConsumers to consume message from a jms
queue
("jmsConsumer:queue:VirtualQueueConsumer.highVolume?concurrentConsumers=5000"),
I have created a threadPool in camel of 2000 threads.
Maybe this value is too high but I would like to see if I can consume more
quickly messages from a queue that I send to a log endpoint

        <route startupOrder="1" id="ConsumeMessages" trace="false">
            <from ref="JmsQueueConsumer" />
            <threads executorServiceRef="myPool">
                <to ref="Log" />
            </threads>
        </route>

In this case, I get this error :

ATTENTION: Error occurred while shutting down routes. This exception will be
ignored.
java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:597)
        at
java.util.concurrent.ThreadPoolExecutor.addIfUnderCorePoolSize(ThreadPoolExecutor.java:703)
        at
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:652)
        at
java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:78)
        at
java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:600)
        at
org.apache.camel.impl.DefaultShutdownStrategy.doShutdown(DefaultShutdownStrategy.java:120)
        at
org.apache.camel.impl.DefaultShutdownStrategy.shutdown(DefaultShutdownStrategy.java:92)
        at
org.apache.camel.impl.DefaultShutdownStrategy.shutdown(DefaultShutdownStrategy.java:84)
        at
org.apache.camel.impl.DefaultCamelContext.doStop(DefaultCamelContext.java:1318)
        at org.apache.camel.impl.ServiceSupport.stop(ServiceSupport.java:98)
        at
org.apache.camel.impl.ServiceSupport.stop(ServiceSupport.java:122)
        at
org.apache.camel.core.xml.AbstractCamelContextFactoryBean.destroy(AbstractCamelContextFactoryBean.java:531)
        at
org.springframework.beans.factory.support.DisposableBeanAdapter.destroy(DisposableBeanAdapter.java:184)
        at
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroyBean(DefaultSingletonBeanRegistry.java:487)
        at
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingleton(DefaultSingletonBeanRegistry.java:463)
        at
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingletons(DefaultSingletonBeanRegistry.java:431)
        at
org.springframework.context.support.AbstractApplicationContext.destroyBeans(AbstractApplicationContext.java:1048)
        at
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:433)
        at
org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:139)
        at
org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:93)
        at
org.apache.camel.spring.Main.createDefaultApplicationContext(Main.java:219)
        at org.apache.camel.spring.Main.doStart(Main.java:173)
        at
org.apache.camel.impl.ServiceSupport.start(ServiceSupport.java:65)
        at
org.apache.camel.impl.ServiceSupport.start(ServiceSupport.java:52)
        at org.apache.camel.impl.MainSupport.run(MainSupport.java:114)
        at org.apache.camel.impl.MainSupport.run(MainSupport.java:291)
        at org.apache.camel.spring.Main.main(Main.java:97)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.camel.maven.RunMojo$1.run(RunMojo.java:415)
        at java.lang.Thread.run(Thread.java:619)

Remark : 1024M of memory is allowed to start the route and the error appears
at the starting up of the camel routes

Regards,

Charles Moulliard

Senior Enterprise Architect (J2EE, .NET, SOA)
Apache Camel - Karaf - ServiceMix Committer
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Blog : http://cmoulliard.blogspot.com |  Twitter :
http://twitter.com/cmoulliard
Linkedin : http://www.linkedin.com/in/charlesmoulliard | Skype: cmoulliard

Re: ThreadPool limit !!

Posted by Claus Ibsen <cl...@gmail.com>.
This is insane. Having 5000 threads is not idea for the JVM to handle.

Its not like you got a CPU with 5000 cores (at least not yet).



On Fri, Aug 27, 2010 at 3:20 PM, Charles Moulliard <cm...@gmail.com> wrote:
> Hi,
>
> To be able to allow 5000 concurentConsumers to consume message from a jms
> queue
> ("jmsConsumer:queue:VirtualQueueConsumer.highVolume?concurrentConsumers=5000"),
> I have created a threadPool in camel of 2000 threads.
> Maybe this value is too high but I would like to see if I can consume more
> quickly messages from a queue that I send to a log endpoint
>
>        <route startupOrder="1" id="ConsumeMessages" trace="false">
>            <from ref="JmsQueueConsumer" />
>            <threads executorServiceRef="myPool">
>                <to ref="Log" />
>            </threads>
>        </route>
>
> In this case, I get this error :
>
> ATTENTION: Error occurred while shutting down routes. This exception will be
> ignored.
> java.lang.OutOfMemoryError: unable to create new native thread
>        at java.lang.Thread.start0(Native Method)
>        at java.lang.Thread.start(Thread.java:597)
>        at
> java.util.concurrent.ThreadPoolExecutor.addIfUnderCorePoolSize(ThreadPoolExecutor.java:703)
>        at
> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:652)
>        at
> java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:78)
>        at
> java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:600)
>        at
> org.apache.camel.impl.DefaultShutdownStrategy.doShutdown(DefaultShutdownStrategy.java:120)
>        at
> org.apache.camel.impl.DefaultShutdownStrategy.shutdown(DefaultShutdownStrategy.java:92)
>        at
> org.apache.camel.impl.DefaultShutdownStrategy.shutdown(DefaultShutdownStrategy.java:84)
>        at
> org.apache.camel.impl.DefaultCamelContext.doStop(DefaultCamelContext.java:1318)
>        at org.apache.camel.impl.ServiceSupport.stop(ServiceSupport.java:98)
>        at
> org.apache.camel.impl.ServiceSupport.stop(ServiceSupport.java:122)
>        at
> org.apache.camel.core.xml.AbstractCamelContextFactoryBean.destroy(AbstractCamelContextFactoryBean.java:531)
>        at
> org.springframework.beans.factory.support.DisposableBeanAdapter.destroy(DisposableBeanAdapter.java:184)
>        at
> org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroyBean(DefaultSingletonBeanRegistry.java:487)
>        at
> org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingleton(DefaultSingletonBeanRegistry.java:463)
>        at
> org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingletons(DefaultSingletonBeanRegistry.java:431)
>        at
> org.springframework.context.support.AbstractApplicationContext.destroyBeans(AbstractApplicationContext.java:1048)
>        at
> org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:433)
>        at
> org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:139)
>        at
> org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:93)
>        at
> org.apache.camel.spring.Main.createDefaultApplicationContext(Main.java:219)
>        at org.apache.camel.spring.Main.doStart(Main.java:173)
>        at
> org.apache.camel.impl.ServiceSupport.start(ServiceSupport.java:65)
>        at
> org.apache.camel.impl.ServiceSupport.start(ServiceSupport.java:52)
>        at org.apache.camel.impl.MainSupport.run(MainSupport.java:114)
>        at org.apache.camel.impl.MainSupport.run(MainSupport.java:291)
>        at org.apache.camel.spring.Main.main(Main.java:97)
>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>        at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>        at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>        at java.lang.reflect.Method.invoke(Method.java:597)
>        at org.apache.camel.maven.RunMojo$1.run(RunMojo.java:415)
>        at java.lang.Thread.run(Thread.java:619)
>
> Remark : 1024M of memory is allowed to start the route and the error appears
> at the starting up of the camel routes
>
> Regards,
>
> Charles Moulliard
>
> Senior Enterprise Architect (J2EE, .NET, SOA)
> Apache Camel - Karaf - ServiceMix Committer
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> Blog : http://cmoulliard.blogspot.com |  Twitter :
> http://twitter.com/cmoulliard
> Linkedin : http://www.linkedin.com/in/charlesmoulliard | Skype: cmoulliard
>



-- 
Claus Ibsen
Apache Camel Committer

Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus