Multiple Threads Created WHen Using OnCompletion

[dougly <dm...@gmail.com>]

Hi,
I have this route which uses a onCompletion processor. I recetnly run into
OutOfMemory issue after a week or so.
When profiling the memory dump, I see a lot of big instances in
ThreadLocalMap. Each big instance is refrenecing a onCompletionProcessor
thread. Please see the attached screenshot.
Is this multiple onCompletion threads causing the out of memory issue? And
is there a way I can restrict the # of onCompletion thread?

Thanks http://old.nabble.com/file/p28897414/outofmem.png 

onException(Exception.class).process(new Processor() {
            @Override
            public void process(Exchange ex) throws Exception
            {
                System.out.println("Exception: " + ex.getIn().getBody());
            }
        });
       
        from("quartz://mail/mailReminder?cron=0/30+*+*+*+*+?").process(new
Processor()
        {
            @Override
            public void process(Exchange ex) throws Exception
            {
                System.out.println("Polling Exchange Server WS");
                List<ItemType> items = poller.getMailItems();
                ProducerTemplate prod =
getContext().createProducerTemplate();
                for (ItemType item : items)
                {
                    prod.sendBody("seda:mail", item);
                }
            }
        });
       
        from("seda:mail")
            // on completion, delete that email from exchange server
            .onCompletion().onCompleteOnly()
            .process(new Processor()
            {
                @Override
                public void process(Exchange exchange) throws Exception
                {
                ItemIdType itemId = emailProcessor.deleteMessage(exchange);
                    poller.deleteItem(itemId);
                }
            })
            .end() http://old.nabble.com/file/p28897414/outofmem.PNG
outofmem.PNG 
            // this is the original route
            .process(
            new Processor()
            {
                @Override
                public void process(Exchange exchange) throws Exception
                {
                    emailProcessor.process(exchange);
                }
            }
        ).to("test-activemq:queue:" + queueName); 
-- 
View this message in context: http://old.nabble.com/Multiple-Threads-Created-WHen-Using-OnCompletion-tp28897414p28897414.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Multiple Threads Created WHen Using OnCompletion

[Claus Ibsen <cl...@gmail.com>]

Hi

OnCompletion will use a thread pool with a upper size of 10 threads in the pool.
You can customize a ExecutorService on it if you want to define a custom pool.

This is a bit easier to setup in 2.3
http://camel.apache.org/threading-model.html

I would assume its maybe something of your custom logic which holds a
reference. Are you using a ThreadLocal?

To my knowledge we only use ThreadLocal one place in Camel and its an
optimization in XPath. And you gotta remember to clear the thread
local when you are done using it.

And btw you should not create a new instance of ProducerTemplate as
you do. See the FAQ
http://camel.apache.org/why-does-camel-use-too-many-threads-with-producertemplate.html



On Wed, Jun 16, 2010 at 1:03 AM, dougly <dm...@gmail.com> wrote:
>
> Hi,
> I have this route which uses a onCompletion processor. I recetnly run into
> OutOfMemory issue after a week or so.
> When profiling the memory dump, I see a lot of big instances in
> ThreadLocalMap. Each big instance is refrenecing a onCompletionProcessor
> thread. Please see the attached screenshot.
> Is this multiple onCompletion threads causing the out of memory issue? And
> is there a way I can restrict the # of onCompletion thread?
>
> Thanks http://old.nabble.com/file/p28897414/outofmem.png
>
> onException(Exception.class).process(new Processor() {
>            @Override
>            public void process(Exchange ex) throws Exception
>            {
>                System.out.println("Exception: " + ex.getIn().getBody());
>            }
>        });
>
>        from("quartz://mail/mailReminder?cron=0/30+*+*+*+*+?").process(new
> Processor()
>        {
>            @Override
>            public void process(Exchange ex) throws Exception
>            {
>                System.out.println("Polling Exchange Server WS");
>                List<ItemType> items = poller.getMailItems();
>                ProducerTemplate prod =
> getContext().createProducerTemplate();
>                for (ItemType item : items)
>                {
>                    prod.sendBody("seda:mail", item);
>                }
>            }
>        });
>
>        from("seda:mail")
>            // on completion, delete that email from exchange server
>            .onCompletion().onCompleteOnly()
>            .process(new Processor()
>            {
>                @Override
>                public void process(Exchange exchange) throws Exception
>                {
>                ItemIdType itemId = emailProcessor.deleteMessage(exchange);
>                    poller.deleteItem(itemId);
>                }
>            })
>            .end() http://old.nabble.com/file/p28897414/outofmem.PNG
> outofmem.PNG
>            // this is the original route
>            .process(
>            new Processor()
>            {
>                @Override
>                public void process(Exchange exchange) throws Exception
>                {
>                    emailProcessor.process(exchange);
>                }
>            }
>        ).to("test-activemq:queue:" + queueName);
> --
> View this message in context: http://old.nabble.com/Multiple-Threads-Created-WHen-Using-OnCompletion-tp28897414p28897414.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>



-- 
Claus Ibsen
Apache Camel Committer

Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus