You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Aida <ai...@gmail.com> on 2012/12/03 16:57:05 UTC
Re: [FTP + ConsumerTemplate] Threads remaining alive
Hi again,
I fear that after upgrading to Camel 2.9.4 this problem still remains, at
least for me (I see that the issue was fixed for 2.9.2 and 2.10.0). I
couldn´t test it with Camel 2.10.0 because of cxf incompatibilities.
I have been doing some tests in real and I have been able to "produce" 5000
threads living at the same time (and never dying), one per endpoint that the
consumerTemplate tried to consume. At first I thought that the threads could
have something to do with the LRUCache that the DefaultConsumerTemplate has,
but since its maximun size is 1000 by default and I had about 5000 threads I
discarded that option and I'm looking for other solution (without success
for the moment).
As a reminder of this forum thread, my problem was that when consuming a
file (from a file/ftp endpoint) with the consumerTemplate, a thread remained
alive for each different endpoint that was consumed.
I'm using JBoss running in a Windows machine, and the JConsole to see the
threads that are alive.
If anyone could help or tell me what I can try, or what would be the next
step to keep on looking for the solution would be great.
Thanks in advance.
Aida.
--
View this message in context: http://camel.465427.n5.nabble.com/FTP-ConsumerTemplate-Threads-remaining-alive-tp5548338p5723551.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: [FTP + ConsumerTemplate] Threads remaining alive
Posted by Aida <ai...@gmail.com>.
Hi,
I have been investigating more deeply, and I have this stacktrace. The
problem is solved for me removing the "disconnect" option from the endpoints
consumed by the consumerTemplate (but I add here the stacktrace in case it
can help to somebody else). Otherwise, sometimes a NullPointerException is
thrown and a thread remains alive.
Thanks.
-------------------------------------------------------------------------------------
012-12-07 14:09:05,487 FATAL [myPackage.processor.MyProcessor]
(pool-27-thread-8) Exception launched while stopping consumerTemplate: null
java.lang.NullPointerException
at org.apache.commons.net.ftp.FTP.__send(FTP.java:496)
at org.apache.commons.net.ftp.FTP.sendCommand(FTP.java:470)
at org.apache.commons.net.ftp.FTP.sendCommand(FTP.java:547)
at org.apache.commons.net.ftp.FTP.sendCommand(FTP.java:596)
at org.apache.commons.net.ftp.FTP.quit(FTP.java:804)
at org.apache.commons.net.ftp.FTPClient.logout(FTPClient.java:941)
at
org.apache.camel.component.file.remote.FtpOperations.disconnect(FtpOperations.java:197)
at
org.apache.camel.component.file.remote.RemoteFileConsumer.disconnect(RemoteFileConsumer.java:113)
at
org.apache.camel.component.file.remote.RemoteFileConsumer.doStop(RemoteFileConsumer.java:100)
at org.apache.camel.support.ServiceSupport.stop(ServiceSupport.java:91)
at org.apache.camel.util.ServiceHelper.stopService(ServiceHelper.java:109)
at
org.apache.camel.impl.EventDrivenPollingConsumer.doStop(EventDrivenPollingConsumer.java:153)
at org.apache.camel.support.ServiceSupport.stop(ServiceSupport.java:91)
at org.apache.camel.util.ServiceHelper.stopService(ServiceHelper.java:109)
at
org.apache.camel.util.ServiceHelper.stopAndShutdownServices(ServiceHelper.java:177)
at org.apache.camel.impl.ConsumerCache.doStop(ConsumerCache.java:238)
at org.apache.camel.support.ServiceSupport.stop(ServiceSupport.java:91)
at org.apache.camel.util.ServiceHelper.stopService(ServiceHelper.java:109)
at
org.apache.camel.util.ServiceHelper.stopAndShutdownService(ServiceHelper.java:155)
at
org.apache.camel.impl.DefaultConsumerTemplate.doStop(DefaultConsumerTemplate.java:268)
at org.apache.camel.support.ServiceSupport.stop(ServiceSupport.java:91)
at
...
some of my classes
...
at sun.reflect.GeneratedMethodAccessor716.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.camel.component.bean.MethodInfo.invoke(MethodInfo.java:340)
at org.apache.camel.component.bean.MethodInfo$1.proceed(MethodInfo.java:237)
at
org.apache.camel.component.bean.BeanProcessor.process(BeanProcessor.java:166)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)
at
org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:99)
at
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)
at
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:71)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)
at
org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:99)
at
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)
at
org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:91)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)
at
org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:330)
at
org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:220)
at
org.apache.camel.processor.RouteContextProcessor.processNext(RouteContextProcessor.java:45)
at
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)
at
org.apache.camel.processor.interceptor.DefaultChannel.process(DefaultChannel.java:304)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:117)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:80)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)
at
org.apache.camel.processor.ChoiceProcessor.process(ChoiceProcessor.java:85)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)
at
org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:99)
at
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)
at
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:71)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)
at
org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:99)
at
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)
at
org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:91)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)
at
org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:330)
at
org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:220)
at
org.apache.camel.processor.RouteContextProcessor.processNext(RouteContextProcessor.java:45)
at
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)
at
org.apache.camel.processor.interceptor.DefaultChannel.process(DefaultChannel.java:304)
at
org.apache.camel.processor.RouteContextProcessor.processNext(RouteContextProcessor.java:45)
at
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)
at
org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:122)
at
org.apache.camel.processor.RouteInflightRepositoryProcessor.processNext(RouteInflightRepositoryProcessor.java:50)
at
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)
at
org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:99)
at
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)
at
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:71)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)
at
org.apache.camel.component.direct.DirectProducer.process(DirectProducer.java:61)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)
at
org.apache.camel.processor.SendProcessor$2.doInAsyncProducer(SendProcessor.java:117)
at
org.apache.camel.impl.ProducerCache.doInAsyncProducer(ProducerCache.java:291)
at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:112)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)
at
org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:99)
at
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)
at
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:71)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)
at
org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:99)
at
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)
at
org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:91)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)
at
org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:330)
at
org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:220)
at
org.apache.camel.processor.RouteContextProcessor.processNext(RouteContextProcessor.java:45)
at
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)
at
org.apache.camel.processor.interceptor.DefaultChannel.process(DefaultChannel.java:304)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:117)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:80)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)
at
org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:99)
at
org.apache.camel.processor.ChoiceProcessor.process(ChoiceProcessor.java:81)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)
at
org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:99)
at
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)
at
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:71)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)
at
org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:99)
at
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)
at
org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:91)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)
at
org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:330)
at
org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:220)
at
org.apache.camel.processor.RouteContextProcessor.processNext(RouteContextProcessor.java:45)
at
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)
at
org.apache.camel.processor.interceptor.DefaultChannel.process(DefaultChannel.java:304)
at
org.apache.camel.processor.RouteContextProcessor.processNext(RouteContextProcessor.java:45)
at
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)
at
org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:122)
at
org.apache.camel.processor.RouteInflightRepositoryProcessor.processNext(RouteInflightRepositoryProcessor.java:50)
at
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)
at
org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:99)
at
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)
at
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:71)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)
at
org.apache.camel.component.direct.DirectProducer.process(DirectProducer.java:61)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)
at
org.apache.camel.impl.InterceptSendToEndpoint$1.process(InterceptSendToEndpoint.java:154)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)
at
org.apache.camel.processor.SendProcessor$2.doInAsyncProducer(SendProcessor.java:117)
at
org.apache.camel.impl.ProducerCache.doInAsyncProducer(ProducerCache.java:291)
at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:112)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)
at
org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:99)
at
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)
at
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:71)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)
at
org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:99)
at
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)
at
org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:91)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)
at
org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:330)
at
org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:220)
at
org.apache.camel.processor.RouteContextProcessor.processNext(RouteContextProcessor.java:45)
at
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)
at
org.apache.camel.processor.interceptor.DefaultChannel.process(DefaultChannel.java:304)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:117)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:80)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)
at
org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:330)
at
org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:220)
at
org.apache.camel.processor.RouteContextProcessor.processNext(RouteContextProcessor.java:45)
at
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)
at
org.apache.camel.processor.UnitOfWorkProcessor.processAsync(UnitOfWorkProcessor.java:150)
at
org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:117)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:99)
at
org.apache.camel.processor.MulticastProcessor.doProcessParallel(MulticastProcessor.java:703)
at
org.apache.camel.processor.MulticastProcessor.access$200(MulticastProcessor.java:80)
at
org.apache.camel.processor.MulticastProcessor$1.call(MulticastProcessor.java:289)
at
org.apache.camel.processor.MulticastProcessor$1.call(MulticastProcessor.java:274)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
-------------------------------------------------------------------------------------
--
View this message in context: http://camel.465427.n5.nabble.com/FTP-ConsumerTemplate-Threads-remaining-alive-tp5548338p5723821.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: [FTP + ConsumerTemplate] Threads remaining alive
Posted by Aida <ai...@gmail.com>.
Hi,
Thank you very much, as you said doing that the threads are cleaned up :) I
should have tried that before, my apologies ...
I'm sorry to bother you, but after using that approach and consuming files
intensively I have seen that sometimes the consumerTemplate cannot be
stoppped, and, in that case the thread created for that consumption is not
released, neither in the moment to do the stop nor forward in the time (when
the next time the consumer template is used a start/stop is done).
I have seen that a NullpointerException is thrown (with no trace and no
localized message). I think this happen when calling service.shutdown();
from the method stopAndShutdownService of the ServiceHelper class. I haven´t
been able to delimit more the problem for the moment.
Thanks in advance.
--
View this message in context: http://camel.465427.n5.nabble.com/FTP-ConsumerTemplate-Threads-remaining-alive-tp5548338p5723622.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: [FTP + ConsumerTemplate] Threads remaining alive
Posted by Claus Ibsen <cl...@gmail.com>.
On Mon, Dec 3, 2012 at 4:57 PM, Aida <ai...@gmail.com> wrote:
> Hi again,
>
> I fear that after upgrading to Camel 2.9.4 this problem still remains, at
> least for me (I see that the issue was fixed for 2.9.2 and 2.10.0). I
> couldn´t test it with Camel 2.10.0 because of cxf incompatibilities.
>
> I have been doing some tests in real and I have been able to "produce" 5000
> threads living at the same time (and never dying), one per endpoint that the
> consumerTemplate tried to consume. At first I thought that the threads could
> have something to do with the LRUCache that the DefaultConsumerTemplate has,
> but since its maximun size is 1000 by default and I had about 5000 threads I
> discarded that option and I'm looking for other solution (without success
> for the moment).
>
> As a reminder of this forum thread, my problem was that when consuming a
> file (from a file/ftp endpoint) with the consumerTemplate, a thread remained
> alive for each different endpoint that was consumed.
>
> I'm using JBoss running in a Windows machine, and the JConsole to see the
> threads that are alive.
>
> If anyone could help or tell me what I can try, or what would be the next
> step to keep on looking for the solution would be great.
>
> Thanks in advance.
>
You need to stop the consumer template if you are not going to re-use
any endpoints.
Then any running threads etc is cleaned up.
> Aida.
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/FTP-ConsumerTemplate-Threads-remaining-alive-tp5548338p5723551.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
Red Hat, Inc.
FuseSource is now part of Red Hat
Email: cibsen@redhat.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen