You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@servicemix.apache.org by Gert Vanthienen <ge...@skynet.be> on 2007/03/30 09:22:18 UTC
FTP poller stalls...
L.S.,
After some time of well-behaved operations, the FTP poller stalls...
Below, you find a set of thread dumps. If I interpret this data
correctly, the poller threads are waiting on a network operation to
complete, causing the connection pool to become empty --threads waiting
on GenericObjectPool.borrowObject()--. How do I avoid this? Is there a
way to set a time-out somewhere? Or is this a bug and should I raise an
issue for this --and attach a patch afterwards ;) -- ?
Regards,
Gert
1 thread performing timed wait (which is good)
-----------------------------------------------
Name: pool-component.servicemix-ftp-thread-1
State: TIMED_WAITING on
edu.emory.mathcs.backport.java.util.concurrent.locks.CondVar@1335332
Total blocked: 0 Total waited: 429.516
Stack trace:
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:432)
edu.emory.mathcs.backport.java.util.concurrent.TimeUnit.timedWait(TimeUnit.java:301)
edu.emory.mathcs.backport.java.util.concurrent.locks.CondVar.await(CondVar.java:103)
edu.emory.mathcs.backport.java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:345)
org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.accept(DeliveryChannelImpl.java:252)
org.apache.servicemix.common.AsyncBaseLifeCycle.pollDeliveryChannel(AsyncBaseLifeCycle.java:266)
org.apache.servicemix.common.AsyncBaseLifeCycle$1.run(AsyncBaseLifeCycle.java:252)
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:665)
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:690)
java.lang.Thread.run(Thread.java:595)
8 threads that are waiting on some kind of network operation
-------------------------------------------------------------
Name: pool-component.servicemix-ftp-thread-9
State: RUNNABLE
Total blocked: 742.403 Total waited: 35.249
Stack trace:
java.net.PlainSocketImpl.socketAccept(Native Method)
java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
java.net.ServerSocket.implAccept(ServerSocket.java:450)
java.net.ServerSocket.accept(ServerSocket.java:421)
org.apache.commons.net.ftp.FTPClient._openDataConnection_(FTPClient.java:502)
org.apache.commons.net.ftp.FTPClient.initiateListParsing(FTPClient.java:2390)
org.apache.commons.net.ftp.FTPClient.initiateListParsing(FTPClient.java:2364)
org.apache.commons.net.ftp.FTPClient.listFiles(FTPClient.java:2141)
org.apache.servicemix.ftp.FtpPollerEndpoint.pollFileOrDirectory(FtpPollerEndpoint.java:211)
org.apache.servicemix.ftp.FtpPollerEndpoint.pollFileOrDirectory(FtpPollerEndpoint.java:203)
org.apache.servicemix.ftp.FtpPollerEndpoint.poll(FtpPollerEndpoint.java:78)
org.apache.servicemix.common.endpoints.PollingEndpoint$PollSchedulerTask$1.run(PollingEndpoint.java:155)
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:665)
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:690)
java.lang.Thread.run(Thread.java:595)
23 threads that are waiting to get a connection from the pool
-------------------------------------------------------------
Name: pool-component.servicemix-ftp-thread-32
State: WAITING on org.apache.commons.pool.impl.GenericObjectPool@cbe11d
Total blocked: 301 Total waited: 17
Stack trace:
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:474)
org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:748)
org.apache.servicemix.ftp.SocketClientPoolSupport.borrowClient(SocketClientPoolSupport.java:61)
org.apache.servicemix.ftp.FtpPollerEndpoint.borrowClient(FtpPollerEndpoint.java:323)
org.apache.servicemix.ftp.FtpPollerEndpoint.pollFileOrDirectory(FtpPollerEndpoint.java:200)
org.apache.servicemix.ftp.FtpPollerEndpoint.poll(FtpPollerEndpoint.java:78)
org.apache.servicemix.common.endpoints.PollingEndpoint$PollSchedulerTask$1.run(PollingEndpoint.java:155)
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:665)
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:690)
java.lang.Thread.run(Thread.java:595)
Re: FTP poller stalls...
Posted by Gert Vanthienen <ge...@skynet.be>.
No, there are only 5 pollers, so there still is a problem.
I was just being curious as to how these connection pools work...
Guillaume Nodet wrote:
> Opps. yes, for sure ...
> But for a single poller, only one call should be made concurrently.
> Do you have 23 + 8 pollers ?
> And in this case, each poller has its own FTP connection pool,
> so there's no reason why they would all be blocked ...
>
>
> On 3/30/07, Gert Vanthienen <ge...@skynet.be> wrote:
>>
>> Guillaume,
>>
>> Isn't the poll method called concurrently when multiple <ftp:poller/>s
>> exist?
>>
>> Gert
>>
>> Guillaume Nodet wrote:
>> > Hi Guert !
>> >
>> > Yeah, I guess the accept operation on the socket should timeout
>> somehow,
>> > but this would be an enhancement in commons-net (maybe they know what
>> > to do for handling such problems ...).
>> > Another way would be to improve the pooling strategy with timeouts,
>> > aso ...
>> >
>> > But what seems weird is that there are 8 threads blocked in the poll
>> > method,
>> > which should not be called concurrently. So there is at least a bug
>> > there
>> > -- please raise a JIRA for this one :-)
>> >
>> > On 3/30/07, Gert Vanthienen <ge...@skynet.be> wrote:
>> >>
>> >> L.S.,
>> >>
>> >>
>> >> After some time of well-behaved operations, the FTP poller stalls...
>> >> Below, you find a set of thread dumps. If I interpret this data
>> >> correctly, the poller threads are waiting on a network operation to
>> >> complete, causing the connection pool to become empty --threads
>> waiting
>> >> on GenericObjectPool.borrowObject()--. How do I avoid this? Is
>> there
>> a
>> >> way to set a time-out somewhere? Or is this a bug and should I raise
>> an
>> >> issue for this --and attach a patch afterwards ;) -- ?
>> >>
>> >>
>> >> Regards,
>> >>
>> >> Gert
>> >>
>> >>
>> >>
>> >> 1 thread performing timed wait (which is good)
>> >> -----------------------------------------------
>> >> Name: pool-component.servicemix-ftp-thread-1
>> >> State: TIMED_WAITING on
>> >> edu.emory.mathcs.backport.java.util.concurrent.locks.CondVar@1335332
>> >> Total blocked: 0 Total waited: 429.516
>> >>
>> >> Stack trace:
>> >> java.lang.Object.wait(Native Method)
>> >> java.lang.Object.wait(Object.java:432)
>> >>
>> >> edu.emory.mathcs.backport.java.util.concurrent.TimeUnit.timedWait(
>> >> TimeUnit.java:301)
>> >>
>> >> edu.emory.mathcs.backport.java.util.concurrent.locks.CondVar.await(
>> >> CondVar.java:103)
>> >>
>> >>
>> edu.emory.mathcs.backport.java.util.concurrent.ArrayBlockingQueue.poll(
>> >> ArrayBlockingQueue.java:345)
>> >>
>> >> org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.accept(
>> >> DeliveryChannelImpl.java:252)
>> >>
>> >> org.apache.servicemix.common.AsyncBaseLifeCycle.pollDeliveryChannel(
>> >> AsyncBaseLifeCycle.java:266)
>> >>
>> >> org.apache.servicemix.common.AsyncBaseLifeCycle$1.run(
>> >> AsyncBaseLifeCycle.java:252)
>> >>
>> >>
>> >>
>> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask
>>
>> >>
>> >> (ThreadPoolExecutor.java:665)
>> >>
>> >>
>> >>
>> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run
>>
>> >>
>> >> (ThreadPoolExecutor.java:690)
>> >> java.lang.Thread.run(Thread.java:595)
>> >>
>> >> 8 threads that are waiting on some kind of network operation
>> >> -------------------------------------------------------------
>> >> Name: pool-component.servicemix-ftp-thread-9
>> >> State: RUNNABLE
>> >> Total blocked: 742.403 Total waited: 35.249
>> >>
>> >> Stack trace:
>> >> java.net.PlainSocketImpl.socketAccept(Native Method)
>> >> java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
>> >> java.net.ServerSocket.implAccept(ServerSocket.java:450)
>> >> java.net.ServerSocket.accept(ServerSocket.java:421)
>> >>
>> >> org.apache.commons.net.ftp.FTPClient._openDataConnection_(
>> FTPClient.java
>> >> :502)
>> >>
>> >>
>> org.apache.commons.net.ftp.FTPClient.initiateListParsing(FTPClient.java
>> >> :2390)
>> >>
>> >>
>> org.apache.commons.net.ftp.FTPClient.initiateListParsing(FTPClient.java
>> >> :2364)
>> >>
>> org.apache.commons.net.ftp.FTPClient.listFiles(FTPClient.java:2141)
>> >>
>> >> org.apache.servicemix.ftp.FtpPollerEndpoint.pollFileOrDirectory(
>> >> FtpPollerEndpoint.java:211)
>> >>
>> >> org.apache.servicemix.ftp.FtpPollerEndpoint.pollFileOrDirectory(
>> >> FtpPollerEndpoint.java:203)
>> >>
>> >>
>> org.apache.servicemix.ftp.FtpPollerEndpoint.poll(FtpPollerEndpoint.java
>> >> :78)
>> >>
>> >>
>> >>
>> org.apache.servicemix.common.endpoints.PollingEndpoint$PollSchedulerTask$1.run
>>
>> >>
>> >> (PollingEndpoint.java:155)
>> >>
>> >>
>> >>
>> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask
>>
>> >>
>> >> (ThreadPoolExecutor.java:665)
>> >>
>> >>
>> >>
>> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run
>>
>> >>
>> >> (ThreadPoolExecutor.java:690)
>> >> java.lang.Thread.run(Thread.java:595)
>> >>
>> >> 23 threads that are waiting to get a connection from the pool
>> >> -------------------------------------------------------------
>> >> Name: pool-component.servicemix-ftp-thread-32
>> >> State: WAITING on
>> org.apache.commons.pool.impl.GenericObjectPool@cbe11d
>> >> Total blocked: 301 Total waited: 17
>> >>
>> >> Stack trace:
>> >> java.lang.Object.wait(Native Method)
>> >> java.lang.Object.wait(Object.java:474)
>> >>
>> >> org.apache.commons.pool.impl.GenericObjectPool.borrowObject(
>> >> GenericObjectPool.java:748)
>> >>
>> >> org.apache.servicemix.ftp.SocketClientPoolSupport.borrowClient(
>> >> SocketClientPoolSupport.java:61)
>> >>
>> >> org.apache.servicemix.ftp.FtpPollerEndpoint.borrowClient(
>> >> FtpPollerEndpoint.java:323)
>> >>
>> >> org.apache.servicemix.ftp.FtpPollerEndpoint.pollFileOrDirectory(
>> >> FtpPollerEndpoint.java:200)
>> >>
>> >>
>> org.apache.servicemix.ftp.FtpPollerEndpoint.poll(FtpPollerEndpoint.java
>> >> :78)
>> >>
>> >>
>> >>
>> org.apache.servicemix.common.endpoints.PollingEndpoint$PollSchedulerTask$1.run
>>
>> >>
>> >> (PollingEndpoint.java:155)
>> >>
>> >>
>> >>
>> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask
>>
>> >>
>> >> (ThreadPoolExecutor.java:665)
>> >>
>> >>
>> >>
>> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run
>>
>> >>
>> >> (ThreadPoolExecutor.java:690)
>> >> java.lang.Thread.run(Thread.java:595)
>> >>
>> >>
>> >>
>> >
>> >
>>
>>
>
>
Re: FTP poller stalls...
Posted by Guillaume Nodet <gn...@gmail.com>.
Opps. yes, for sure ...
But for a single poller, only one call should be made concurrently.
Do you have 23 + 8 pollers ?
And in this case, each poller has its own FTP connection pool,
so there's no reason why they would all be blocked ...
On 3/30/07, Gert Vanthienen <ge...@skynet.be> wrote:
>
> Guillaume,
>
> Isn't the poll method called concurrently when multiple <ftp:poller/>s
> exist?
>
> Gert
>
> Guillaume Nodet wrote:
> > Hi Guert !
> >
> > Yeah, I guess the accept operation on the socket should timeout somehow,
> > but this would be an enhancement in commons-net (maybe they know what
> > to do for handling such problems ...).
> > Another way would be to improve the pooling strategy with timeouts,
> > aso ...
> >
> > But what seems weird is that there are 8 threads blocked in the poll
> > method,
> > which should not be called concurrently. So there is at least a bug
> > there
> > -- please raise a JIRA for this one :-)
> >
> > On 3/30/07, Gert Vanthienen <ge...@skynet.be> wrote:
> >>
> >> L.S.,
> >>
> >>
> >> After some time of well-behaved operations, the FTP poller stalls...
> >> Below, you find a set of thread dumps. If I interpret this data
> >> correctly, the poller threads are waiting on a network operation to
> >> complete, causing the connection pool to become empty --threads waiting
> >> on GenericObjectPool.borrowObject()--. How do I avoid this? Is there
> a
> >> way to set a time-out somewhere? Or is this a bug and should I raise
> an
> >> issue for this --and attach a patch afterwards ;) -- ?
> >>
> >>
> >> Regards,
> >>
> >> Gert
> >>
> >>
> >>
> >> 1 thread performing timed wait (which is good)
> >> -----------------------------------------------
> >> Name: pool-component.servicemix-ftp-thread-1
> >> State: TIMED_WAITING on
> >> edu.emory.mathcs.backport.java.util.concurrent.locks.CondVar@1335332
> >> Total blocked: 0 Total waited: 429.516
> >>
> >> Stack trace:
> >> java.lang.Object.wait(Native Method)
> >> java.lang.Object.wait(Object.java:432)
> >>
> >> edu.emory.mathcs.backport.java.util.concurrent.TimeUnit.timedWait(
> >> TimeUnit.java:301)
> >>
> >> edu.emory.mathcs.backport.java.util.concurrent.locks.CondVar.await(
> >> CondVar.java:103)
> >>
> >> edu.emory.mathcs.backport.java.util.concurrent.ArrayBlockingQueue.poll(
> >> ArrayBlockingQueue.java:345)
> >>
> >> org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.accept(
> >> DeliveryChannelImpl.java:252)
> >>
> >> org.apache.servicemix.common.AsyncBaseLifeCycle.pollDeliveryChannel(
> >> AsyncBaseLifeCycle.java:266)
> >>
> >> org.apache.servicemix.common.AsyncBaseLifeCycle$1.run(
> >> AsyncBaseLifeCycle.java:252)
> >>
> >>
> >>
> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask
> >>
> >> (ThreadPoolExecutor.java:665)
> >>
> >>
> >>
> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run
> >>
> >> (ThreadPoolExecutor.java:690)
> >> java.lang.Thread.run(Thread.java:595)
> >>
> >> 8 threads that are waiting on some kind of network operation
> >> -------------------------------------------------------------
> >> Name: pool-component.servicemix-ftp-thread-9
> >> State: RUNNABLE
> >> Total blocked: 742.403 Total waited: 35.249
> >>
> >> Stack trace:
> >> java.net.PlainSocketImpl.socketAccept(Native Method)
> >> java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
> >> java.net.ServerSocket.implAccept(ServerSocket.java:450)
> >> java.net.ServerSocket.accept(ServerSocket.java:421)
> >>
> >> org.apache.commons.net.ftp.FTPClient._openDataConnection_(
> FTPClient.java
> >> :502)
> >>
> >> org.apache.commons.net.ftp.FTPClient.initiateListParsing(FTPClient.java
> >> :2390)
> >>
> >> org.apache.commons.net.ftp.FTPClient.initiateListParsing(FTPClient.java
> >> :2364)
> >> org.apache.commons.net.ftp.FTPClient.listFiles(FTPClient.java:2141)
> >>
> >> org.apache.servicemix.ftp.FtpPollerEndpoint.pollFileOrDirectory(
> >> FtpPollerEndpoint.java:211)
> >>
> >> org.apache.servicemix.ftp.FtpPollerEndpoint.pollFileOrDirectory(
> >> FtpPollerEndpoint.java:203)
> >>
> >> org.apache.servicemix.ftp.FtpPollerEndpoint.poll(FtpPollerEndpoint.java
> >> :78)
> >>
> >>
> >>
> org.apache.servicemix.common.endpoints.PollingEndpoint$PollSchedulerTask$1.run
> >>
> >> (PollingEndpoint.java:155)
> >>
> >>
> >>
> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask
> >>
> >> (ThreadPoolExecutor.java:665)
> >>
> >>
> >>
> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run
> >>
> >> (ThreadPoolExecutor.java:690)
> >> java.lang.Thread.run(Thread.java:595)
> >>
> >> 23 threads that are waiting to get a connection from the pool
> >> -------------------------------------------------------------
> >> Name: pool-component.servicemix-ftp-thread-32
> >> State: WAITING on org.apache.commons.pool.impl.GenericObjectPool@cbe11d
> >> Total blocked: 301 Total waited: 17
> >>
> >> Stack trace:
> >> java.lang.Object.wait(Native Method)
> >> java.lang.Object.wait(Object.java:474)
> >>
> >> org.apache.commons.pool.impl.GenericObjectPool.borrowObject(
> >> GenericObjectPool.java:748)
> >>
> >> org.apache.servicemix.ftp.SocketClientPoolSupport.borrowClient(
> >> SocketClientPoolSupport.java:61)
> >>
> >> org.apache.servicemix.ftp.FtpPollerEndpoint.borrowClient(
> >> FtpPollerEndpoint.java:323)
> >>
> >> org.apache.servicemix.ftp.FtpPollerEndpoint.pollFileOrDirectory(
> >> FtpPollerEndpoint.java:200)
> >>
> >> org.apache.servicemix.ftp.FtpPollerEndpoint.poll(FtpPollerEndpoint.java
> >> :78)
> >>
> >>
> >>
> org.apache.servicemix.common.endpoints.PollingEndpoint$PollSchedulerTask$1.run
> >>
> >> (PollingEndpoint.java:155)
> >>
> >>
> >>
> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask
> >>
> >> (ThreadPoolExecutor.java:665)
> >>
> >>
> >>
> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run
> >>
> >> (ThreadPoolExecutor.java:690)
> >> java.lang.Thread.run(Thread.java:595)
> >>
> >>
> >>
> >
> >
>
>
--
Cheers,
Guillaume Nodet
------------------------
Architect, LogicBlaze (http://www.logicblaze.com/)
Blog: http://gnodet.blogspot.com/
Re: FTP poller stalls...
Posted by Gert Vanthienen <ge...@skynet.be>.
Guillaume,
Isn't the poll method called concurrently when multiple <ftp:poller/>s
exist?
Gert
Guillaume Nodet wrote:
> Hi Guert !
>
> Yeah, I guess the accept operation on the socket should timeout somehow,
> but this would be an enhancement in commons-net (maybe they know what
> to do for handling such problems ...).
> Another way would be to improve the pooling strategy with timeouts,
> aso ...
>
> But what seems weird is that there are 8 threads blocked in the poll
> method,
> which should not be called concurrently. So there is at least a bug
> there
> -- please raise a JIRA for this one :-)
>
> On 3/30/07, Gert Vanthienen <ge...@skynet.be> wrote:
>>
>> L.S.,
>>
>>
>> After some time of well-behaved operations, the FTP poller stalls...
>> Below, you find a set of thread dumps. If I interpret this data
>> correctly, the poller threads are waiting on a network operation to
>> complete, causing the connection pool to become empty --threads waiting
>> on GenericObjectPool.borrowObject()--. How do I avoid this? Is there a
>> way to set a time-out somewhere? Or is this a bug and should I raise an
>> issue for this --and attach a patch afterwards ;) -- ?
>>
>>
>> Regards,
>>
>> Gert
>>
>>
>>
>> 1 thread performing timed wait (which is good)
>> -----------------------------------------------
>> Name: pool-component.servicemix-ftp-thread-1
>> State: TIMED_WAITING on
>> edu.emory.mathcs.backport.java.util.concurrent.locks.CondVar@1335332
>> Total blocked: 0 Total waited: 429.516
>>
>> Stack trace:
>> java.lang.Object.wait(Native Method)
>> java.lang.Object.wait(Object.java:432)
>>
>> edu.emory.mathcs.backport.java.util.concurrent.TimeUnit.timedWait(
>> TimeUnit.java:301)
>>
>> edu.emory.mathcs.backport.java.util.concurrent.locks.CondVar.await(
>> CondVar.java:103)
>>
>> edu.emory.mathcs.backport.java.util.concurrent.ArrayBlockingQueue.poll(
>> ArrayBlockingQueue.java:345)
>>
>> org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.accept(
>> DeliveryChannelImpl.java:252)
>>
>> org.apache.servicemix.common.AsyncBaseLifeCycle.pollDeliveryChannel(
>> AsyncBaseLifeCycle.java:266)
>>
>> org.apache.servicemix.common.AsyncBaseLifeCycle$1.run(
>> AsyncBaseLifeCycle.java:252)
>>
>>
>> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask
>>
>> (ThreadPoolExecutor.java:665)
>>
>>
>> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run
>>
>> (ThreadPoolExecutor.java:690)
>> java.lang.Thread.run(Thread.java:595)
>>
>> 8 threads that are waiting on some kind of network operation
>> -------------------------------------------------------------
>> Name: pool-component.servicemix-ftp-thread-9
>> State: RUNNABLE
>> Total blocked: 742.403 Total waited: 35.249
>>
>> Stack trace:
>> java.net.PlainSocketImpl.socketAccept(Native Method)
>> java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
>> java.net.ServerSocket.implAccept(ServerSocket.java:450)
>> java.net.ServerSocket.accept(ServerSocket.java:421)
>>
>> org.apache.commons.net.ftp.FTPClient._openDataConnection_(FTPClient.java
>> :502)
>>
>> org.apache.commons.net.ftp.FTPClient.initiateListParsing(FTPClient.java
>> :2390)
>>
>> org.apache.commons.net.ftp.FTPClient.initiateListParsing(FTPClient.java
>> :2364)
>> org.apache.commons.net.ftp.FTPClient.listFiles(FTPClient.java:2141)
>>
>> org.apache.servicemix.ftp.FtpPollerEndpoint.pollFileOrDirectory(
>> FtpPollerEndpoint.java:211)
>>
>> org.apache.servicemix.ftp.FtpPollerEndpoint.pollFileOrDirectory(
>> FtpPollerEndpoint.java:203)
>>
>> org.apache.servicemix.ftp.FtpPollerEndpoint.poll(FtpPollerEndpoint.java
>> :78)
>>
>>
>> org.apache.servicemix.common.endpoints.PollingEndpoint$PollSchedulerTask$1.run
>>
>> (PollingEndpoint.java:155)
>>
>>
>> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask
>>
>> (ThreadPoolExecutor.java:665)
>>
>>
>> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run
>>
>> (ThreadPoolExecutor.java:690)
>> java.lang.Thread.run(Thread.java:595)
>>
>> 23 threads that are waiting to get a connection from the pool
>> -------------------------------------------------------------
>> Name: pool-component.servicemix-ftp-thread-32
>> State: WAITING on org.apache.commons.pool.impl.GenericObjectPool@cbe11d
>> Total blocked: 301 Total waited: 17
>>
>> Stack trace:
>> java.lang.Object.wait(Native Method)
>> java.lang.Object.wait(Object.java:474)
>>
>> org.apache.commons.pool.impl.GenericObjectPool.borrowObject(
>> GenericObjectPool.java:748)
>>
>> org.apache.servicemix.ftp.SocketClientPoolSupport.borrowClient(
>> SocketClientPoolSupport.java:61)
>>
>> org.apache.servicemix.ftp.FtpPollerEndpoint.borrowClient(
>> FtpPollerEndpoint.java:323)
>>
>> org.apache.servicemix.ftp.FtpPollerEndpoint.pollFileOrDirectory(
>> FtpPollerEndpoint.java:200)
>>
>> org.apache.servicemix.ftp.FtpPollerEndpoint.poll(FtpPollerEndpoint.java
>> :78)
>>
>>
>> org.apache.servicemix.common.endpoints.PollingEndpoint$PollSchedulerTask$1.run
>>
>> (PollingEndpoint.java:155)
>>
>>
>> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask
>>
>> (ThreadPoolExecutor.java:665)
>>
>>
>> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run
>>
>> (ThreadPoolExecutor.java:690)
>> java.lang.Thread.run(Thread.java:595)
>>
>>
>>
>
>
Re: FTP poller stalls...
Posted by Guillaume Nodet <gn...@gmail.com>.
Hi Guert !
Yeah, I guess the accept operation on the socket should timeout somehow,
but this would be an enhancement in commons-net (maybe they know what
to do for handling such problems ...).
Another way would be to improve the pooling strategy with timeouts, aso ...
But what seems weird is that there are 8 threads blocked in the poll method,
which should not be called concurrently. So there is at least a bug there
-- please raise a JIRA for this one :-)
On 3/30/07, Gert Vanthienen <ge...@skynet.be> wrote:
>
> L.S.,
>
>
> After some time of well-behaved operations, the FTP poller stalls...
> Below, you find a set of thread dumps. If I interpret this data
> correctly, the poller threads are waiting on a network operation to
> complete, causing the connection pool to become empty --threads waiting
> on GenericObjectPool.borrowObject()--. How do I avoid this? Is there a
> way to set a time-out somewhere? Or is this a bug and should I raise an
> issue for this --and attach a patch afterwards ;) -- ?
>
>
> Regards,
>
> Gert
>
>
>
> 1 thread performing timed wait (which is good)
> -----------------------------------------------
> Name: pool-component.servicemix-ftp-thread-1
> State: TIMED_WAITING on
> edu.emory.mathcs.backport.java.util.concurrent.locks.CondVar@1335332
> Total blocked: 0 Total waited: 429.516
>
> Stack trace:
> java.lang.Object.wait(Native Method)
> java.lang.Object.wait(Object.java:432)
>
> edu.emory.mathcs.backport.java.util.concurrent.TimeUnit.timedWait(
> TimeUnit.java:301)
>
> edu.emory.mathcs.backport.java.util.concurrent.locks.CondVar.await(
> CondVar.java:103)
>
> edu.emory.mathcs.backport.java.util.concurrent.ArrayBlockingQueue.poll(
> ArrayBlockingQueue.java:345)
>
> org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.accept(
> DeliveryChannelImpl.java:252)
>
> org.apache.servicemix.common.AsyncBaseLifeCycle.pollDeliveryChannel(
> AsyncBaseLifeCycle.java:266)
>
> org.apache.servicemix.common.AsyncBaseLifeCycle$1.run(
> AsyncBaseLifeCycle.java:252)
>
>
> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask
> (ThreadPoolExecutor.java:665)
>
>
> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run
> (ThreadPoolExecutor.java:690)
> java.lang.Thread.run(Thread.java:595)
>
> 8 threads that are waiting on some kind of network operation
> -------------------------------------------------------------
> Name: pool-component.servicemix-ftp-thread-9
> State: RUNNABLE
> Total blocked: 742.403 Total waited: 35.249
>
> Stack trace:
> java.net.PlainSocketImpl.socketAccept(Native Method)
> java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
> java.net.ServerSocket.implAccept(ServerSocket.java:450)
> java.net.ServerSocket.accept(ServerSocket.java:421)
>
> org.apache.commons.net.ftp.FTPClient._openDataConnection_(FTPClient.java
> :502)
>
> org.apache.commons.net.ftp.FTPClient.initiateListParsing(FTPClient.java
> :2390)
>
> org.apache.commons.net.ftp.FTPClient.initiateListParsing(FTPClient.java
> :2364)
> org.apache.commons.net.ftp.FTPClient.listFiles(FTPClient.java:2141)
>
> org.apache.servicemix.ftp.FtpPollerEndpoint.pollFileOrDirectory(
> FtpPollerEndpoint.java:211)
>
> org.apache.servicemix.ftp.FtpPollerEndpoint.pollFileOrDirectory(
> FtpPollerEndpoint.java:203)
>
> org.apache.servicemix.ftp.FtpPollerEndpoint.poll(FtpPollerEndpoint.java
> :78)
>
>
> org.apache.servicemix.common.endpoints.PollingEndpoint$PollSchedulerTask$1.run
> (PollingEndpoint.java:155)
>
>
> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask
> (ThreadPoolExecutor.java:665)
>
>
> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run
> (ThreadPoolExecutor.java:690)
> java.lang.Thread.run(Thread.java:595)
>
> 23 threads that are waiting to get a connection from the pool
> -------------------------------------------------------------
> Name: pool-component.servicemix-ftp-thread-32
> State: WAITING on org.apache.commons.pool.impl.GenericObjectPool@cbe11d
> Total blocked: 301 Total waited: 17
>
> Stack trace:
> java.lang.Object.wait(Native Method)
> java.lang.Object.wait(Object.java:474)
>
> org.apache.commons.pool.impl.GenericObjectPool.borrowObject(
> GenericObjectPool.java:748)
>
> org.apache.servicemix.ftp.SocketClientPoolSupport.borrowClient(
> SocketClientPoolSupport.java:61)
>
> org.apache.servicemix.ftp.FtpPollerEndpoint.borrowClient(
> FtpPollerEndpoint.java:323)
>
> org.apache.servicemix.ftp.FtpPollerEndpoint.pollFileOrDirectory(
> FtpPollerEndpoint.java:200)
>
> org.apache.servicemix.ftp.FtpPollerEndpoint.poll(FtpPollerEndpoint.java
> :78)
>
>
> org.apache.servicemix.common.endpoints.PollingEndpoint$PollSchedulerTask$1.run
> (PollingEndpoint.java:155)
>
>
> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask
> (ThreadPoolExecutor.java:665)
>
>
> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run
> (ThreadPoolExecutor.java:690)
> java.lang.Thread.run(Thread.java:595)
>
>
>
--
Cheers,
Guillaume Nodet
------------------------
Architect, LogicBlaze (http://www.logicblaze.com/)
Blog: http://gnodet.blogspot.com/