You are viewing a plain text version of this content. The canonical link for it is here.
Posted to droids-dev@incubator.apache.org by Eugen Paraschiv <ha...@gmail.com> on 2011/04/09 18:09:49 UTC
socketRead0 hangs on thread
I'm experiencing some very strange behavior when using Droids - it seems
that at some point in the crawling process, the crawling thread just hangs.
A thread dump shows where:
"myExecutor-9" prio=6 tid=0x0000000007c21800 nid=0x9fc runnable
[0x000000000bfde000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at
org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:130)
at
org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:127)
at
org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:233)
at
org.apache.http.impl.conn.DefaultResponseParser.parseHead(DefaultResponseParser.java:98)
at
org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:210)
at
org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:271)
at
org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:227)
at
org.apache.http.impl.conn.AbstractClientConnAdapter.receiveResponseHeader(AbstractClientConnAdapter.java:209)
at
org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:292)
at
org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:126)
at
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:483)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:641)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:576)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:554)
at
org.apache.droids.protocol.http.HttpProtocol.load(HttpProtocol.java:67)
at
org.apache.droids.robot.crawler.CrawlingWorkerMine.execute(CrawlingWorkerMine.java:57)
at
org.apache.droids.robot.crawler.CrawlingWorkerMine.execute(CrawlingWorkerMine.java:28)
at
org.apache.droids.impl.SequentialTaskMaster.processAllTasks(SequentialTaskMaster.java:84)
- locked <0x000000078192e180> (a
org.apache.droids.impl.SequentialTaskMaster)
at
org.apache.droids.robot.crawler.CrawlingDroid.start(CrawlingDroid.java:70)
I've seen some discussion on this sometimes back on a droids mailing list
but it doesn't seem to have lead anywhere. Also, socketRead0 hanging the
thread seems to be a common problem that happens in case the network is
overloaded. Seeing how there is not mechanism (that I'm aware of) in Droids
to prevent a network overload, and seeing how socketRead0 is blocking,
that's a real problem. I have stumbled upon this pretty much every time I
run my code, so I thought I'd ask - is this a known issue, is it something I
can do to fix it? If not, should we think about introducing some sort of
timeout or sleep so that the network doesn't get overloaded?
Any feedback is appreciated.
Thanks.
Eugen.
Re: socketRead0 hangs on thread
Posted by Chapuis Bertil <bc...@agimem.com>.
I personally never encountered this problem.
http://stackoverflow.com/questions/4606237/thread-hangs-in-socketread0-java
On 9 April 2011 20:09, Eugen Paraschiv <ha...@gmail.com> wrote:
> I'm experiencing some very strange behavior when using Droids - it seems
> that at some point in the crawling process, the crawling thread just hangs.
> A thread dump shows where:
>
> "myExecutor-9" prio=6 tid=0x0000000007c21800 nid=0x9fc runnable
> [0x000000000bfde000]
> java.lang.Thread.State: RUNNABLE
> at java.net.SocketInputStream.socketRead0(Native Method)
> at java.net.SocketInputStream.read(SocketInputStream.java:129)
> at
>
> org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:130)
> at
>
> org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:127)
> at
>
> org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:233)
> at
>
> org.apache.http.impl.conn.DefaultResponseParser.parseHead(DefaultResponseParser.java:98)
> at
>
> org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:210)
> at
>
> org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:271)
> at
>
> org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:227)
> at
>
> org.apache.http.impl.conn.AbstractClientConnAdapter.receiveResponseHeader(AbstractClientConnAdapter.java:209)
> at
>
> org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:292)
> at
>
> org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:126)
> at
>
> org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:483)
> at
>
> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:641)
> at
>
> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:576)
> at
>
> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:554)
> at
> org.apache.droids.protocol.http.HttpProtocol.load(HttpProtocol.java:67)
> at
>
> org.apache.droids.robot.crawler.CrawlingWorkerMine.execute(CrawlingWorkerMine.java:57)
> at
>
> org.apache.droids.robot.crawler.CrawlingWorkerMine.execute(CrawlingWorkerMine.java:28)
> at
>
> org.apache.droids.impl.SequentialTaskMaster.processAllTasks(SequentialTaskMaster.java:84)
> - locked <0x000000078192e180> (a
> org.apache.droids.impl.SequentialTaskMaster)
> at
> org.apache.droids.robot.crawler.CrawlingDroid.start(CrawlingDroid.java:70)
>
> I've seen some discussion on this sometimes back on a droids mailing list
> but it doesn't seem to have lead anywhere. Also, socketRead0 hanging the
> thread seems to be a common problem that happens in case the network is
> overloaded. Seeing how there is not mechanism (that I'm aware of) in Droids
> to prevent a network overload, and seeing how socketRead0 is blocking,
> that's a real problem. I have stumbled upon this pretty much every time I
> run my code, so I thought I'd ask - is this a known issue, is it something
> I
> can do to fix it? If not, should we think about introducing some sort of
> timeout or sleep so that the network doesn't get overloaded?
> Any feedback is appreciated.
> Thanks.
> Eugen.
>
--
Bertil Chapuis
Agimem Sàrl
http://www.agimem.com