You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by Jon Shoberg <jo...@shoberg.net> on 2005/10/02 06:01:21 UTC
Re: fetcher hangs and thead lifetime

Jeff Pettenski wrote:
> Check
> http://www.mail-archive.com/nutch-developers@lists.sourceforge.net/msg04491.html
> 
> There is a problem with the PDFBox in nutch 0.7. It may hang.
> 
> I D/L the 0.7.2 version from http://www.pdfbox.org/, replaced the jar file
> in the plug-in directory and updated the plugin.xml to refer to the new jar
> file. Seems to be running better. Not done running my test yet.
> 
> -j.p.
> 

Not quite ... Im running the pdfbox patched version.  The past five 
minutes I've been looking at this ... when my script should be moving to 
the next fetch.  I'm guessing that "fetcher 17" is caught on something.

--

Full thread dump Java HotSpot(TM) 64-Bit Server VM (1.5.0_04-b05 mixed 
mode):

"MultiThreadedHttpConnectionManager cleanup" daemon prio=1 
tid=0x00002aabb3dfaf20 nid=0x3e99 in Object.wait() 
[0x000000004722a000..0x000000004722adb0]
         at java.lang.Object.wait(Native Method)
         - waiting on <0x00002aaaf51f1c50> (a 
java.lang.ref.ReferenceQueue$Lock)
         at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
         - locked <0x00002aaaf51f1c50> (a java.lang.ref.ReferenceQueue$Lock)
         at 
org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$ReferenceQueueThread.run(MultiThreadedHttpConnectionManager.java:1100)

"fetcher17" prio=1 tid=0x00002aabb544fd70 nid=0x3e46 runnable 
[0x0000000041ed7000..0x0000000041ed7e30]
         at java.net.SocketInputStream.socketRead0(Native Method)
         at java.net.SocketInputStream.read(SocketInputStream.java:129)
         at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
         at java.io.BufferedInputStream.read1(BufferedInputStream.java:256)
         at java.io.BufferedInputStream.read(BufferedInputStream.java:313)
         - locked <0x00002aaaf4dcc428> (a java.io.BufferedInputStream)
         at 
org.apache.commons.httpclient.ContentLengthInputStream.read(ContentLengthInputStream.java:169)
         at 
org.apache.commons.httpclient.ContentLengthInputStream.read(ContentLengthInputStream.java:183)
         at 
org.apache.commons.httpclient.ChunkedInputStream.exhaustInputStream(ChunkedInputStream.java:368)
         at 
org.apache.commons.httpclient.ContentLengthInputStream.close(ContentLengthInputStream.java:117)
         at java.io.FilterInputStream.close(FilterInputStream.java:159)
         at 
org.apache.commons.httpclient.AutoCloseInputStream.notifyWatcher(AutoCloseInputStream.java:176)
         at 
org.apache.commons.httpclient.AutoCloseInputStream.close(AutoCloseInputStream.java:140)
         at 
org.apache.nutch.protocol.httpclient.HttpResponse.<init>(HttpResponse.java:125)
         at 
org.apache.nutch.protocol.httpclient.Http.getProtocolOutput(Http.java:204)
         at 
org.apache.nutch.fetcher.Fetcher$FetcherThread.run(Fetcher.java:135)

"Low Memory Detector" daemon prio=1 tid=0x00002aabb3d04ec0 nid=0x3e33 
runnable [0x0000000000000000..0x0000000000000000]

"CompilerThread1" daemon prio=1 tid=0x00002aabb3d03400 nid=0x3e32 
waiting on condition [0x0000000000000000..0x0000000040ac2510]

"CompilerThread0" daemon prio=1 tid=0x00002aabb3d02000 nid=0x3e31 
waiting on condition [0x0000000000000000..0x00000000409c16d0]

"AdapterThread" daemon prio=1 tid=0x00002aabb3d00b90 nid=0x3e30 waiting 
on condition [0x0000000000000000..0x0000000000000000]

"Signal Dispatcher" daemon prio=1 tid=0x00002aabb0cd7990 nid=0x3e2f 
waiting on condition [0x0000000000000000..0x0000000000000000]

"Finalizer" daemon prio=1 tid=0x00002aabb0cc5480 nid=0x3e2e in 
Object.wait() [0x00000000406bf000..0x00000000406bfe30]
         at java.lang.Object.wait(Native Method)
         - waiting on <0x00002aaaf4c44c28> (a 
java.lang.ref.ReferenceQueue$Lock)
         at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
         - locked <0x00002aaaf4c44c28> (a java.lang.ref.ReferenceQueue$Lock)
         at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
         at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)

"Reference Handler" daemon prio=1 tid=0x00002aabb0cc22e0 nid=0x3e2d in 
Object.wait() [0x00000000405be000..0x00000000405bebb0]
         at java.lang.Object.wait(Native Method)
         - waiting on <0x00002aaaf4c47640> (a java.lang.ref.Reference$Lock)
         at java.lang.Object.wait(Object.java:474)
         at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
         - locked <0x00002aaaf4c47640> (a java.lang.ref.Reference$Lock)

"main" prio=1 tid=0x0000000040115bc0 nid=0x3e25 waiting on condition 
[0x00007fffffcfb000..0x00007fffffcfbc80]
         at java.lang.Thread.sleep(Native Method)
         at org.apache.nutch.fetcher.Fetcher.run(Fetcher.java:351)
         at org.apache.nutch.fetcher.Fetcher.main(Fetcher.java:488)

"VM Thread" prio=1 tid=0x00000000401bab30 nid=0x3e2c runnable

"GC task thread#0 (ParallelGC)" prio=1 tid=0x00000000401a85c0 nid=0x3e2a 
runnable

"GC task thread#1 (ParallelGC)" prio=1 tid=0x00000000401a8c60 nid=0x3e2b 
runnable

"VM Periodic Task Thread" prio=1 tid=0x00002aabb3d06c70 nid=0x3e34 
waiting on condition