Tomcat index locking

[Matt Wiseley <mw...@embarcgroup.com>]

Hi,

I've implemented the basic "intranet" search example using Nutch and
integrated it into a web site. However, when I try to re-index the site,
Tomcat has a lock on the index database files and the index fails. My
work-around is to:

1. Write the index to a temporary folder
2. Stop tomcat
3. Replace the real index folder with the temporary folder
4. Restart tomcat

This is obviously not ideal. I've noticed that the NutchBean stores an
instance of itself in the ServletContext. I've commented this out so a
new NutchBean gets created for every search, but still there is a lock.

Any ideas how to force NutchBean to release this lock without restarting
Tomcat?

Thanks much,
Matt

JobTracker dies when running index on an old segment

[Gal Nitzan <gn...@usa.net>]

Hi,

I have run the indexer on an old segment.

the JobTracker died and in the taskTracker log I found the following:

051011 225239 task_r_jp47mf 0.5% reduce > sort
051011 225340 Server connection on port 63524 from 206.222.12.010: exiting
051011 225340 task_r_jp47mf Child Error
java.io.IOException: Task process exit with nonzero status.
        at org.apache.nutch.mapred.TaskRunner.runChild(TaskRunner.java:132)
        at org.apache.nutch.mapred.TaskRunner.run(TaskRunner.java:92)
051011 225339 task_r_jp47mf  Parent died.  Exiting task_r_jp47mf
051011 225342 Server handler on 63524 caught: java.net.SocketException: 
Socket closed
java.net.SocketException: Socket closed
        at 
java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:99)
        at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
        at 
java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:66)
        at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:124)
        at java.io.DataOutputStream.flush(DataOutputStream.java:101)
        at org.apache.nutch.ipc.Server$Handler.run(Server.java:214)
051011 225339 task_r_sjhf6d  Parent died.  Exiting task_r_sjhf6d
051011 225342 Server handler on 63524 caught: java.net.SocketException: 
Socket closed
java.net.SocketException: Socket closed
        at 
java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:99)
        at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
        at 
java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:66)
        at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:124)
        at java.io.DataOutputStream.flush(DataOutputStream.java:101)
        at org.apache.nutch.ipc.Server$Handler.run(Server.java:214)
051011 225342 task_r_jp47mf Error reading child output
java.io.IOException: Bad file descriptor
        at java.io.FileInputStream.readBytes(Native Method)
        at java.io.FileInputStream.read(FileInputStream.java:194)
        at 
sun.nio.cs.StreamDecoder$CharsetSD.readBytes(StreamDecoder.java:408)
        at 
sun.nio.cs.StreamDecoder$CharsetSD.implRead(StreamDecoder.java:450)
        at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:182)
        at java.io.InputStreamReader.read(InputStreamReader.java:167)
        at java.io.BufferedReader.fill(BufferedReader.java:136)
        at java.io.BufferedReader.readLine(BufferedReader.java:299)
        at java.io.BufferedReader.readLine(BufferedReader.java:362)
        at org.apache.nutch.mapred.TaskRunner.logStream(TaskRunner.java:157)
        at org.apache.nutch.mapred.TaskRunner.access$100(TaskRunner.java:31)
        at org.apache.nutch.mapred.TaskRunner$1.run(TaskRunner.java:125)
051011 225342 task_r_sjhf6d Child Error
java.io.IOException: Task process exit with nonzero status.
        at org.apache.nutch.mapred.TaskRunner.runChild(TaskRunner.java:132)
        at org.apache.nutch.mapred.TaskRunner.run(TaskRunner.java:92)
051011 225342 Server connection on port 63524 from 206.222.12.010: exiting


Any clue?

Gal