You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by Stefan Groschupf <sg...@media-style.com> on 2005/10/03 15:23:15 UTC

tasks is not killed

Hi,
I notice that can happen that a task is still running when the job  
already was killed.
The web gui says there is no running job and process hold the nodes  
busy.
I haven't found the source of the problem yet.


Stefan

box 1
051003 092000 task_r_brnupd 0.16666667% reduce > copy >  
task_m_qxbya3@cluster05.local.upd:7030
051003 092001 Server handler on 7030 caught:  
java.io.FileNotFoundException: /tmp/nutch/mapred/local/task_m_targ1p/ 
part-2.out (No such file or directory)
java.io.FileNotFoundException: /tmp/nutch/mapred/local/task_m_targ1p/ 
part-2.out (No such file or directory)
         at java.io.FileInputStream.open(Native Method)
         at java.io.FileInputStream.<init>(FileInputStream.java:106)
         at org.apache.nutch.mapred.MapOutputFile.write 
(MapOutputFile.java:93)
         at org.apache.nutch.io.ObjectWritable.writeObject 
(ObjectWritable.java:117)
         at org.apache.nutch.io.ObjectWritable.write 
(ObjectWritable.java:64)
         at org.apache.nutch.ipc.Server$Handler.run(Server.java:213)
051003 092001 task_r_brnupd 0.16666667% reduce > copy >  
task_m_qxbya3@cluster05.local.upd:7030

box 2

051003 091957 Server handler on 7030 caught:  
java.io.FileNotFoundException: /tmp/nutch/mapred/local/task_m_40legb/ 
part-2.out (No such file or directory)
java.io.FileNotFoundException: /tmp/nutch/mapred/local/task_m_40legb/ 
part-2.out (No such file or directory)
         at java.io.FileInputStream.open(Native Method)
         at java.io.FileInputStream.<init>(FileInputStream.java:106)
         at org.apache.nutch.mapred.MapOutputFile.write 
(MapOutputFile.java:93)
         at org.apache.nutch.io.ObjectWritable.writeObject 
(ObjectWritable.java:117)
         at org.apache.nutch.io.ObjectWritable.write 
(ObjectWritable.java:64)
         at org.apache.nutch.ipc.Server$Handler.run(Server.java:213)
051003 091957 task_r_gzhr0m copy failed: task_m_6ps2ww from  
cluster05.local.upd/192.168.200.105:7030

box 3

051003 092101 Server handler on 7030 caught:  
java.io.FileNotFoundException: /tmp/nutch/mapred/local/task_m_qojrr6/ 
part-2.out (No such file or directory)
java.io.FileNotFoundException: /tmp/nutch/mapred/local/task_m_qojrr6/ 
part-2.out (No such file or directory)
         at java.io.FileInputStream.open(Native Method)
         at java.io.FileInputStream.<init>(FileInputStream.java:106)
         at org.apache.nutch.mapred.MapOutputFile.write 
(MapOutputFile.java:93)
         at org.apache.nutch.io.ObjectWritable.writeObject 
(ObjectWritable.java:117)
         at org.apache.nutch.io.ObjectWritable.write 
(ObjectWritable.java:64)
         at org.apache.nutch.ipc.Server$Handler.run(Server.java:213)

Re: tasks is not killed

Posted by Doug Cutting <cu...@nutch.org>.
Stefan Groschupf wrote:
> I notice that can happen that a task is still running when the job  
> already was killed.
> The web gui says there is no running job and process hold the nodes  busy.
> I haven't found the source of the problem yet.

I have seen this too.  I think the solution is that, when the 
tasktracker reports task status to the jobtracker in a heartbeat 
message, the jobtracker should return an indication of which tasks are 
no longer valid and should be killed.  Please file a bug report.

Thanks,

Doug