You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by Cam Bazz <ca...@gmail.com> on 2011/07/10 23:46:18 UTC
exception while fetching
Hello,
I am getting some exception while fetching:
2011-07-10 23:25:21,427 WARN mapred.LocalJobRunner - job_local_0001
org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find
taskTracker/jobcache/job_local_0001/attempt_local_0001_m_000000_0/output/spill0.out
in any of the configured local directories
at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:389)
at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:138)
at org.apache.hadoop.mapred.MapOutputFile.getSpillFile(MapOutputFile.java:94)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.mergeParts(MapTask.java:1443)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1154)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:359)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
2011-07-10 23:25:22,279 FATAL fetcher.Fetcher - Fetcher:
java.io.IOException: Job failed!
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1252)
at org.apache.nutch.fetcher.Fetcher.fetch(Fetcher.java:1107)
at org.apache.nutch.fetcher.Fetcher.run(Fetcher.java:1145)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.nutch.fetcher.Fetcher.main(Fetcher.java:1116)
What should I do? What happens if I restart the fetch job?
Best Regards,
C.B.
Re: exception while fetching
Posted by Markus Jelsma <ma...@openindex.io>.
A recurring question: you're most likely out of disk space in /tmp. Consider
using another location for hadoop.tmp.dir with plenty of room for large
transient files or using a Hadoop cluster.
> Hello,
>
> I am getting some exception while fetching:
>
> 2011-07-10 23:25:21,427 WARN mapred.LocalJobRunner - job_local_0001
> org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find
> taskTracker/jobcache/job_local_0001/attempt_local_0001_m_000000_0/output/sp
> ill0.out in any of the configured local directories
> at
> org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToR
> ead(LocalDirAllocator.java:389) at
> org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocato
> r.java:138) at
> org.apache.hadoop.mapred.MapOutputFile.getSpillFile(MapOutputFile.java:94)
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.mergeParts(MapTask.java:1
> 443) at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1154)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:359) at
> org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
> 2011-07-10 23:25:22,279 FATAL fetcher.Fetcher - Fetcher:
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1252)
> at org.apache.nutch.fetcher.Fetcher.fetch(Fetcher.java:1107)
> at org.apache.nutch.fetcher.Fetcher.run(Fetcher.java:1145)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.nutch.fetcher.Fetcher.main(Fetcher.java:1116)
>
> What should I do? What happens if I restart the fetch job?
>
> Best Regards,
> C.B.