You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Bai Shen <ba...@gmail.com> on 2011/10/28 17:18:12 UTC
Could not find taskTracker/jobcache
I'm using hadoop through nutch 1.4 I'm using the native libraries, but this
happens without them as well. I'm only using one local node. Sometimes
when running a nutch fetch job, I get the following error. There doesn't
seem to be any rhyme or reason to it AFAIK.
org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find
taskTracker/jobcache/job_local_0001/attempt_local_0001_m_000000_0/output/spill0.out
in any of the configured local directories
at
org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:389)
at
org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:138)
at
org.apache.hadoop.mapred.MapOutputFile.getSpillFile(MapOutputFile.java:94)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.mergeParts(MapTask.java:1443)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1154)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:359)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
2011-10-28 10:49:44,415 ERROR fetcher.Fetcher - Fetcher:
java.io.IOException: Job failed!
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1252)
at org.apache.nutch.fetcher.Fetcher.fetch(Fetcher.java:1202)
at org.apache.nutch.fetcher.Fetcher.run(Fetcher.java:1238)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.nutch.fetcher.Fetcher.main(Fetcher.java:1211)
Re: Could not find taskTracker/jobcache
Posted by Bai Shen <ba...@gmail.com>.
NM. Looks like it's an issue with running multiple jobs at once.
http://mail-archives.apache.org/mod_mbox/nutch-user/201009.mbox/%3C431DE8D6-C9B9-47C8-9301-BD0AB00405FE@gmail.com%3E
On Fri, Oct 28, 2011 at 11:18 AM, Bai Shen <ba...@gmail.com> wrote:
> I'm using hadoop through nutch 1.4 I'm using the native libraries, but
> this happens without them as well. I'm only using one local node.
> Sometimes when running a nutch fetch job, I get the following error. There
> doesn't seem to be any rhyme or reason to it AFAIK.
>
> org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find
> taskTracker/jobcache/job_local_0001/attempt_local_0001_m_000000_0/output/spill0.out
> in any of the configured local directories
>
> at
> org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:389)
>
> at
> org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:138)
>
> at
> org.apache.hadoop.mapred.MapOutputFile.getSpillFile(MapOutputFile.java:94)
>
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.mergeParts(MapTask.java:1443)
>
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1154)
>
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:359)
>
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
>
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
>
> 2011-10-28 10:49:44,415 ERROR fetcher.Fetcher - Fetcher:
> java.io.IOException: Job failed!
>
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1252)
>
> at org.apache.nutch.fetcher.Fetcher.fetch(Fetcher.java:1202)
>
> at org.apache.nutch.fetcher.Fetcher.run(Fetcher.java:1238)
>
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>
> at org.apache.nutch.fetcher.Fetcher.main(Fetcher.java:1211)
>