You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Jason Venner <ja...@attributor.com> on 2008/07/18 20:10:17 UTC

help request: 0.16.0 java.io.IOException: Filesystem closed & org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find task_....

I am seeing an odd mix of errors in a job we have running on a particular cluster of machines.
Has anyone seen this before and what is actually the problem?
We are running linux (Centos51, on 8 way xeons, with all disks under raid 5) and GigE switches between the machines.
The namenode machine does not run a datanode or a tasktracker and is essentially idle.

Thanks!


2008-07-18 09:25:43,626 WARN org.apache.hadoop.mapred.TaskTracker: Error running child
java.io.IOException: Filesystem closed
	at org.apache.hadoop.dfs.DFSClient.checkOpen(DFSClient.java:158)
	at org.apache.hadoop.dfs.DFSClient.access$500(DFSClient.java:58)
	at org.apache.hadoop.dfs.DFSClient$DFSInputStream.close(DFSClient.java:1095)
	at java.io.FilterInputStream.close(FilterInputStream.java:155)
	at org.apache.hadoop.mapred.LineRecordReader$LineReader.close(LineRecordReader.java:97)
	at org.apache.hadoop.mapred.LineRecordReader.close(LineRecordReader.java:277)
	at org.apache.hadoop.mapred.KeyValueLineRecordReader.close(KeyValueLineRecordReader.java:113)
	at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.close(MapTask.java:155)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:212)
	at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2071)

and

2008-07-18 04:26:41,056 WARN org.apache.hadoop.mapred.TaskTracker: Error running child
org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find task_200807141819_0007_m_001981_1/spill0.out in any of the configured local directories
	at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:359)
	at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:138)
	at org.apache.hadoop.mapred.MapOutputFile.getSpillFile(MapOutputFile.java:77)
	at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpillToDisk(MapTask.java:464)
	at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:713)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:209)
	at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2071)


-- 
Jason Venner
Attributor - Program the Web <http://www.attributor.com/>
Attributor is hiring Hadoop Wranglers and coding wizards, contact if 
interested