Shuffle error. Reducers are not finding map i/p in 2 node cluster....

[Sid123 <it...@gmail.com>]

The problem is not going away but i have a lead....
Here is a detailed diagnostic of what goes wrong:
1) 2 cluster setup on Ubuntu machines (data node and tasktracker is running
on both...)
2) The reducer tries to look for a non existent file as described below:
The problem happens only on my machine(slave) when it runs the
reducer....When the slave executes a reducer this problem does not occur.... 
The map tasks seem to be running fine...

First this happens:
--2009-04-27 19:54:09,901 INFO org.apache.hadoop.mapred.TaskTracker:
org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find
taskTracker/jobcache/job_200904271951_0002/attempt_200904271951_0002_m_000000_0/output/file.out
in any of the configured local directories
--2009-04-27 19:54:46,671 INFO org.apache.hadoop.mapred.TaskTracker:
org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find
taskTracker/jobcache/job_200904271951_0002/attempt_200904271951_0002_r_000000_0/output/file.out
in any of the configured local directories.
Then this:
Repeated attempts to start another reducer fail in the same manner...
    
So Is there a reducer specific setting that may be wrong on my machine? The
task tracker seems to be off...



The only inference I am able to make here is that the reducer on my machine
is not able to find the mapper outputs at all...
I have tried to delete the contents of mapred.local.dir and formatted the
name node... 
So what could be wrong....??

-- 
View this message in context: http://www.nabble.com/Shuffle-error.-Reducers-are-not-finding-map-i-p-in-2-node-cluster....-tp23271298p23271298.html
Sent from the Hadoop core-user mailing list archive at Nabble.com.