Distributed Cache File Not Found Exception

[Shuja Rehman <sh...@gmail.com>]

Hi,
I m getting FileNotFound Exception while using distribute cache. here is the
details.

Configuration config = new Configuration();
config.clear();
config.set("hbase.zookeeper.quorum", HBASE_ZOOKEEPER_QUORUM);
config.set("hbase.zookeeper.property.clientPort",
HBASE_ZOOKEEPER_PROPERTY_CLIENTPORT);
FileSystem fs = FileSystem.get(config);
Path temp = new Path("tmp/extract", UUID.randomUUID().toString());
ObjectOutputStream os = new ObjectOutputStream(fs.create(temp));
os.writeObject("t70");
os.close();
// Register the file in the DC. Open the local file "targets"
DistributedCache.addCacheFile(new URI(temp+"#targets"), config);
DistributedCache.createSymlink(config);

Job job = new Job(config, "My Job");
job.setJarByClass(DetectorDriver.class);
job.setInputFormatClass(TableInputFormat.class);
job.setNumReduceTasks(0);
job.setOutputFormatClass(NullOutputFormat.class);

startRow = Bytes.toBytes(start);
endRow = Bytes.toBytes(end);
Scan s = new Scan(startRow,endRow);

TableMapReduceUtil.initTableMapperJob(HBASE_HOURLY_MAPPER_TABLE, s,
Mapper.class, Text.class, Result.class, job);

System.exit(job.waitForCompletion(true) ? 0 : 1)

Whenever job will run, it will always throw FileNotFound Exception.Here is
the trace.

SEVERE: null
java.io.FileNotFoundException: File does not exist: /home/shuja/lib/Myfile
at
org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:533)
at
org.apache.hadoop.filecache.DistributedCache.getFileStatus(DistributedCache.java:185)
at
org.apache.hadoop.filecache.TrackerDistributedCacheManager.determineTimestamps(TrackerDistributedCacheManager.java:804)
at
org.apache.hadoop.mapred.JobClient.copyAndConfigureFiles(JobClient.java:746)
at
org.apache.hadoop.mapred.JobClient.copyAndConfigureFiles(JobClient.java:637)
at org.apache.hadoop.mapred.JobClient.access$300(JobClient.java:170)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:848)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:833)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:833)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:476)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:506)
at detector.DetectorDriver.runJob(DetectorDriver.java:154)
at detector.DetectorDriver.main(DetectorDriver.java:172)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:186)

if i try to add jars using this line.

DistributedCache.addFileToClassPath(new
Path("/home/shuja/lib/UIDataTransporter.jar"),config);

then same FileNotFound Exception comes.
Any solution???



-- 
Regards
Shuja-ur-Rehman Baig
<http://pk.linkedin.com/in/shujamughal>