Staging area isn't cleaned

[Ron Sher <ro...@gmail.com>]

Hi,

I'm using Hadoop version 2.0.0-cdh4.1.2,
rf0b53c81cbf56f5955e403b49fcd27afd5f082de
I'm using MRV1 (not yarn).

Whenever I run a job it says it's about to clean the staging area, but it
doesn't.

I tried some hack in which I do the cleaning up myself using the code below.
This works fine while there are no exceptions in the job, but if there are
(for example, when there are no input files to work on) I get the exception
and my cleanup code doesn't run.

Is there some way I can guarantee that the staging area get cleaned?

Thanks for you help,
Ron
String stagingDirToRemove = getStagingDir(conf, job);
 Path stagingDir = new Path(stagingDirToRemove);

LOG.info("about to remove " + stagingDirToRemove);

dfs.delete(stagingDir, true);
}


private static String getStagingDir(Configuration conf, RunningJob job) {
 String stagingRoot = conf.get("mapreduce.jobtracker.staging.root.dir");
String userName = System.getProperty("user.name");
 String jobID = job.getID().toString();
String result = stagingRoot + "/" + userName + "/.staging/" + jobID;

return result;
}