You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org> on 2008/10/09 22:38:44 UTC
[jira] Resolved: (HADOOP-3214) JobInProgress.garbageCollect delete
twice.
[ https://issues.apache.org/jira/browse/HADOOP-3214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE resolved HADOOP-3214.
--------------------------------------------
Resolution: Duplicate
The was fixed by HADOOP-3135.
> JobInProgress.garbageCollect delete <job-dir> twice.
> ----------------------------------------------------
>
> Key: HADOOP-3214
> URL: https://issues.apache.org/jira/browse/HADOOP-3214
> Project: Hadoop Core
> Issue Type: Bug
> Reporter: Tsz Wo (Nicholas), SZE
>
> In JobInProgress.garbageCollect, the following codes delete <job-dir> twice.
> {code}
> // JobClient always creates a new directory with job files
> // so we remove that directory to cleanup
> FileSystem fs = FileSystem.get(conf);
> fs.delete(new Path(profile.getJobFile()).getParent(), true);
>
> // Delete temp dfs dirs created if any, like in case of
> // speculative exn of reduces.
> Path tempDir = new Path(conf.getSystemDir(), jobId);
> fs.delete(tempDir, true);
> {code}
> Below is the clean-up trace copied from HADOOP-3182:
> * FileSystem.delete <job-dir> by JobTracker as user_account
> at org.apache.hadoop.mapred.JobInProgress.garbageCollect(JobInProgress.java:1637)
> at org.apache.hadoop.mapred.JobInProgress.isJobComplete(JobInProgress.java:1396)
> at org.apache.hadoop.mapred.JobInProgress.completedTask(JobInProgress.java:1357)
> at org.apache.hadoop.mapred.JobInProgress.updateTaskStatus(JobInProgress.java:565)
> at org.apache.hadoop.mapred.JobTracker$TaskCommitQueue.run(JobTracker.java:2270)
> <job-dir> is obtained by *profile.getJobFile().getParent()*
> * FileSystem.delete <job-dir> again by JobTracker as user_account
> at org.apache.hadoop.mapred.JobInProgress.garbageCollect(JobInProgress.java:1642)
> at org.apache.hadoop.mapred.JobInProgress.isJobComplete(JobInProgress.java:1396)
> at org.apache.hadoop.mapred.JobInProgress.completedTask(JobInProgress.java:1357)
> at org.apache.hadoop.mapred.JobInProgress.updateTaskStatus(JobInProgress.java:565)
> at org.apache.hadoop.mapred.JobTracker$TaskCommitQueue.run(JobTracker.java:2270)
> <job-dir> is obtained by *new Path(conf.getSystemDir(), jobId)*
> Is there any case that these two paths are distinct?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.