[jira] Resolved: (HADOOP-1612) listing of an output directory shortly after job completion fails

["Owen O'Malley (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HADOOP-1612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Owen O'Malley resolved HADOOP-1612.
-----------------------------------

    Resolution: Won't Fix

This seems to have  been caused by two parts:
  1. HADOOP-1576
  2. Non-winning speculative execution tasks are not cleaned up before the job is marked complete. This is difficult to fix, because you need to wait for the task to report in, which may take more than 10 minutes in the case of an unresponsive task tracker. Users should work around this by ensuring that they ignore all contents of the output directory that start with "_".

> listing of an output directory shortly after job completion fails
> -----------------------------------------------------------------
>
>                 Key: HADOOP-1612
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1612
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.14.0
>            Reporter: Christian Kunz
>            Assignee: Arun C Murthy
>            Priority: Blocker
>             Fix For: 0.14.0
>
>
> Sometimes, after a job finishes, and another application wants to rename dfs files created by that job, listing of the output directory containing the newly created files fails. File creation and directory listing is done via libhdfs, but it is unlikely that this makes any difference, therefore, I add this to the mapred component.
> It might be a race condition: does the job complete before the files in the output directory are promoted?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.