[jira] [Commented] (MAPREDUCE-4788) Job are marking as FAILED even if there are no failed tasks in it

["Jeff Schott (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13569940#comment-13569940 ] 

Jeff Schott commented on MAPREDUCE-4788:
----------------------------------------

We are seeing the exact same thing. After a lot of experimenting, it seems to happen consistently when we set the number of reducers very large for a given job/data set. 

After increasing the number of slots with these parameters:
yarn.nodemanager.resource.memory-mb
mapreduce.tasktracker.map.tasks.maximum
mapreduce.tasktracker.reduce.tasks.maximum

We were able to run over 100 reducers for wordcount. It now occurs for wordcount when reducers are over 500. However it also occurs with other larger jobs and only 30 reducers.

Any help, is much appreciated. 
                
> Job are marking as FAILED even if there are no failed tasks in it
> -----------------------------------------------------------------
>
>                 Key: MAPREDUCE-4788
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4788
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 2.0.1-alpha, 2.0.2-alpha
>            Reporter: Devaraj K
>            Assignee: Devaraj K
>
> Sometimes Jobs are marking as FAILED and some the tasks are marking as KILLED in it. 
> In MRAppMaster, JobFinishEvent is triggering and waiting for the 5000 millis. If any tasks final state is unknown by this time those tasks are marking as KILLED and Job state is marking as FAILED.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira