You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Sandy Ryza (JIRA)" <ji...@apache.org> on 2013/05/02 04:17:14 UTC
[jira] [Commented] (MAPREDUCE-4366) mapred metrics shows negative
count of waiting maps and reduces
[ https://issues.apache.org/jira/browse/MAPREDUCE-4366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647200#comment-13647200 ]
Sandy Ryza commented on MAPREDUCE-4366:
---------------------------------------
Oof this was a grueling one, but I think I've figured out the problem. At the start of a job, waiting tasks is incremented by the number of tasks, but then decremented each time an attempt launches, even if the attempt is speculative. Also, waiting tasks was incremented whenever a task failed, even if it was a speculative one that failed after a job had completed. This could cause a positive number of waiting tasks to stick around after the job completed. Uploading a patch that fully excludes speculative tasks from waiting tasks.
> mapred metrics shows negative count of waiting maps and reduces
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-4366
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4366
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 1.0.2
> Reporter: Thomas Graves
> Assignee: Thomas Graves
>
> Negative waiting_maps and waiting_reduces count is observed in the mapred metrics. MAPREDUCE-1238 partially fixed this but it appears there is still issues as we are seeing it, but not as bad.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira