You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Ramya R (JIRA)" <ji...@apache.org> on 2009/10/27 12:06:59 UTC
[jira] Created: (MAPREDUCE-1158) running_maps is not decremented
when the tasks of a job is killed/failed
running_maps is not decremented when the tasks of a job is killed/failed
-------------------------------------------------------------------------
Key: MAPREDUCE-1158
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1158
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: jobtracker
Affects Versions: 0.20.1
Reporter: Ramya R
Fix For: 0.20.2
running_maps counter in the metrics is not decremented when the tasks of a job is killed/failed. Below are the exact steps to reproduce the problem:
* Initially running_maps=0
* Submit a job with 5 maps. running_maps is set to 5
* Kill 2 attempts of a map task
* Fail 4 attempts of the same map task so that the job is finally marked killed.
* Once the job is marked killed, running_maps is set to 3 and not 0.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1158) running_maps is not decremented
when the tasks of a job is killed/failed
Posted by "Sharad Agarwal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sharad Agarwal updated MAPREDUCE-1158:
--------------------------------------
Attachment: 1158_yahoo.patch
Y! 0.20 patch.
> running_maps is not decremented when the tasks of a job is killed/failed
> -------------------------------------------------------------------------
>
> Key: MAPREDUCE-1158
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1158
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.22.0
> Reporter: Ramya R
> Assignee: Sharad Agarwal
> Fix For: 0.22.0
>
> Attachments: 1158.patch, 1158_yahoo.patch
>
>
> running_maps counter in the metrics is not decremented when the tasks of a job is killed/failed. Below are the exact steps to reproduce the problem:
> * Initially running_maps=0
> * Submit a job with 5 maps. running_maps is set to 5
> * Kill 2 attempts of a map task
> * Fail 4 attempts of the same map task so that the job is finally marked killed.
> * Once the job is marked killed, running_maps is set to 3 and not 0.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1158) running_maps is not decremented
when the tasks of a job is killed/failed
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12770920#action_12770920 ]
Hadoop QA commented on MAPREDUCE-1158:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12423417/1158.patch
against trunk revision 829529.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/97/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/97/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/97/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/97/console
This message is automatically generated.
> running_maps is not decremented when the tasks of a job is killed/failed
> -------------------------------------------------------------------------
>
> Key: MAPREDUCE-1158
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1158
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.22.0
> Reporter: Ramya R
> Assignee: Sharad Agarwal
> Fix For: 0.22.0
>
> Attachments: 1158.patch
>
>
> running_maps counter in the metrics is not decremented when the tasks of a job is killed/failed. Below are the exact steps to reproduce the problem:
> * Initially running_maps=0
> * Submit a job with 5 maps. running_maps is set to 5
> * Kill 2 attempts of a map task
> * Fail 4 attempts of the same map task so that the job is finally marked killed.
> * Once the job is marked killed, running_maps is set to 3 and not 0.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (MAPREDUCE-1158) running_maps is not decremented
when the tasks of a job is killed/failed
Posted by "Sharad Agarwal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sharad Agarwal reassigned MAPREDUCE-1158:
-----------------------------------------
Assignee: Sharad Agarwal
> running_maps is not decremented when the tasks of a job is killed/failed
> -------------------------------------------------------------------------
>
> Key: MAPREDUCE-1158
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1158
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.1
> Reporter: Ramya R
> Assignee: Sharad Agarwal
> Fix For: 0.20.2
>
> Attachments: 1158.patch
>
>
> running_maps counter in the metrics is not decremented when the tasks of a job is killed/failed. Below are the exact steps to reproduce the problem:
> * Initially running_maps=0
> * Submit a job with 5 maps. running_maps is set to 5
> * Kill 2 attempts of a map task
> * Fail 4 attempts of the same map task so that the job is finally marked killed.
> * Once the job is marked killed, running_maps is set to 3 and not 0.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1158) running_maps is not decremented
when the tasks of a job is killed/failed
Posted by "Sharad Agarwal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sharad Agarwal updated MAPREDUCE-1158:
--------------------------------------
Attachment: 1158.patch
Currently runningMaps/runningReduces metrics is captured in JobInProgress. It was discussed in offline discussion with Hemanth and Amareshwari that right place would be to do in JobTracker where slot metrics is captured. Attaching patch doing the same.
> running_maps is not decremented when the tasks of a job is killed/failed
> -------------------------------------------------------------------------
>
> Key: MAPREDUCE-1158
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1158
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.1
> Reporter: Ramya R
> Fix For: 0.20.2
>
> Attachments: 1158.patch
>
>
> running_maps counter in the metrics is not decremented when the tasks of a job is killed/failed. Below are the exact steps to reproduce the problem:
> * Initially running_maps=0
> * Submit a job with 5 maps. running_maps is set to 5
> * Kill 2 attempts of a map task
> * Fail 4 attempts of the same map task so that the job is finally marked killed.
> * Once the job is marked killed, running_maps is set to 3 and not 0.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1158) running_maps is not decremented
when the tasks of a job is killed/failed
Posted by "Sharad Agarwal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sharad Agarwal updated MAPREDUCE-1158:
--------------------------------------
Status: Patch Available (was: Open)
> running_maps is not decremented when the tasks of a job is killed/failed
> -------------------------------------------------------------------------
>
> Key: MAPREDUCE-1158
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1158
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.22.0
> Reporter: Ramya R
> Assignee: Sharad Agarwal
> Fix For: 0.22.0
>
> Attachments: 1158.patch
>
>
> running_maps counter in the metrics is not decremented when the tasks of a job is killed/failed. Below are the exact steps to reproduce the problem:
> * Initially running_maps=0
> * Submit a job with 5 maps. running_maps is set to 5
> * Kill 2 attempts of a map task
> * Fail 4 attempts of the same map task so that the job is finally marked killed.
> * Once the job is marked killed, running_maps is set to 3 and not 0.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1158) running_maps is not decremented
when the tasks of a job is killed/failed
Posted by "Sharad Agarwal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sharad Agarwal updated MAPREDUCE-1158:
--------------------------------------
Affects Version/s: (was: 0.20.1)
0.22.0
Fix Version/s: (was: 0.20.2)
0.22.0
> running_maps is not decremented when the tasks of a job is killed/failed
> -------------------------------------------------------------------------
>
> Key: MAPREDUCE-1158
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1158
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.22.0
> Reporter: Ramya R
> Assignee: Sharad Agarwal
> Fix For: 0.22.0
>
> Attachments: 1158.patch
>
>
> running_maps counter in the metrics is not decremented when the tasks of a job is killed/failed. Below are the exact steps to reproduce the problem:
> * Initially running_maps=0
> * Submit a job with 5 maps. running_maps is set to 5
> * Kill 2 attempts of a map task
> * Fail 4 attempts of the same map task so that the job is finally marked killed.
> * Once the job is marked killed, running_maps is set to 3 and not 0.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1158) running_maps is not decremented
when the tasks of a job is killed/failed
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12771068#action_12771068 ]
Hudson commented on MAPREDUCE-1158:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #100 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/100/])
. Fix JT running maps and running reduces metrics. Contributed by Sharad Agarwal.
> running_maps is not decremented when the tasks of a job is killed/failed
> -------------------------------------------------------------------------
>
> Key: MAPREDUCE-1158
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1158
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.22.0
> Reporter: Ramya R
> Assignee: Sharad Agarwal
> Fix For: 0.22.0
>
> Attachments: 1158.patch, 1158_yahoo.patch
>
>
> running_maps counter in the metrics is not decremented when the tasks of a job is killed/failed. Below are the exact steps to reproduce the problem:
> * Initially running_maps=0
> * Submit a job with 5 maps. running_maps is set to 5
> * Kill 2 attempts of a map task
> * Fail 4 attempts of the same map task so that the job is finally marked killed.
> * Once the job is marked killed, running_maps is set to 3 and not 0.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1158) running_maps is not decremented
when the tasks of a job is killed/failed
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12771503#action_12771503 ]
Hudson commented on MAPREDUCE-1158:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk #127 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/127/])
. Fix JT running maps and running reduces metrics. Contributed by Sharad Agarwal.
> running_maps is not decremented when the tasks of a job is killed/failed
> -------------------------------------------------------------------------
>
> Key: MAPREDUCE-1158
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1158
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.22.0
> Reporter: Ramya R
> Assignee: Sharad Agarwal
> Fix For: 0.22.0
>
> Attachments: 1158.patch, 1158_yahoo.patch
>
>
> running_maps counter in the metrics is not decremented when the tasks of a job is killed/failed. Below are the exact steps to reproduce the problem:
> * Initially running_maps=0
> * Submit a job with 5 maps. running_maps is set to 5
> * Kill 2 attempts of a map task
> * Fail 4 attempts of the same map task so that the job is finally marked killed.
> * Once the job is marked killed, running_maps is set to 3 and not 0.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1158) running_maps is not decremented
when the tasks of a job is killed/failed
Posted by "Sharad Agarwal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12770863#action_12770863 ]
Sharad Agarwal commented on MAPREDUCE-1158:
-------------------------------------------
test patch passed:
{code}
+1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 3 new or modified tests.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec]
[exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
{code}
> running_maps is not decremented when the tasks of a job is killed/failed
> -------------------------------------------------------------------------
>
> Key: MAPREDUCE-1158
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1158
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.22.0
> Reporter: Ramya R
> Assignee: Sharad Agarwal
> Fix For: 0.22.0
>
> Attachments: 1158.patch
>
>
> running_maps counter in the metrics is not decremented when the tasks of a job is killed/failed. Below are the exact steps to reproduce the problem:
> * Initially running_maps=0
> * Submit a job with 5 maps. running_maps is set to 5
> * Kill 2 attempts of a map task
> * Fail 4 attempts of the same map task so that the job is finally marked killed.
> * Once the job is marked killed, running_maps is set to 3 and not 0.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1158) running_maps is not decremented
when the tasks of a job is killed/failed
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12770829#action_12770829 ]
Amareshwari Sriramadasu commented on MAPREDUCE-1158:
----------------------------------------------------
Changes look fine to me.
> running_maps is not decremented when the tasks of a job is killed/failed
> -------------------------------------------------------------------------
>
> Key: MAPREDUCE-1158
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1158
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.22.0
> Reporter: Ramya R
> Assignee: Sharad Agarwal
> Fix For: 0.22.0
>
> Attachments: 1158.patch
>
>
> running_maps counter in the metrics is not decremented when the tasks of a job is killed/failed. Below are the exact steps to reproduce the problem:
> * Initially running_maps=0
> * Submit a job with 5 maps. running_maps is set to 5
> * Kill 2 attempts of a map task
> * Fail 4 attempts of the same map task so that the job is finally marked killed.
> * Once the job is marked killed, running_maps is set to 3 and not 0.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1158) running_maps is not decremented
when the tasks of a job is killed/failed
Posted by "Sharad Agarwal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sharad Agarwal updated MAPREDUCE-1158:
--------------------------------------
Resolution: Fixed
Release Note: Fix Jobtracker running maps/reduces metrics.
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
I just committed this.
> running_maps is not decremented when the tasks of a job is killed/failed
> -------------------------------------------------------------------------
>
> Key: MAPREDUCE-1158
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1158
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.22.0
> Reporter: Ramya R
> Assignee: Sharad Agarwal
> Fix For: 0.22.0
>
> Attachments: 1158.patch
>
>
> running_maps counter in the metrics is not decremented when the tasks of a job is killed/failed. Below are the exact steps to reproduce the problem:
> * Initially running_maps=0
> * Submit a job with 5 maps. running_maps is set to 5
> * Kill 2 attempts of a map task
> * Fail 4 attempts of the same map task so that the job is finally marked killed.
> * Once the job is marked killed, running_maps is set to 3 and not 0.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1158) running_maps is not decremented
when the tasks of a job is killed/failed
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12770897#action_12770897 ]
Hemanth Yamijala commented on MAPREDUCE-1158:
---------------------------------------------
Changes look fine to me.
> running_maps is not decremented when the tasks of a job is killed/failed
> -------------------------------------------------------------------------
>
> Key: MAPREDUCE-1158
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1158
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.22.0
> Reporter: Ramya R
> Assignee: Sharad Agarwal
> Fix For: 0.22.0
>
> Attachments: 1158.patch
>
>
> running_maps counter in the metrics is not decremented when the tasks of a job is killed/failed. Below are the exact steps to reproduce the problem:
> * Initially running_maps=0
> * Submit a job with 5 maps. running_maps is set to 5
> * Kill 2 attempts of a map task
> * Fail 4 attempts of the same map task so that the job is finally marked killed.
> * Once the job is marked killed, running_maps is set to 3 and not 0.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.