You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Ramya Sunil (JIRA)" <ji...@apache.org> on 2011/08/10 02:33:27 UTC
[jira] [Created] (MAPREDUCE-2795) [MR-279] AppsKilled is never
incremented
[MR-279] AppsKilled is never incremented
----------------------------------------
Key: MAPREDUCE-2795
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2795
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.0
Reporter: Ramya Sunil
Fix For: 0.23.0
AppsKilled metrics is never incremented even though there are killed jobs in the system.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-2795) [MR-279] AppsKilled is never
incremented
Posted by "Devaraj K (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj K reassigned MAPREDUCE-2795:
------------------------------------
Assignee: Devaraj K
> [MR-279] AppsKilled is never incremented
> ----------------------------------------
>
> Key: MAPREDUCE-2795
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2795
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Devaraj K
> Priority: Blocker
> Fix For: 0.23.0
>
>
> AppsKilled metrics is never incremented even though there are killed jobs in the system.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2795) [MR-279] AppsKilled is never
incremented
Posted by "Vinod Kumar Vavilapalli (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13124055#comment-13124055 ]
Vinod Kumar Vavilapalli commented on MAPREDUCE-2795:
----------------------------------------------------
That is by design. In case of mapreduce running over YARN, as of now, one mapreduce job is mapped to one YARN application. So, if the application has not yet started running, we kill the application itself so as to not start the job (the AM precisely). In which case, YARN duely increments the killedApps metric, correctly.
> [MR-279] AppsKilled is never incremented
> ----------------------------------------
>
> Key: MAPREDUCE-2795
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2795
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Devaraj K
> Priority: Blocker
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MAPREDUCE-2795.patch
>
>
> AppsKilled metrics is never incremented even though there are killed jobs in the system.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-2795) [MR-279] AppsKilled is never
incremented
Posted by "Devaraj K (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj K resolved MAPREDUCE-2795.
----------------------------------
Resolution: Invalid
As per the above discussions, I am closing it as Invalid.
> [MR-279] AppsKilled is never incremented
> ----------------------------------------
>
> Key: MAPREDUCE-2795
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2795
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Devaraj K
> Priority: Blocker
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MAPREDUCE-2795.patch
>
>
> AppsKilled metrics is never incremented even though there are killed jobs in the system.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2795) [MR-279] AppsKilled is never
incremented
Posted by "Devaraj K (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj K updated MAPREDUCE-2795:
---------------------------------
Fix Version/s: 0.24.0
Status: Patch Available (was: Open)
> [MR-279] AppsKilled is never incremented
> ----------------------------------------
>
> Key: MAPREDUCE-2795
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2795
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Devaraj K
> Priority: Blocker
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MAPREDUCE-2795.patch
>
>
> AppsKilled metrics is never incremented even though there are killed jobs in the system.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2795) [MR-279] AppsKilled is never
incremented
Posted by "Vinod Kumar Vavilapalli (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121186#comment-13121186 ]
Vinod Kumar Vavilapalli commented on MAPREDUCE-2795:
----------------------------------------------------
Ramya, can you please look at the above comment and close this if you find the explanation valid? Thanks!
> [MR-279] AppsKilled is never incremented
> ----------------------------------------
>
> Key: MAPREDUCE-2795
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2795
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Devaraj K
> Priority: Blocker
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MAPREDUCE-2795.patch
>
>
> AppsKilled metrics is never incremented even though there are killed jobs in the system.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2795) [MR-279] AppsKilled is never
incremented
Posted by "Devaraj K (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj K updated MAPREDUCE-2795:
---------------------------------
Attachment: MAPREDUCE-2795.patch
> [MR-279] AppsKilled is never incremented
> ----------------------------------------
>
> Key: MAPREDUCE-2795
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2795
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Devaraj K
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2795.patch
>
>
> AppsKilled metrics is never incremented even though there are killed jobs in the system.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2795) [MR-279] AppsKilled is never
incremented
Posted by "Devaraj K (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120688#comment-13120688 ]
Devaraj K commented on MAPREDUCE-2795:
--------------------------------------
Here the problem was, the AppMaster state is sending to the Metrics instead of the App state. In case of a app is killed or failed, the AppMaster state is finished(FINISHED) and the same is sent to the metrics, instead of Killed or failed(i.e App State), so the metrics were not updating properly.
> [MR-279] AppsKilled is never incremented
> ----------------------------------------
>
> Key: MAPREDUCE-2795
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2795
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Devaraj K
> Priority: Blocker
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MAPREDUCE-2795.patch
>
>
> AppsKilled metrics is never incremented even though there are killed jobs in the system.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2795) [MR-279] AppsKilled is never
incremented
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120694#comment-13120694 ]
Hadoop QA commented on MAPREDUCE-2795:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12497753/MAPREDUCE-2795.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 6 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.mapreduce.v2.app.TestRuntimeEstimators
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/944//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/944//console
This message is automatically generated.
> [MR-279] AppsKilled is never incremented
> ----------------------------------------
>
> Key: MAPREDUCE-2795
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2795
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Devaraj K
> Priority: Blocker
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MAPREDUCE-2795.patch
>
>
> AppsKilled metrics is never incremented even though there are killed jobs in the system.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2795) [MR-279] AppsKilled is never
incremented
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-2795:
-----------------------------------------------
Status: Open (was: Patch Available)
The patch has got it wrong.
Application is a YARN level concept, while Job is a MapReduce concept. When a mapreduce job is killed, it goes through the AM. So for the platform(aka YARN), the application finished cleanly even though the job itself is killed by the user.
I see that AppsKilled is updated correctly, only when an application is forcefully terminated on the RM via ClientRMProtocol.forceKillApplication(). If you want to figure out the number of killed jobs, you should go to the JobHistoryServer.
If you are satisfied with the explanation, please close the ticket as invalid. Thanks.
> [MR-279] AppsKilled is never incremented
> ----------------------------------------
>
> Key: MAPREDUCE-2795
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2795
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Devaraj K
> Priority: Blocker
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MAPREDUCE-2795.patch
>
>
> AppsKilled metrics is never incremented even though there are killed jobs in the system.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2795) [MR-279] AppsKilled is never
incremented
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-2795:
-------------------------------------
Priority: Blocker (was: Major)
> [MR-279] AppsKilled is never incremented
> ----------------------------------------
>
> Key: MAPREDUCE-2795
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2795
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Priority: Blocker
> Fix For: 0.23.0
>
>
> AppsKilled metrics is never incremented even though there are killed jobs in the system.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2795) [MR-279] AppsKilled is never
incremented
Posted by "Devaraj K (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123979#comment-13123979 ]
Devaraj K commented on MAPREDUCE-2795:
--------------------------------------
Hi Vinod,
Thanks for the explanation.
{code:title=YARNRunner.java|borderStyle=solid}
public void killJob(JobID arg0) throws IOException, InterruptedException {
/* check if the status is not running, if not send kill to RM */
JobStatus status = clientCache.getClient(arg0).getJobStatus(arg0);
if (status.getState() != JobStatus.State.RUNNING) {
resMgrDelegate.killApplication(TypeConverter.toYarn(arg0).getAppId());
return;
}
try {
/* send a kill to the AM */
clientCache.getClient(arg0).killJob(arg0);
long currentTimeMillis = System.currentTimeMillis();
long timeKillIssued = currentTimeMillis;
while ((currentTimeMillis < timeKillIssued + 10000L) && (status.getState()
!= JobStatus.State.KILLED)) {
try {
Thread.sleep(1000L);
} catch(InterruptedException ie) {
/** interrupted, just break */
break;
}
currentTimeMillis = System.currentTimeMillis();
status = clientCache.getClient(arg0).getJobStatus(arg0);
}
} catch(IOException io) {
LOG.debug("Error when checking for application status", io);
}
if (status.getState() != JobStatus.State.KILLED) {
resMgrDelegate.killApplication(TypeConverter.toYarn(arg0).getAppId());
}
}
{code}
If I refer to YARNRunner.killJob(JobID) method, then I see that the killing of the job is handled in two different ways.
1. If JobStatus.State. is not RUNNING then ResourceMgrDelegate.killApplication(ApplicationId)
2. Otherwise ClientServiceDelegate.killJob(JobID)
Is it good idea to kill the application itself, if it not in running state? In this case it is incrementing the apps killed metrics even if the user trying to kill the job.
> [MR-279] AppsKilled is never incremented
> ----------------------------------------
>
> Key: MAPREDUCE-2795
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2795
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Devaraj K
> Priority: Blocker
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MAPREDUCE-2795.patch
>
>
> AppsKilled metrics is never incremented even though there are killed jobs in the system.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2795) [MR-279] AppsKilled is never
incremented
Posted by "Devaraj K (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120822#comment-13120822 ]
Devaraj K commented on MAPREDUCE-2795:
--------------------------------------
The above test failure is not because of the patch changes. These are passing in my local environment. Please find the report below.
{code:xml}
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.281 sec
Running org.apache.hadoop.mapreduce.v2.app.metrics.TestMRAppMetrics
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.093 sec
Running org.apache.hadoop.mapreduce.v2.app.speculate.TestDataStatistics
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec
Running org.apache.hadoop.mapreduce.v2.app.TestFail
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.513 sec
Running org.apache.hadoop.mapreduce.v2.app.TestFetchFailure
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.03 sec
Running org.apache.hadoop.mapreduce.v2.app.TestKill
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.093 sec
Running org.apache.hadoop.mapreduce.v2.app.TestMRApp
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.686 sec
Running org.apache.hadoop.mapreduce.v2.app.TestMRAppMaster
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.016 sec
Running org.apache.hadoop.mapreduce.v2.app.TestMRClientService
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.906 sec
Running org.apache.hadoop.mapreduce.v2.app.TestRecovery
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.888 sec
Running org.apache.hadoop.mapreduce.v2.app.TestRMContainerAllocator
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.31 sec
Running org.apache.hadoop.mapreduce.v2.app.TestRuntimeEstimators
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 57.454 sec
Running org.apache.hadoop.mapreduce.v2.app.webapp.TestAMWebApp
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.467 sec
Results :
Tests run: 42, Failures: 0, Errors: 0, Skipped: 0
{code}
> [MR-279] AppsKilled is never incremented
> ----------------------------------------
>
> Key: MAPREDUCE-2795
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2795
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Devaraj K
> Priority: Blocker
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MAPREDUCE-2795.patch
>
>
> AppsKilled metrics is never incremented even though there are killed jobs in the system.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira